Logging Options
In a MapR cluster, the logging option that you configure defines how the logs are stored and accessed.
- Centralized logging. The logs are written to local volumes on the MapR-FS.
- YARN log aggregation. The logs are written to the local file system and then the container logs from each node are aggregated and stored on the MapR-FS.
-
Local logging. The log files for each job or
application are written to the local file system. This is the
default behavior for MapReduce v1
jobs, MapReduce v2 applications, and non-MapReduce
applications.
- For MapReduce v1 jobs, logs are written to the following
directory on the local filesystem:
/opt/mapr/hadoop/hadoop-0.20.0/logs/userlogs
- For MapReduce v2 or other applications that run on YARN, the
logs are written to the following directory on the local
filesystem:
/opt/mapr/hadoop/hadoop-2.x.x/logs/userlogs/
- For MapReduce v1 jobs, logs are written to the following
directory on the local filesystem:
The logging options that you can choose from are determined by the type of jobs or applications that you run:
Type of Job or Application | Available Logging Options |
---|---|
MapReduce v2 |
|
YARN applications (non- MapReduce) |
|
MapReduce v1 |
|
If you enable centralized logging for MapReduce v2, the MapReduce applications will use centralized logging, while the other YARN applications in the cluster will use local logging.
Select a logging option that stores the logs on the MapR-FS for the following reasons:
- Prevent job or application failures due to a lack of space on the local file system for logs.
-
Prevent the loss or inaccessibility of logs due to node failure. Logs stored in a local volume are two-way replicated.