hadoop-metrics.properties

The hadoop-metrics.properties files direct MapR where to output service metric reports: to an output file (FileContext) or to Ganglia 3.1 (MapRGangliaContext31). A third context, NullContext, disables metrics. To direct metrics to an output file, comment out the lines pertaining to Ganglia and the NullContext; for the chosen service; to direct metrics to Ganglia, comment out the lines pertaining to the metrics file and the NullContext. See Service Metrics.

There are two hadoop-metrics.properties files:

  • /opt/mapr/hadoop/hadoop-<version>/conf/hadoop-metrics.properties specifies output for standard Hadoop services
  • /opt/mapr/conf/hadoop-metrics.properties specifies output for MapR-specific services

The following table describes the parameters for each service in the hadoop-metrics.properties files.

Parameter

Example Values

Description

<service>.class
  • org.apache.hadoop.metrics.spi.NullContextWithUpdateThread
  • apache.hadoop.metrics.file.FileContext
  • com.mapr.fs.cldb.counters.MapRGangliaContext31

The class that implements the interface responsible for sending the service metrics to the appropriate handler. When implementing a class that sends metrics to Ganglia, set this property to the class name.

<service>.period
  • 10
  • 60

The interval between 2 service metrics data exports to the appropriate interface. This is independent of how often are the metrics updated in the framework.

<service>.fileName

/tmp/cldbmetrics.log

The path to the file where service metrics are exported when the cldb.class property is set to FileContext.

<service.servers

localhost:8649

The location of the gmon or gmeta that is aggregating metrics for this instance of the service, when the cldb.class property is set to GangliaContext.

<service>.spoof

1

Specifies whether the metrics being sent out from the server should be spoofed as coming from another server. All our fileserver metrics are also on cldb, but to make it appear to end users as if these properties were emitted by fileserver host, we spoof the metrics to Ganglia using this property. Currently only used for the FileServer service.

Examples

The hadoop-metrics.properties files are organized into sections for each service that provides metrics. Each section is divided into subsections for the three contexts.

/opt/mapr/hadoop/hadoop-<version>/conf/hadoop-metrics.properties 
# Configuration of the "dfs" context for null
dfs.class=org.apache.hadoop.metrics.spi.NullContext

# Configuration of the "dfs" context for file
#dfs.class=org.apache.hadoop.metrics.file.FileContext
#dfs.period=10
#dfs.fileName=/tmp/dfsmetrics.log

# Configuration of the "dfs" context for ganglia
# Pick one: Ganglia 3.0 (former) or Ganglia 3.1 (latter)
# dfs.class=org.apache.hadoop.metrics.ganglia.GangliaContext
# dfs.class=org.apache.hadoop.metrics.ganglia.GangliaContext31
# dfs.period=10
# dfs.servers=localhost:8649


# Configuration of the "mapred" context for null
mapred.class=org.apache.hadoop.metrics.spi.NullContext

# Configuration of the "mapred" context for file
#mapred.class=org.apache.hadoop.metrics.file.FileContext
#mapred.period=10
#mapred.fileName=/tmp/mrmetrics.log

# Configuration of the "mapred" context for ganglia
# Pick one: Ganglia 3.0 (former) or Ganglia 3.1 (latter)
# mapred.class=org.apache.hadoop.metrics.ganglia.GangliaContext
# mapred.class=org.apache.hadoop.metrics.ganglia.GangliaContext31
# mapred.period=10
# mapred.servers=localhost:8649


# Configuration of the "jvm" context for null
#jvm.class=org.apache.hadoop.metrics.spi.NullContext

# Configuration of the "jvm" context for file
#jvm.class=org.apache.hadoop.metrics.file.FileContext
#jvm.period=10
#jvm.fileName=/tmp/jvmmetrics.log

# Configuration of the "jvm" context for ganglia
# jvm.class=org.apache.hadoop.metrics.ganglia.GangliaContext
# jvm.period=10
# jvm.servers=localhost:8649


# Configuration of the "ugi" context for null
ugi.class=org.apache.hadoop.metrics.spi.NullContext


# Configuration of the "fairscheduler" context for null
#fairscheduler.class=org.apache.hadoop.metrics.spi.NullContext

# Configuration of the "fairscheduler" context for file
#fairscheduler.class=org.apache.hadoop.metrics.file.FileContext
#fairscheduler.period=10
#fairscheduler.fileName=/tmp/fairschedulermetrics.log

# Configuration of the "fairscheduler" context for ganglia
# fairscheduler.class=org.apache.hadoop.metrics.ganglia.GangliaContext
# fairscheduler.period=10
# fairscheduler.servers=localhost:8649
#
/opt/mapr/conf/hadoop-metrics.properties 
###########################################################################################################################
                                                      hadoop-metrics.properties
###########################################################################################################################

#CLDB metrics config - Pick one out of null,file or ganglia.
#Uncomment all properties in null, file or ganglia context, to send cldb metrics to that context

# Configuration of the "cldb" context for null
#cldb.class=org.apache.hadoop.metrics.spi.NullContextWithUpdateThread
#cldb.period=10

# Configuration of the "cldb" context for file
#cldb.class=org.apache.hadoop.metrics.file.FileContext
#cldb.period=60
#cldb.fileName=/tmp/cldbmetrics.log

# Configuration of the "cldb" context for ganglia
cldb.class=com.mapr.fs.cldb.counters.MapRGangliaContext31
cldb.period=10
cldb.servers=localhost:8649
cldb.spoof=1

#FileServer metrics config - Pick one out of null,file or ganglia.
#Uncomment all properties in null, file or ganglia context, to send fileserver metrics to that context

# Configuration of the "fileserver" context for null
#fileserver.class=org.apache.hadoop.metrics.spi.NullContextWithUpdateThread
#fileserver.period=10

# Configuration of the "fileserver" context for file
#fileserver.class=org.apache.hadoop.metrics.file.FileContext
#fileserver.period=60
#fileserver.fileName=/tmp/fsmetrics.log

# Configuration of the "fileserver" context for ganglia
fileserver.class=com.mapr.fs.cldb.counters.MapRGangliaContext31
fileserver.period=37
fileserver.servers=localhost:8649
fileserver.spoof=1

################################################################################################################