Enable Fair Scheduler
When changing any parameters in this section, a JobTracker restart is required.
|
Parameter |
Description |
|---|---|
|
mapred.fairscheduler.allocation.file |
Specifies the path to the allocation file. Example value: |
|
mapred.jobtracker.taskScheduler |
The class responsible for task scheduling. Example value: |
|
mapred.fairscheduler.assignmultiple |
A Boolean property that allows the scheduler to assign both a map task and a reduce task on each heartbeat. This improves cluster throughput when there are many small tasks to run Example value: |
|
mapred.fairscheduler.eventlog.enabled |
Enable scheduler logging in ${HADOOP_LOG_DIR}/fairscheduler/ Example value: false |
|
mapred.fairscheduler.smalljob.schedule.enable |
Set this property's value to False to disable fast scheduling for small jobs in FairScheduler. TaskTrackers can reserve an ephemeral slot for small jobs when the cluster is under load. Example value: |
|
mapred.fairscheduler.smalljob.max.maps |
Small job definition. Max number of maps allowed in small job. Example value: |
|
mapred.fairscheduler.smalljob.max.reducers |
Small job definition. Max number of reducers allowed in small job. Example value: |
|
mapred.fairscheduler.smalljob.max.inputsize |
Small job definition. Max input size in bytes allowed for a small job. Default is 10GB. Default value: |
|
mapred.fairscheduler.smalljob.max.reducer.inputsize |
Small job definition. Max estimated input size for a reducer allowed in small job. Default is 1GB per reducer. Default value: |
|
mapred.cluster.ephemeral.tasks.memory.limit.mb |
Small job definition. Max memory in mbytes reserved for an ephermal slot. Default is 200mb. This value must be same on JobTracker and TaskTracker nodes. Default value: |