Using a Different Scheduler
Myriad uses MyriadFairScheduler by default. However, MyriadCapacityScheduler and MyriadFifoScheduler can also be used to run jobs.
About this task
The Myriad Scheduler property is set in
the
yarn-site.xml
file:
<property>
<name>yarn.resourcemanager.scheduler.class</name>
<value>org.apache.myriad.scheduler.yarn.MyriadFairScheduler</value>
<description>You can configure other schedulers from following list:
org.apache.myriad.scheduler.yarn.MyriadCapacityScheduler,
org.apache.myriad.scheduler.yarn.MyriadFifoScheduler</description>
</property>
IMPORTANT: Fine-grained scaling only works with a MyriadFairScheduler
configuration.
To use either MyriadCapacityScheduler or MyriadFifoScheduler:
Procedure
- Edit the
/opt/mapr/hadoop/hadoop-2.7.0/etc/hadoop/yarn-site.xml
file. -
For the
yarn.resourcemanager.scheduler.class
property, specify eitherorg.apache.myriad.scheduler.yarn.MyriadCapacityScheduler
ororg.apache.myriad.scheduler.yarn.MyriadFifoScheduler
-
Modify the YARN scheduler minimum allocation properties.
The configure.sh sets these properties to zero (0) by default. When using a different scheduler, the properties must be set (as a minimum) to the following values:
yarn.scheduler.minimum-allocation-mb = 1024 yarn.scheduler.minimum-allocation-vcores = 1 yarn.scheduler.minimum-allocation-disks = 0
-
Copy or update the
yarn-site.xml
file on every node in the cluster. - Restart the initial Resource Manager from Marathon.