Cluster Resource Allocation
In a MapR Hadoop cluster, warden sets the default resource allocation for the operating system, MapR-FS, MapR Hadoop services, and MapReduce v1 and YARN applications. Warden allocates resources to MapR Hadoop services and applications based on the roles installed on a node. For example, warden allocates resource for YARN applications on nodes with NodeManager role installed and it allocates resources to MapReduce v1 applications on nodes with the TaskTracker role installed.
In general, you should not need to override the values set in the default configuration files and by warden. However, users can provide updated values by adding or updating parameters in the Hadoop site configuration files or warden files. To override parameter values for a single job, the option can be overridden in the command line when submitting a YARN or MapReduce v1 application to the cluster.
To determine the current value
of a hadoop parameter, run hadoop conf | grep
<ParameterName>. In
the following example, the hadoop conf command was
used to get the value of mapreduce.map.memory.md:
# hadoop conf | grep mapreduce.map.memory.mb
<property><name>mapreduce.map.memory.mb</name><value>1024</value><source>mapred-site.xml</source></property>Alternatively, run
hadoop conf
without the grep
command to get a full list of the current parameter values. To
determine the current value of a warden parameter, open the warden
files located in the following directories:
/opt/mapr/conf/conf.d and
/opt/mapr/conf.
In some cases, the current value of the parameter can only be seen in the MCS or in Resource Manager or Task Tracker UI.
You can perform the following procedures to allocate memory and resources in a MapR cluster: