Configure Authentication for Spark on YARN
As of Spark 1.5.2-1603 and Spark 1.6.1-1604, when the cluster is secure, authentication via a shared secret is enabled by default between all of the SparkMaster and SparkWorker nodes. When authentication is enabled, authentication keys are randomly generated for each job. When the cluster is not secure or for previous versions of Spark, complete the following step to manually configure authentication.
About this task
Procedure
spark.authenticate true
The spark-defaults.conf file is in the following location:
/opt/mapr/spark/spark-<version>/conf/