Configure SASL Encryption for Spark on YARN
As of Spark 1.6.1-1605, when the cluster is secure, SASL encryption between all the SparkMaster and SparkWorker nodes is enabled by default. When the cluster is not secure or for previous versions of Spark, complete the following steps to manually enable SASL encryption.
About this task
Procedure
-
Verify that authentication is enabled or configure authentication. SASL
encryption uses the same authentication keys.
For more information, see Configure Authentication for Spark on YARN
-
Configure the following property in the spark-defaults.conf file on each spark
node.
spark.authenticate.enableSaslEncryption true
The spark-defaults.conf file is in the following location:
/opt/mapr/spark/spark-<version>/conf/