Pre-Upgrade Steps for Spark

About this task

You need to back up the configuration files if you made configuration changes that you want to carry over to the next version. Shut down services depending on your Spark platform (standalone or YARN).

Procedure

  1. Copy configuration files from /opt/mapr/spark/spark-<version>/conf to a location outside of the MapR installation directory.
    For example, if Spark SQL is configured to work with Hive, copy the /opt/mapr/spark/spark-<version>/conf/hive-site.xml file to a backup directory.
  2. Shut down services.
    • Standalone Spark:
      1. Shut down the spark-master, spark-historyserver services (if the spark-historyserver is running):
        maprcli node services -nodes <node-ip> -name spark-master -action stop
maprcli node services -nodes <node-ip> -name spark-historyserver -action stop
      2. As mapr user, stop the slaves:

        /opt/mapr/spark/spark-<version>/sbin/stop-slaves.sh

    • Spark on YARN:

      Shut down the spark-historyserver services (if the spark-historyserver is running):

      maprcli node services -nodes <node-ip> -name spark-historyserver -action stop