Spark 1.6.1-1609 Release Notes

The notes below relate specifically to the MapR Converged Data Platform. You may also be interested in the open source Spark 1.6.1 Release Notes

Spark Version 1.6.1
Release Date September 30, 2016
MapR Version Interoperability See the Interoperability Matrices, Ecosystem Support Matrix (Pre-5.2 releases), and Spark Support Matrix.
Source on GitHub https://github.com/mapr/spark/tree/1.6.1-mapr-1609
GitHub Release Tag 1.6.1-mapr-1609
Package Names The following packages are associated with this release:
  • mapr-spark-1.6.1.201609271200-1.noarch.rpm
  • mapr-spark_1.6.1.201609271200_all.deb
  • mapr-spark-historyserver-1.6.1.201609271200-1.noarch.rpm
  • mapr-spark-historyserver_1.6.1.201609271200_all.deb
  • mapr-spark-master-1.6.1.201609271200-1.noarch.rpm
  • mapr-spark-master_1.6.1.201609271200_all.deb

Important Notes

  • If you want to integrate Spark 1.6.1-1609 with HPE Ezmeral Data Fabric Streams, you must install the Kafka 0.9.0-1607 package.
  • Full support of HPE Ezmeral Data Fabric Streams is available only on MapR 5.2 clusters.

Hive Support

This version of Spark supports integration with Hive. However, note the following exceptions:

Fixes

This release by MapR includes the following new fixes since the previous release of MapR Spark 1.6.1. For complete details, refer to the commit log for this project in GitHub.

GitHub Commit Date (YYYY-MM-DD) Comments
2b4ae57 2016-09-19 [MAPR-24603] This fix fixes the issue where a beeline shell could not be launched after starting Spark.
b2d53e7 2016-09-20 [MAPR-24491] This fix fixes the issue where the HBase classpath might contain Hive libraries.
b9de0fb 2016-09-26 [MAPR-24678] This fix fixes the issue where PySpark is unable to consume from HPE Ezmeral Data Fabric Streams.

Known Issues

  • MAPR-17271: On secure clusters, the MapR Control System (MCS) does not display links for Spark-Master and Spark-HistoryServer.
  • MAPR-19761: On a secure cluster, MapR does not support the Spark SQL Thrift JDBC server. When the cluster is secure, the Spark Thrift server will not start.
  • Spark versions up to and including 2.3.0 have the following security vulnerability: CVE-2018-1334 Apache Spark local privilege escalation vulnerability