Spark 1.6.1-1609 Release Notes
The notes below relate specifically to the MapR Converged Data Platform. You may also be interested in the open source Spark 1.6.1 Release Notes
Spark Version | 1.6.1 |
Release Date | September 30, 2016 |
MapR Version Interoperability | See the Interoperability Matrices, Ecosystem Support Matrix (Pre-5.2 releases), and Spark Support Matrix. |
Source on GitHub | https://github.com/mapr/spark/tree/1.6.1-mapr-1609 |
GitHub Release Tag | 1.6.1-mapr-1609 |
Package Names | The following packages are associated with this release:
|
Important Notes
- If you want to integrate Spark 1.6.1-1609 with HPE Ezmeral Data Fabric Streams, you must install the Kafka 0.9.0-1607 package.
- Full support of HPE Ezmeral Data Fabric Streams is available only on MapR 5.2 clusters.
Hive Support
This version of Spark supports integration with Hive. However, note the following exceptions:
- Hive on Spark is not supported.
- Spark-SQL is supported, but it is not fully compatible with Hive; see the Apache Spark documentation and MapR Spark documentation for details.
Fixes
This release by MapR includes the following new fixes since the previous release of MapR Spark 1.6.1. For complete details, refer to the commit log for this project in GitHub.
GitHub Commit | Date (YYYY-MM-DD) | Comments |
---|---|---|
2b4ae57 | 2016-09-19 | [MAPR-24603] This fix fixes the issue where a beeline shell could not be launched after starting Spark. |
b2d53e7 | 2016-09-20 | [MAPR-24491] This fix fixes the issue where the HBase classpath might contain Hive libraries. |
b9de0fb | 2016-09-26 | [MAPR-24678] This fix fixes the issue where PySpark is unable to consume from HPE Ezmeral Data Fabric Streams. |
Known Issues
- MAPR-17271: On secure clusters, the MapR Control System (MCS) does not display links for Spark-Master and Spark-HistoryServer.
- MAPR-19761: On a secure cluster, MapR does not support the Spark SQL Thrift JDBC server. When the cluster is secure, the Spark Thrift server will not start.
- Spark versions up to and including 2.3.0 have the following security vulnerability: CVE-2018-1334 Apache Spark local privilege escalation vulnerability