Spark 3.5.1.0 (EEP 9.3.0) Release Notes

This section provides reference information, including new features, patches, and known issues for Spark 3.5.1.0.

The notes below relate specifically to the Hewlett Packard Enterprise Distribution for Apache Hadoop. For more information, you may also want to consult the open-source Spark 3.5.1 Release Notes.

These release notes contain only Hewlett Packard Enterprise specific information and are not necessarily cumulative in nature. For information about how to use the release notes, see Ecosystem Component Release Notes.

Spark Version 3.5.1.0
Release Date July 2024
HPE Version Interoperability See Component Versions for Released EEPs and EEP Components and OS Support.
Source on GitHub https://github.com/mapr/spark
GitHub Release Tag 3.5.1.0-eep-930
Maven Artifacts https://repository.mapr.com/maven/
Package Names Navigate to https://package.ezmeral.hpe.com/releases/MEP/ and select your EEP and OS to view the list of package names.

Hive Support

  • Starting from Spark 3.1.2, Spark supports Hive 2.3.

New in This Release

Spark 3.3.5.1 introduces the following enhancements or HPE platform-specific behavior changes:

  • Bug fixes
  • CVE fixes

Fixes

This HPE release includes the following new fixes on the base Apache release. For details, refer to the commit log for this project in GitHub.

GitHub Commit Date (YYYY-MM-DD) Comment
e21e9c6 13/06/2024 MapR [SPARK-1300] ssl_keystore/ssl_truststore copied to incorrect location
d39d95e 12/06/2024 MapR [SPARK-1292] configure.sh -R add new line to spark-defaults.conf
3e81f5d 07/06/2024 MapR [SPARK-1298] RDD Bulk load from hbase table fails
4b5bf84 06/06/2024 MapR [SPARK-1294] DStream doesn't start on Spark 3.5.1 and Core 7.8.0 combination
b894b8d 30/05/2024 MapR [SPARK-1297] [Spark 3.5.1 + Java 17] Beeline command execution fails
4c4e059 22/05/2024 MapR [SPARK-1293] Spark 3.5.1 doesn't work with Kerberos
e3af08a 06/05/2024 MapR [SPARK-1286] Pyspark + Panda integration doesn't work
6c58f91 03/05/2024 MapR [SPARK-1285] Upgrade from Spark 2.4.x to Spark 3.3.2 causes the EOF handling of Kafka Consumers to fail
d5bb2ab 16/04/2024 MapR [SPARK-1267] Cross Validation fails with "Caused by: java.lang.IllegalArgumentException: Null user"
05d6229 05/04/2024 [EZAF-4620] Update Go to 1.21.5
1b523a2 28/03/2024 MapR [SPARK-1272] Backport the SPARK-1097 into 9.1.1 and 9.2.1
65cae5f 28/03/2024 MapR [SPARK-1103] Excessive logs for spark beeline
942fbae 27/03/2024 MapR [SPARK-1261] Update Spark to v3.5.1

Known Issues and Limitations

  • FIPS in mixed mode not supported with enabled SSL for WebUI.
  • SPARK-1099: Non-mapr user is unable to insert values into Hive table by using Spark Thrift Server

Resolved Issues

  • None.