Spark 3.3.3.0 (EEP 9.2.1) Release Notes

This section provides reference information, including new features, patches, and known issues for Spark 3.3.3.0.

The notes below relate specifically to the Hewlett Packard Enterprise Distribution for Apache Hadoop. For more information, you may also want to consult the open-source Spark 3.3.3 Release Notes.

These release notes contain only Hewlett Packard Enterprise specific information and are not necessarily cumulative in nature. For information about how to use the release notes, see Ecosystem Component Release Notes.

Spark Version 3.3.3.0
Release Date January 2024
HPE Version Interoperability See Component Versions for Released EEPs and EEP Components and OS Support.
Source on GitHub https://github.com/mapr/spark
GitHub Release Tag 3.3.3.0-eep-921
Maven Artifacts https://repository.mapr.com/maven/
Package Names Navigate to https://package.ezmeral.hpe.com/releases/MEP/ and select your EEP and OS to view the list of package names.

Hive Support

  • Starting from Spark 3.1.2, Spark supports Hive 2.3.

Delta Lake Support

Spark 3.2.0 and later provides Delta Lake support on HPE Ezmeral Data Fabric. See Apache Spark Feature Support.

New in This Release

Fixes

This HPE release includes the following new fixes since the latest Spark release. For details, refer to the commit log for this project in GitHub.

GitHub Commit Date (YYYY-MM-DD) Comment
fa4a702 28/12/2023 [SPARK-1246] Spark read from parquet is failing
652ee53 19/12/2023 [SPARK-1226] Run, analyze and fix CVE issues based on scan of Spark package
86ffc3f 14/12/2023 [SPARK-1241] Update Guava version to the latest to fix the CVEs
c44454e 13/12/2023 [SPARK-1243] Prepare Spark 3.3.3 to weekly release
1e11cc2 21/11/2023 [SPARK-1225] Move dist dir to devops/dist
d9cf0bb 21/11/2023 [SPARK-1237] "No such file or directory" when installing Spark 3.3.3 packages
6c09a9c 14/11/2023 [SPARK-1225] Invoke deploy during first build
8973558 14/11/2023 [DFDEVOPS-3051] Update scala-maven-plugin to prevent StackOverflow failures during build
372f649 14/11/2023 [DFDEVOPS-3022] Add an option to easily modify build images per component
903a3d9 14/11/2023 [SPARK-1225] Final improvements for build scripts and add some documentation of build workflow
3f44c4f 14/11/2023 [EZAF-1127] Fix name of Spark Classpath Filter package
af1f597 08/11/2023 [SPARK-1228] "Warning: Ignoring non-Spark config property: maprfs" message appears when spark session is starting
53ce510 06/11/2023 [SPARK-1227] Backport Spark-3.3.3 to EEP
7e1d633 02/11/2023 [EZAF-3538] EEP Spark logging is different from Apache
310ca51 01/11/2023 MapR [SPARK-1220] Reported vulnerabilities in MEP 6.3.0 in mapr-spark-2.4.4
438d76f 26/10/2023 [SPARK-1165] Handling of MapR-ES re-subscription in Spark job
5e327d3 25/10/2023 [SPARK-1029] Symlinks are not working with Parquet files
8b9f639 25/10/2023 [SPARK-1140] Spark job fails on standalone cluster mode
f71f47f 09/10/2023 [SPARK-1216] java.net.URISyntaxException: Relative path in absolute URI: ${system:user.name%7D
9b4c72b 09/10/2023 [SPARK-1121] Incorrect value INSTALL_DIR=/home//__spark-internal__/security_keys
f77b049 09/10/2023 [SPARK-1144] Spark job fails on FIPS cluster

Known Issues and Limitations

  • FIPS in mixed mode not supported with enabled SSL for WebUI.
  • SPARK-1099: Non-mapr user is unable to insert values into Hive table by using Spark Thrift Server

Resolved Issues

  • None.