Spark 3.3.3.0 (EEP 9.2.1) Release Notes
This section provides reference information, including new features, patches, and known issues for Spark 3.3.3.0.
The notes below relate specifically to the Hewlett Packard Enterprise Distribution for Apache Hadoop. For more information, you may also want to consult the open-source Spark 3.3.3 Release Notes.
These release notes contain only Hewlett Packard Enterprise specific information and are not necessarily cumulative in nature. For information about how to use the release notes, see Ecosystem Component Release Notes.
Spark Version | 3.3.3.0 |
Release Date | January 2024 |
HPE Version Interoperability | See Component Versions for Released EEPs and EEP Components and OS Support. |
Source on GitHub | https://github.com/mapr/spark |
GitHub Release Tag | 3.3.3.0-eep-921 |
Maven Artifacts | https://repository.mapr.com/maven/ |
Package Names | Navigate to https://package.ezmeral.hpe.com/releases/MEP/ and select your EEP and OS to view the list of package names. |
Hive Support
- Starting from Spark 3.1.2, Spark supports Hive 2.3.
Delta Lake Support
Spark 3.2.0 and later provides Delta Lake support on HPE Ezmeral Data Fabric. See Apache Spark Feature Support.
New in This Release
- For a complete list of new features, see the open-source Spark 3.3.3 Release Notes.
- Bug fixes.
- CVE fixes.
Fixes
This HPE release includes the following new fixes since the latest Spark release. For details, refer to the commit log for this project in GitHub.
GitHub Commit | Date (YYYY-MM-DD) | Comment |
fa4a702 | 28/12/2023 | [SPARK-1246] Spark read from parquet is failing |
652ee53 | 19/12/2023 | [SPARK-1226] Run, analyze and fix CVE issues based on scan of Spark package |
86ffc3f | 14/12/2023 | [SPARK-1241] Update Guava version to the latest to fix the CVEs |
c44454e | 13/12/2023 | [SPARK-1243] Prepare Spark 3.3.3 to weekly release |
1e11cc2 | 21/11/2023 | [SPARK-1225] Move dist dir to devops/dist |
d9cf0bb | 21/11/2023 | [SPARK-1237] "No such file or directory" when installing Spark 3.3.3 packages |
6c09a9c | 14/11/2023 | [SPARK-1225] Invoke deploy during first build |
8973558 | 14/11/2023 | [DFDEVOPS-3051] Update scala-maven-plugin to prevent StackOverflow failures during build |
372f649 | 14/11/2023 | [DFDEVOPS-3022] Add an option to easily modify build images per component |
903a3d9 | 14/11/2023 | [SPARK-1225] Final improvements for build scripts and add some documentation of build workflow |
3f44c4f | 14/11/2023 | [EZAF-1127] Fix name of Spark Classpath Filter package |
af1f597 | 08/11/2023 | [SPARK-1228] "Warning: Ignoring non-Spark config property: maprfs" message appears when spark session is starting |
53ce510 | 06/11/2023 | [SPARK-1227] Backport Spark-3.3.3 to EEP |
7e1d633 | 02/11/2023 | [EZAF-3538] EEP Spark logging is different from Apache |
310ca51 | 01/11/2023 | MapR [SPARK-1220] Reported vulnerabilities in MEP 6.3.0 in mapr-spark-2.4.4 |
438d76f | 26/10/2023 | [SPARK-1165] Handling of MapR-ES re-subscription in Spark job |
5e327d3 | 25/10/2023 | [SPARK-1029] Symlinks are not working with Parquet files |
8b9f639 | 25/10/2023 | [SPARK-1140] Spark job fails on standalone cluster mode |
f71f47f | 09/10/2023 | [SPARK-1216] java.net.URISyntaxException: Relative path in absolute URI: ${system:user.name%7D |
9b4c72b | 09/10/2023 | [SPARK-1121] Incorrect value INSTALL_DIR=/home//__spark-internal__/security_keys |
f77b049 | 09/10/2023 | [SPARK-1144] Spark job fails on FIPS cluster |
Known Issues and Limitations
- FIPS in mixed mode not supported with enabled SSL for WebUI.
- SPARK-1099: Non-mapr user is unable to insert values into Hive table by using Spark Thrift Server
Resolved Issues
- None.