Livy 0.7.0.200 - 2201 (EEP 8.1.0) Release Notes

The notes below relate specifically to the HPE Ezmeral Data Fabric Distribution for Apache Hadoop. You can find additional information on the Livy release notes page or Livy homepage.

These release notes contain only HPE-specific information and are not necessarily cumulative in nature. For information about how to use the release notes, see Ecosystem Component Release Notes.

Version 0.7.0.200
Release Date January 2022
HPE Version Interoperability See EEP Components and OS Support
Source on GitHub https://github.com/mapr/livy/tree/0.7.0.200-eep-810
GitHub Release Tag 0.7.0.200-eep-810
Package Names

Navigate to https://package.ezmeral.hpe.com/releases/MEP/ and select your EEP(MEP) and OS to view the list of package names.

New in This Release

  • MLIVY-96: Updated dependencies in Livy to be compatible with Spark 3.2 .
  • MLIVY-98: Fixed incompatibility of Livy Python modules with Python 3.8.
  • MLIVY-97, MLIVY-99: Ensured Livy worked on FIPS-enabled cluster. Added support of SCRAM-SHA-256 SASL mechanism for communication between Livy server and Livy session Spark Applications.
  • MLIVY-92: Updated dependencies in Livy to resolve CVE vulnerabilities.
  • MLIVY-100: Updated log4j 1.2.17 to log4j 1.3.1-mapr to resolve vulnerabilities.

Fixes

This HPE release includes the following fixes on the base release:

GitHub Commit Number Date (YYYY-MM-DD) HPE Fix Number and Description
1daea08 2021-12-09 MLIVY-96 Update dependencies to be compatible with Spark 3.2
96d6457 2021-12-24 Mock ast.Module to work with Python 3.8
78fc627 2022-01-04 MLIVY-70 Switch to log4j 1.3.0-mapr
7b018f7 2022-01-21 MLIVY-97 Fix Livy on FIPS-enabled cluster
f28d269 2022-01-24 MLIVY-99 Fix Livy for older cores
80417bf 2022-01-24 MLIVY-92 CVE fixes
4922f62 2022-01-25 MLIVY-92 Fix Livy after last round of CVE fixing
5c2abf2 2022-01-25 MLIVY-92 Resolve dependencies issues
ad6697d 2022-01-25 MLIVY-100 Update log4j v1 to the 1.3.1-mapr
700c1bb 2022-01-27 MLIVY-97 Change the way of enabling SCRAM-SHA-256 on FIPS setup
70263f8 2022-02-02 MLIVY-101 Build Livy ECO EEP 8.1.0 components with DF v 6.2.0

For complete details, refer to the commit log for this project in GitHub.

Resolved issues

  • MLIVY-98: Fixed incompatibility of Livy Python modules with Python 3.8.

Known Issues and Limitations

  • When you enable the SSL in a mixed (FIPS and non-FIPS) configuration, Spark application run fails. To run Spark applications, set spark.ssl.ui.enabled option to false in spark-defaults.conf configuration file.
  • Hive-compatible JDBC / ODBC server introduced in Livy 0.7 is not available in HPE distribution.