What's New in Release 8.0

Describes the new features in release 8.0 and provides links to more information.

Following are some new features and capabilities that distinguish release 8.0.0 from the previous release (7.10.0):

Product branding changes

HPE Ezmeral Data Fabric is HPE Data Fabric from release 8.0.0.

HPE Ezmeral Ecosystem Pack is Data Fabric Ecosystem Pack from Data Fabric Ecosystem Pack release 10.0.0.

For details on changes to product branding, see Data Fabric Branding Changes.

AI readiness with HPE Data Fabric MCP (Model Context Protocol) server

The HPE Data Fabric MCP server facilitates a secure model-to-data interaction and provides compatibility with diverse AI tools. Any MCP-compliant client and LLM can securely query HPE Data Fabric by using a MCP architecture and Spark-powered execution.

Every request is verified, scoped, and enforced at multiple layers to ensure explicit trust between services, users, or components.

The HPE Data Fabric MCP server is integrated with Keycloak, and supports LDAP, and any enterprise IAM. The HPE Data Fabric MCP server supports JWT tokens for fabrics that are integrated with Keycloak as the SSO provider.

All access flows are routed through the HPE Data Fabric MCP server.

Every access request is tied to a validated user identity and all access flows through the HPE Data Fabric MCP server.

See Model Context Protocol (MCP) for more information.

Java version support

With release 8.0.0, HPE Data Fabric requires JDK 17 as the minimum JDK version. If the JDK version is lower than 17, some of the critical services, including the Data Fabric UI, do not come up.

Before upgrading from an earlier HPE Data Fabric version to release 8.0, ensure that you install JDK 17. The java, javac commands must point to JDK 17.

In rare scenarios, the upgrade to JDK 17 could impact customer applications that are not compatible with JDK 17.

HPE Data Fabric 8.0.0 supports JDK 21.

See Java support matrix for information on JDK versions supported by different HPE Ezmeral Data Fabric releases.

Data Fabric UI improvements

For release 8.0.0, following is a summary of changes with respect to the Data Fabric UI.

Restructured layout for cleaner, unified and more intuitive user experience
Unified experience for all users and roles
Support for feature enablement based on user permissions, ensuring personalized access control
Support for various user roles and enable/disable features and Data Fabric UI components
Custom configuration for cloud deployment that enables you to define a custom or suitable cloud instance type, disk size and node counts suited for your workload, to meet performance, memory usage and cost requirements.
Custom database configurations for Data Fabric Ecosystem Pack (DEP) service installation while creating a fabric.
Upgrade a DEP service without recreating the fabric on which the service is installed.

All the existing functionality remains unchanged.

See Working with the Data Fabric UI for details.

NOTE

With release 7.3.0 and later, you can use the Data Fabric UI on Data Fabric clusters. To understand the limitations and benefits of doing so, see Data Fabric UI.

ATTENTION

Data Fabric Ecosystem Pack (DEP) components do not support IPv6.

gfsck scaling

The gfsck utility has new parameters introduced for better performance.

See gfsck for new parameters.

See Using Global File System Checking for gfsck behavior related to object size.

Polaris support

HPE Data Fabric 8.0 supports Polaris as a metadata catalog for Apache Iceberg table data.

A Data Fabric user can use engines like Flink or Dremio on Iceberg tables within Data Fabric through the Polaris catalog. The centralized metadata management in Polaris ensures atomic operations and consistency across distributed datasets, reducing errors in analytics pipelines.

Polaris’s REST API support enables seamless integration with diverse engines like Dremio, Snowflake, and Flink. This aligns with easy integration with Presto, Ray.

See Polaris 1.0.1.0 - 2510 (DEP 10.0.0) Release Notes and Apache Polaris for more information.

Flink support

HPE Data Fabric 8.0 supports Apache Flink distributed processing engine for stateful computations over unbounded and bounded data streams.

See Flink 2.0.0.0 - 2510 (DEP 10.0.0) Release Notes and Apache Flink for more information.

Data Fabric Ecosystem Pack version support

HPE Data Fabric 8.0.0 works with Data Fabric Ecosystem Pack 10.0.0.

See Ecosystem Pack 10.0.0 Release Notes and What's New in DEP 10.0.0 for more information.

See Component Versions for Released Ecosystem Packs for details on the ecosystem component versions that comprise Data Fabric Ecosystem Pack 10.0.0.

Keycloak support

Keycloak installation is optional when you install a fabric. You can choose to not install Keycloak when you are installing HPE Data Fabric.

Important features introduced in the earlier releases

The major features introduced in the earlier Data Fabric releases are listed below for your reference.

Support for HPE Alletra Storage MP X10000 as an External S3 Storage

Data Fabric supports the import of HPE Alletra MP X10000 object storage as an external S3 server into a cluster group or global namespace by using the maprcli and the Data Fabric UI. For more information, see clustergroup addexternal. To import HPE Alletra MP X10000 from the Data Fabric UI, see Importing an external S3 server.

Librdkafka Updated to Version 2.6.0

Beginning with release 7.10.0 and EEP 9.4.0, librdkafka is now updated to version 2.6.0.

IMPORTANT

*Librdkafka 2.6.0 in core 7.10 is not compatible with HPE Data Fabric Stream clients for HPE Data Fabric Streams C#/.NET Applications. Nodes running C# applications should not be upgraded to HPE Data Fabric 7.10.

The Data Fabric 7.10 release supports C API Librdkafka 2.6.0 for HPE Data Fabric Streams. See the following topics for details:

IPv6 Support

Release 7.6.1 introduced support for IPv6 addresses and support for clusters that have a mix of IPv6 and IPv4 addresses. A new option in the configure.sh script enables the configuration of IPv6 support. For more information, see IPv6 Support in Data Fabric and configure.sh.

Iceberg Support

Release 7.6.1 introduced support for the Iceberg table format. You can use Apache Spark to query Iceberg tables stored in the Data Fabric and perform other operations to take advantage of Iceberg features. For more information, see Iceberg Support and Getting Started with Iceberg.

Support for Third-Party Storage Solutions

With release 7.6.1, the HPE Data Fabric offers global namespace support for NFS- and S3-protocol operations performed against WEKA, VAST Data, Scality, and other third-party, object-storage solutions.

For more information, see Third-Party Storage Solutions. External S3 Support

Release 7.6.1 introduced support for mounting an external NFS server or an external S3 server (including third-party storage solutions) into a cluster group or global namespace. For more information, see clustergroup addexternal.

Assigning Multiple Security Policies to a Volume

Release 7.6.1 introduced support for attaching multiple security policies to one, or more than one, volume in a cluster. For more information, see policy attach.

Support for Third-Party Storage Solutions

With release 7.6.1, Data Fabric offers global namespace support for NFS- and S3-protocol operations performed against WEKA, VAST Data, Scality, and other third-party, object-storage solutions.

For more information, see Third-Party Storage Solutions.

Monitoring Packages Moved to Core Repository

Beginning with release 7.8.0 and EEP 9.3.0, the monitoring components are provided in the Data Fabric (core) repository and are not present in the EEP repository. Monitoring packages include the following:

mapr-collectd
mapr-elasticsearch
mapr-fluentd
mapr-grafana
mapr-kibana
mapr-opentsdb

For example, the RHEL monitoring packages for release 7.8.0 can be found at:

https://package.ezmeral.hpe.com/releases/v7.8.0/redhat/

The RHEL monitoring packages are not present in the repository for EEP 9.3.0 at:

https://package.ezmeral.hpe.com/releases/MEP/MEP-9.3.0/redhat/

The same is true for the monitoring packages for Rocky Linux, Ubuntu, SLES, and Oracle Enterprise Linux.

New Repository for NFS v4 on SLES 15

In release 7.8.0, the /releases directory of package.ezmeral.hpe.com includes two suse repositories:

The suse15.5 repository is new and enables support for NFS or NFS v4 for installations on SLES 15 SP3 and SP5. To use the old and new suse repositories, see this table:

Repository	Use this respository to install	NFS Support
`suse`	SLES 15 SP2	NFS
`suse15.5`	SLES 15 SP3 or SP4 or SP5	NFS or NFS v4

New Key for Signature Verification for Data Fabric Files

Release 7.6.1 implemented a new key for .rpm, .tar.gz, .zip, and .tgz files for the following Data Fabric products:

core 7.6.1 and later
Data Fabric clients
Ecosystem Pack (EEP) 9.2.1 and later
Installer 1.18.0.5 and later

Before you install these products, you must import the HPE GPG public keys. This is a one-time operation on each node where packages are installed. Importing the keys allows you to, optionally, verify the GPG and RPM signatures for the products. A verified GPG or RPM signature attests that the product you received has been signed with digital private keys held only by HPE. Successful signature verification also ensures that the file has not been altered after it was signed and released by HPE.

For more information, see HPE GPG Public Keys for GPG or RPM Signature Verification.

New Repository for Data Fabric Software

On August 1, 2023, Hewlett Packard Enterprise introduced a new download repository for the HPE Data Fabric core and ecosystem software packages. https://package.ezmeral.hpe.com/ is the new repository for HPE Ezmeral Data Fabric downloads. For all Data Fabric releases, https://package.ezmeral.hpe.com/ replaces two older repositories:

Description	URL	Authentication Required?
New repository	https://package.ezmeral.hpe.com/	Yes
Old repositories	https://package.mapr.com/ https://package.mapr.hpe.com	Yes¹

¹Beginning October 2023, the old repositories are redirected to the new repository URL, which requires authentication. The new repository requires you to provide the email and token for your HPE Passport account. Software that points to the old repositories must be updated to include your HPE Passport email and token. For more information about using the new repository, see Using the HPE Data Fabric Token-Authenticated Internet Repository.

If you plan to use the Data Fabric Installer, you must update the Installer to the most current 1.18.0.3 version or later. Earlier versions of the Installer will not work with the new repository. See Updating the Installer.

HPE Data Fabric Ecosystem Pack Support

Release 7.8.0 requires EEP 9.3.0 or later. EEP 9.3.0 is new for this release and delivers updates to various ecosystem components but no new components. EEP 9.3.0 can only be used with release 7.8.0.

For more information about new features delivered as part of the Ecosystem Pack, see What's New in EEP 9.3.0. For reference information, see EEP 9.3.0 Reference Information.

For information about the EEPs that can be used with different versions of core releases, see DEP Support and Lifecycle Status.

SSO Support for Keycloak

Release 7.8.0 supports SSO when configured with the Keycloak identity and access management (IAM) solution. Other IAM solutions are not currently supported.

Configuring SSO is optional. If you do not configure SSO, you must use Data Fabric user names and passwords for access to the fabric. While SSO is supported for Data Fabric core, it is not currently supported for ecosystem components or the Installer.

Beginning with release 7.5.0, Keycloak is preconfigured and preinstalled if you:

Install the mapr-keycloak package as part of cluster creation
Specify the -keycloak option when you run the configure.sh script

For more information, see Configuring SSO.

Data Access Gateway 6.3.0.1 Support

For the gateway to lightweight client applications, release 7.8.0 requires Data Access Gateway 6.3.0.1. Data Access Gateway 6.3.0.1 can be used with core 7.8.0, 7.7.0, 7.6.1, 7.5.0, 7.4.0, 7.3.0, and 7.2.0 with some restrictions. However, Data Access Gateway 6.3.0.1 is part of EEP 9.3.0, and EEP 9.3.0 is currently supported only for use with core 7.8.0.

CAUTION

Streams users who upgrade from release 7.1.0 (DAG 5.0) or release 7.2.0 (DAG 5.1) to one of the following releases will not be able to access topics configured using the pre-DAG 6.0 mapping rules:

7.8.0 (DAG 6.3.0.1)
7.7.0 (DAG 6.3)
7.6.1 (DAG 6.2)
7.5.0 (DAG 6.2)
7.4.0 (DAG 6.1)
7.3.0 (DAG 6.0)

For more information, see Understanding the HPE Data Fabric Data Access Gateway and the Data Access Gateway 6.3 Release Notes.

JDK 17 Support

As indicated in the Java Support Matrix, release 7.8.0 can be used in JDK 11 or in JDK 17 installations. EEP 9.3.0 can also be used in JDK 11 or JDK 17 installations. However, the Installer is supported only on JDK 11, and if new cluster nodes require a JDK at installation time, the Installer can only install JDK 11.