About Release 7.9.0
This site contains documentation for HPE Ezmeral Data Fabric release 7.9.0, including installation, configuration, administration, and reference content, as well as content for the associated ecosystem components and drivers.
7.9.0 Installation
This section contains information about installing and upgrading HPE Ezmeral Data Fabric software. It also contains information about how to migrate data and applications from an Apache Hadoop cluster to a HPE Ezmeral Data Fabric cluster.
7.9.0 Data Fabric
HPE Ezmeral Data Fabric is the industry-leading data platform for AI and analytics that solves enterprise business needs.
7.9.0 Administration
This section describes how to manage the nodes and services that make up a cluster.
- Administering Users and Clusters
  Lists topics that help manage a data-fabric cluster.
- Administering Nodes
  Provides a synopsis of managing nodes in a cluster.
- Administering Volumes
  This section provide information about how to organize and manage data using volumes, a unique feature of HPE Ezmeral Data Fabric clusters.
- Administering Files and Directories
- Administering Tables
  Administration of the HPE Ezmeral Data Fabric Database is done primarily via the command line (maprcli) or with the Managed Control System (MCS). Regardless of whether the HPE Ezmeral Data Fabric Database table is used for binary files or JSON documents, the same types of commands are used with slightly different parameter options. HPE Ezmeral Data Fabric Database administration is associated with tables, columns and column families, and table regions.
- Administering Streams
- Administering Data Fabric Gateways
  A HPE Ezmeral Data Fabric gateway mediates one-way communication between a source HPE Ezmeral Data Fabric cluster and a destination cluster. You can replicate HPE Ezmeral Data Fabric Database tables (binary and JSON) and HPE Ezmeral Data Fabric Streams streams. HPE Ezmeral Data Fabric gateways also apply updates from JSON tables to their secondary indexes and propagate Change Data Capture (CDC) logs.
- Administering Services
- Monitoring the Cluster
  This section describes how to monitor the health and performance of a MapR cluster.
  - Monitoring Using the Control System and the CLI
    Describes the Overview page in the Control System, which displays information about the cluster.
  - Using HPE Ezmeral Data Fabric Monitoring (Spyglass Initiative)
    HPE Ezmeral Data Fabric Monitoring (part of the Spyglass initiative) provides the ability to collect, store, and view metrics and logs for nodes, services, and jobs/applications.
    - HPE Ezmeral Data Fabric Monitoring Architecture
      HPE Ezmeral Data Fabric Monitoring integrates with open-source components to collect, aggregate, store, and visualize metrics and logs.
    - Metric Collection
      Metrics are collected from each node in the cluster so that administrators can use the data to monitor the cluster. In general, the collectd service collects metrics every 10 seconds. The exception is volume metrics which are collected every 10 minutes.
    - Configure the OpenTSDB Service Heap Size
      By default, the OpenTSDB service is configured to use a default heap size of 6 gigabytes. The default heap size can be adjusted by modifying certain configuration files.
    - Metric Visualization
      Use dashboards to visualize metrics across multiple nodes and clusters.
    - Log Collection
      Fluentd collects log events from each node in the cluster and stores them in a centralized location so that administrators can search the logs when troubleshooting issues in the cluster. The process that fluentd uses to parse and send log events to Elasticsearch differs based on the formatting of log events in each log file.
    - Log Aggregation and Storage
      Fluentd uses a round-robin approach when writing logs to Elasticsearch nodes. If an Elasticsearch node in unavailable, Fluentd can fail over log storage to another Elasticsearch node.
      - Configure Log Retention
        By default, Elasticsearch indexes 2 days of logs. Based on your requirements, you can configure a different retention period for Elasticsearch.
      - Configure Log Rotation Policies for Monitoring Services
        New log files are created based on the log rotation policy. By default, each Monitoring service has a log rotation policy. In most cases, you can change the policy based on your requirements.
      - Configure the Elasticsearch Service Heap Size
        The Elasticsearch service is memory-intensive. By default, the Elasticsearch service is configured to use a minimum and maximum heap size of 2 GB. You can override these default values by making changes in the Elasticsearch Warden configuration file and the jvm.options file. Restart Elasticsearch after you modify the settings.
      - Configure Fluentd Services to Write to Elasticsearch Nodes on the Same Rack
        On clusters with high-density racks, ensure you have at least one Elasticsearch server per rack and configure each Fluentd service to write to Elasticsearch nodes that run on the same rack as the Fluentd service. This configuration minimizes the impact of log aggregation on other processes that run on the cluster and in particular, minimizes the amount of backbone bandwidth used by the log aggregation.
      - 60-mapr_elasticsearch.conf
      - 60-mapr_fluentd.conf
    - Log Visualization
      Use dashboards to visualize the logs across multiple nodes and clusters.
    - HPE Ezmeral Data Fabric Monitoring Tips and Troubleshooting
      Lists the nuances of monitoring clusters.
    - Reconfiguring Data Fabric Monitoring
      Changes to an existing cluster, such as the addition of services, may require additional steps to enable the collection of metrics and logs.
  - Configuring Data Fabric to Track User Behavior
    Describes how to configure Data Fabric to track user behavior.
- Configuring Security
  Describes how to configure security and manage secure clusters.
- Managing Secure Clusters
  Provides procedures that will enable you to use Data Fabric clusters securely.
- Administering the Data Access Gateway
  The HPE Ezmeral Data Fabric Data Access Gateway is a service that acts as a proxy and gateway for translating requests between lightweight client applications and the HPE Ezmeral Data Fabric cluster. This section describes considerations when upgrading the service, how to modify configuration settings, and how to administer and manage the service.
- Planning for High Availability
- Administrator's Reference
  This section contains in-depth reference information for the administrator.
- Troubleshooting Cluster Administration
  Lists the common errors and their solutions.
- Best Practices for Backing Up HPE Ezmeral Data Fabric Information
  Lists the best practices and performance considerations to follow when backing up HPE Ezmeral Data Fabric information.
- IPv6 Support in Data Fabric
  Describes the IPv6 support feature for Data Fabric.
7.9.0 Development
This section contains information related to application development for Ezmeral ecosystem components and HPE Ezmeral Data Fabric products, including the file system, Database (Key-Value and JSON), and Event Streams.
Other Docs
This section contains release-independent information, including: Installer documentation, Ecosystem release notes, interoperability matrices, security vulnerabilities, and links to other data-fabric version documentation.
Glossary
Definitions for commonly used terms in MapR Converged Data Platform environments.

Configure Log Rotation Policies for Monitoring Services

New log files are created based on the log rotation policy. By default, each Monitoring service has a log rotation policy. In most cases, you can change the policy based on your requirements.

OpenTSDB Log Rotation Policy

By default, OpenTSDB creates a new log file when each log file reaches the maximum file size of 128MB. After 4 log files are generated, it deletes the oldest log file.

To change the log rotation policy, edit the following file: /opt/mapr/opentsdb/opentsdb-<version>/etc/opentsdb/logback.xml. For more information, see the OpenTSDB Logging documentation.

Fluentd and CollectD Log Rotation Policy

By default, Fluentd and Collectd create a new log file each day and they both retain 30 log files. Log rotation for Fluentd and Collectd logs is managed by logrotate.

To change the log rotation policies, edit the following files: /etc/logrotate.d/fluentd and etc/logrotate.d/collectd. For details on how to update the log rotation policy, see the logrotate documentation.

Elasticsearch Log Rotation Policy

By default, Elasticsearch creates a new log file each day and it retains 7 days of logs.

To change the log rotation policy, edit the following file /opt/mapr/elasticsearch/elasticsearch-<version>/etc/elasticsearch/logging.yml. For details on how to update logging.yml, see the Elasticsearch documentation.

Grafana Log Rotation Policy

By default, Grafana creates a new log file whenever the current log file exceeds the 256MB. It retains log files that were generated in the last 7 days.

To change the log rotation policy, edit the [log] section of the following file: /opt/mapr/grafana/grafana-<version>/etc/grafana/grafana.ini. For details on how to update the grafana.ini, see the Grafana documentation.

Kibana Log Rotation Policy

Each time you start Kibana, it logs data to its log file and it does not automatically delete old log files. A new log file is created when you restart Kibana. To purge the log files, see Configure Log Retention.

Partners Support Dev-Hub Community Training ALA Privacy Policy Glossary

HPE Ezmeral Data Fabric – Customer-Managed 7.9.0 Documentation
Abstract	This site contains documentation for the customer-managed platform of the HPE Ezmeral Data Fabric version 7.9.0 including installation, configuration, administration, and reference content, as well as content for the associated bundled ecosystem components and drivers.
Published	November 2024
Edition	7.9.0