Editing the Retention Settings of Aggregated Logs

About this task

By default, aggregate logs are stored on the HPE Ezmeral Data Fabric file system for 30 days. The retention time for aggregated logs also applies to centralized logs.

To edit the retention settings, add or edit the following properties in the yarn-site.xml file:

Procedure

  1. Set the value of yarn.log-aggregation.retain-seconds to the duration that the logs are maintained. If you set a negative value for yarn.log-aggregation.retain-seconds, logs arre not deleted.
    NOTE
    The duration specified by yarn.log-aggregation.retain-seconds starts from the time that the application starts running. Therefore, if you configure the duration, be sure to consider how long you want the log to remain in addition to the amount of time that the application will take to run. For example, if you expect most applications to take 20 seconds to run, do not set the value of this property to 20 seconds because the log might be deleted as soon as the applications completes.
  2. Optionally, set the yarn.log-aggregation.retain-check-interval-seconds to specify how often the log retention check should be run. By default, it is one-tenth of the log retention time.

Results

For more details about the properties that impact the YARN container logs and the aggregation option, see yarn-site.xml.