Spark History Server

This topic provides an overview of Spark History Server.

Spark History Server provides a web UI to monitor and view the status of submitted Spark Applications. It shows the status of Running, Completed, and Failed (completed but failed) Spark Applications.

Spark History Server pulls the details of Spark Applications from event logs directory when you enable the event logging to submit the Spark Applications. The event logs are written to the event log directory on persistent storage.

You can access the Spark History Server web UI through node IP address and node port number as exposed by NodePort Service at

The default node port number is 18080.

For more details about Spark History Server, see Apache Monitoring and Instrumentation.

Figure 1. Spark History Server Architecture