Ports Used by Spark
To run a Spark job from a client node, ephemeral ports should be opened in the cluster for the client from which you are running the Spark job.
If you do not want to open all the ephemeral ports, you can use the configuration parameter to specify the range of ports.
To set ports to special values, use the spark.driver.port, spark.blockManager.port, and spark.port.maxRetries properties.
The spark.port.maxRetries property is 16 by default.
For example, if you need to open port 200 for spark.blockManager.port from 40000, set spark.blockManager.port = 40000 and spark.port.maxRetries = 200.
For a list of Web UIs ports dynamically used when starting spark contexts, see the open source documentation.
The default port numbers that need to be opened on the firewall behind the client and MapR
cluster nodes for Spark jobs to operate in YARN client, YARN cluster, and standalone modes are
as follows:
* refers to ports for secure clusters
| Service | Port Number |
|---|---|
| Spark Standalone Master (RPC) | 7077 |
| Spark Standalone Master (Web UI) | 8580, 8980* |
| Spark Standalone Worker | 8581, 8981* |
| Spark Thrift Server | 2304 |
| Spark History Server | 18080,18480* |
| Spark External Shuffle Service (if yarn shuffle service is enabled) | 7337 |
| CLDB | 7222 |
| ZooKeeper | 5181 |
| Nodes running ResourceManager | 8032 |
| MapR Filesystem Server | 5660, 5692 |