Lists the ports used by Data Fabric
services.
Avoiding Port Conflicts
To avoid trouble with port conflicts on your Data Fabric
clusters, try these tips:
- Remap the ports for the HBaseMaster and HBaseRegionServer services to ports below
32768
.
- Set the ephemeral port range to stop at
50029
by changing the value
in the file /proc/sys/net/ipv4/ip_local_port_range
. Note that this
setting changes the available number of ephemeral ports from the default of
28233
ports to 17233
.
Ports Needed for POSIX Clients and File System to Communicate With Each
Other
POSIX clients communicate with the CLDB and server components of the Data Fabric filesystem. You need to open the relevant ports for TCP
connectivity from POSIX clients to the Data Fabric
file-system cluster nodes. Open the CLDB, file-system server, and file-system server
instances ports, as detailed in the following section.
Services and Ports Quick Reference
The following list defines the ports used by a Data Fabric
cluster, along with the default port numbers. All the ports used by Data Fabric software are TCP ports.
- Airflow Webserver
- Source IP: Nodes/clients accessing Airflow webserver
- Destination IP: Nodes running Airflow webserver
- Ports: 8780
- Purpose: Used by Airflow webserver clients to access the Airflow
webserver
- Parameter and File where Port is Configured: Not Applicable
- Airflow Scheduler
- Source IP: Airflow webserver
- Destination IP: Nodes running Airflow scheduler
- Ports: 8793
- Purpose: Used by Airflow webserver to access the Airflow scheduler
- Parameter and File where Port is Configured: Not Applicable
- API Server (apiserver)
- Source IP: Cluster nodes running apiserver
- Destination IP: Cluster nodes running apiserver
- Ports:
- Purpose:Clustering support
- Parameter and File where Port is Configured: Not Applicable
- CLDB
- Source IP: Nodes running any Data Fabric
services, clients interacting with the file system
- Destination IP: Cluster nodes running CLDB services
- Ports: 7222
A client reads CLDB IP and port number from the
/opt/mapr/conf/mapr-clusters.conf
file. The client
initially tries to communicate with CLDB on port 7222
. Once it
establishes the connection, it fetches the additional CLDB IPs and ports from
the connected CLDB.
By default, CLDB listens on ports
7222
and 7223
. For performance reasons,
additional ports may be opened, depending on the configuration parameter
cldb.num.rpc.threads
in the
/opt/mapr/conf/cldb.conf
file. For example, setting
cldb.num.rpc.threads=3
, opens up ports
7222
, 7223
and 7224
.
NOTE
The
cldb.num.rpc.threads
parameter is hard-coded with a
default value of 3. To change this value, add this parameter with the new
value to the
/opt/mapr/conf/cldb.conf
file.
NOTE
If you upgrade from Core 5.2.x or Core 6 to Core 6.1 and above, the value
of
cldb.num.rpc.threads
is not changed. The default remains
as 3, which means three ports are open for each CLDB node.
The client tries connecting to the CLDBs till timeout occurs in the case of
soft mount, while the client indefinitely retries in the case of hard mount. If
a client cannot connect to a CLDB port, the CLDB is marked unreachable. For
example, assume a CLDB with IP 10.10.10.10 and 3 ports 7222
,
7223
and 7224
. If a client fails to
connect to the CLDB say on port 7223
, the CLDB 10.10.10.10 is
marked unreachable, and the client will not try the two other ports for the
next few minutes. It tries to connect with the next CLDB entry in the
list.
For load balancing at CLDB, a client will always pick a random port
among the available CLDB ports.
- Purpose: file system API calls
- Parameter and File where Port is Configured:
/opt/mapr/conf/cldb.conf
/opt/mapr/conf/warden.conf
/opt/mapr/conf/mapr-clusters.conf
- CLDB JMX Monitor Port
- Source IP: Nodes running CLDB services
- Destination IP: CLDB JMX monitor port
- Ports: 7220
- Purpose: The port on which
Collectd
gathers CLDB metrics
through JMX.
- Parameter and File where Port is Configured: Not Applicable
- CLDB web port
- Source IP: Nodes/clients connecting to the CLDB GUI
- Destination IP: Cluster nodes running CLDB services
- Ports: 7221
- Purpose: CLDB GUI for a cluster with security disabled. For a secure
cluster, the port is
7443
as defined by the
maprlogin
utility.
- Parameter and File where Port is Configured:
/opt/mapr/conf/cldb.conf
- maprlogin utility
- Source IP: Connections using the maprlogin utility
- Destination IP: Cluster nodes running CLDB services
- Ports: 7443
- Purpose: When security is enabled for a cluster, the CLDB listens for
connections on port 7443. If security is disabled, the
maprlogin
utility is unable to reach the CLDB.
- Parameter and File where Port is Configured: Not Applicable
- Data Access Gateway
- Source IP: Clients using the HPE Ezmeral Data Fabric Database JSON
REST API with HTTPS
- Destination IP:Not Applicable
- Ports: 8243
- Purpose: The port used to connect to the Data Access Gateway using
HTTPS
- Parameter and File where Port is Configured:
rest.https.port
in
/opt/mapr/data-access-gateway/conf/properties.cfg
- Data Access Gateway
- Source IP: Node.js OJAI client
- Destination IP: Cluster nodes running the Data Access Gateway service
- Ports: 5678
- Purpose: The port used to connect the OJAI client to the Data Access
Gateway
- Parameter and File where Port is Configured:
grpc.service.port
in
/opt/mapr/data-access-gateway/conf/properties.cfg
- Data Access Gateway
- Source IP: Python OJAI client
- Destination IP: Cluster nodes running the Data Access Gateway service
- Ports: 5678
- Purpose: The port used to connect the OJAI client to the Data Access
Gateway
- Parameter and File where Port is Configured:
grpc.service.port
in
/opt/mapr/data-access-gateway/conf/properties.cfg
- Data Access Gateway
- Source IP: Go OJAI client
- Destination IP: Cluster nodes running the Data Access Gateway service
- Ports: 5678
- Purpose: The port used to connect the OJAI client to the Data Access
Gateway
- Parameter and File where Port is Configured:
grpc.service.port
in
/opt/mapr/data-access-gateway/conf/properties.cfg
- Data Access Gateway
- Source IP: C# OJAI client
- Destination IP: Cluster nodes running the Data Access Gateway service
- Ports: 5678
- Purpose: The port used to connect the OJAI client to the Data Access
Gateway
- Parameter and File where Port is Configured:
grpc.service.port
in
/opt/mapr/data-access-gateway/conf/properties.cfg
- Data Access Gateway
- Source IP: Java OJAI thin client
- Destination IP: Cluster nodes running the Data Access Gateway service
- Ports: 5678
- Purpose: The port used to connect the OJAI client to the Data Access
Gateway
- Parameter and File where Port is Configured:
grpc.service.port
in
/opt/mapr/data-access-gateway/conf/properties.cfg
- Apache Kafka Wire Protocol Service
- Source IP: Apache Kafka Client
- Destination IP: Cluster nodes running the Data Access Gateway service
- Ports: 9092
- Purpose: The port used by Apache Kafka Wire Protocol Service to connect
the Kafka client to the Data Access Gateway service.
- Parameter and File where Port is Configured:
port
in
/opt/mapr/data-access-gateway/conf/kafka-server.conf
- Drill JMX Port
- Source IP: Nodes running the Drillbit service
- Destination IP: Drill JMX Port
- Ports: 6090
- Purpose: The port on which
Collectd
gathers Drill metrics
via JMX.
- Parameter and File where Port is Configured: Not Applicable
- Drill Web UI
- Source IP: Nodes running the Drillbit service
- Destination IP: Nodes running the Drillbit service
- Ports: 8047
- Purpose: TCP port needed for the Drill Web UI and clients using REST API
and nodes running the Drillbit service.
- Parameter and File where Port is Configured:
drill.exec.http.port
in
/opt/mapr/drill/drill-<version>/conf/drill-override.conf
- Drill (User Port)
- Source IP: Nodes running the Drillbit service and clients using
JDBC/ODBC
- Destination IP: Nodes running the Drillbit service
- Ports: 31010
- Purpose: TCP user port address. Used between nodes in a Drill cluster.
Needed for an external client, such as Tableau, to connect into the cluster nodes.
Also needed for the Drill Web UI. You can also use this port to connect directly
to a Drillbit.
- Parameter and File where Port is Configured:
drill.exec.rpc.user.server.port
in
/opt/mapr/drill/drill-<version>/conf/drill-override.conf
- Drill (Control Port)
- Source IP: Nodes running the Drillbit service
- Destination IP: Nodes running the Drillbit service
- Ports: 31011
- Purpose: TCP port that controls the port address. Used between nodes in a
Drill cluster. Needed for multi-node installation of Drill.
- Parameter and File where Port is Configured:
drill.exec.rpc.bit.server.port
in
/opt/mapr/drill/drill-<version>/conf/drill-override.conf
- Drill (Data Port)
- Source IP: Nodes running the Drillbit service
- Destination IP: Nodes running the Drillbit service
- Ports: 31012
- Purpose: TCP data port address. Used between nodes in a Drill cluster.
Needed for multi-node installation of Drill.
- Parameter and File where Port is Configured:
drill.exec.rpc.bit.server.port + 1
in
/opt/mapr/drill/drill-<version>/conf/drill-override.conf
- Drill (ZooKeeper Port)
- Source IP: Clients using JDBC/ODBC and nodes running ZooKeeper
services
- Destination IP: Nodes running the Drillbit service
- Ports: 5181
- Purpose: ZooKeeper port used to connect to Drill through the JDBC
driver.
- Parameter and File where Port is Configured:
See the ZooKeeper
entry in this list.
- Elasticsearch (Components Communication Port)
- Source IP: Non-Elasticsearch components, such a web browser, curl, and
Kibana, that connect to Elasticsearch.
- Destination IP: Nodes running Elasticsearch for monitoring use cases
- Ports: 9200
- Purpose: Non-Elasticsearch components use this port when communicating
with Elasticsearch.
- Parameter and File where Port is Configured: You can configure a different
port for monitoring use cases when you run the configure.sh script with the
-ES
parameter.
- Elasticsearch (Daemons Communication Port)
- Source IP: Nodes running Elasticsearch
- Destination IP: Nodes running Elasticsearch for monitoring use cases
- Ports: 9300
- Purpose: Elasticsearch uses this port for communications between
Elasticsearch daemons.
- Parameter and File where Port is Configured: Not Applicable
- Gateway
- Source IP: Nodes sending operations to replicate
- Destination IP: Nodes running the gateway service
- Ports: 7660
- Purpose: The port used by gateway services to listen for incoming
replication operations.
- Parameter and File where Port is Configured: Not Applicable
- Grafana
- Source IP: Web Browsers
- Destination IP: Nodes running Grafana for monitoring
- Ports: 3000
- Purpose: Web browsers use this port when connecting to Grafana.
- Parameter and File where Port is Configured: Not Applicable
- HBase Master
- Source IP: HBase Clients
- Destination IP: Nodes running HBase Master services
- Ports: 16000
- Purpose: HBase API and HBase shell use this port to connect to HBase
Master
- Parameter and File where Port is Configured:
/opt/mapr/hbase/hbase-<version>/conf/hbase-site.xml
- HBase Master Web UI
- Source IP: HBase Master Web UI clients
- Destination IP: Nodes running HBase Master services
- Ports: 16010
- Purpose: Information Web UI of HBase Master
- Parameter and File where Port is Configured:
/opt/mapr/hbase/hbase-<version>/conf/hbase-site.xml
- HBase Thrift Server
- Source IP: HBase Thrift Server clients
- Destination IP: Nodes running HBase Thrift Server
- Ports: 9090
- Purpose: The HBase client uses this port to connect to HBase, using the
Thrift protocol
- Parameter and File where Port is Configured:
/opt/mapr/hbase/hbase-<version>/conf/hbase-site.xml
- HBase Thrift Web UI
- Source IP: HBase Thrift Web UI clients
- Destination IP: Nodes running HBase Thrift
- Ports: 9095
- Purpose: Information Web UI of HBase Thrift
- Parameter and File where Port is Configured:
/opt/mapr/hbase/hbase-<version>/conf/hbase-site.xml
- HBase REST Server
- Source IP: HBase REST Server clients
- Destination IP: Nodes running HBase REST Server
- Ports: 8080
- Purpose: The HBase client uses this port to connect to HBase using the
HTTP protocol
- Parameter and File where Port is Configured:
/opt/mapr/hbase/hbase-<version>/conf/hbase-site.xml
- HBase REST Web UI
- Source IP: HBase REST Web UI clients
- Destination IP: Nodes running HBase REST
- Ports: 8086
- Purpose: Information Web UI of HBase REST
- Parameter and File where Port is Configured:
/opt/mapr/hbase/hbase-<version>/conf/hbase-site.xml
- HBase Regionserver
- Source IP: HBase Clients
- Destination IP: Nodes running HBase Regionserver services
- Ports: 16020
- Purpose: HBase API and HBase shell use this port to connect to HBase
RegionServer
- Parameter and File where Port is Configured:
/opt/mapr/hbase/hbase-<version>/conf/hbase-site.xml
- HBase Regionserver UI
- Source IP: HBase Regionserver Web UI clients
- Destination IP: Nodes running HBase Regionserver
- Ports: 16030
- Purpose: Information Web UI of HBase Regionserver
- Parameter and File where Port is Configured:
/opt/mapr/hbase/hbase-<version>/conf/hbase-site.xml
- HistoryServer RPC
- Source IP: Not Applicable
- Destination IP: Nodes running MapReduce JobHistory Server
- Ports: 10020
- Purpose: Not Applicable
- Parameter and File where Port is Configured: Not Applicable
- HistoryServer Web UI and REST APIs
- Source IP: Clients that access Job History Server UI in a
non-secure cluster
- Destination IP: Secure nodes running MapReduce JobHistory Server in a
non-secure cluster
- Ports: 19888
- Purpose: Non-secure HistoryServer Web UI and REST APIs
- Parameter and File where Port is Configured: See mapred-site.xml
- HistoryServer Web UI and REST APIs
- Source IP: Clients that access Job History Server UI in a secure
cluster
- Destination IP: Secure nodes running MapReduce JobHistory Server in a
secure cluster
- Ports: 19890
- Purpose: Secure HistoryServer Web UI and REST APIs
- Parameter and File where Port is Configured: See mapred-site.xml
- Hive Metastore
- Source IP: Nodes/clients performing Hive queries/operations
- Destination IP: Nodes running the Hive metastore services
- Ports: 9083
- Purpose: Used by Hive clients to query/access the Hive metastore
- Parameter and File where Port is Configured:
/opt/mapr/hive/hive-<version>/conf/hive-site.xml
- Hiveserver2
- Source IP: Nodes or clients performing hive queries using JDBC/ODBC
- Destination IP: Nodes running Hiveserver2
- Ports: 10000
- Purpose: Port through which clients perform hive queries
- Parameter and File where Port is Configured: Not Applicable
- Hiveserver2 Web UI
- Source IP: Not Applicable
- Destination IP: Nodes running Hiverserver2 Web UI
- Ports: 10002
- Purpose: Provides access to Hive configuration settings, local logs,
metrics, and information about active sessions and queries.
- Parameter and File where Port is Configured: Not Applicable
- Hoststats
- See
hoststats.port
and hs.port
in the Warden configuration file.
- Httpfs
- Source IP: Nodes/clients accessing httpfs services
- Destination IP: Nodes running httpfs services
- Ports: 14000
- Purpose: Used by httpfs file clients to access the httpfs server
- Parameter and File where Port is Configured:
/opt/mapr/hadoop/hadoop-<version>/etc/hadoop/httpfs-env.sh
- Hue Webserver
- Source IP: Nodes/clients accessing Hue web services
- Destination IP: Nodes running Hue web services
- Ports: 8888
- Purpose: Used by Hue webserver clients to access the Hue webserver
- Parameter and File where Port is Configured:
/opt/mapr/hue/hue*/desktop/conf/hue.ini
- KSQL
- Source IP: All cluster nodes
- Destination IP: Nodes running KSQL
- Ports: 8084
- Purpose: KSQL
- Parameter and File where Port is Configured:
$KSQL_INSTALL_DIR/etc/ksql/ksqlserver.properties
- Kafka Connect
- Source IP: All cluster nodes
- Destination IP: Nodes running Kafka Connect
- Ports: 8083
- Purpose: Kafka Connect REST API calls
- Parameter and File where Port is Configured:
/opt/mapr/kafka/kafka-<version>/config/connect-distributed.properties
- Kafka REST
- Source IP: All cluster nodes
- Destination IP: Nodes running Kafka REST
- Ports: 8082
- Purpose: Kafka Connect REST API calls
- Parameter and File where Port is Configured:
/opt/mapr/kafka-rest/kafka-rest-<version>/config/kafka-rest.properties
- Kafka Schema Registry
- Source IP: All cluster nodes
- Destination IP: Nodes running Kafka Schema Registry
- Ports: 8087
- Purpose: Kafka Schema Registry API calls
- Parameter and File where Port is Configured:
/opt/mapr/schema-registry/schema-registry-<version>/config/schema-registry.properties
- Kibana
- Source IP: Web browsers
- Destination IP: Nodes running Kibana for monitoring use cases
- Ports: 5601
- Purpose: Web browsers use this port when connecting to Grafana.
- Parameter and File where Port is Configured: Not Applicable
- MAST Gateway
- Source IP: Nodes running MAST Gateway service
- Destination IP: Nodes running MAST Gateway service
- Ports: 8660
- Purpose: Data Fabric clients use this port to
connect to the MAST Gateway
- Parameter and File where Port is Configured:
/opt/mapr/conf/mastgateway.conf
- file system server
- Source IP: Nodes running any Data Fabric
services, clients interacting with the file system
- Destination IP: Nodes running FileServer services
- Ports: 5660, 5692, 5724, and 5756
- Purpose: The filesystem is a random read-write distributed filesystem that
allows applications to concurrently read and write directly to disk. Clients use
these ports to access the file-system server.
- Parameter and File where Port is Configured: Not Applicable
- file system server
- Source IP: Nodes running the gateway service
- Destination IP: Nodes running the file system
- Ports: 6660
- Purpose: The port on which gateway nodes send replicated operations to
nodes in destination clusters.
- Parameter and File where Port is Configured: Not Applicable
- file system server instances
- Source IP: Not Applicable
- Destination IP: Not Applicable
- Ports: See Working with Multiple Instances of the File System
- Purpose: Multiple file system instances
- Parameter and File where Port is Configured: Not Applicable
- Object Store
- Source IP: Nodes accessing MOSS
- Destination IP: Nodes running MOSS
- Ports: 9000
- Purpose: Port for MOSS
- Parameter and File where Port is Configured: The moss.port option
in
/opt/mapr/conf/moss.conf
.CAUTION
The default port for
S3 Gateway is also 9000. If you run
S3 Gateway and Object Store,
change one of the ports to avoid conflicts.
- S3 Gateway
- Source IP: Nodes accessing the S3 Gateway server
- Destination IP: Nodes running the S3 Gateway server
- Ports: 9000
- Purpose: Port for the S3 Gateway server
- Parameter and File where Port is Configured: The ports option in
/opt/mapr/objectstore-client/objectstore-client-<version>/conf/minio.json
- NFS
- Source IP: Nodes/clients accessing the filesystem via the NFS
protocol
- Destination IP: Nodes running Data Fabric NFS
Services
- Ports: 2049
- Purpose: NFSv3 or NFSv4 access to the file system
- Parameter and File where Port is Configured: Not Applicable
- NFS
- Source IP: Nodes running NFS services
- Destination IP: Nodes running NFS services
- Ports: 9997, 9998
- Purpose: NFS VIP Management
- Parameter and File where Port is Configured:
/opt/mapr/conf/nfsserver.conf
- NodeManager JMX Port
- Source IP: Nodes running NodeManager
- Destination IP: NodeManager JMX Port
- Ports: 8027
- Purpose: The port on which Collectd gathers metrics from NodeManager nodes
via JMX.
- Parameter and File where Port is Configured:Not Applicable
- NodeManager
- Source IP: Not Applicable
- Destination IP: Nodes running NodeManager
- Ports: 8099
- Purpose: The node manager manages the health of each node in the
cluster.
- Parameter and File where Port is
Configured:
yarn.nodemanager.address
in
/opt/mapr/hadoop/hadoop-<version>/etc/hadoop/yarn-site.xml
- NodeManager Localizer RPC
- Source IP: Not Applicable
- Destination IP: Nodes running NodeManager
- Ports: 8040
- Purpose: The port that node manager uses to localize resources for a node.
With localization, remote resources are downloaded to the local filesystem for
access.
- Parameter and File where Port is Configured:
yarn.nodemanager.localizer.address
in
/opt/mapr/hadoop/hadoop-<version>/etc/hadoop/yarn-site.xml
- NodeManager Web UI and REST APIs
- Source IP: External Web browsers and REST clients accessing NodeManager
services in a non-secure cluster
- Destination IP: Nodes running NodeManager services in a non-secure
cluster
- Ports: 8042
- Purpose: NodeManager HTTP port
- Parameter and File where Port is Configured:
yarn.nodemanager.webapp.address
in
/opt/mapr/hadoop/hadoop-<version>/etc/hadoop/yarn-site.xml
- NodeManager Web UI and REST APIs
- Source IP: External Web browsers and REST clients accessing NodeManager
services in a secure cluster
- Destination IP: Nodes running NodeManager services in a secure
cluster
- Ports: 8044
- Purpose: NodeManager HTTPS port
- Parameter and File where Port is Configured:
yarn.nodemanager.webapp.https.address
in
/opt/mapr/hadoop/hadoop-<version>/etc/hadoop/yarn-site.xml
- OpenTSDB
- Source IP: OpenTSDB clients, such as
Collectd
.
- Destination IP: Nodes running OpenTSDB for monitoring use cases.
- Ports: 4242
- Purpose:
Collectd
uses this port to write metrics to
OpenTSDB.
- Parameter and File where Port is Configured: You can configure a different
port for monitoring use cases when you run configure.sh script with the
-OT
parameter.
- Port Mapper
- Source IP: Nodes running Data Fabric NFS
Services
- Destination IP: Nodes/clients accessing the filesystem using the NFS
protocol
- Ports: 111
- Purpose: RPC Portmap services used to connect to the file system using NFSv3
- Parameter and File where Port is Configured: Not Applicable
- Ranger (nonsecure Admin UI for http)
- Source IP: Nodes or clients accessing the Ranger Admin (UI/API)
service
- Destination IP: Nodes running the Ranger Admin service
- Ports: 6080
- Purpose: Used by Ranger Admin clients to access the Ranger Admin
service
- Parameter and File where Port is Configured:
ranger.service.http.port
in
/opt/mapr/ranger/ranger-2.3.0/ranger-admin/conf/ranger-admin-site.xml
- Ranger (secure Admin UI for https)
- Source IP: Nodes or clients accessing the Ranger Admin (UI/API)
service
- Destination IP: Nodes running the Ranger Admin service
- Ports: 6182
- Purpose: Used by Ranger Admin clients to access the Ranger Admin
service
- Parameter and File where Port is Configured:
ranger.service.https.port
in
/opt/mapr/ranger/ranger-2.3.0/ranger-admin/conf/ranger-admin-site.xml
- Ranger Usersync
- Source IP: Nodes or clients accessing the Ranger Usersync service
- Destination IP: Nodes running the Ranger Usersync service
- Ports: 5151
- Purpose: Used by the Ranger Admin to access the Ranger Usersync
service
- Parameter and File where Port is Configured:
ranger.usersync.port
in
/opt/mapr/ranger/ranger-2.3.0/ranger-usersync/conf/ranger-ugsync-site.xml
- ResourceManager JMX Port
- Source IP: Nodes running ResourceManager
- Destination IP: ResourceManager JMX port
- Ports: 8025
- Purpose: The port on which
Collectd
gathers metrics from
the ResourceManager using JMX.
- Parameter and File where Port is Configured: Not Applicable
- ResourceManager Admin RPC
- Source IP: Applications that access the ResourceManager
- Destination IP: Nodes running ResourceManager
- Ports: 8033
- Purpose: The port that applications use to access the ResourceManager
RPC
- Parameter and File where Port is Configured:
yarn.resourcemanager.admin.address
in
/opt/mapr/hadoop/hadoop-<version>/etc/hadoop/yarn-site.xml
- ResourceManager Client RPC
- Source IP: Clients that submit YARN applications
- Destination IP: Nodes running ResourceManager
- Ports: 8032
- Purpose: The port that clients use to access the YARN applications
- Parameter and File where Port is Configured:
yarn.resourcemanager.address
in
/opt/mapr/hadoop/hadoop-<version>/etc/hadoop/yarn-site.xml
- ResourceManager Resource Tracker RPC (for NodeManagers)
- Source IP: Applications that access the ResourceManager
- Destination IP: Nodes running ResourceManager
- Ports: 8031
- Purpose: The port that applications use to access the Resource Manager
Tracker RPC
- Parameter and File where Port is Configured:
yarn.resourcemanager.resource-tracker.address
in
/opt/mapr/hadoop/hadoop-<version>/etc/hadoop/yarn-site.xml
- ResourceManager Scheduler RPC (for ApplicationMasters)
- Source IP: Applications that access the ResourceManager
- Destination IP: Nodes running ResourceManager
- Ports: 8030
- Purpose: The port on which the applications in the cluster talk to the
ResourceManager.
- Parameter and File where Port is Configured:
yarn.resourcemanager.scheduler.address
in
/opt/mapr/hadoop/hadoop-<version>/etc/hadoop/yarn-site.xml
- ResourceManager Web UI (HTTP)
- Source IP: Clients that access ResourceManager UI in a non-secure
cluster
- Destination IP: Nodes running ResourceManager master in a non-secure
cluster
- Ports: 8088
- Purpose: ResourceManager Web UI
- Parameter and File where Port is Configured:
yarn.resourcemanager.webapp.address
in
/opt/mapr/hadoop/hadoop-<version>/etc/hadoop/yarn-site.xml
- ResourceManager Web UI (HTTPS)
- Source IP: Clients that access ResourceManager UI in a secure
cluster
- Destination IP: Nodes running ResourceManager master in a secure
cluster
- Ports: 8090
- Purpose: ResourceManager Web UI
- Parameter and File where Port is Configured:
yarn.resourcemanager.webapp.address
in
/opt/mapr/hadoop/hadoop-<version>/etc/hadoop/yarn-site.xml
- Shuffle HTTP
- Source IP: Not Applicable
- Destination IP: Shuffle HTTP
- Ports: 13562
- Purpose: The port that MapReduce Shuffle uses. Transferring the map
outputs to reducer inputs in sorted form is the shuffle operation.
- Parameter and File where Port is Configured: Not Applicable
- Spark Standalone Master (RPC)
- Source IP: Not Applicable
- Destination IP: Not Applicable
- Ports: 7077
- Purpose: The port on which to submit jobs in a Spark standalone
cluster.
- Parameter and File where Port is Configured:
SPARK_MASTER_PORT
in
SPARK_HOME/conf/spark-env.sh
- Spark Standalone Master (Web UI)
- Source IP: Nodes/clients accessing Spark services in a non-secure
cluster
- Destination IP: Nodes running Spark services in a non-secure cluster
- Ports: 8580
- Purpose: The port on which browsers connect to Spark master in a
non-secure Spark standalone cluster.
- Parameter and File where Port is Configured:
SPARK_MASTER_WEBUI_PORT
in
SPARK_HOME/conf/spark-env.sh
- Spark Standalone Master (Web UI)
- Source IP: Nodes/clients accessing Spark services in a secure
cluster
- Destination IP: Nodes running Spark services in a secure cluster
- Ports: 8980
- Purpose: The port on which browsers connect to a Spark master in a secure
Spark standalone cluster.
- Parameter and File where Port is Configured:
SPARK_MASTER_WEBUI_PORT
in
SPARK_HOME/conf/spark-env.sh
- Spark Standalone Worker
- Source IP: Not Applicable
- Destination IP: Not Applicable
- Ports: 8081
- Purpose: The port on which browsers connect to Spark workers in a Spark
standalone cluster.
- Parameter and File where Port is Configured:
SPARK_WORKER_WEBUI_PORT
in
SPARK_HOME/conf/spark-env.sh
- Spark Thrift Server (if start and stop server using Spark scripts)
- Source IP: Not Applicable
- Destination IP: Not Applicable
- Ports: 10000
- Purpose: The port on which JDBC clients connect to Spark Thrift
server.
- Parameter and File where Port is Configured:
hive.server2.thrift.port
in
SPARK_HOME/conf/hive-site.xml
- Spark Thrift Server (if start and stop server through Warden, starting in
EEP 4.0)
- Source IP: Not Applicable
- Destination IP: Not Applicable
- Ports: 2304
- Purpose: The port on which JDBC clients connect to Spark Thrift
server.
- Parameter and File where Port is Configured:
hive.server2.thrift.port
in
SPARK_HOME/conf/hive-site.xml
- Spark History Server
- Source IP: Clients that access Spark Job History in a non-secure
cluster
- Destination IP: Nodes running Spark History Server in a non-secure
cluster
- Ports: 18080
- Purpose: The port on which browsers connect to a non-secure Spark history
server.
- Parameter and File where Port is Configured:
spark.history.ui.port
in
SPARK_HOME/conf/spark-default.conf"
- Spark History Server
- Source IP: Clients that access Spark Job History in a secure
cluster
- Destination IP: Nodes running Spark History Server in a secure
cluster
- Ports: 18480
- Purpose: The port on which browsers connect to a secure Spark history
server.
- Parameter and File where Port is Configured:
spark.ssl.historyServer.port
in
SPARK_HOME/conf/spark-defaults.conf (starting from
Spark-2.2.1)
- Spark External Shuffle Service
- Source IP: Not Applicable
- Destination IP: Not Applicable
- Ports: 7337
- Purpose: The port on which Spark jobs connect to External Shuffle
server.
- Parameter and File where Port is Configured:
spark.shuffle.service.port
in
SPARK_HOME/conf/spark-default.conf
- Tez Shuffle
- Source IP: Not Applicable
- Destination IP: Not Applicable
- Ports: 13563
- Purpose: Port to communicate with the Tez Shuffler. A Tez specific shuffle
handler allows data to be shuffled in a way that takes advantage of the new
features in Tez
- Parameter and File where Port is Configured: Not Applicable
- Timeline Server
- Source IP: Not Applicable
- Destination IP: Not Applicable
- Ports: 10200
- Purpose: Hadoop IPC port used for internal communication in Hadoop
- Parameter and File where Port is Configured: Not Applicable
- Timeline Server Web Interface (HTTP)
- Source IP: Not Applicable
- Destination IP: Not Applicable
- Ports: 8188
- Purpose: Non-secure web access for the Timeline Server. The Timeline
Server allows storage and retrieval of an application’s current and historic
information in a generic fashion.
- Parameter and File where Port is Configured: Not Applicable
- Timeline Server Web Interface (HTTPS)
- Source IP: Not Applicable
- Destination IP: Not Applicable
- Ports: 8190
- Purpose: Secure web access for the Timeline Server
- Parameter and File where Port is Configured: Not Applicable
- Tomcat Port (Hive-on-Tez UI)
- Source IP: Not Applicable
- Destination IP: Not Applicable
- Ports: 9383
- Purpose: The non-secure port to access the Tez UI. Hive-on-Tez speeds up
execution of Hive queries.
- Parameter and File where Port is Configured: Not Applicable
- Tomcat SSL Port (Hive-on-Tez UI)
- Source IP: Not Applicable
- Destination IP: Not Applicable
- Ports: 9393
- Purpose: The secure port to access the Tez UI.
- Parameter and File where Port is Configured: Not Applicable
- Web UI
- Source IP: External web browser accessing either a non-secure or a
secure cluster
- Destination IP: Nodes running the Control System Web UI in a non-secure or
a secure cluster
- Ports: 8443
- Purpose: Control System Web UI
- Parameter and File where Port is Configured:
/opt/mapr/apiserver/conf/properties.cfg
- Zeppelin
- Source IP: Not Applicable
- Destination IP: Not Applicable
- Ports: 9995
- Purpose: The port to connect to the Zeppelin Docker container
- Parameter and File where Port is Configured: Configurable by setting
ZEPPELIN_SSL_PORT
when running the Zeppelin Docker image
- ZooKeeper
- Source IP: Nodes running ZooKeeper services, clients executing ZooKeeper
API calls
- Destination IP: Nodes running ZooKeeper services
- Ports: 5181
- Purpose: ZooKeeper API calls
- Parameter and File where Port is Configured:
/opt/mapr/zookeeper/zookeeper-<version>/conf/zoo.cfg
/opt/mapr/conf/warden.conf, /opt/mapr/conf/cldb.conf
/opt/mapr/hbase/hbase-<version>/conf/hbase-site.xml
/opt/mapr/hive/hive-<version>/conf/hive-site.xml
- ZooKeeper follower-to-leader Communication
- Source IP: Nodes running ZooKeeper services
- Destination IP: Nodes running ZooKeeper services
- Ports: 2888
- Purpose: ZooKeeper Server > Server Communication
- Parameter and File where Port is Configured:
/opt/mapr/zookeeper/zookeeper-<version>/conf/zoo.cfg
- ZooKeeper Leader Election
- Source IP: Nodes running ZooKeeper services
- Destination IP: Nodes running ZooKeeper services
- Ports: 3888
- Purpose: ZooKeeper Server > Server Communication
- Parameter and File where Port is Configured:
/opt/mapr/zookeeper/zookeeper-<version>/conf/zoo.cfg