Multiple Drill Clusters within YARN
You can define multiple Drill clusters within a single YARN cluster. Each Drill cluster is a collection of Drillbits that work as an independent unit. For example, you might define one test Drill cluster that consists of a few machines on the same physical cluster that runs larger Drill clusters for development and marketing.
You must assign each Drill cluster a distinct ZooKeeper entry. Drill uses ZooKeeper to coordinate activities. Each Drill cluster also needs a distinct set of ports because YARN may launch Drillbits from different clusters on the same physical node.
The following steps summarize the process for defining multiple Drill clusters on a single YARN
cluster:
- Create a new site directory.
- Configure Drill.
- Configure Drill-on-YARN.
- Start the cluster.
The following task provides instructions for each of the steps required to configure and run multiple Drill clusters under YARN: