Using JobSubmissionClient to Submit Ray Jobs

Describes how to connect to Ray cluster and submit Ray jobs using JobSubmissionClient.

The Ray Client has multithreading and connection issues which impact its reliability and submitting Ray job using Ray Client is an outdated method. Hewlett Packard Enterprise recommends using JobSubmissionClient to submit Ray jobs.

To submit Ray jobs using JobSubmissionClient, you must specify entry point resources as follows:
  • For CPU, set entrypoint_num_cpus to 1 or <M>
  • For GPU, set entrypoint_num_gpus to 1 or <M>
NOTE
The failure to specify entry point resources before submitting any jobs in the Ray cluster results in unexpected behavior.

To learn how to submit Ray jobs using JobSubmissionClient, see Independent Tune Trials.

Example:

The following code block shows the sample code for connecting to the Ray cluster and submitting Ray Jobs using JobSubmissionClient:
import ray
from ray.job_submission import JobSubmissionClient
import time

# Ray cluster information
ray_head_ip = "kuberay-head-svc.kuberay.svc.cluster.local"
ray_head_port = 8265
ray_address = f"http://{ray_head_ip}:{ray_head_port}"

# Submit Ray job using JobSubmissionClient
client = JobSubmissionClient(ray_address)
job_id = client.submit_job(
    entrypoint="python demo.py",
    runtime_env={
        "working_dir": "./",
        # "excludes": ['']
    },
    entrypoint_num_cpus = 3
)

print(client.__dict__)
print(f"Ray job submitted with job_id: {job_id}")