Polaris Federation

Polaris supports federation with external Iceberg REST catalogs (for example, catalogs served by other Polaris deployments). Federation enables clients to access tables that are managed by remote Iceberg REST catalog services through the local Polaris instance. In this model, Polaris exposes the remote catalog through its own REST endpoint, so engines connect only to the local Polaris service while transparently accessing remote tables. Authorization is enforced on both the local and remote catalogs, so effective privileges are determined by the intersection of permissions in each system.

To use Polaris federation:
  1. Enable the federation feature on the local Polaris instance where external catalogs will be created:
    # add to <polaris_home>/server/conf/application.properties
polaris.features."ENABLE_CATALOG_FEDERATION"=true
  2. Create an external catalog that points to the remote Iceberg REST catalog:
    polaris --client-id ${POLARIS_CLIENT} \
--client-secret ${POLARIS_SECRET} \
catalogs create \
--type EXTERNAL \
--storage-type <storage_type> \
--default-base-location "<path_to_catalog>" \
--allowed-location "<path_to_catalog>" \
--catalog-connection-type iceberg-rest \
--iceberg-remote-catalog-name <remote_polaris_catalog_name> \
--catalog-uri "https://<remote_polaris_url>:8181/api/catalog" \
--catalog-authentication-type OAUTH \
--catalog-token-uri "https://<remote_polaris_url>:8181/api/catalog/v1/oauth/tokens" \
--catalog-client-id "<remote_polaris_client_id>" \
--catalog-client-secret "<remote_polaris_secret>" \
--catalog-client-scope "PRINCIPAL_ROLE:ALL" \
<catalog_local_name>

  3. Create (or reuse) Polaris principals, roles, and catalog roles, then assign the appropriate catalog roles to the external catalog you created in Step 2.

    This step must ensure permissions are correct in both the local and remote Polaris instances. See the Note below.

  4. Use an Iceberg engine/catalog implementation that supports REST catalogs. Example using spark-shell:
    spark-shell \
--conf spark.sql.extensions=org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions \
--conf spark.sql.catalog.spark_demo.warehouse=<catalog_local_name> \
--conf spark.sql.catalog.spark_demo.type=rest \
--conf spark.sql.catalog.spark_demo.header.X-Iceberg-Access-Delegation=vended-credentials \
--conf spark.sql.catalog.spark_demo=org.apache.iceberg.spark.SparkCatalog \
--conf spark.sql.catalog.spark_demo.uri=https://<local_polaris_url>:8181/api/catalog \
--conf spark.sql.catalog.spark_demo.oauth2-server-uri=https://<local_polaris_url>:8181/api/catalog/v1/oauth/tokens \
--conf spark.sql.catalog.spark_demo.rest.auth.type=oauth2 \
--conf spark.sql.catalog.spark_demo.credential='<credentials_for_local_principal>' \
--conf spark.sql.catalog.spark_demo.scope='PRINCIPAL_ROLE:ALL'
NOTE
  • Iceberg-only requirement: For federation, you must use Iceberg’s org.apache.iceberg.spark.SparkCatalog (or an equivalent Iceberg REST implementation). Federation does not support Polaris “generic tables”.

  • Authorization is enforced in both places: Local and remote principals must have appropriate permissions. Effective access is constrained by what is allowed on both instances.

    For example, if the local principal has admin, and the remote principal has read-only, the user can read only. If both are admin , the user can perform all operations allowed by the remote service.

  • Connectivity is validated at catalog creation time: Polaris does not perform lazy probing of the remote service; catalog creation fails if the remote REST endpoint is unreachable or authentication is rejected.