Data Fabric MCP Server Settings

This topic describes various configuration settings required for MCP.

Data Fabric MCP Server Settings

The Spark MCP endpoints can operate in two modes, either with impersonation enabled or impersonation disabled.

Impersonation-enabled mode:

Impersonation enabled is the default mode, and is the recommended mode.

In this mode, all Spark sessions are impersonated according to the credentials of the specific user. A separate session gets created for each user, with a default idle time of 30 minutes. As each session reserves cluster resources, the number of concurrent active sessions is limited. You can configure this limit.

Before accessing Spark for the first time, you must call the create_sql_session tool. This tool must also be used to recreate expired sessions.

Impersonation-disabled mode

The impersonation-disabled mode is not a recommended mode, as it carries security risks. In this mode, all users have access to data that is available to the session-user.

In Impersonation-disabled mode, sessions are pre-created (by default, three) under the user specified as session-user in the configuration. However, there is no latency in creating sessions, and all sessions in this mode will be re-created automatically if they crash or expire.

To add the required spark parameters, you need to first create a Spark section in /opt/mapr/data-access-gateway/conf/mcp-server.conf file.

Following are the list of various Spark parameters:
NOTE
To use any Spark endpoint you must have installed mapr-livy, mapr-spark, mapr-hive, mapr-hivemetastore packages.
NOTE

For information about configuring the MCP server to use Apache Polaris as an Iceberg catalog backend for Spark SQL queries, including Keycloak SSO authentication and Polaris role-based access control, see Configuring Polaris as a metastore for Spark.

MCP Spark Endpoint Options

Table 1. MCP Spark Endpoint Options
Parameter Description
mcp.spark.enabled To enable Spark endpoint.
mcp.spark.session-use The user that will create sessions in impersonation-disabled mode
mcp.spark.pool.size Number of sessions pre-created for impersonation-disabled mode.

Default value: 3
mcp.spark.userpool.size Number of available concurrent sessions/users for impersonation mode.

Default value: 3
mcp.spark.session.ttl ttl for Spark sessions for impersonation-disabled mode.

Default value: 10 minutes
mcp.spark.session.userttl ttl for Spark sessions for impersonation-enabled mode.

Default value: 30 minutes
mcp.spark.impersonation-enabled To enable or disable impersonation mode.

Default value: true
mcp.spark.livy.auth-type How the MCP server authenticates to Livy. Set to bearer to forward the caller's own Keycloak token to Livy. Livy must accept Bearer tokens (for example, with multi-auth-sso).

Default value: bearer

Spark options

Table 2. Spark options (these values are passed to Spark engine):
Parameter Description
mcp.spark.driver.memory Driver memory for Spark session.

Default value: 1 GB
mcp.spark.driver.cores Number of driver cores per Spark session.

Default value: 1
mcp.spark.executor.memory Executor memory per Spark session.

Default value: 1 GB
mcp.spark.executor.cores Number of executor cores per Spark session.

Default value: 1
mcp.spark.executor.instances Number of executors per Spark session.

Default value: 1

Iceberg options for a single Iceberg catalog

Table 3. Iceberg options for a single Iceberg catalog
Parameter Description
mcp.spark.iceberg.extensions Extension to use for Iceberg integration.

Default value: org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions
mcp.spark.iceberg.catalog Iceberg catalog to use (org.apache.iceberg.spark.SparkCatalog by default)

Default value: org.apache.iceberg.spark.SparkCatalog
mcp.spark.iceberg.catalog.type Catalog type.

Default value: hive
mcp.spark.iceberg.catalog.uri Catalog URI.

Default value: thrift://localhost:9083
mcp.spark.iceberg.useBuiltIn Use built in Iceberg library.

Default value: true

Configuring multiple Iceberg Catalogs

Starting with DEP 10.2.0, the Spark endpoint of the MCP server supports multiple Iceberg catalogs. You can configure both Polaris and Hive catalogs in the iceberg section of mcp-server.conf.

Existing configurations are backward compatible. If you already have a configuration, you don't have to change it.
Table 4. Iceberg options for a multiple Iceberg Catalogs
Parameter Description
mcp.spark.iceberg.default-catalog The catalog the Spark endpoint uses by default. Give one of the catalog names listed under polaris.catalogs or hive.catalogs.
mcp.spark.iceberg.polaris.catalogs List of Polaris catalog names, for example ["sales", "finance"]. Authentication defaults to sso-token, which forwards the caller's token to Polaris.
mcp.spark.iceberg.polaris.uri Optional. The Polaris catalog URI. If you leave it out, the MCP server finds it from the running Polaris service: https://<host>:8181/api/catalog.
mcp.spark.iceberg.hive.catalogs List of Hive catalog names, for example ["analytics"].
mcp.spark.iceberg.hive.uri Optional. The Hive metastore URI. If you leave it out, the MCP server finds it from the running hivemeta service: thrift://<host>:9083.

Example

spark {
    ## Enables the MCP interface for Spark SQL queries
    enabled = true
    ## Per-user Livy sessions: the Spark driver runs as the authenticated caller.
    impersonation-enabled = true

    livy {
        ## Forward the caller's own Keycloak token to Livy (multiauth-sso accepts Bearer).
        auth-type = "bearer"
    }

    iceberg {
      default-catalog = "sales"

      polaris {
        ## uri omitted: discovered from the running Polaris service (https://<host>:8181/api/catalog).
        ## auth defaults to sso-token: the caller's token is forwarded to Polaris.
        catalogs = ["sales", "finance"]
      }

      hive {
        ## uri omitted: discovered from the running hivemeta service (thrift://<host>:9083).
        catalogs = ["analytics"]
      }
    }
}

HTTP / Authentication

Table 5. HTTP / Authentication
Parameter Description
mcp.http.auth.introspection.enabled Enables live OAuth 2.0 token introspection ([RFC 7662] (https://datatracker.ietf.org/doc/html/rfc7662)) against the identity provider (Keycloak). When enabled, the gateway verifies that access tokens have not been revoked and that user sessions remain active. When set to false, the server performs only offline JWT signature and expiration verification.

Default value: true
mcp.http.auth.introspection.cache-ttl-ms The duration in milliseconds to cache token introspection results (active / inactive). Set to 0 to disable caching and query Keycloak on every incoming request.

Default value: 60000 (60 seconds)