Managing Kafka Schema Registry

Describes how to manage the internal stream for Kafka Schema Registry.

Schema Registry Internal Stream

By default, the schema-registry-internal-stream topic stores the schemas. The schema-registry-internal-stream topic is located in /apps/schema-registry/schema-registry-internal-stream. The kafkastore.stream property in the SR_CONF_DIR/schema-registry.properties file sets the internal stream topic. By default, the kafkastore.stream property is set to /apps/schema-registry/schema-registry-internal-stream.

The internal stream is automatically created if it does not already exist. If the internal stream already exists and has the same permissions as the default, the system returns a warning. If kafkastore.stream is set to a value other than the default, the system returns an error if the stream does not exist. You must explicitly set permissions on the stream you designate as the Schema Registry internal stream.

You can use Warden and /opt/mapr/server/configure.sh to manage and configure Kafka Schema Registry. Any time you change Schema Registry configurations, run configure.sh for changes to take effect.

Secure Cluster Default Permissions on the Internal Stream

By default, the internal stream is readable by all the users and writable only by the cluster administrator. For secure clusters, the default permissions are:
  • -produceperm u:$CLUSTER_ADMIN
  • -consumeperm p
  • -topicperm u:$CLUSTER_ADMIN
The cluster admin is typically mapr or the value set for MAPR_USER.

Log Compaction for the Schema Registry Internal Stream

Log compaction is the process of purging messages previously published to a topic partition while retaining the latest version. Log compaction for the Schema Registry internal stream is enabled by default.
IMPORTANT
Log compaction requires a gateway on the same cluster as the Schema Registry internal stream. Also, TTL should be disabled because it can interfere with log compaction.

Installing a Gateway

Run the command appropriate for your system:
  • RedHat/CentOS
    yum install mapr-gateway
  • Ubuntu
    apt-get install mapr-gateway
  • SLES
    zypper install mapr-gateway