Configuration Parameters
This section provides the Kafka REST Proxy for HPE Ezmeral Data Fabric Streams parameters.
kafka-rest.properties file.
/opt/mapr/kafka-rest/kafka-rest-<version>/config/kafka-rest.properties| Parameter | Description |
|---|---|
| api.v2.enable | Enables the REST Proxy v2 API when set to true. Default is
true. This parameter is available starting in Kafka REST
6.0.0.0. |
| api.v3.enable | Enables the REST Proxy v3 API when set to true. Default is
false. This parameter is available starting in Kafka REST
6.0.0.0. |
| advertised.listeners | List of advertised listeners used when generating absolute URLs in responses. Supports http and https protocols. Each listener must include the protocol, hostname, and port. For example: http://myhost:8080, https://0.0.0.0:8081. This parameter is available starting in Kafka REST 6.0.0.0. |
| schema.registry.enable | Enables Avro serialization and deserialization support with Schema Registry. Starting in Schema Registry 6.0.0.0, enables JSON Schema, and Protobuf serialization and deserialization as well as Avro. |
| schema.registry.url | The base URL for the schema registry for use by the Avro serializer. Starting in Schema Registry 6.0.0.0, also for use by the JSON Schema and Protobuf serializers, as well as Avro. This setting is ignored if schema.registry.enable is set to false. The default value is resolved from Zookeeper. |
| schema.registry.service.id | Indicates the ID of the schema registry service. Default: default_ |
| schema.registry.discovery.timeout | The timeout in milliseconds for request to Schema Registry URL storage. Default: 60000 |
| schema.registry.discovery.retries | The number of retries for Schema Registry URL discovery. Default: 6 |
| schema.registry.discovery.interval | The interval in milliseconds between retries for Schema Registry URL discovery. Default: 15000 |
| streams.default.stream | The default stream the consumer should poll messages from and the producer should send messages to. If the topic name does not specify the stream path, and the property has a valid value, then this topic name is found in the default stream. |
| id |
Unique ID for this REST server instance. This is used in generating unique IDs for consumers that do not specify their ID. The ID is empty by default, which makes a single server setup easier to get up and running, but is not safe for multi-server deployments where automatic consumer IDs are used. Type: string. Default: empty |
| consumer.threads | The number of threads to run consumer requests on. Type: int. Default: 1 |
| simpleconsumer.cache.max.records |
Maximum number of records that can be stored in a single cache. Records with higher offsets replace records with lower ones. The value must be greater than 0. Type: int. Default: 1000. |
| simpleconsumer.max.caches.num |
Maximum number topic-partition combinations for which records are cached. If this parameter is set to 0, then caching is disabled and extra records are thrown away. Cache improves performance if records are fetched sequentially thus increasing offsets.
A pool of caches are available to store extra fetch records by a KafkaConsumer for a particular TopicPartition. The cache increases performance when records are fetched from a particular topic partition in a sequential manner. For example, every next request will start with the following offset after the offset of the latest fetched record in the previous request. Type: int. Default: 0 |
| simpleconsumer.max.poll.time |
Specifies the maximum number of milliseconds that are spent for polling records by a simpleconsumer. The greater the value means greater latency but higher throughput. Type: int. Default: 1000 |
| simpleconsumer.pool.size.max |
Maximum number of SimpleConsumers that can be instantiated. If 0, then the pool size is not limited. Type: int. Default: 25 |
| simpleconsumer.pool.timeout.ms |
Amount of time to wait for an available SimpleConsumer from the pool before failing. Use 0 for no timeout. Type: int. Default: 1000 |
| consumer.instance.timeout.ms | Amount of idle time (in milliseconds) before a consumer instance is automatically destroyed. Type: int. Default: 300000 (5 minutes) |
| consumer.iterator.backoff.ms |
Amount of time (in milliseconds) to backoff when an iterator runs out of data. If a consumer has a dedicated worker thread, this is effectively the maximum error for the entire request timeout. This parameter should be small enough to closely target the timeout, but large enough to avoid busy waiting. Type: int. Default: 50 |
| consumer.request.max.bytes |
Maximum number of bytes in unencoded message keys and values returned by a single request. This can be used by administrators to limit the memory used by a single consumer and to control the memory usage required to decode responses on clients that cannot perform a streaming decode. Note that the actual payload will be larger due to overhead from base64 encoding the response data and from JSON encoding the entire response. Type: long. Default: 6710884 |
| consumer.request.timeout.ms |
The maximum total time (in milliseconds) to wait for messages for a request if the maximum number of messages has not yet been reached. Type: int. Default: 1 |
| producer.threads |
Number of threads to run producer requests on. Type: int. Default: 5 |
| producer.streams.buffer.max.time.ms | Buffers messages in the producer for the maximum time specified time. A thread flushes all the messages that have been buffered beyond the time specified. Default: 1 |
| producers.max.caches.num | Maximum number user names for which producers are cached. If 0, then caching is disabled and producer will be created for each request. Default: 20 |
| request.logger.name |
Name of the SLF4J logger to write the NCSA Common Log Format request log. Type: string. Default: io.confluent.rest-utils.requests. |
| response.mediatype.default |
The default response media type that should be used if no specify types are requested in an Accept header. Type: string. Default: application/vnd.kafka.v1+json |
| response.mediatype.preferred |
An ordered list of the server's preferred media types used for responses, from most preferred to least. Type: list. Default: application/vnd.kafka.v1+json, application/vnd.kafka+json, application/json |
| access.control.allow.methods |
Sets the value to the Jetty Access-Control-Allow-Origin header for specified methods. Type: string. Default: empty |
| access.control.allow.origin |
Sets the value for the Jetty Access-Control-Allow-Origin header. Type: string. Default: empty |
| host.name |
The host name used to generate absolute URLs in responses. If empty, the default canonical hostname is used. Type: string. Default: empty |
| debug |
Boolean indicating whether extra debugging information is generated in some error response entities. Type: Boolean. Default: false |
| shutdown.graceful.ms |
Amount of time to wait after a shutdown request for outstanding requests to complete. Type: int. Default: 1000 |
| metric.reporters |
A list of classes to use as metrics reporters. Implementing the MetricReporterinterface allows plugging in classes that will be notified of new metric creation. The JmxReporter is always included to register JMX statistics. Type: list. Default: empty |
| metrics.jmx.prefix |
Prefix to apply to metric names for the default JMX reporter. Type: string. Default: kafka.rest |
| metrics.num.samples |
The number of samples maintained to compute metrics. Type: int. Default: 2 |
| metrics.sample.window.ms |
The metrics system maintains a configurable number of samples over a fixed window size. This configuration controls the size of the window. For example, used to maintain two samples each measured over a 30 second period. When a window expires, the oldest window is erased and overwritten. Type: long. Default: 30000 |