stream create

Creates a new stream.

After you create a stream, you can edit the values of its parameters with the command maprcli stream edit.

To see the value of a stream's parameters, use the command maprcli stream info.

To run this command, your user ID must have write permission on the directory in which you want to create a stream.

Permissions Required

To run this command, your user ID must have the following permissions:
  • readAce and writeAce on the volume
  • lookupdir on directories in the path
NOTE
The mapr user is not treated as a superuser. HPE Ezmeral Data Fabric Streams does not allow the mapr user to run this command unless that user is given the relevant permission or permissions with access-control expressions.

Syntax

CLI 
maprcli stream create
	 -path <Stream Path> 
	[ -ttl <Time to live in second> default:604800 ]
	[ -autocreate <Auto create topics> default:true ]
	[ -defaultpartitions <Default partitions per topic> default:1 ]
	[ -compression off|lz4|lzf|zlib. default: inherit from parent directory ]
	[ -produceperm <Producer access control expression> default u:creator ]
	[ -consumeperm <Consumer access control expression> default u:creator ]
	[ -topicperm <Topic CRUD access control expression> default u:creator ]
	[ -copyperm <Stream copy access control expression> default u:creator ]
	[ -adminperm <Stream administration access control expression> default u:creator ]
	[ -copymetafrom <Stream to copy attributes from> default:none ]
       [ -ischangelog <true|false> default: false ]
       [ -defaulttimestamptype timestamp type: createtime | logappendtime. default: createtime ]
       [ -pidexpirysecs <Producer ID expiry time in seconds. Default: 6048000> ]
       [ -mincompactionlag <Set time in melliseconds for which a message remains uncompacted. Default: 0> ]                
       [ -deleteretention <Set the time in milliseconds for which delete records are retained. 
                    Default: 86400000> ]
REST http[s]://<host>:<port>/rest/stream/create?path=<path>

Parameters

Parameter Description
path The path and name of the stream to create.

The path to the stream can include any character allowed by Data Fabric. For example, /my/path/with:/to/mystream is valid, but /my/path/with:/to/mystream:withcolon is invalid.

The name of the stream cannot include a colon ( : ) or a forward slash ( / ).
ttl Specifies the number of seconds to elapse between the publication of a message in a topic in this stream and the expiration of that message.

Consumers do not see messages that have expired.

Messages that have expired are deleted during the next purge process. See Time-to-Live for Messages for details.

A value of 0 causes messages to be retained indefinitely.

autocreate Specifies whether to create a topic automatically when a producer tries to write the first message to it. Values are true and false. The default is true.
defaultpartitions Specifies the default number of partitions to allocate to new topics in the stream.
compression

Specifies the compression setting to use for the stream. Producer client libraries can bundle messages that are to be published on the same partition and compress them. The messages are sent to the server compressed, are stored compressed, are replicated to other containers compressed, and (if stream replication is configured) replicated to replica streams compressed. Consumer client libraries receive compressed data, decompresses it, and passes it to client applications.

Valid options are off, lzf, lz4, and zlib. The default setting is the type of compression that is set for the directory in which the stream is located.

For more information, see Compression.

produceperm Specifies the access-control expression that controls who can publish messages to topics in the stream. See ACE Syntax.
consumeperm Specifies the access-control expression that controls who can who can listen to topics in the stream. See ACE Syntax.
topicperm Specifies the access-control expression that controls who can create, edit, or remove topics in the stream. See ACE Syntax.
copyperm Specifies the access-control expression that controls who can use mapr copystream or mapr diffstreams on the stream. See ACE Syntax.
adminperm Determines which users can modify ACEs for a stream, set up replication of a stream, and modify other attributes of a stream. By default, the stream owner and the Data Fabric user can modify this setting. See ACE Syntax.

This permission includes the topicperm permission.

copymetafrom If you plan to replicate messages to this stream from another stream, specify the path to that other stream. The metadata from that stream will be copied to the new stream when the new stream is created.
ischangelog Specifies whether the stream is for the Change Data Capture feature's changed data records. Value: true|false. Default: false.
  • If you want to use a non-default partition value (Default: 1) for the topic, use the maprcli stream create -path </mypath/stream:topic> -ischangelog true -defaultpartitions < value other than 1 > command to create the stream and then create the topic with the maprcli streams topic create command.
  • If you want to use the default partitions value (1) for the Change Data Capture feature, use the maprcli stream create -path </mypath/stream> -ischangelog true command to create the stream and then use the maprcli table changelog add -path <mypath/stream:topic command to set up the Change Data Capture feature and create the topic.
defaulttimestamptype Specifies the type of timestamp stored in the topic's message. Value: createtime | logappendtime Default: createtime. The topic inherits the default value from the stream unless the topic sets the timestamp type to a different value.

A createtime value is the time defined by the user or application (when creating the message). If user or application does not define this value (or passes null), the client uses the current system timestamp.

A logappendtime value is the time when the message (log) was appended to the server.
pidexpirysecs Specifies the expiration time for the Producer ID. This parameter fixes the lifetime for the Producer ID. Default: 604800
mincompactionlag Sets the minimum delay (in milliseconds) before which messages are not compacted. It is the minimum time that the messages are available for consumption. Beyond this time period, the messages may be compacted. Default: 0

The lag is calculated from the time that a message was produced to the stream topic-partition.

NOTE
Compaction is set only when you edit the stream. See stream edit.
deleteretention Sets the minimum time (in milliseconds) before which deleted records are removed. It is the minimum time that the deleted records are still available. Beyone this time period, the deleted messages may be removed. Default: 86400000

Used with log compaction.