Topic Creation

Topics are created in streams and contain logical collections of messages. They can be created either automatically through your producer application or manually through the MCS or maprcli.

Automatic Creation

If the topic does not already exist, a topic is created automatically when a producer first publishes a message to it. This is the default behavior.

For example, you created the stream anonymous_usage that you intend to use to collect data about the usage of a software application that is soon to be released. However, you did not create any stream topics because the topics were not known at the time. After the software is released to the public, at some point, a producer application starts publishing messages to a topic that is created based on the range within which the producer's IP address falls. At another point in time, another producer application starts publishing messages to a topic based on a different range of IP addresses. Eventually, the stream contains a number of topics for different IP address ranges.

Note: Automatic creation of topics can be turned off by setting the autocreate parameter to false either when creating the stream or by editing the stream. See the maprcli stream create or stream edit command for more information. If you turn off automatic creation, you must manually create the stream topic, otherwise, the publishing of a message fails.

Manual Creation

To create topics manually, use either the MCS or maprcli. See Administering Streams for information about managing MapR-ES streams, topics and replication. See the stream topic create command for specific information about creating stream topics with maprcli.

For example, you created a stream called systemMetrics that you intend to use to collect operational metrics from systems in your enterprise. You did create several topics based on system, location, company department, project, or some other criterion. In this case, you could create topics in advance because they were pre-planned.

Note: When you manually create a topic, you can have the option of customizing the number of topic partitions used, otherwise, the default number of partitions is inherited from the stream.