MapR-ES and Apps

MapR-ES brings integrated publish and subscribe messaging to the MapR Converged Data Platform.

Topics in MapR-ES are grouped into streams, to which administrators can apply security, retention, and replication policies. Combined with MapR-FS and MapR-DB in the MapR Converged Data Platform, using these streams enables organizations to create a centralized, secure data lake that unifies files, database tables, and message topics.

Producer applications can publish messages to topics, which are logical collections of messages, that are managed by MapR-ES. Consumer applications can then read those messages at their own pace. All messages published to MapR-ES are persisted, allowing future consumers to “catch-up” on processing and analytics applications to process historical data.

Use Cases

MapR-ES is ideal for a variety of use cases, including the following:
Application event pipelines
Many types of applications generate event or log data that must be centrally stored and analyzed to gain insights about user activity or application performance. MapR-ES simplifies these pipelines by transporting events to a central location, from which they can undergo event-by-event transformation and analysis.
Database change capture
Most modern databases enable users to generate an event each time an entry is added or modified. These events can be published to MapR-ES to keep systems like search indexes and caches synchronized, as well as to feed security or notification applications.
Internet of Things
The explosion in the number of smart devices and sensors has created many situations in which billions of data points are created by millions of geographically dispersed sensors. MapR-ES provides a reliable, global transport for these messages, enabling you to perform analytics both at the source and at a central location.

Replication

In addition to reliably delivering messages to applications within a single data center, MapR-ES can continuously replicate data between multiple clusters, delivering messages globally. Like other MapR services, MapR-ES has a distributed, scale-out design, allowing it to scale to billions of messages per second, millions of topics, and millions of producer and consumer applications.

Security

MapR-ES supports user impersonation through the Java API. See MapR-ES Java Applications for more information. MapR-ES does not support user impersonation through the C API or Python API.
Note: Kafka REST supports outbound user impersonation. See User Impersonation in Kafka REST for more information.