We are Sponsoring Kafka Summit 2020

Sheryl Sage, Director Technology Partners and Marketing Blog, Technology

Modern data architectures don’t work in isolation. They are made up of multiple technologies each serving a different purpose in the data ecosystem.  Apache Kafka® fits naturally as a messaging layer for real-time streaming data pipelines to reliably get data between many independent systems or applications.

One of Aerospike’s superpowers is fast data access, so it is no surprise that it is often part of a larger event-driven architecture consisting of one or more data stores, messaging systems and information processing systems,

With this in mind, in 2019 we released Aerospike Connect for Kafka to exchange data bi-directionally between Aerospike and transactional systems at the edge and core.  Aerospike Connect for Kafka makes it easy to integrate an Aerospike database into a larger system’s architecture. Aerospike Connect for Kafka includes two connectors:

  • Inbound for streaming data from Kafka into an Aerospike database.
    The inbound Aerospike Kafka Sink Connector streams data from Kafka and persists it to an Aerospike database. Each Kafka record is converted to an Aerospike record and persisted to an Aerospike cluster. In standalone mode all work is performed in a single process. Distributed mode handles automatic balancing of work, supports automatic scaling, and offers fault tolerance both in the active tasks and for configuration and offset commit data.
    Inbound for streaming data from Kafka into an Aerospike database
  • Outbound for updates from an Aerospike database to Kafka.
    The outbound connector receives data from the Aerospike server via the Change Notification Framework. Each message contains either an updated/inserted record or a record deletion notification.
    Outbound for updates from an Aerospike database to Kafka

Aerospike supports JSON, Flat JSON, Avro, Kafka Avro and MessagePack message formats.  A REST Client can be utilized as an easy and standard interface for the Aerospike database.

In partnership with Confluent, Aerospike is excited to sponsor Kafka Summit 2020, which kicks off next week with an action-packed virtual program led by prominent experts to learn more about Apache Kafka®  — a highly flexible streaming platform for real time digital businesses.

Kiran MattyOn Monday August 24th, 11:30AM PT, we invite you to join us for a deep dive into Distributed Data Storage & Streaming for Real-time Decisioning using Kafka, Spark and Aerospike, delivered by Kiran Matty, Director Product Management at Aerospike. In this talk Kiran will cover how Aerospike Connect for Kafka and Spark is used for real time analytics including reviewing a real world trading application and an HPE COVID-19 project architecture.

Kiran is the Director of Product Management at Aerospike, where he manages ecosystem products that relate to AI/ML, Streaming, and Aerospike DB access. His experience spans big data and ML infrastructure, data security, and analytics. Prior to Aerospike, he was a product manager at Visa, Hortonworks, HPE, and Cisco. His interests include large scale distributed systems and AI/ML.

If you’re ready to take your Kafka skills to the next level join us at Kafka Summit, August 24-25,

Share:

About Author

mm

    Sheryl Sage, Director Technology Partners and Marketing

    All posts by this author
    Sheryl is the Director of Technology Partners and Marketing at Aerospike. She is a partner and ecosystems development strategist with a track record of successfully building integrated solutions, strategic alliances and GTM programs with global technology partners. Sheryl has held positions in startup and large global technology companies including Redis Labs, MapR, Informatica and VMware where she built several global partner programs including the VMware Solution Exchange, Technology Alliance Partner (TAP) and VMware Ready programs.