Adform: Transforming Batch-based Data Processing Pipeline

Andrius Mažeiva, Head of Solutions Architecture, Adform

Adform is an Ad Tech company which provides an integrated advertising platform both for sell and buy-side users. Aerospike is used very widely in our company. We use it starting from our real time bidding. We use it in the sell side for building profiles. We use it for processing. We are using it for the joining different streams, so we have really wide use cases where actually Aerospike helps us do business.

The main challenge before Aerospike was the speed of operations, how fast we can get we needed records. Previously we have been using relational database and we needed actually key value functionality. And Aerospike in this case provides really great speed, availability and scalability features for such functionality.

The main thing I loved about Aerospike is the performance on both the amount of operations that it can perform and on the quite limited amount of servers. So you can, you actually can get most of the servers by using Aerospike on top of it. The fewer servers a cluster has the easier it is to maintain it, where smaller is the possibility, that some fail, some several will fail. So that’s why having less servers, actually makes all the solution cheaper to operate.

Aerospike solution surprised me about how fast you actually can get things working, how fast you can actually start a setup in the cluster and start working on a solution.

The thing which I most like in Aerospike is performance. Performance you get while getting the records and the flexibility about actually storing what you need. When we started to work with Aerospike, we started from Community Edition and switching to Enterprise Edition actually gave us success to the support which helped us to solve other problems much, much faster.

One of the use cases where we use Aerospike is joining the streams, so we use Aerospike in-memory mode for actually separating the joining into two separate processes. One for just feeding the data to Aerospike and the another one for getting the data by keys. And performance of Aerospike in this case allow us to do whatever kinds of joins we need, as big windows as we need.

Aerospike helps us to have a one integrated platform because we can use it for many different things and it allows us to build a core component, which is supplying all of our business. I believe Aerospike is the only product, only key value store, which can satisfy the needs of Ad Tech.