Intel® Optane™ SSDs and Aerospike Database: a New Level of Fast in Operational Stores for Hybrid Transactional/Analytical Processing

Aerospike’s founding was predicated on building a hybrid memory, ultra-fast operational database that could take advantage of non-volatile memory (NVM) from the ground up. By relying on non-relational constructs popularized as “NoSQL”, key-value (primary key) access is fast and parallel. Since rows are locked individually, without the requirement of maintaining relational structures, analysis capability improves dramatically.

As the NAND Flash market matured, products were offered for low-write “data lake” uses. However, operational databases continue to require high write rates. A use case for which Intel® Optane™ SSDs is uniquely suited comes from the financial services industry. Capital markets order management requires breaking an individual trade into child trades, settling them in different exchanges, understanding the moment-by-moment risk profile of the broker, and applying post-trade analytics in order to understand exchange efficiency and place the next trade efficiently. Legacy order management systems were built to gauge risk over a 24-hour period; they now require risk to be recalculated in minutes. The new multitude of trading exchanges requires analytics to determine where and how to place trades. Further analytics—such as a more detailed look at fraud schemes—may soon be required. Further desires by end customers—for instance, seeing the status of in-flight trades, and displaying them in rich mobile environments—are upcoming requirements.

These new capabilities require a trading system with a data layer that can handle millions of operations per day—both to update their status and to make this status immediately available for analysis. Bringing to bear new analytics on in-flight transactional data is the focus of HTAP (Hybrid Transactional/Analytical Processing), which is where ultra-fast non-relational analytical processing systems are headed. The random behavior of the workload, the heavy writes needed to keep up with trades, and the many downstream analytics systems that require the data in sub-1 millisecond are all highly demanding. Relational (SQL) databases are not write-capable in this manner: their writes take too many milliseconds, they don’t scale out well, and they are now only mostly relevant for the system of record and accounting aspects they were originally designed for. Thus, a non-relational approach is required. And in order to reach this new level of processing and business commitment capability, new non-volatile memories are required as well.

Intel^Ⓡ Optane™ SSDs provides characteristics that match these new requirements: the storage media provides persistence, uniform write/read latency, and world-class responsiveness. The media itself is the breakthrough, thus making the network and application logic truly become the “next” bottleneck.

A financial use case with Aerospike Database

In order to gauge the benefits of Intel® Optane™ technology in a real-world test, Aerospike took the requirements of a retail brokerage customer and created an end-to-end test that takes into account not just the benefits of the Optane™ media, but also the system-wide latencies of client software, server software, network, and operating system layers. The use case has a mix of object sizes and phases—read-only, write-only, and the core intra-day mixed (read/write) workload.

How much faster is the Intel® Optane™ SSD solution for these workloads on a per-server basis when the requirements call for 0.5 millisecond end-to-end database responsiveness? Unlike NAND drives, whose responsiveness rapidly degrades, Intel’s Optane™ SSD technology can be pushed hard. In fact, according to a test run by Aerospike on Optane™-enabled Intel servers, Intel® Optane™ performs about 3x faster than NAND drives at the required latency level.

Latency results of Optane testing

Transactional results of Optane testing

Naturally, no benchmark is perfect. This benchmark reflects an attempt to realistically model the production requirements of an existing customer. As Brian Bulkowski, Aerospike’s Co-founder and CTO, confidently tells his customers,

“Our testing shows that Intel® Optane™ SSDs deliver half a million transactions per second per server with real-world responsiveness, no matter the write pressure or reasonable object size growth.”

Intel says two simple words: “write on”.