Aerospike announces our latest releases, Aerospike v3.13 and v3.14, which contain eagerly awaited improvements to the operational usability of Aerospike.
Popular for high-performance operational uses at scale, Aerospike has seen ever-increasing adoption thanks to its ability to provide scale and agility while ensuring availability, uptime, and unmatched performance. We continue to improve the core functionality of this system, whose Enterprise Edition version has been in production for years at some of the most demanding Internet companies, and whose open source Community Edition version is available free to all users.
Releases 3.13 and 3.14 contain three key features, which are explained below:
Large Clusters
Dynamic Namespaces
Rapid Upgrade
These new releases build on our core mission of providing not just a fast database, but also one that can survive and be managed in challenging real-world Internet use cases. In v3.13, we added critical operational capabilities such as a radical improvement to our core consensus algorithm for cluster formation, as well as the ability to dynamically add and remove namespaces. We’ve also made changes to our data synchronization and migration system in order to speed up in-production version updates.
Required “Jump” Release
Due to our changes in core clustering protocols, before they can apply 3.14, existing clusters must first upgrade to 3.13 and undergo a specific operational procedure to switch to the new protocol. The 3.14 release removes the old clustering protocols and is the recommended release for all new clusters. Please read our release notes to learn how to operationally apply these changes to existing clusters.
Rack-Aware Configuration Changes
If you have been using the Aerospike rack-aware functionality to separate out data replication among multiple racks, you will be pleased to see simplified configuration along with Aerospike Cluster Manager v5 (described below). Please follow our guide to upgrade your rack-aware configuration as you upgrade through 3.13 to 3.14.
Aerospike Cluster Manager Version 5
Aerospike focuses on clusters of 10 to 80 nodes. At this scale—with today’s networking and Flash hardware—it is possible to attain upwards of 50M TPS on 500T of replicated storage and still maintain the capacity to grow tenfold by organically adding nodes.
Over the last few releases, Aerospike has been improving cluster formation. Aerospike v3.10 included a major upgrade to our “heartbeat” system for clustering information. In Aerospike v3.13, we did a significant rewrite of the system to provide new functionality, increase cluster sizes, and provide more stability for larger clusters. We call this functionality “Aerospike Cluster Manager Version 5” (ACM v5).
A classic problem in distributed systems concerns cases where network connectivity is not symmetric. In these cases, some nodes can connect to a subset of nodes, but some nodes may be able to connect to all nodes. Network “brownouts” may cause a large number of simultaneous, but transitory, faults. By arranging changes into a “microbatch” of transitions and maintaining and distributing all interconnection information, the cluster can make far more informed choices about which nodes should be included, and how requests and data are routed.
In our 3.13 and 3.14 releases, Aerospike vastly improved the protocols by which we exchange information regarding clustering. By improving efficiency by 100x, we’ve both allowed more information to be exchanged, and increased the practical number of nodes per cluster. The system now easily supports the 128 nodes that constitute our current configuration limit—even in demanding public cloud environments—and positions us well to achieve a 10x cluster size growth in future releases.
ACM v5 also supports greater precision during cluster changes and enables the “Rapid Upgrade” feature described below. Moreover, it supports additional flexibility in namespace configuration (also defined below).
We’ve applied this core improvement to both the Aerospike Community Edition (dubbed CE, the open source version) and the Aerospike Enterprise Edition (EE). Please note that the CE is limited to 32 nodes and the EE to 128 nodes per cluster.
While newly created clusters using 3.14 will automatically use ACM v5, pre-existing clusters require an extra operational step, as they must first upgrade through 3.13. For instructions on how to upgrade, please review the operational guide.
Dynamic Namespaces
Aerospike’s configuration allows multiple namespaces, representing differing storage configurations, storage capacities, and replication factors.
In the past, adding and removing namespaces was a difficult operational procedure, which often required either cluster downtime or complex operational procedures. Adding namespaces or changing the storage capacity of a cluster was difficult.
With the improvements in our latest releases, namespaces can now be added dynamically. When you change a node’s configuration to either add or remove a node, and then restart, the namespace is changed. If only a subset of the nodes serve that namespace, then those nodes are allowed to participate. During a transition, you may have a subset of the nodes—say, 10 out of 20—serving a particular namespace. The only supported configuration is running with all nodes serving each namespace; other configurations are expected to be transitional.
This feature, which has been broadly requested by Aerospike’s existing user base, is available immediately in both flavors of Aerospike (CE and EE). To access it, you will need to transition to the ACM v5 clustering mechanism described above.
Please see our operational guide for more information regarding dynamic namespace addition and removal.
Rapid Upgrade
As a highly available (AP) database, Aerospike commonly recommends a software upgrade process whereby individual servers are restarted only after the cluster has resynchronized its data. Doing otherwise could result in dropped and lost writes.
As part of our advanced cluster management system (ACM v5, described above), Aerospike now allows the rapid upgrading of cluster nodes. A wide variety of simulations and testing have validated that with our new clustering and partition management changes, writes are not lost during a rapid cluster upgrade.
This change is transparent to users. The benefit in reliability will only be observed either (a) once all nodes have been updated to 3.13 and the clustering has been upgraded to ACM v5, or (b) once Aerospike 3.14 is in use. Like other features in this release, it is immediately available in both the CE and EE editions.
Reminder: Large Data Type (LDT) and Set Delete Deprecation
The large data type functionality has been deprecated in both the Community Edition and the Enterprise Edition; 3.14 will be the last release to include the LDT feature. If you use LDTs, please migrate any application functionality from this interface. Please contact Aerospike if you need any advice regarding the best application design for meeting your need for large data types.
Aerospike’s “Set Delete” functionality has been superseded by a feature we call “Set Truncate”. Truncating is a much faster and more reliable mechanism for achieving the same result—dropping tables. Set Delete is supported in 3.13, but was removed in 3.14. To learn how to migrate from Set Delete to Set Truncate, please read our guide on this topic.
Other Notable Improvements
Aerospike Spark 1.3. In this release, we’ve added additional methods to segment queries and workloads, as well as the ability to “AeroJoin” to provide exceptionally fast join methodology. Now more than ever, this makes Aerospike an excellent database choice within your Spark environment.
Please visit the download section on our website to download Aerospike 3.13 or Aerospike 3.14.
As always, we look forward to your input and help to continue to improve and enhance Aerospike. Feel free to contribute your feedback, ideas, and questions to our user forum, file Github issues, or create a pull request for the next great feature you’d like to contribute to the Aerospike user community!
