We are excited to announce the release of Aerospike 3.10. This release builds on our core mission of improving reliability for applications that demand predictable performance with high availability and also provides support for key new deployment environments such as IPv6.
SortedMap General Availability
Among our new features is the general availability (GA) of our SortedMap functionality. This feature was introduced as experimental in the 3.9.0 release and complements the already existing List functionality.SortedMap GA introduces in-database manipulation of Maps, allowing for network-efficient operation on data structures. The feature is now supported by Aerospike drivers for Java, C, Node, Python, Go, and Ruby. By supporting a multitude of languages, Aerospike provides efficient cross-language data structure operations for the most used data types.
Durable Delete
Aerospike now supports two methods of explicitly deleting records. The new delete method is “durable,” which means that the delete survives cold restarts and acts correctly as nodes are added to and removed from the cluster.
In this initial release, the new mechanism of durable delete writes a tombstone to storage when an item is deleted. This provides a durability guarantee and allows a greater range of merge policies when merging between deleted records and existing records. With this information, if a node joins a cluster with existing data, the merge policy can determine whether the tombstone is most recent or whether the data in the entering node is more correct.
Each delete writes to storage as opposed to just changing the RAM index, so performance is lower than the previous mechanism. There may be times when Aerospike has many tombstones but no explicit data records, resulting in a full database with zero records. Using durable deletes will complicate management of Aerospike. The new system requires a tombstone management background process. This process sweeps the datastore, looking for cases where a tombstone can finally be removed as no other copy of the data exists on the local disk. The policy of minimum dwell time must be maintained to handle the case of an out-of-cluster node that attempts to rejoin as there may be older versions of the data record that have not yet caught up to the tombstone. This extra background process will require storage and CPU resources.
The old expunge mechanism will be maintained and results in higher performance. This mechanism deletes the index entry to the data, thereby freeing the index space and storage space immediately. However, it does not durably write a tombstone marker record to storage, so deleted records can be restored with a full cold reboot of the cluster or network partition scenarios.
When considering whether to use the durable delete mechanism or the expunge delete mechanism, several factors should be considered:
Resilience of the application data model to older data
Frequency of multi-server cold restarts from storage
Frequency of network partitions
Importance of delete performance
Requirement to immediately reuse deleted storage space
In order to use the Durable Delete feature, you must use the new API policy detailed for Java here. As the Durable Delete feature is only available on Aerospike Enterprise, using this new client policy against a Community Edition server will result in a “not supported” error.
This feature is the first of many in this release to extend the reliability of the Aerospike database, but it does come with some tradeoffs. Because of these tradeoffs, we recommend that initial release users reach out to their Aerospike support contact to determine how to monitor and manage tombstones. Community Edition users who wish to access the Durable Delete feature may upgrade to the Aerospike Enterprise edition via an Enterprise Trial or QuickStart or contact Aerospike support. See our documentation for more information on the Durable Delete feature.
IPv6
Aerospike now supports operations in an IPv6 network environment, in response to specific requests from telecom equipment providers.
IPv6 is the next generation of the IP protocol, built to directly connect staggeringly huge numbers of computers directly to the Internet. The advent of mobile devices, which pushes up the number of IP addresses per person, has played a significant role in the acceleration of IPv6 technology.
In this initial release, Aerospike supports a variety of configurations of IPv6 technology, including using raw IPv6 addresses in configuration files, using DNS addresses that resolve to IPv6, and supporting “ANY” listeners.
Initially, Aerospike’s IPv6 work will be available for Java, C, and C# clients, with more to come. The server component is available only on Aerospike Enterprise Edition.
In order to use Aerospike on a network that is entirely IPv6, you will need to follow the configuration and deployment guide.
As IPv6 is a relatively new technology, we request you engage with Aerospike’s support team while configuring your IPv6-aware Aerospike cluster.
Improved Cluster Management
We are in the process of updating and improving several aspects of Aerospike cluster deployment and management. These updates support Aerospike’s leadership position of providing high predictable performance at the lowest hardware cost. In order to do so, we have improved several of the key protocols that announce cluster members to clients.
I. Reliability through Cluster Names
One such change, cluster_name, is notable because it improves reliability by preventing transactions from being applied to unintended servers when an IP address is reused, which occurs frequently in cloud environments like Amazon EC2. You can use this feature by adding heartbeat protocol v3 and a cluster_name field to your aerospike.conf configuration files. You should also specify the cluster_name in the client’s cluster_create call, which ensures the cluster you are connecting to is the correct cluster. The combination of implementing both client and server checks validates that the client is attaching to the intended cluster. If the cluster_name does not match, nodes will not be allowed to join that cluster.
II. Improved Cluster Heartbeat Implementation
Aerospike’s gossip protocol for determining cluster membership has been improved. While many of the changes involve code refactoring to simplify the implementation and remove inaccuracies seen through network failure simulations, other changes include:
Simplified management: Older management commands such as snubbing, dunning, and undunning are no longer supported. A reset command will trigger a new round of cluster validation and formation.
Lower bandwidth use: The protocol has been designed for massively improved efficiency, and compression is applied to packets.
Faster cluster formation.
An increase in practical cluster sizes: Previously, bandwidth and correctness issues would preclude the use of larger clusters.
Enabling the use of DNS addresses throughout the heartbeat and cluster management system.
III. Flexible Cluster Sizes
The new cluster protocol also removes the configuration setting MAX_CLUSTER_SIZE. Previously, changing this variable, which defaulted to 32 cluster in the Community Edition and 128 in the Enterprise Edition, required complex operator intervention. Furthering Aerospike’s goal to eliminate maintenance windows, refactoring the code eliminates this configuration. Instead, nodes can be added to the default cluster size limit, and changes from Community to Enterprise are seamless, even when expanding the cluster beyond the default Community cluster size limit.
Updated Network Naming
To simplify Aerospike cluster configuration and deployment, we have improved how to specify networks and interfaces within the configuration file by:
Adding support for DNS and network interface names, eliminating the need to maintain raw IP addresses.
Enabling consistent specification between our three different network protocol configurations—client-server, heartbeat, and fabric.
Extending the concept of bind and access addresses from single addresses to lists.
Configuration directives that take IP addresses now support:
IPv4 address (e.g., 1.2.3.4)
IPv6 address (e.g., 2001::1234)
DNS names (e.g.,cluster.aerospike.com)
Network interface names (e.g., eth0), only for bind addresses
Check out our documentation for more information on how to use the updated network naming features.
Other Improvements
Release 3.10 incorporates other improvements, such as:
Improved object histograms: To enable better performance tuning, new histograms show the size of read and write operations at any given time. This must be activated via the existing enable-benchmarks-storage config set.
Scan request timeout: Scan requests that are unable to run will now timeout after 10 seconds instead of blocking subsequent scans.
Aggregation performance improvement: To improve overall aggregation processing time for users with maps containing a large number of elements, we have improved transient map allocation, thus improving aggregation performance.
Backward Compatibility
Due to improvements made to the wire protocol to discover cluster nodes rolled out in 2015, Aerospike is removing support for some older clients. For the following languages, please make sure your clients have been upgraded to at least the specified version so they are backward compatible:
C: 3.0.26 (September 11, 2013)
C#/.NET: 3.1.2 (April 27, 2015)
C Libevent: 2.1.20 (September 24, 2013)
Go: 1.0 (no upgrade required)
Java: 3.1.1 (April 15, 2015)
Node.js: 1.0.27 (January 29, 2015)
Python: 1.0.35 (January 8, 2015)
Ruby: 1.0 (no upgrade required)
If you are not using these client versions or better, they will not be able to connect to a 3.10 cluster. During an upgrade, any non-compatible clients may cause proxy requests.
Perl and Erlang clients are still looking for community maintainers and have not been upgraded; thus, they should not be used with Aerospike 3.10. If you are interested in maintaining these clients, please contact us.
If you have written your own client for the Aerospike wire protocol, please contact Aerospike through support channels or the user forum to get advice on how to proceed.
What’s Next?
With our next release (3.11), we’re hoping to introduce several new features. Please consider the below list as a preview, not a commitment!
Transport Layer Security (SSL) encryption
Continued network clustering algorithm improvement
Support for adding and/or deleting namespaces on rolling restarts
If you’d like to download Aerospike 3.10, visit our download page.As always, we look forward to your input and help to continue to improve and enhance Aerospike. Feel free to contribute your feedback, ideas, and questions to our user forum, file Github issues, or create a pull request for the next great feature you’d like to contribute to the Aerospike user community!
