7

Is there some kind of library or framework to support a redundantly distributed collection of Java data objects? By 'redundantly', I mean a complete set of data is maintained on every node of the distributed team of computers.

Basically I am looking for an alternative to a database server. A database can be overkill for the equivalent of a single table with a simple schema with little data churn such as merely adding a single row ever few minutes. Another problem with a database is that each server must use polling to get updated data. I need a distributed collection that informs its peers whenever a item is added/removed/updated in the collection.

I've been introduced to HazelCast technology which implements collections (lists, maps, queues, and more) shared across JVMs (Java virtual machines). Any of the JVMs (nodes) can perform a Get or Put operation against the entire collection unified in memory across the multiple JVMs. Interesting technology, but not quite what I need.

Sharding is used in the HazelCast implementation, with each node holding a subset of the collection’s objects plus a partial backup of other nodes’ objects. This approach using sharding assumes that the app running on the various nodes does not regularly need access to all the objects in the collection.

In my case, I have collections of a million objects, each a sample of scientific data generated each minute in a time series. I have multiple JVMs for redundancy. Each JVM needs a full set of data stored in memory, for quick frequent access. If any JVM adds an object to the collection, the other nodes should be identically updated across the internet.

3

SharedHashMap (Github) is worth to be added to your watchlist. It fits your description. SharedHashMap is java.util.Map implementation which stores the entries off-heap and replicates them across nodes via sockets. Keys and values should be Serializable or implement specialized marshalling interface.

It isn't production-ready solution yet, but it is very actively developed right now.

P. S. Note I am affiliated with OpenHFT.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.