3

I'm looking for recommendations for libraries that help me create an efficient and fast object value cache in Java.

My needs are as follows:

  • Write: There are up to like 50000 writes per second to the cache. The good news is that all writes happen from a single thread. Inserts/deletes are rare, what actually changes are the values of the individual objects in the cache. (see below)
  • Read: Read's are quite infrequent. Perhaps once every 10 second. Reads can happen from any thread. There aren't many readers. (most of the time just one, but up to let's say 3 readers)
  • Cache management: There's no need for automated eviction. In fact, any eviction is always explicit (a delete operation, so to speak)
  • Memory size: Not much of a problem. I may have 20k objects in the cache, maximum, and they can easily fit inside any heap.

Each object in the cache is essentially a wrapper around a key/value pair map. Through the lifetime of a cached object this map doesn't expand, nor shrink (meaning keys are always the same). What changes are the values in the key/value pair map. Writes must be done in an atomic operation, meaning that a reader must not be able to read from an object while it is being written to.

My questions:

  1. Are there any libraries I should look at ?
  2. Are there any techniques I should look at? (for example in terms of reducing lock contention)

All in all you can think of an analogy of a database table. The cache is equivalent to the table in this analogy. There aren't many INSERT/DELETE operations on the table, but there's a tremendous amount of UPDATE operations. And we will not allow dirty reads.

(no, my use case has nothing to do with databases, I do not need persistence, I only mention database table because it is an good analogy)

I'm sure problems similar to this has been tackled before so I would rather like to piggy-back on someone else experience. However most of the libraries I have been able to find seem to cater for the case where there's some backend which is slow and hence it makes sense to cache a result. (which means you need to deal with things like eviction, staleness, off-heap storage, sharding, distributed, and what not, ... things that are completely irrelevant in my case)

12
  • Do you need persistence? Or does this data live only for the duration of the app's runtime? Sep 1, 2017 at 5:59
  • You seem to be conflating two different thread-safety issues. There is the safety of making structural modifications to the mapping, adding/removing keys or values from multiple threads. Separate is the question of changing the content within any particular value object from multiple threads. Please clarify in your Question which you mean. This first issue appears to be what you mean in your "What changes…" sentence. The second issue appears to be what you mean in your "Writes must be done…" sentence. Sep 1, 2017 at 6:38
  • @BasilBourque. No need for persistence.
    – peterh
    Sep 1, 2017 at 8:05
  • @BasilBourque. I've added some more information in my question using a database table analogy. Hope it helps, but you already seem to have a good understanding of the problem. :-) I don't really have doubt that the outermost part of the cache is a ConcurrentHashMap. It is performant enough. It is the contention on the individual object I'm worried about (if using the database analogy: row lock contention)
    – peterh
    Sep 1, 2017 at 8:12
  • Your Question is still not clear on the core critical issue: Are you replacing the value object mapped to a key, or are you changing the content inside the value object mapped to a key? These are two very different issues with very different solutions. Sep 1, 2017 at 15:54

3 Answers 3

5

tl;dr

No need for a library. The built-in Java libraries suffice.

Details

Probably not appropriate to post programming details here.

If you post this same kind on Question on Stack Overflow, but asking how to do this in general rather than asking for a library recommendation, then I can post code for a working example that executes eight million( 8,000,000 ) put operations per second (more than enough for your 50,000 per second) while having 3 threads reading every several seconds.

  • Being a ConcurrentMap means it can handle concurrent readers accessing the map as a writer may be swapping out a value object with a modified clone, all the while maintaining happen-before behavior.
  • Being a NavigableMap means the keys are kept sorted for rapid get. This is appropriate since you said the keys are rarely replaced; we are doing vastly more reading of keys than writing.
0

MapDb might work for your purpose. It features drop-in replacement for Maps, Lists, Queues and other collections. Also provides RDBM like transactions support.

0

Two packages may be of interest:

  • Apache Geode. What used to be Gemfire is now open sourced as an Apache project. This is very mature and highly performant software used in many financial trading companies.
  • Hazelcast. Also very mature, similar in scope to Geode.

Try both to see which works best for you.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.