The nodes that will be accessing the data are separated from the server by high latency low speed connections. I would like each node to have an LRU cache using around 250MB of ram and 20GB disk to minimize redundant communication with the server. The dataset on the server will grow by about a terabyte a month. A small (maybe 1GB) in-memory cache on the server would be nice, but is not required. As each node generates data it needs to be written to the server immediately so that other nodes may access it. Average record size is 16.5KB, ranging from 1KB to 32KB. Data is never deleted nor changed, so each node's disk cache should be persistent. The application is in Java, so a Java API or implementation would be ideal. Currently uses a home-grown Java DB implementation that needs to be thrown out. I'm hoping for a package that implements both the caching and database server components (and communication between the two). Open source for Linux would be ideal. Thank you.

No cache invalidation is needed other than LRU when full. The only functions needed are put(key,value) and get(key). Reads are user driven so latency is important but not critical (seconds are fine, minutes not so much). Writes occur throughout the day, often in small batches. It would be nice if write latency was measured in seconds or minutes, not hours.

  • Welcome to Software Recommendations, great question! Do you need perfect cache invalidation? What kind of requests need to be served? What request latency is acceptable? How often do writes happen, like do they happen in batch once a day, or little-by-little all the time?
    – Nicolas Raoul
    Feb 24 '16 at 10:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.