2

I have evaluated Beaker and Dogpile caches, but due to their backends they're not very fast when it comes to "batch" style reading and writing multiple keys.

I need to cache some values for particular database keys (computing results for them is expensive, while I do not want to "clutter" database schema with ad-hoc cache colummns in tables themselves on top of having to handle expiration myself).

The problem: suppose you have computed some HTML snippets (that should expire overnight) for 10,000 RDBMS table keys.

I want to cache them and retrieve cached values for say 2,000 particular keys resulting from SQL query. I also need to cache them on disk (it's too much content to keep it all in memory). Obviously, storing and retrieving entire set of 10,000 computed values (everything in a cache region) just to get this subset of 2,000 keys is undesirable.

Reading them one-by-one from cache like Dogpile or Beaker is not very fast (it's so bad that computing them on the fly is actually faster than retrieving from cache..). Something like this would be better:

http://flask.pocoo.org/snippets/87/

I was thinking about modifying this snippet myself to use SELECT ... WHERE KEY_COLUMN IN (1, 100, 500, 1000, ...) for SQLite, but obviously that's just a snippet, not a cache of this kind available off the shelf (with nice stuff such as Dogpile's "regions").

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.