0

Background

I'm looking to implement a pass-through cache for a file-serving application. The users would make requests for files to the cache, if found, read from cache, otherwise the cache will fetch the file from a backend, then serve the file.

Basically, the cache will have 2 main functions:

  • Accessing files implies that certain permissions and other business logic need to be validated, and I have implemented this in the cache logic. This is something I have already available.
  • The cache needs to maintain the cached files in a local file system, and handle storage quotas, evictions, etc. This is the main concern for this question.

Requirements

The cache component needs to fulfill the following requirements:

  • I should be able to write and read files to/from some FS location (e.g. /cache/...).
  • The cache should be configurable with quotas (e.g. max storage in bytes or % of available disk).
  • The cache should handle evictions, either by LFU or LRU (or some other similar policy).
  • The cache should be ephemeral. Since I'll be filling the cache by my own logic, I don't want any permanent storage backing the cache. It should be safe for any files in the cache to be evicted at any time.
  • The cache needs to be able to run in Linux / Docker.

This seems like a common-enough problem that I imagine could be solved with some off-the-shelf tool, but I'm not familiar with any solution that can help.

Alternatives I've considered

  • Minio has a gateway feature that can be used with a local cache, but it's unclear if/how the cache can be used without a permanent storage backing it up.
  • Roll my own, with a combination of cronjobs, script magic and luck. I want to avoid coming with my own self-made solution if I can help it, mostly to avoid dealing with the cache house-keeping.

Question

Is there an off-the-shelf alternative for the described problem, a simple file-system cache that can be used for this application?

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.