2

or "Polipo, but also caching HTTPS"

My friend is looking for a caching proxy to make the most of his PC's unreliable Internet connection. He often sees ping times measured in thousands of milliseconds. To work around this, he caches the web aggressively. He currently runs Polipo listening on localhost, which caches legacy unencrypted HTTP and tunnels HTTPS without caching. But as more sites go all HTTPS all the time, with HSTS and 301 redirects to enforce HTTPS use as well as sensitive features available only in secure contexts, Polipo is falling back to a tunnel more and more often, causing his cache miss rate to increase.

I've suggested to him that he use a different caching proxy, one that can act as a man in the middle for HTTPS using a local private CA. He wants one meeting the following specs:

  • Runs on GNU/Linux
  • Preferably free software
  • Caches both HTTP and HTTPS, with easy-to-understand configuration of the private CA
  • Can handle a very unreliable outgoing network with high latency and dropouts, such as satellite or cellular with marginal reception
  • "Latency more important than throughput"
  • "Can censor HTTP request headers for reasons" and rewrite Referer to avoid anti-hotlinking measures (such as by always sending the root url as the Referer)
  • Can dedicate 1 GB of RAM (out of the 8 GB in the machine) and 20 GB of disk space to the cache

He found Squid "a nightmare" to set up, and it didn't improve perceived latency, nor could it resume a partial download.

He has found that the built-in caches of major free web browsers (Chromium and Firefox) lack detailed configuration features suitable for connections that aren't unmetered wired broadband. They delete fairly early because of capacity and/or Expires:. They can't share one cache between machines on his home LAN. And many sites he visits fail to follow cache control best practices. With a separate caching proxy, "I only have to trawl through one poorly-written manual instead of half a dozen before I find out that the feature I'm looking for is not supported." In particular, he wants to trust Last-Modified and ETag over Expires.

I've been working with him on his home forum to elicit enough detail to seek a recommendation.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.