Note: Please read the question carefully - it may present itself as general / wide-audience or possibly asked with illicit purpose when in fact it's intended for a narrow use-case and I strived to research it well.
In short: I'm looking to tunnel traffic via a custom (own) server such that it's shielded from the ISP, and the tunnelling is available selectively to applications without requiring a rooted device.
Long version - motivation, approach and things tried
Motivation: While the majority of use-cases in the area of VPN, tunnels and proxies concern privacy, bypassing internet censorship or playing games, my use-case has to do with some ISPs not being able to route certain packets, specifically those required for establishing a TLS v1.3 handshake (long related read here). However because internet censorship is a problem for so many, the space is littered with poor quality solutions, questions, and applications, some of which are likely to contain scam or even malware.
Technique: the best way I found to do this is the following:
- have an AWS machine (or different cloud provider) with fixed IP and key-based SSH access, to route traffic through. EC2 machines come out of the box like this (they give you a PEM file)
- on the Android device create a SOCKS proxy by maintaining a
ssh -Dport forwarding connection to your server (aka dynamic application-level port forwarding). Traditionally this would be bound to 127.0.0.1:1080, i.e. for local use only.
- also on the Android device, route outbound traffic of specific apps (or all apps) through the tunnel. I'm lacking expertise here but there seem to be 2 methods to do this routing - the "iptables way" and the "VPN way" (read this excellent answer in the Android SO). The former requires root, the latter doesn't, so is preferred.
- Looking for a VPN app to do this is a nightmare because regularly they come as fully fledged solutions (not via local SOCKS but with some paid or free inbuilt servers). In my use-case VPN is a misnomer because the app merely uses Android's non-root VPN API for the purposes of application "SOCKSification" i.e. the routing of an app's traffic via a proxy without the app having been written to support that.
- many tunnelling apps exist but they may lack the necessary customisability, convenience, or may not offer a decent way to use them (contain ads which literally take over your phone, I'm fine with paying but many don't offer a premium version).
Note - there are reports of being able to do this with either one single app or two (one to create the SOCKS proxy and one for the VPN part). I'm fine with 2 apps but that comes with caveats (need to start in a certain order etc), see below.
For the SSH tunnel:
- (A1) key-based auth (private key file)
- (A2) possibility to use your own server, username, port (obviously)
- (A3) start on boot
- (A4) resilient to connection loss, including when switching between WiFi and Mobile Data
For the VPN:
- (B1) possibility to connect to a local server (127.0.0.1:1080)
- (B2) possibility to pick which apps use the VPN (this can be done via package names). This is a requirement when using a 2-apps solution since obvs the SOCKS tunnel should not be routed through the VPN, so should be added as an exception. But this is very convenient regardless, e.g. one may not want browsing (or might ONLY want browsing) to be tunnelled.
- (B3) resilient to connection loss, including when switching between WiFi and Mobile Data (all VPNs can be configured as always-on so they would start on boot)
Apps tried: These apps seem designed to do both tasks in one but neither worked for some reason.
- iSocks VPN; does a connect/disconnect loop
- HTTP SOCKS PROXY; connects the tunnel but the VPN part doesn't work
- Socks injector; connects the tunnel but Permission denied in the logs on the VPN part (related to tun2socks)
- TLS TUNNEL; works globally (for all apps) but no app selection, no key-based auth, ridiculous ads and no premium version
There are many 2-app combos to try, main difficulties are
- many SSH clients will do SOCKS proxying but don't offer a convenient way to persist the connection
- VPN apps generally don't let you pick your custom server (since they want you to pay for one)
Has anyone done this conveniently?
Edit: I found a very good paper showing how poor and sometimes malicious apps of this type can be.