I need to copy a few hundred GB of data between two machines. Due to limited diskspace and time I would far prefer known-good transfers on the first go, so I want something that verifies the data as it's transferring. (I will most definitely still be checksumming the files when they're done to make sure they really did transfer okay, but in-flight verification significantly minimizes the likelihood anything will have gone wrong, and I highly value this.)
The machine with the data on it is running Linux; the destination is a laptop running Windows 7 (at least until I get Linux installed on it as well). So, the tool needs to at least support Win7 and Linux.
My Wi-Fi is largely fairly stable but has the occasional hitch/jitter; it also does abruptly go out to lunch every so often, and of course power cycling it kills all active connections. So, I'm looking for something that gracefully handles "completely disconnected" and continually retries to reconnect until successful.
I've weighed up encryption vs no encryption and I've decided that while I don't really need it, I also don't see the point in making do with (or suggesting) programs that don't implement encryption, given the ubiquity of HTTPS.
Transparent compression is okay but not critically needed; I'm unsure if the data I'm transferring will compress well.
I'm currently banging together a barebones file-transfer script that fulfills my requirements, but if there's something already out there that does this I'd love to know because the script is taking a bit more time than I expected it would.
Here's what I've already looked into:
BitTorrent: perfect (client designed around network transience and poor quality), except for the lack of inherent file encryption and the fact that P2P is widely frowned upon, so I generally avoid using it as a rule
FTP/FTPS, HTTP/HTTPS: don't support verification :(
Metalink: the protocol is perfect (and it runs over HTTPS!), but I'm unaware of any clients that actually support piece hashes. I don't really think the whole idea of "...and...done! wait, your 100GB file's hash is wrong, we need to start over" makes any sense whatsoever.
Resilio Sync: closed-source, probably implements tracking; very reluctant to use :/