I have a single 1.3gb html file that contains the entirety of the Encyclopedia Britannica. I want to convert it to markdown in a way that preserves subheadings, tables, and UTF-8 special characters.
Unfortunately, this file is so incredibly massive that all the usual conversion tools fail. Pandoc runs for a couple of hours before getting SIGKiLLED by my OS for using more than 60 GB of RAM. Html2text is somewhat less RAM hungry, but after running for more than 48 hours it still hadn't completed, and given the lack of a progress display I had to assume that it was hanging.
I was able to successfully use links to convert the file to plain text, but I would greatly prefer markdown. Subheads and especially tables are formatted terribly this way.
My machine has a Skylake processor clocked at 4.4 Ghz as well as 20 GB of RAM, so I have a decent amount of resources to work with. macOS-compatible software is preferred, but at this point, I wouldn't complain if I had to install and boot into Haiku. Anything to get this file converted.