0

I need a tool that can pull out all the useful content from a multi-page webpage and converts it to one Markdown file.

I need to save the material that I learning from and document it in a specific markup format so I can add my new experiences and make connections between things.

I can manually write a program for different websites but I cannot find a general solution for converting multi-page to one Markdown file. I don't want to do it manually.

For example, I have converted all Java documentation into one Markdown file with a basic program and have added my experiences and create connections between with my computer science notes. These notes save me a lot of pain because search engines such as Google are not saving me every time.

|improve this question
  • usually Markdown is used to generate HTML, not the other way around. – NH. Dec 12 '18 at 16:58
  • No, you can convert HTML to markdown. Markdown is the lightweight version of the HTML. And can be converted tools such as pandoc. – my-lord Dec 12 '18 at 17:03
  • You seem to know the answer yourself -- pandoc is the tool for the job. For multi-page sites, you'll probably need to use some combo of tools: a web crawler plus pandoc plus some simple Bash scripting. – tonysdg Dec 12 '18 at 17:52
  • Yes. But the real problem starts when I try to fetch data from web pages has different structures. I already build a java program that I really need but every documentation web page needs 2-3 hours of writing structure rules and getting useful data from the web page. I need a general solution for parsing that data. – my-lord Dec 12 '18 at 18:22
  • 1
    my-lord, please either A) post an answer describing both the benefits and drawbacks of pandoc, or B) say in your question why pandoc is not a good solution for you. Thanks! :-) – Nicolas Raoul♦ Dec 13 '18 at 3:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.