0

I need a tool that can pull out all the useful content from a multi-page webpage and converts it to one Markdown file.

I need to save the material that I learning from and document it in a specific markup format so I can add my new experiences and make connections between things.

I can manually write a program for different websites but I cannot find a general solution for converting multi-page to one Markdown file. I don't want to do it manually.

For example, I have converted all Java documentation into one Markdown file with a basic program and have added my experiences and create connections between with my computer science notes. These notes save me a lot of pain because search engines such as Google are not saving me every time.

|improve this question
  • usually Markdown is used to generate HTML, not the other way around. – NH. Dec 12 '18 at 16:58
  • No, you can convert HTML to markdown. Markdown is the lightweight version of the HTML. And can be converted tools such as pandoc. – my-lord Dec 12 '18 at 17:03
  • You seem to know the answer yourself -- pandoc is the tool for the job. For multi-page sites, you'll probably need to use some combo of tools: a web crawler plus pandoc plus some simple Bash scripting. – tonysdg Dec 12 '18 at 17:52
  • Yes. But the real problem starts when I try to fetch data from web pages has different structures. I already build a java program that I really need but every documentation web page needs 2-3 hours of writing structure rules and getting useful data from the web page. I need a general solution for parsing that data. – my-lord Dec 12 '18 at 18:22
  • 1
    my-lord, please either A) post an answer describing both the benefits and drawbacks of pandoc, or B) say in your question why pandoc is not a good solution for you. Thanks! :-) – Nicolas Raoul♦ Dec 13 '18 at 3:32

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Browse other questions tagged or ask your own question.