Having a website, I'd like to extract content from it. So far I've been using Regex in Notepad++, but

  1. we all know we should not use Regex to parse HTML
  2. due to a redesign of the website, getting the Regex right is almost impossible

I have read about possible answers 1, 2, 3 and 4 but they all boil down to Python+Scrapy or JSoup.

  1. I'm looking for a tool for non-programmers which
  2. allows the user to select elements visually (not necessarily in rendered HTML, could also be in HTML source)
  3. constructs a list of "XPaths" (or similar, XML XPath understanding is available)
  4. can later re-run the extraction without GUI in a scheduled job

Other requirements

  1. Windows or Ubuntu
  2. commercial solution is ok

tool for non-programmers

constructs a list of "XPaths"

Almost contradictive requirements, yet still soome prominent scraping software are good in both. See the list of them:

Their short descriptions.

  • Welcome on Software Recommendations. One of the rules here is that you should have experience with the tools you suggest. Another rules is that you should mention how the software fulfills the requirements. Do all of them provide a XPath-like language and provide a command line tool to re-run the extract? – Thomas Weller Dec 3 '15 at 11:04
  • Lists of answers without a detailed description of the product are generally frowned upon. Might want to check out What is required for an answer to be high quality? – Huey Dec 3 '15 at 11:05
  • 1
    @Thomas, I've updated according to your comment. – Igor Savinkin Dec 3 '15 at 12:23
  • CloudScrape is a web-service, not an application. "Everything in the cloud" – Thomas Weller Dec 4 '15 at 8:38
  • @Thomas, true; yet it matches all the other criterias... – Igor Savinkin Dec 4 '15 at 9:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.