3

Having a website, I'd like to extract content from it. So far I've been using Regex in Notepad++, but

  1. we all know we should not use Regex to parse HTML
  2. due to a redesign of the website, getting the Regex right is almost impossible

I have read about possible answers 1, 2, 3 and 4 but they all boil down to Python+Scrapy or JSoup.

  1. I'm looking for a tool for non-programmers which
  2. allows the user to select elements visually (not necessarily in rendered HTML, could also be in HTML source)
  3. constructs a list of "XPaths" (or similar, XML XPath understanding is available)
  4. can later re-run the extraction without GUI in a scheduled job

Other requirements

  1. Windows or Ubuntu
  2. commercial solution is ok

1 Answer 1

-1

tool for non-programmers

constructs a list of "XPaths"

Almost contradictive requirements, yet still soome prominent scraping software are good in both. See the list of them:

Their short descriptions.

5
  • Welcome on Software Recommendations. One of the rules here is that you should have experience with the tools you suggest. Another rules is that you should mention how the software fulfills the requirements. Do all of them provide a XPath-like language and provide a command line tool to re-run the extract? Dec 3, 2015 at 11:04
  • Lists of answers without a detailed description of the product are generally frowned upon. Might want to check out What is required for an answer to be high quality?
    – Huey
    Dec 3, 2015 at 11:05
  • 1
    @Thomas, I've updated according to your comment. Dec 3, 2015 at 12:23
  • CloudScrape is a web-service, not an application. "Everything in the cloud" Dec 4, 2015 at 8:38
  • @Thomas, true; yet it matches all the other criterias... Dec 4, 2015 at 9:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.