2

I am looking for a good library to help me scrape information off of a certain website. The path the scraping has to follow can be rather dynamic, meaning that if the initial scrape is on a page of type 1, the next one will be on a page A, otherwise on a page B, etc; i.e. depending on the scraped info, it should follow another logic.

I have looked at Selenium as a possible help here, but there might be better options for it, specific to scraping, rather than testing.

2

jsoup is a really nice library to parse and scrape HTML pages.

  • It has a simple and easy to use API
  • It is actively developed and open source
  • It has no dependencies to other libraries

I used it in a project and it worked great.

0

Some open source web scraping libraries for Java are:

  1. Jsoup (Recommend)
  2. Webscrap4j
  3. Mechanize

But, Jsoup and Webscrap4j are very user friendly, and convenient to use with nice documentation.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.