2

I am looking for a good library to help me scrape information off of a certain website. The path the scraping has to follow can be rather dynamic, meaning that if the initial scrape is on a page of type 1, the next one will be on a page A, otherwise on a page B, etc; i.e. depending on the scraped info, it should follow another logic.

I have looked at Selenium as a possible help here, but there might be better options for it, specific to scraping, rather than testing.

Improve this question

2 Answers 2

Reset to default
2

jsoup is a really nice library to parse and scrape HTML pages.

  • It has a simple and easy to use API
  • It is actively developed and open source
  • It has no dependencies to other libraries

I used it in a project and it worked great.

Improve this answer
0
0

Some open source web scraping libraries for Java are:

  1. Jsoup (Recommend)
  2. Webscrap4j
  3. Mechanize

But, Jsoup and Webscrap4j are very user friendly, and convenient to use with nice documentation.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.