I am looking for a good library to help me scrape information off of a certain website. The path the scraping has to follow can be rather dynamic, meaning that if the initial scrape is on a page of type 1, the next one will be on a page A, otherwise on a page B, etc; i.e. depending on the scraped info, it should follow another logic.

I have looked at Selenium as a possible help here, but there might be better options for it, specific to scraping, rather than testing.