I am searching for a software that I can use to get information from websites.
I've seen some 'scraping' and 'RPA' tools that you can make rules based on CSS or XPaths.
The thing is that there are many different websites with different structures, so I need flexibility to build more complex rules. for example, maybe see the order of the tags or the words that are inside the text.
Another desired feature would be to be able to scrape not only the given page, but also the links inside the given page.
If possible I don't want to have to build the logic using a library like 'BeautifulSoup' because it would make harder to make it configurable for the client.
Is there a tool that would give me these features?