I'm currently using PHP DOM with CURL and Simple HTML DOM Parser for HTML scraping. I need to scrape large set of contents and it need to stored. I am using PHP as my main language and MySQL as storage DB however I am interested in options in any language and can integrate it into my workflow.
Are there any other HTML scraping packages I should look into? I have heard about web scraping libraries named 'Scrapy' and 'Beautiful Soup' which are based on Python.
Suggestions are highly appreciated.! :)