1

Want to build a automatic web scrapper/crawler. At first I thought to build it with ‘CasperJS and PhantomJS’ then I was trying to study a bit before starting but it seams ‘CasperJS’ is a bit old and may have not much support now a days, in 2023. Then I found, maybe ‘Selenium with Python’ maybe a good idea but someone suggested me about ‘Scrapy with Python’ and now I am really confused and need some suggestions if you please.

What I intend to do:

  1. Search through an eCommerce website for a specific product automatically after every 5 minutes or so, without a provided API.
  2. If I got my desired product in the search the fill a form and submit automatically in that shop.

** I would love to remain in the domain of JavaScript (and, or PHP).

Thanks in advance.

1 Answer 1

1

Selenium can be used with JavaScript as well, with many most popular languages to be more detailed. Just use the proper webdriver for it:

npm install selenium-webdriver

It should be more than sufficient for simple tasks like finding some HTML and submitting a form, although it's more of a testing library.

3
  • Thank you for your answer. Can you please let me chose the better one between 'selenium' and "puppeteer" ? Puppeteer is in so many packages!!
    – Rajib Paul
    Commented Sep 29, 2023 at 14:51
  • 1
    No problem. I've already given my recommendation. Pupeteer will likely be fine too, but it's a newer tool that I haven't really used yet. Selenium definitely has advantage of supporting many more browsers, but Pupeteer is reported to be faster.
    – Destroy666
    Commented Sep 29, 2023 at 15:37
  • But, Selenium, being older, will have a wider user base, more support, coding examples, etc. In fact, it will probably be easier for ChatGPT to code the OP's requiements for Sleeniumm than for Puppeteer. If oit's only running once evey 5 minutes, then speed is not important, so I am throwing my weight behind your recommendation of Selenium
    – Mawg
    Commented Feb 27 at 15:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.