I need to build a competitors prices tracking tool for 100,000 products from around 15 competitors? What would be the best way to do it? Please recommend?


1 Answer 1


Read the web scraping chapter of the excellent (and free) eBook "Automate the boring stuff with Python".

That will tell you everything that you need to know.

  • This is insightful. But my problem is I don't have competitors URLs for each product. I just have product IDs that are consistent with competitors. The solution should search each individual product ID on the competitors' website, extract prices and keep track of prices. (For Example, I have QWER-1234 as a product id. The solution should search on Amazon, Alibaba,ebay etc. for QWER-1234 and should give me current prices). May 23, 2019 at 14:45
  • That's not going to happen without some human intervention May 24, 2019 at 6:45
  • About the only good news is that it you have to do it manually once for each site then you can automate it aft er that. Since it's only 15 sites, that's not too bad, as I would expect them top have the same layout on every page. May 24, 2019 at 6:46
  • Can you please elaborate a bit May 24, 2019 at 13:22
  • 1
    done I have coded it. Jun 2, 2019 at 3:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.