I need to build a competitors prices tracking tool for 100,000 products from around 15 competitors? What would be the best way to do it? Please recommend?
1 Answer
Read the web scraping chapter of the excellent (and free) eBook "Automate the boring stuff with Python".
That will tell you everything that you need to know.
-
This is insightful. But my problem is I don't have competitors URLs for each product. I just have product IDs that are consistent with competitors. The solution should search each individual product ID on the competitors' website, extract prices and keep track of prices. (For Example, I have QWER-1234 as a product id. The solution should search on Amazon, Alibaba,ebay etc. for QWER-1234 and should give me current prices). Commented May 23, 2019 at 14:45
-
-
About the only good news is that it you have to do it manually once for each site then you can automate it aft er that. Since it's only 15 sites, that's not too bad, as I would expect them top have the same layout on every page.– MawgCommented May 24, 2019 at 6:46
-
-
1