I'm writing an application that navigates to a website, clicks a button on the website (which updates it) and then extracts information from the website. The application requires a headless browser, but I'm only using it for clicking a button. I'm currently using puppeteer because it seems highly renowned in the webscraping community, but I've noticed that it is somewhat slow and extremely large (difficult to push to github). Is there a lighter alternative to puppeteer that would accomplish my goal?

Thanks for your help!

  • Not a software recommendation, hence as comment: check what the action of that button is. Usually it's some POST or GET. Then just perform that, no clicking needed. Of course things look a bit different if the click just triggers some AJAX action… – Izzy Jun 21 '19 at 6:08
  • @Izzy the button accesses an external database to which I do not have access in order to update all of the information on the page. – CodeMonkey JD Jun 26 '19 at 20:51
  • I didn't mean what happens when you press it, but rather what action/call is triggered (like the FORM target). It must trigger either a post/get or at least some AJAX call. If you can find out which that is, you could perform that directly. – Izzy Jun 27 '19 at 7:12
  • @Izzy Sorry I'm so new at this... How do I do that? I inspected the page and tracked down the onclick function call and it is: onclick="$.OP.GG.summoner.renewBtn.start(this, '80749755');" Is there something else that I should be looking for? – CodeMonkey JD Jun 29 '19 at 1:05
  • Now you'd need to locate the JQuery function called. I'm not really familiar with that (JQuery), and it's OT here anyway, sorry. I was rather expecting it would be a "simple form", not a button with JS events. – Izzy Jun 29 '19 at 5:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.