I'm working on a chatbot, and I need a Python 3 module (or library) for searching the web. I tried pygoogle, but it gave me just URLs; I tried xgoogle, but it's a Python 2 module that 2to3 didn't convert correctly; I tried GoogleScraper, but it gave me a 403. I can scrape Google myself with BeautifulSoup or lxml, but I thought there was probably already a module out there that can do it for me.

  • Most importantly, I need a URL, its title, and its description.
  • Speed is pretty important. This is in chat, where a bot is not worth using if opening a new tab is just as fast.

It isn't important that I search Google. I tried pws to search Bing, but the problem was that it threw an error whenever there weren't any related searches.

  • You can do it using the requests (or urllib*) modules, and possibly beautifulsoup if need be. – boardrider Oct 5 '16 at 17:05
  • @boardrider: Yes, I know, but I thought there might be another module already available that has everything optimized. There might be other websites besides Google that have an API to make it faster. I'm just trying to find a fast option, and I thought I was more likely to get speed with a module designed for it then trying to scrape Google. – zondo Oct 5 '16 at 22:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.