Retrieve Google Scholar results

Question

I am looking for a library that can send a query to Google Scholar and retrieve all the results in some exploitable format (JSON, XML, etc.). A CLI program is OK too provided there is no interaction needed, i.e. that I can easily call it from another program. I am OK if the library only takes an author's name as the query. If possible, free and works with Windows 7 SP1 x64 Ultimate.

I know that there are general-purpose libraries like FancyURLopener or BeautifulSoup that would make the job easier, but I want an end-to-end solution just like Publish or Perish (which isn't CLI) as Google Scholar's policies change from time to time, and I don't want to have to maintain my own library code. Google Scholar provides no API, hence the need to scrape.

Answer 1

Have a look at this python command line tool:

https://github.com/ckreibich/scholar.py

Example usage if you want a bibtex file:

$ scholar.py -c 1 --author "albert einstein" --phrase "quantum theory" --citation bt
@article{einstein1917quantum,
  title={On the quantum theory of radiation},
  author={Einstein, Albert},
  journal={Phys. Z},
  volume={18},
  pages={121--128},
  year={1917}
}

It can also count citations:

$ scholar.py --txt-globals --author "albert einstein" | grep '\[G\]' | grep Results
[G]    Results 4190