2

Google, and as far as I have been able to find, all other web search sites do not allow scraping in any form, including just finding the number of hits.

In fact I'm pretty sure that none even offer a web API to retrieve this info any more, at least not for free.

I realize this is a service that can be very easily abused, which is why it is not offered any more. But there are many other services which are offered various limits using API keys and throttling and quotas.

But I haven't been able to find a service for "web search hits" that's free, even with limits.

Requirements:

  • Free, as in no fee.
  • Registration, throttling, daily limits, any other kind of restriction is OK.
  • Service with the fewest restrictions is, however, preferred over those with more restrictions.
  • Service provided by or using Google is preferred, but other providers are OK.
  • Any format is OK, though urlencoded and JSON are expected.
  • Must be available live on-line of course!
  • Batching strongly preferred.
    (Ability to get the number of hits for two or many terms with one call.)
  • Services with more "advanced search options" are preferred over services with fewer such options.
    (Options such as restricting search to certain TLDs, supporting an "OR" operator, etc.)

One use case is doing first approximations of comparative popularity of synonyms or spelling variants. I'm sure there are plenty of other use cases.

Improve this question
3
  • Comparing popularity of synonyms and spelling variants can be viewed with the Google Ngram Viewer. It looks like some of you other bits can be done using [Google Trends!(google.com/trends) which also has an API. Can you give us a more exact user-story or use-case for what you are trying to accomplish?
    – Caleb
    Commented Feb 28, 2014 at 10:44
  • My personal use cases are usually for popularity contests, but for all kinds of languages such as Japanese kanji vs hiragana vs katakana, Chinese variants including but not limited to simplified vs traditional as used in mainland China, Taiwan, Hong Kong, and Singapore, etc. In the last six months I've used it also for Mongolian and Lao. But I didn't want to over restrict my question to the point it's not useful for anyone besides me and I know other people have wanted a program-accessible equivalent of "Google fight".
    – hippietrail
    Commented Feb 28, 2014 at 11:16
  • If Google Trends really hs a public API that might be an answer. The Wikipedia article seems to contradict itself. There are questions about it on Quora and Stack Overflow. There is an unofficial js lib to access it on GitHub but Google has been known to block operation with such libraries when they get too popular.
    – hippietrail
    Commented Feb 28, 2014 at 11:28

1 Answer 1

Reset to default
1

You could try Faroo http://www.faroo.com/hp/api/api.html .

It matches your requirements. However, when I used it (about half a year ago) their index was too small for us, but we were looking for niche results, for popularity contests this might be a very good source.

Improve this answer
3
  • It looks promising after the first few clicks. I'm going to play with this for sure.
    – hippietrail
    Commented Mar 11, 2014 at 13:10
  • One downside is there doesn't seem to be a way to play with it without registering. And you need to register it to an app so "playing with it" isn't really an app so not sure where to go from there. Note that in fact even the direct API link you've now included doesn't work from here.
    – hippietrail
    Commented Mar 12, 2014 at 9:55
  • That must have changed very recently then, I tested it this morning and now it is not working here either...
    – Rhand
    Commented Mar 12, 2014 at 10:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.