python - 使用 Google 自定义搜索 API 时为 "daily limit exceeded"

Question

我想为大约 2000 个查询抓取 200 个结果，但它给我一个“超出每日限制”的错误。

我想确认我们每天可以抓取多少结果。有什么解决方案可以解决这个问题吗？或者唯一的方法是每天抓取一小部分查询...？

我抓取google的代码如下:

def crawl(query_list):
    http = httplib2.Http()

    # Construct the service object for the interacting with the CustomSearch API.
    service = discovery.build('customsearch', 'v1',  developerKey='my api key', http=http)

    res_list = []
    for query in query_list:
        json_res = service.cse().list(q = query, cx = 'my search engine id', num = 200,).execute()
        res_list.append(json_res)

谢谢!

Answer 1

根据这个post :

The first 100 queries per day are free. Any more, then you have to pay $5 per 1000 queries, for up to 10,000 queries per day, just enable billing to do so. Each query returns a maximum of 10 results, so you can retrieve 1000 URL’s from your search per day for free.