python - 当查询结果超过一定数量时，Freebase + GoogleAPI 查询返回错误

Question

我正在尝试在 Freebase 上查询所有美国县及其地理位置(经度 + 纬度)。我注意到有时查询会起作用，但在其他尝试中它会返回以下内容:<“请求时出现 HttpError 503...返回“后端错误”>。

我尝试更改查询结果限制，但我发现查询失败的限制有所不同；有时在“limit”:2900时有效，有时在“limit”:1200时返回上述错误。

这是我到目前为止编写的代码:

    from itertools import islice    from apiclient import discovery    from apiclient import model    import json    from CREDENTIALS import FREEBASE_KEY    from pandas import DataFrame, Series    DEVELOPER_KEY = FREEBASE_KEY    model.JsonModel.alt_param = ""    freebase = discovery.build('freebase', 'v1', developerKey=DEVELOPER_KEY)    query_json = """    [{      "id": null,      "name": null,      "/location/us_county/fips_6_4_code": [],      "/location/location/geolocation": {        "latitude": null,        "longitude": null      },      "limit": 3050    }]""".replace("\n", " ")    query = json.loads(query_json)    response = json.loads(freebase.mqlread(query=json.dumps(query)).execute())    results = list()    for result in islice(response['result'], None):        results.append( {'id': result['id'],                         'name': result['name'],                         'latitude': float(result['/location/location/geolocation']['latitude']),                         'longitude': float(result['/location/location/geolocation']['longitude']),                         'fips': result['/location/us_county/fips_6_4_code'],                         } )    states = DataFrame(results)    plt.scatter(states["longitude"], states["latitude"])

这似乎不是配额问题，其他人在 Freebase 邮件列表上注意到了类似的问题:http://lists.freebase.com/pipermail/freebase-discuss/2011-December/007710.html但这是针对另一种类型的数据，因此他们的解决方案似乎不适用于我正在研究的内容。

<小时/>

[编辑]我使用游标来迭代数据，效果很好。这是我使用的最终代码:

    from itertools import islice    from apiclient import discovery    from apiclient import model    import json    from CREDENTIALS import FREEBASE_KEY    from pandas import DataFrame, Series    DEVELOPER_KEY = FREEBASE_KEY    model.JsonModel.alt_param = ""    freebase = discovery.build('freebase', 'v1', developerKey=DEVELOPER_KEY)    query = [{      "id": None,      "name": None,      "type": "/location/us_county",      "/location/location/geolocation": {        "latitude": None,        "longitude": None      }    }]    results = []    count = 0    def do_query(cursor=""):        response = json.loads(freebase.mqlread(query=json.dumps(query), cursor=cursor).execute())        for result in islice(response['result'], None):            results.append( {'id': result['id'],                             'name': result['name'],                             'latitude': result['/location/location/geolocation']['latitude'],                             'longitude': result['/location/location/geolocation']['longitude'],                             } )        return response.get("cursor")    cursor = do_query()    while(cursor):        cursor = do_query(cursor)        # Check how many iterations this loop has gone through.        #print count        count+=1    # Plug results into a pandas DataFrame and plot.    states = DataFrame(results)    plt.scatter(states["longitude"], states["latitude"])

Answer 1

这是一个相对简单的查询，但从角度来看，默认限制是 100，这比您要求的要低很多。我建议使用下限和光标来翻阅结果(并提交错误报告，因为它不应该返回通用的“后端错误”，而是某种 MQL 特定错误)