python - HttpError 401 尝试通过 CloudStorage Pipeline 访问 BigQuery 网址

Question

这个问题本质上是我上一个问题的后续问题，您可以找到 here 。处理完遇到的编码错误后，我遇到了新的 HttpError 401 问题。我的开发应用程序服务器日志的跟踪如下:

ERROR    2012-12-20 03:10:46,312 pipeline.py:2237] Generator main.CloudStorageToBigQuery(*([u'/gs/jibdancsvtest/Datastore Mapper main.streamdata-1582389642294FE5DBBCE-output'],), **{})#2d3dba4f4a5211e2bf0ac3cee1488e47 raised exception. HttpError: <HttpError 401 when requesting https://www.googleapis.com/bigquery/v2/projects/1093XXXXXXXXX/jobs?alt=json returned "Invalid Credentials">
Traceback (most recent call last):
  File "C:\Users\Tank\Documents\Aptana Studio 3 Workspace\jibdantest-bq\mapreduce\lib\pipeline\pipeline.py", line 2030, in evaluate
    self, pipeline_key, root_pipeline_key, caller_output)
  File "C:\Users\Tank\Documents\Aptana Studio 3 Workspace\jibdantest-bq\mapreduce\lib\pipeline\pipeline.py", line 1064, in _run_internal
    return self.run(*self.args, **self.kwargs)
  File "C:\Users\Tank\Documents\Aptana Studio 3 Workspace\jibdantest-bq\main.py", line 77, in run
    result.execute()
  File "C:\Users\Tank\Documents\Aptana Studio 3 Workspace\jibdantest-bq\oauth2client\util.py", line 120, in positional_wrapper
    return wrapped(*args, **kwargs)
  File "C:\Users\Tank\Documents\Aptana Studio 3 Workspace\jibdantest-bq\apiclient\http.py", line 678, in execute
    raise HttpError(resp, content, uri=self.uri)
HttpError: <HttpError 401 when requesting https://www.googleapis.com/bigquery/v2/projects/1093XXXXXXXXX/jobs?alt=json returned "Invalid Credentials">

我还想我应该发布来自 url 的 json 响应。这是 json:

{
 "error": {
  "errors": [
   {
    "domain": "global",
    "reason": "required",
    "message": "Login Required",
    "locationType": "header",
    "location": "Authorization"
   }
  ],
  "code": 401,
  "message": "Login Required"
 }
}

如果您点击 stackoverflow 上我上一个问题的链接，您将在我的 main.py 文件中找到运行所有内容的代码。但如果我需要在这里重新发布它，我可以这样做。

所以，我认为这是一个 OAuth 问题，但我相信我已经正确设置了所有这些。对于我可以做/调查的其他事情的任何建议都会很棒。

非常感谢。

Answer 1

当您在本地开发服务器上运行时，您需要以不同的方式设置您的凭据:

首先，您需要设置本地凭据存储。从这里使用 get_refresh_token.py:https://codereview.appspot.com/5362041/diff/1/samples/oauth2cmdline/get_refresh_token.py

在 Google API 控制台上为 Web 应用程序设置客户端 ID，并将参数传递给上面的脚本(将启动到浏览器中进行 OAuth2 身份验证)。这将创建一个包含凭据的本地文件，然后您可以将其传递给 API 客户端以获取经过身份验证的请求:

from oauth2client.client import Storage, Credentials
from oauth2client.appengine import AppAssertionCredentials

class FileStorage(Storage):
    def __init__(self, filepath):
        self._filepath = filepath

    def locked_get(self):
        with open(self._filepath, 'r') as f:
            json = f.read()
        credentials = Credentials.new_from_json(json)
        return credentials

if not os.environ.get('SERVER_SOFTWARE', '').startswith('Development'):
    credentials = AppAssertionCredentials(scope='https://www.googleapis.com/auth/bigquery')
else:
    filepath = #Path to credentials file here
    storage = FileStorage(filepath)
    credentials = storage.get()

希望这有帮助!