python - 谷歌云存储出现错误 pwd.getpwuid

Question

我 pip install --upgrade google-cloud-storage -t libs 到我的应用引擎应用。

在 appengine_config.py 中，我添加了:

vendor.add('libs')
vendor.add(os.path.join(os.path.dirname(os.path.realpath(__file__)), 'libs'))

它适用于在线 App Engine，但不适用于本地 App Engine 沙箱。

ERROR    2016-10-03 00:22:01,311 wsgi.py:263] 
Traceback (most recent call last):
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/runtime/wsgi.py", line 240, in Handle
    handler = _config_handle.add_wsgi_middleware(self._LoadHandler())
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/runtime/wsgi.py", line 299, in _LoadHandler
    handler, path, err = LoadObject(self._handler)
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/runtime/wsgi.py", line 85, in LoadObject
    obj = __import__(path[0])
  File "/Users/charlesng/Documents/Codes/python/web/myapp/src/main.py", line 19, in <module>
    from handlers import page_handlers, user_handlers, repo_handlers, doc_handlers
  File "/Users/charlesng/Documents/Codes/python/web/myapp/src/handlers/repo_handlers.py", line 28, in <module>
    from google.cloud import storage
  File "/Users/charlesng/Documents/Codes/python/web/myapp/src/libs/google/cloud/storage/__init__.py", line 42, in <module>
    from google.cloud.storage.batch import Batch
  File "/Users/charlesng/Documents/Codes/python/web/myapp/src/libs/google/cloud/storage/batch.py", line 29, in <module>
    from google.cloud.exceptions import make_exception
  File "/Users/charlesng/Documents/Codes/python/web/myapp/src/libs/google/cloud/exceptions.py", line 24, in <module>
    from google.cloud._helpers import _to_bytes
  File "/Users/charlesng/Documents/Codes/python/web/myapp/src/libs/google/cloud/_helpers.py", line 62, in <module>
    _USER_ROOT = os.path.expanduser('~')
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/posixpath.py", line 262, in expanduser
    userhome = pwd.getpwuid(os.getuid()).pw_dir
KeyError: 'getpwuid(): uid not found: 429123195'

文件夹结构:

myapp/
   /src/main.py
   /src/libs
   /env/(virtualenv files)
   /env/lib

通常，如果您 p​​ip 一个库，这些文件位于 lib/中，但对于 App Engine 第三方库，我们必须 pip -t libs ，以便它们位于 libs 而不是 lib 中。

当我使用 python2 或 python3 时，从 google.cloud 导入存储，它们很好，但不运行 appengine 沙箱，因为它调用 src/libs/google/cloud 而不是 env/lib/google/cloud。

我应该如何解决这个问题？任何建议或指导将不胜感激。

Answer 1

不确定这是否完全回答了您的问题，但是，我假设您正在使用 Google App Engine SDK在您的计算机上本地启动该应用程序。这使用了经过修补的 os 模块进行沙箱:

/tmp/google_appengine$ cat google/appengine/tools/devappserver2/python/sandbox.py
...
...
...

  def apply_policy(self, module_dict):
    """Apply this policy to the provided module dict.

    In order, one of the following will apply:
    - Symbols in overrides are set to the override value.
    - Symbols in deletes are removed.
    - Whitelisted symbols and symbols with a constant type are unchanged.
    - If a default stub is set, all other symbols are replaced by it.
    - If default_pass_through is True, all other symbols are unchanged.
    - If default_pass_through is False, all other symbols are removed.

    Args:
      module_dict: The module dict to be filtered.
    """
    for symbol in module_dict.keys():
      if symbol in self.overrides:
        module_dict[symbol] = self.overrides[symbol]
      elif symbol in self.deletes:
        del module_dict[symbol]
      elif not (symbol in self.whitelist or
                isinstance(module_dict[symbol], self.constant_types) or
                (symbol.startswith('__') and symbol.endswith('__'))):
        if self.default_stub:
          module_dict[symbol] = self.default_stub
        elif not self.default_pass_through:
          del module_dict[symbol]

_MODULE_OVERRIDE_POLICIES = {
    'os': ModuleOverridePolicy(
        default_stub=stubs.os_error_not_implemented,
        whitelist=['altsep', 'curdir', 'defpath', 'devnull', 'environ', 'error',
                   'fstat', 'getcwd', 'getcwdu', 'getenv', '_get_exports_list',
                   'name', 'open', 'pardir', 'path', 'pathsep', 'sep',
                   'stat_float_times', 'stat_result', 'strerror', 'sys',
                   'walk'],
        overrides={
            'access': stubs.fake_access,
            'listdir': stubs.RestrictedPathFunction(os.listdir),
            # Alias lstat() to stat() to match the behavior in production.
            'lstat': stubs.RestrictedPathFunction(os.stat),
            'open': stubs.fake_open,
            'stat': stubs.RestrictedPathFunction(os.stat),
            'uname': stubs.fake_uname,
            'getpid': stubs.return_minus_one,
            'getppid': stubs.return_minus_one,
            'getpgrp': stubs.return_minus_one,
            'getgid': stubs.return_minus_one,
            'getegid': stubs.return_minus_one,
            'geteuid': stubs.return_minus_one,
            'getuid': stubs.return_minus_one,
            'urandom': stubs.fake_urandom,
            'system': stubs.return_minus_one,
            },
        deletes=['execv', 'execve']),
    'signal': ModuleOverridePolicy(overrides={'__doc__': None}),
    'locale': ModuleOverridePolicy(
        overrides={'setlocale': stubs.fake_set_locale},
        default_pass_through=True),
    'distutils.util': ModuleOverridePolicy(
        overrides={'get_platform': stubs.fake_get_platform},
        default_pass_through=True),
    # TODO: Stub out imp.find_module and friends.
    }

如您所见，os.getuid() 将始终返回 -1:

/tmp/google_appengine$ grep -A1 return_minus_one google/appengine/tools/devappserver2/python/stubs.py
def return_minus_one(*unused_args, **unused_kwargs):
  return -1

并且 -1 被转换为 429123195 因为，在 python 源代码 (Modules/pwdmodule.c) 中...

static PyObject *
pwd_getpwuid(PyObject *self, PyObject *args)
{
    uid_t uid;
    struct passwd *p;
    if (!PyArg_ParseTuple(args, "O&:getpwuid", _Py_Uid_Converter, &uid)) {
        if (PyErr_ExceptionMatches(PyExc_OverflowError))
            PyErr_Format(PyExc_KeyError,
                         "getpwuid(): uid not found");
        return NULL;
    }
    if ((p = getpwuid(uid)) == NULL) {
        if (uid < 0)
            PyErr_Format(PyExc_KeyError,
                         "getpwuid(): uid not found: %ld", (long)uid);
        else
            PyErr_Format(PyExc_KeyError,
                         "getpwuid(): uid not found: %lu", (unsigned long)uid);
        return NULL;
    }
    return mkpwent(p);
}

... uid_t 类型转换为 long

截至今天(2016 年 10 月 3 日)，google app engine knowledge base文章说(在 Python 部分下):

The system does not allow you to invoke subprocesses, as a result some os module methods are disabled