python - 如何将函数发送到远程 Pyro 对象

Question

我正在尝试使用 Pyro 设置一些代码来处理远程主机上的 python 代码函数并取回结果。启动名称服务器后，我将在远程主机上执行此代码(实际上仍在本地主机上):

import Pyro4

class Server(object):
    def evaluate(self, func, args):
        return func(*args)

def main():
    server = Server()
    Pyro4.Daemon.serveSimple(
            {
                server: "server"
            },
            ns=True)

if __name__ == '__main__':
    main()

在客户端，我有这段代码，这是我尝试设置的行为示例。

import Pyro4

remoteServer = Pyro4.Proxy('PYRONAME:server')

def square(x): 
    return x**2

print remoteServer.evaluate(square, 4)

但是，此代码会导致以下异常:

/usr/lib/python2.7/site-packages/Pyro4/core.py:155: UserWarning: HMAC_KEY not set,
protocol data may not be secure
warnings.warn("HMAC_KEY not set, protocol data may not be secure")
Traceback (most recent call last):
  File "/home/davide/Projects/rempy/example-api-pyro.py", line 7, in <module>
    print remoteServer.evaluate(square, 4)
  File "/usr/lib/python2.7/site-packages/Pyro4/core.py", line 149, in __call__
    return self.__send(self.__name, args, kwargs)
  File "/usr/lib/python2.7/site-packages/Pyro4/core.py", line 289, in _pyroInvoke
    raise data
AttributeError: 'module' object has no attribute 'square'

在我看来，函数对象被正确地 pickle 并被发送到远程主机上的服务器实例，但是命名空间中存在一些问题。

我该如何解决这个问题？

谢谢

Answer 1

我想我知道你的问题:

定义函数的模块被调用

'__main__'

它存在于所有运行的 python 版本中。

pickle 不传源码，传一个引用

__main__.square

所以你有两种可能性:

源代码尽可能短，使主模块尽可能短，例如:

# main.py

def square(x): 
   return x**2

import Pyro4
def main():
    remoteServer = Pyro4.Proxy('PYRONAME:server')


    print remoteServer.evaluate(square, 4)

和:

# __main__.py
import main
main.main()

然后服务器可以从文件中导入完全相同的模块。

或者用我的代码创建一个模块:

class ThisShallNeverBeCalledError(Exception):
    pass

class _R(object):
    def __init__(self, f, *args):
        self.ret = (f, args)
    def __reduce__(self):
        return self.ret
    def __call__(self, *args):
        raise ThisShallNeverBeCalledError()

    @classmethod
    def fromReduce(cls, value):
        ret = cls(None)
        ret.ret = value
        return ret


def dump_and_load(obj):
    '''pickle and unpickle the object once'''
    s = pickle.dumps(obj)
    return pickle.loads(s)

# this string creates an object of an anonymous type that can
# be called to create an R object or that can be reduced by pickle
# and creates another anonymous type when unpickled
# you may not inherit from this MetaR object because it is not a class
PICKLABLE_R_STRING= "type('MetaR', (object,), " \
                    "       {'__call__' : lambda self, f, *args: "\
                    "          type('PICKLABLE_R', "\
                    "               (object,), "\
                    "               {'__reduce__' : lambda self: (f, args), "\
                    "                '__module__' : 'pickleHelp_', "\
                    "                '__name__'   : 'PICKLABLE_R', "\
                    "                '__call__'   : lambda self: None})(), "\
                    "        '__reduce__' : lambda self: "\
                    "           self(eval, meta_string, "\
                    "                {'meta_string' : meta_string}).__reduce__(), "\
                    "        '__module__' : 'pickleHelp_', "\
                    "        '__name__' : 'R'})()".replace('  ', '')
PICKLABLE_R = _R(eval, PICKLABLE_R_STRING, \
                {'meta_string' : PICKLABLE_R_STRING})
R = dump_and_load(PICKLABLE_R)
del PICKLABLE_R, PICKLABLE_R_STRING

PICKLABLE___builtins__ = R(vars, R(__import__, '__builtin__'))
PICKLABLE_FunctionType = R(type, R(eval, 'lambda:None'))

##R.__module__ = __name__
##R.__name__ = 'PICKLABLE_R'


def packCode(code, globals = {}, add_builtins = True, use_same_globals = False, \
             check_syntax = True, return_value_variable_name = 'obj',
             __name__ = __name__ + '.packCode()'):
    '''return an object that executes code in globals when unpickled
use_same_globals
    if use_same_globals is True all codes sent through
    one pickle connection share the same globals
    by default the dont
return_value_variable_name
    if a variable with the name in return_value_variable_name exists
    in globals after the code execution
    it is returned as result of the pickling operation
    if not None is returned
__name__

'''
    if check_syntax:
        compile(code, '', 'exec')
    # copying locals is important
    # locals is transferred through pickle for all code identical
    # copying it prevents different code from beeing executed in same globals
    if not use_same_globals:
        globals = globals.copy()
    if add_builtins:
        globals['__builtins__'] = PICKLABLE___builtins__
    globals.setdefault('obj', None)
    # get the compilation code
    # do not marshal or unmarshal code objects because the platforms may vary
    code = R(compile, code, __name__, 'exec')
    # the final object that can reduce, dump and load itself
    obj = R(R(getattr, tuple, '__getitem__'), (
            R(R(PICKLABLE_FunctionType, code, globals)),
            R(R(getattr, type(globals), 'get'), globals, \
              returnValueVariableName, None)
            ), -1)
    return obj

然后发送给对方:

packCode('''
def square(...):
    ...
''', return_value_variable_name = 'square')

并且该函数将在另一端出来，不需要模块代码将此 python 函数传输到另一端。

如果有什么不对请告诉我。