python - 在使用 ctypes 加载时，在 C 中嵌入 Python 解释器会导致段错误

Question

我尝试将 Python 解释器嵌入到 C 中。为了对此进行测试，我创建了一个共享库并尝试使用 ctypes 在 Python 中加载这个。不幸的是这不工作，我想了解原因。

这是一个示例 c - 代码:

#ifdef __cplusplus
extern "C" {
#endif

#include <Python.h>


int run_py(void);
int run_py2(void);

int
run_py(void)
{
    printf("hello from run_py\n");
    return 42;
}

int
run_py2(void)
{
    printf("entering c-function: run_py()\n");
    Py_Initialize();
    PyRun_SimpleString("print('hello world')");
    return 0;
}

#ifdef __cplusplus
}
#endif

所以我用 gcc 将其编译成“mylib.so”，并使用 python3.7-config --cflags 和 --ldflags 进行链接等工作。

这是我用来加载它的 Python 代码..

import ctypes as c
import os
import sys


if __name__ == '__main__':
    print("running shared-lib integration test with python:\n{}".format(sys.version))

    path = os.path.dirname(os.path.realpath(__file__))
    dllfile = os.path.join(path, 'mylib.so')
    dll = c.CDLL(str(dllfile))

    print("loaded CDLL")
    dll.run_py.restype  = c.c_int
    dll.run_py2.restype  = c.c_int

    print("now calling dll.run_py()...")
    rv = dll.run_py()
    print("called dll.run_py: rv={}".format(rv))

    print("now calling dll.run_py2()...")
    rv2 = dll.run_py2()
    print("called dll.run_py2: rv={}".format(rv2))

所以这只是加载函数 run_py 和 run_py2并处决他们。这是输出...

running shared-lib integration test with python:
3.7.1 (default, Oct 22 2018, 10:41:28) 
[GCC 8.2.1 20180831]
loaded CDLL
now calling dll.run_py()...
hello from run_py
called dll.run_py: rv=42
now calling dll.run_py2()...
entering c-function: run_py()
Segmentation fault (core dumped)

所以基本上这会在调用 run_py2 时导致段错误。原因是 PyRun_SimpleString 的调用。但是，如果我将其编译为独立的 C 程序一切似乎都很好。我真的想了解为什么会发生这种情况......但目前我out ouf ideas 所以任何反馈在这里真的很感激。

BR jrsm

Answer 1

我稍微修改了你的代码。此外，我正在 Win 上进行测试(因为此时它对我来说更方便)，但我确信 Nix 中的情况是一样的。

dll.c:

#include <stdio.h>
#include <Python.h>

#define PRINT_MSG_0() printf("From C - [%s] (%d) - [%s]\n", __FILE__, __LINE__, __FUNCTION__)

#if defined(_WIN32)
#define DLL_EXPORT_API __declspec(dllexport)
#else
#define DLL_EXPORT_API
#endif

#if defined(__cplusplus)
extern "C" {
#endif

DLL_EXPORT_API int test0(void);
DLL_EXPORT_API int test1(void);

#if defined(__cplusplus)
}
#endif


int test0(void) {
    PRINT_MSG_0();
    return 42;
}


int test1(void) {
    PRINT_MSG_0();
    Py_Initialize();
    PRINT_MSG_0();
    PyRun_SimpleString("print(\"Hello world!!!\")");
    PRINT_MSG_0();
    return 0;
}

代码.py:

#!/usr/bin/env python3

import sys
from ctypes import CDLL,\
    c_int


DLL = "./dll.so"


def main():
    dll_dll = CDLL(DLL)
    test0_func = dll_dll.test0
    test0_func.argtypes = None
    test0_func.restype = c_int
    test1_func = dll_dll.test1
    test1_func.argtypes = None
    test1_func.restype = c_int

    print("Calling {:}...".format(test0_func.__name__))
    res = test0_func()
    print("{:} returned {:d}".format(test0_func.__name__, res))
    print("Calling {:}...".format(test1_func.__name__))
    res = test1_func()
    print("{:} returned {:d}".format(test1_func.__name__, res))


if __name__ == "__main__":
    print("Python {:s} on {:s}\n".format(sys.version, sys.platform))
    main()

输出:

(py35x64_test) e:\Work\Dev\StackOverflow\q053609932>"c:\Install\x86\Microsoft\Visual Studio Community\2015\vc\vcvarsall.bat" x64

(py35x64_test) e:\Work\Dev\StackOverflow\q053609932>dir /b
code.py
dll.c

(py35x64_test) e:\Work\Dev\StackOverflow\q053609932>cl /nologo /DDLL /MD /Ic:\Install\x64\Python\Python\3.5\include dll.c  /link /NOLOGO /DLL /OUT:dll.so /LIBPATH:c:\Install\x64\Python\Python\3.5\libs
dll.c
   Creating library dll.lib and object dll.exp

(py35x64_test) e:\Work\Dev\StackOverflow\q053609932>dir /b
code.py
dll.c
dll.exp
dll.lib
dll.obj
dll.so

(py35x64_test) e:\Work\Dev\StackOverflow\q053609932>"e:\Work\Dev\VEnvs\py35x64_test\Scripts\python.exe" code.py
Python 3.5.4 (v3.5.4:3f56838, Aug  8 2017, 02:17:05) [MSC v.1900 64 bit (AMD64)] on win32

Calling test0...
From C - [dll.c] (26) - [test0]
test0 returned 42
Calling test1...
From C - [dll.c] (32) - [test1]
From C - [dll.c] (34) - [test1]
Traceback (most recent call last):
  File "code.py", line 30, in <module>
    main()
  File "code.py", line 24, in main
    res = test1_func()
OSError: exception: access violation reading 0x0000000000000010

问题重现。首先，我认为是 [Python 3]: void Py_Initialize()称呼。但后来我想起了[Python 3]: class ctypes.PyDLL(name, mode=DEFAULT_MODE, handle=None) (强调 是我的)其中指出:

Instances of this class behave like CDLL instances, except that the Python GIL is not released during the function call, and after the function execution the Python error flag is checked. If the error flag is set, a Python exception is raised.

Thus, this is only useful to call Python C api functions directly.

在 code.py 中用 PyDLL 替换 CDLL，产生:

(py35x64_test) e:\Work\Dev\StackOverflow\q053609932>"e:\Work\Dev\VEnvs\py35x64_test\Scripts\python.exe" code.py
Python 3.5.4 (v3.5.4:3f56838, Aug  8 2017, 02:17:05) [MSC v.1900 64 bit (AMD64)] on win32

Calling test0...
From C - [dll.c] (26) - [test0]
test0 returned 42
Calling test1...
From C - [dll.c] (32) - [test1]
From C - [dll.c] (34) - [test1]
Hello world!!!
From C - [dll.c] (36) - [test1]
test1 returned 0