Python ctypes cdll.LoadLibrary，实例化一个对象，执行其方法，私有(private)变量地址被截断

Question

我用c写了一个dll库，用vs2017 64位编译，用python3.6 64位加载试试。但是，对象的成员变量地址被截断为 32 位。

这是我的 sim.c 文件，它被编译为 sim.dll:

class Detector {
public:
    Detector();
    void process(int* pin, int* pout, int n);

private:
    int member_var;
};

Detector::Detector()
{
    memset(&member_var, 0, sizeof(member_var));
    myfile.open("addr_debug.txt");
    myfile << "member_var init address: " << &member_var << endl;
}
void Detector::process(int* pin, int* pout, int n);
{
    myfile << "member_var process address: " << &member_var << endl;
    myfile.close();
}

#define DllExport   __declspec( dllexport )  

extern "C" {
    DllExport Detector* Detector_new() { return new Detector(); }
    DllExport void Detector_process(Detector* det, int* pin, int* pout, int n)
    {
        det->process(pin, pout, n);
    }
}

这是我的 python 脚本:

from ctypes import cdll
lib = cdll.LoadLibrary(r'sim.dll')

class Detector(object):
    def __init__(self):
        self.obj = lib.Detector_new()

    def process(self,pin, pout, n):
        lib.Detector_process(self.obj,pin, pout, n)

detector = Detector()

n = 1024
a = np.arange(n, dtype=np.uint32)
b = np.zeros(n, dtype=np.int32)

aptr = a.ctypes.data_as(ctypes.POINTER(ctypes.c_int))
bptr = b.ctypes.data_as(ctypes.POINTER(ctypes.c_int))

detector.process(aptr, bptr, n)

这里是 addr_debug.txt 中 member_var 的地址:

member_var init address:    0000025259E123C4
member_var process address: 0000000059E123C4

因此访问它会触发内存访问错误:

OSError: exception: access violation reading 0000000059E123C4

---

我试图理解这个问题的一些尝试:

• 将 member_var 定义为公共(public)而不是私有(private)，无济于事，地址仍然被截断。
• 将member_var定义为全局变量，地址就可以了。所以我猜 member_var 地址截断发生在将对象返回给 python 或将对象传递回 dll 时。

Answer 1

始终(正确地)为 C 中定义的函数指定 argtypes 和 restype，否则( C89 风格)它们将默认为 int(通常是 32bit)，生成 !!!未定义的行为 !!!。在 64 位 上，地址(大于 2 GiB)将被截断(这正是您遇到的情况)。检查[SO]: C function called from Python via ctypes returns incorrect value (@CristiFati's answer)了解更多详情。

此外，遇到问题时，不要忘记 [Python.Docs]: ctypes - A foreign function library for Python .

下面是您的代码的改编版本。

检测器.cpp:

#include <stdio.h>
#include <memory.h>

#include <fstream>

#define SIM_EXPORT __declspec(dllexport)


#define C_TAG "From C"
#define PRINT_MSG_3SPI(ARG0, ARG1, ARG2) printf("%s - [%s] (%d) - [%s]:  %s: 0x%0p(%d)\n", C_TAG, __FILE__, __LINE__, __FUNCTION__, ARG0, ARG1, ARG2)


using std::endl;

std::ofstream outFile;


class Detector {
    public:
        Detector();
        void process(int *pIn, int *pOut, int n);

    private:
        int m_var;
};


Detector::Detector()
: m_var(25) {
    outFile.open("addr_debug.txt");
    outFile << "m_var init address: " << &m_var << endl;
    PRINT_MSG_3SPI("&m_var(m_var)", &m_var, m_var);
}

void Detector::process(int *pIn, int *pOut, int n)
{
    outFile << "m_var process address: " << &m_var << endl;
    outFile.close();
    PRINT_MSG_3SPI("&m_var(m_var)", &m_var, m_var);
}


#if defined(__cplusplus)
extern "C" {
#endif

SIM_EXPORT Detector* DetectorNew() { return new Detector(); }

SIM_EXPORT void DetectorProcess(Detector *pDet, int *pIn, int *pOut, int n)
{
    pDet->process(pIn, pOut, n);
}

SIM_EXPORT void DetectorDelete(Detector *pDet) { delete pDet; }

#if defined(__cplusplus)
}
#endif

code00.py:

#!/usr/bin/env python

import ctypes as ct
import sys

import numpy as np


IntPtr = ct.POINTER(ct.c_int)

sim_dll = ct.CDLL("./sim.dll")

detector_new_func = sim_dll.DetectorNew
detector_new_func.argtypes = ()
detector_new_func.restype = ct.c_void_p

detector_process_func = sim_dll.DetectorProcess
detector_process_func.argtypes = (ct.c_void_p, IntPtr, IntPtr, ct.c_int)
detector_process_func.restype = None

detector_delete_func = sim_dll.DetectorDelete
detector_delete_func.argtypes = (ct.c_void_p,)
detector_delete_func.restype = None


class Detector():
    def __init__(self):
        self.obj = detector_new_func()

    def process(self, pin, pout, n):
        detector_process_func(self.obj, pin, pout, n)

    def __del__(self):
        detector_delete_func(self.obj)


def main(*argv):
    detector = Detector()

    n = 1024
    a = np.arange(n, dtype=np.uint32)
    b = np.zeros(n, dtype=np.int32)

    aptr = a.ctypes.data_as(IntPtr)
    bptr = b.ctypes.data_as(IntPtr)

    detector.process(aptr, bptr, n)


if __name__ == "__main__":
    print("Python {:s} {:03d}bit on {:s}\n".format(" ".join(elem.strip() for elem in sys.version.split("\n")),
                                                   64 if sys.maxsize > 0x100000000 else 32, sys.platform))
    rc = main(*sys.argv[1:])
    print("\nDone.")
    sys.exit(rc)

注意事项:

• 正如我在开头所述，问题是 argtypes 和 restype 未指定(例如对于 DetectorNew:评论 detector_new_func.restype = ct.c_void_p，又会遇到这个问题)

• 问题中的代码缺少部分(#includes, imports, ...)，还有一些语法错误，所以它没有't 编译，因此不遵循 [SO]: How to create a Minimal, Complete, and Verifiable example (mcve)准则。请在询问时确保有MCVE

• 您分配的对象 (new Detector()) 也必须被释放(否则会产生内存泄漏)，所以我添加了从 (Python) Detector 的析构函数中调用的函数(DetectorDelete - 这样做)

• 其他(非关键)更改(标识符重命名、一些重构、打印到 stdout，...)

输出:

(py35x64_tes1) e:\Work\Dev\StackOverflow\q052268294>"c:\Install\x86\Microsoft\Visual Studio Community\2015\vc\vcvarsall.bat" x64

(py35x64_test) e:\Work\Dev\StackOverflow\q052268294>dir /b
code00.py
detector.cpp

(py35x64_test) e:\Work\Dev\StackOverflow\q052268294>cl /nologo /DDLL /EHsc detector.cpp  /link /DLL /OUT:sim.dll
detector.cpp
   Creating library sim.lib and object sim.exp

(py35x64_test) e:\Work\Dev\StackOverflow\q052268294>dir /b
code00.py
detector.cpp
detector.obj
sim.dll
sim.exp
sim.lib

(py35x64_test) e:\Work\Dev\StackOverflow\q052268294>"e:\Work\Dev\VEnvs\py35x64_test\Scripts\python.exe" ./code.py
Python 3.5.4 (v3.5.4:3f56838, Aug  8 2017, 02:17:05) [MSC v.1900 64 bit (AMD64)] 064bit on win32

From C - [detector.cpp] (28) - [Detector::Detector]:  &m_var: 0x0000020CE366E270
From C - [detector.cpp] (34) - [Detector::process]:  &m_var: 0x0000020CE366E270

Done.