java - C中的简单数据序列化

Question

我目前正在重新设计一个应用程序，偶然发现了一些数据序列化的问题。

假设我有一个大小为 mxn 的数组

双**数据;

我想序列化成一个

char *dataSerialized

使用简单的分隔符(一个用于行，一个用于元素)。

反序列化非常简单，计算分隔符并为要存储的数据分配大小。但是，序列化函数呢，比如说

serialize_matrix(double **data, int m, int n, char **dataSerialized);

确定 char 数组所需大小并为其分配适当内存的最佳策略是什么？

也许在字符串中使用一些固定宽度的 double 指数表示？是否可以将 double 的所有字节转换为 char 并具有 sizeof(double) 对齐的 char 数组？我如何保持数字的准确性？

注意:

我需要字符数组中的数据，而不是二进制文件中的数据。

序列化数据将使用 ZeroMQ 在 C 服务器和 Java 客户端之间通过网络发送。是否有可能，给定数组维度和 sizeof(double)，它总是可以在这两者之间准确重建？

Answer 1

Java 对读取原始字节和转换成你想要的任何东西都有很好的支持。您可以决定一种简单的有线格式，然后在 C 中对此进行序列化，并在 Java 中反序列化。

这是一个极其简单的格式示例，其中包含用于反序列化和序列化的代码。

我写了一个稍微大一点的测试程序，如果你愿意，我可以把它转储到某个地方；它在 C 中创建一个随机数据数组，序列化，将 base64 编码的序列化字符串写入标准输出。然后，更小的 java 程序读取、解码和反序列化它。

要序列化的 C 代码:

/* 
I'm using this format:
32 bit signed int                   32 bit signed int                   See below
[number of elements in outer array] [number of elements in inner array] [elements]

[elements] is buildt like
[element(0,0)][element(0,1)]...[element(0,y)][element(1,0)]...

each element is sendt like a 64 bit iee754 "double". If your C compiler/architecture is doing something different with its "double"'s, look forward to hours of fun :)

I'm using a couple non-standard functions for byte-swapping here, originally from a BSD, but present in glibc>=2.9.
*/

/* Calculate the bytes required to store a message of x*y doubles */
size_t calculate_size(size_t x, size_t y)
{
    /* The two dimensions in the array  - each in 32 bits - (2 * 4)*/
    size_t sz = 8;  
    /* a 64 bit IEE754 is by definition 8 bytes long :) */
    sz += ((x * y) * 8);    
    /* and a NUL */
    sz++;
    return sz;
}

/* Helpers */
static char* write_int32(int32_t, char*);
static char* write_double(double, char*);
/* Actual conversion. That wasn't so hard, was it? */
void convert_data(double** src, size_t x, size_t y, char* dst)
{

    dst = write_int32((int32_t) x, dst);    
    dst = write_int32((int32_t) y, dst);    

    for(int i = 0; i < x; i++) {
        for(int j = 0; j < y; j++) {
            dst = write_double(src[i][j], dst);
        }
    }
    *dst = '\0';
}


static char* write_int32(int32_t num,  char* c)
{
    char* byte; 
    int i = sizeof(int32_t); 
    /* Convert to network byte order */
    num = htobe32(num);
    byte = (char*) (&num);
    while(i--) {
        *c++ = *byte++;
    }
    return c;
}

static char* write_double(double d, char* c)
{
    /* Here I'm assuming your C programs use IEE754 'double' precision natively.
    If you don't, you should be able to convert into this format. A helper library most likely already exists for your platform.
    Note that IEE754 endianess isn't defined, but in practice, normal platforms use the same byte order as they do for integers.
*/
    char* byte; 
    int i = sizeof(uint64_t);
    uint64_t num = *((uint64_t*)&d);
    /* convert to network byte order */
    num = htobe64(num);
    byte = (char*) (&num);
    while(i--) {
        *c++ = *byte++; 
    }
    return c;
}

要反序列化的 Java 代码:

/* The raw char array from c is now read into the byte[] `bytes` in java */
DataInputStream stream = new DataInputStream(new ByteArrayInputStream(bytes));

int dim_x; int dim_y;
double[][] data;

try {   
    dim_x = stream.readInt();
    dim_y = stream.readInt();
    data = new double[dim_x][dim_y];
    for(int i = 0; i < dim_x; ++i) {
        for(int j = 0; j < dim_y; ++j) {
            data[i][j] = stream.readDouble();
        }
    }

    System.out.println("Client:");
    System.out.println("Dimensions: "+dim_x+" x "+dim_y);
    System.out.println("Data:");
    for(int i = 0; i < dim_x; ++i) {
        for(int j = 0; j < dim_y; ++j) {
            System.out.print(" "+data[i][j]);
        }
        System.out.println();
    }


} catch(IOException e) {
    System.err.println("Error reading input");
    System.err.println(e.getMessage());
    System.exit(1);
}