julia - 将 OrderedDict 保存为 Julia 数据格式

Question

我希望使用 JLD将 OrderedDict 写入文件的包，以便我随后可以原样读回它。

这是我的第一个尝试:

using JLD, HDF5, DataStructures

function testjld()  
    res = OrderedDict("A" => 1, "B" => 2)
    filename = "c:/temp/test.jld"
    save(File(format"JLD", filename), "res", res)
    res2 = load(filename)["res"]
    #Check if round-tripping works
    res == res2
end

但是“往返”不起作用 - 函数返回 false。它还会发出警告:

julia> testjld()
┌ Warning: type JLD.AssociativeWrapper{Core.String,Core.Int64,OrderedCollections.OrderedDict{Core.String,Core.Int64}} not present in workspace; reconstructing
└ @ JLD C:\Users\Philip\.julia\packages\JLD\1BoSz\src\jld_types.jl:703
false

阅读文档后，我认为 JLD 不支持 OrderedDict “开箱即用”，但支持 Dict 并且我可以用这个事实写我自己的 custom serialisation对于 OrderedDict。像这样:

struct OrderedDictSerializer
    d::Dict
end

JLD.writeas(data::OrderedDict) = OrderedDictSerializer(Dict("contents" => convert(Dict, data), 
                                                            "keyorder" => [k for (k, v) in data]))

function JLD.readas(serdata::OrderedDictSerializer)
    unordered = serdata.d["contents"]
    keyorder = serdata.d["keyorder"]
    OrderedDict((k, unordered[k]) for k in keyorder)
end

这不是一个详尽的测试，但这似乎确实有效:

julia> testjld()
true

我认为我需要为 OrderedDict 编写自己的序列化程序是否正确，我的序列化程序是否可以改进？

编辑

我的问题“我的序列化程序可以改进吗？”的答案似乎是“它必须是，虽然我还不知道怎么做。”

考虑以下两个测试函数:

function testjld2()
    res = OrderedDict("A" => [1.0,2.0],"B" => [3.0,4.0])
    #check if round-tripping of readas and writeas methods works:
    JLD.readas(JLD.writeas(res)) == res
end

function testjld3()
    res = OrderedDict("A" => [1.0,2.0],"B" => [3.0,4.0])
    filename = "c:/temp/test.jld"
    save(File(format"JLD", filename), "res", res)
    res2 = load(filename)["res"]
    #Check if round-tripping to jld file and back works
    res == res2
end

testjld2 显示我的 writeas 和 readas 方法正确地往返于 OrderedDict{String,Array{Float64,1 }} 有 2 个条目

julia> testjld2()
true

然而 testjld3 根本不起作用，但会产生错误:

julia> testjld3()
HDF5-DIAG: Error detected in HDF5 (1.10.5) thread 0:
  #000: E:/mingwbuild/mingw-w64-hdf5/src/hdf5-1.10.5/src/H5Tfields.c line 60 in H5Tget_nmembers(): not a datatype
    major: Invalid arguments to routine
    minor: Inappropriate type
HDF5-DIAG: Error detected in HDF5 (1.10.5) thread 0:
  #000: E:/mingwbuild/mingw-w64-hdf5/src/hdf5-1.10.5/src/H5Tfields.c line 60 in H5Tget_nmembers(): not a datatype
    major: Invalid arguments to routine
    minor: Inappropriate type
ERROR: Error getting the number of members
Stacktrace:
 [1] error(::String) at .\error.jl:33
 [2] h5t_get_nmembers at C:\Users\Philip\.julia\packages\HDF5\rF1Fe\src\HDF5.jl:2279 [inlined]
 [3] _gen_h5convert!(::Any) at C:\Users\Philip\.julia\packages\JLD\1BoSz\src\jld_types.jl:638
 [4] #s27#9(::Any, ::Any, ::Any, ::Any, ::Any, ::Any) at C:\Users\Philip\.julia\packages\JLD\1BoSz\src\jld_types.jl:664
 [5] (::Core.GeneratedFunctionStub)(::Any, ::Vararg{Any,N} where N) at .\boot.jl:524
 [6] #write_compound#24(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::typeof(JLD.write_compound), ::JLD.JldGroup, ::String, ::JLD.AssociativeWrapper{String,Any,Dict{String,Any}}, ::JLD.JldWriteSession) at C:\Users\Philip\.julia\packages\JLD\1BoSz\src\JLD.jl:700
 [7] write_compound at C:\Users\Philip\.julia\packages\JLD\1BoSz\src\JLD.jl:694 [inlined]
 [8] #_write#23 at C:\Users\Philip\.julia\packages\JLD\1BoSz\src\JLD.jl:690 [inlined]
 [9] _write at C:\Users\Philip\.julia\packages\JLD\1BoSz\src\JLD.jl:690 [inlined]
 [10] write_ref(::JLD.JldFile, ::Dict{String,Any}, ::JLD.JldWriteSession) at C:\Users\Philip\.julia\packages\JLD\1BoSz\src\JLD.jl:658
 [11] macro expansion at C:\Users\Philip\.julia\packages\JLD\1BoSz\src\jld_types.jl:648 [inlined]
 [12] h5convert!(::Ptr{UInt8}, ::JLD.JldFile, ::OrderedDictSerializer, ::JLD.JldWriteSession) at C:\Users\Philip\.julia\packages\JLD\1BoSz\src\jld_types.jl:664
 [13] #write_compound#24(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::typeof(JLD.write_compound), ::JLD.JldFile, ::String, ::OrderedDictSerializer, ::JLD.JldWriteSession) at C:\Users\Philip\.julia\packages\JLD\1BoSz\src\JLD.jl:700
 [14] write_compound at C:\Users\Philip\.julia\packages\JLD\1BoSz\src\JLD.jl:694 [inlined]
 [15] #_write#23 at C:\Users\Philip\.julia\packages\JLD\1BoSz\src\JLD.jl:690 [inlined]
 [16] _write at C:\Users\Philip\.julia\packages\JLD\1BoSz\src\JLD.jl:690 [inlined]
 [17] #write#17(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::typeof(write), ::JLD.JldFile, ::String, ::OrderedDict{String,Array{Float64,1}}, ::JLD.JldWriteSession) at C:\Users\Philip\.julia\packages\JLD\1BoSz\src\JLD.jl:514
 [18] write at C:\Users\Philip\.julia\packages\JLD\1BoSz\src\JLD.jl:514 [inlined]
 [19] #35 at C:\Users\Philip\.julia\packages\JLD\1BoSz\src\JLD.jl:1223 [inlined]
 [20] #jldopen#14(::Base.Iterators.Pairs{Symbol,Bool,Tuple{Symbol,Symbol},NamedTuple{(:compatible, :compress),Tuple{Bool,Bool}}}, ::typeof(jldopen), ::getfield(JLD, Symbol("##35#36")){String,OrderedDict{String,Array{Float64,1}},Tuple{}},
::String, ::Vararg{String,N} where N) at C:\Users\Philip\.julia\packages\JLD\1BoSz\src\JLD.jl:246
 [21] testjld3() at .\none:0
 [22] top-level scope at REPL[48]:1

Answer 1

改用 JLD2:

using JLD2, DataStructures, FileIO
function testjld2()
    res = OrderedDict("A" => 1, "B" => 2)
    myfilename = "c:/temp/test.jld2"
    save(myfilename, "res", res)
    res2 = load(myfilename)["res"]
    #Check if round-tripping works
    res == res2
end

测试:

julia> testjld2()
true

就个人而言，无论何时我都可以使用 BJSON:

using DataStructures, BSON, OrderedCollections
function testbson()
    res = OrderedDict("A" => 1, "B" => 2)
    myfilename = "c:/temp/test.bjson"
    BSON.bson(myfilename, Dict("res" => res))
    res2 = BSON.load(myfilename)["res"]
    #Check if round-tripping works
    res == res2
end

julia> testbson()
true