gpt4 book ai didi

java - Avro 模式不支持向后兼容性

转载 作者:塔克拉玛干 更新时间:2023-11-03 03:24:11 41 4
gpt4 key购买 nike

我有这个 avro 架构

{
"namespace": "xx.xxxx.xxxxx.xxxxx",
"type": "record",
"name": "MyPayLoad",
"fields": [
{"name": "filed1", "type": "string"},
{"name": "filed2", "type": "long"},
{"name": "filed3", "type": "boolean"},
{
"name" : "metrics",
"type":
{
"type" : "array",
"items":
{
"name": "MyRecord",
"type": "record",
"fields" :
[
{"name": "min", "type": "long"},
{"name": "max", "type": "long"},
{"name": "sum", "type": "long"},
{"name": "count", "type": "long"}
]
}
}
}
]
}

这是我们用来解析数据的代码

public static final MyPayLoad parseBinaryPayload(byte[] payload) {
DatumReader<MyPayLoad> payloadReader = new SpecificDatumReader<>(MyPayLoad.class);
Decoder decoder = DecoderFactory.get().binaryDecoder(payload, null);
MyPayLoad myPayLoad = null;
try {
myPayLoad = payloadReader.read(null, decoder);
} catch (IOException e) {
logger.log(Level.SEVERE, e.getMessage(), e);
}

return myPayLoad;
}

现在我想在模式中再添加一个字段,这样模式如下所示

 {
"namespace": "xx.xxxx.xxxxx.xxxxx",
"type": "record",
"name": "MyPayLoad",
"fields": [
{"name": "filed1", "type": "string"},
{"name": "filed2", "type": "long"},
{"name": "filed3", "type": "boolean"},
{
"name" : "metrics",
"type":
{
"type" : "array",
"items":
{
"name": "MyRecord",
"type": "record",
"fields" :
[
{"name": "min", "type": "long"},
{"name": "max", "type": "long"},
{"name": "sum", "type": "long"},
{"name": "count", "type": "long"}
]
}
}
}
{"name": "agentType", "type": ["null", "string"], "default": "APP_AGENT"}
]
}

请注意已添加的字段,并且还定义了默认值。问题是,如果我们收到使用旧模式写入的数据,我会收到此错误

java.io.EOFException: null
at org.apache.avro.io.BinaryDecoder.ensureBounds(BinaryDecoder.java:473) ~[avro-1.7.4.jar:1.7.4]
at org.apache.avro.io.BinaryDecoder.readInt(BinaryDecoder.java:128) ~[avro-1.7.4.jar:1.7.4]
at org.apache.avro.io.BinaryDecoder.readIndex(BinaryDecoder.java:423) ~[avro-1.7.4.jar:1.7.4]
at org.apache.avro.io.ResolvingDecoder.doAction(ResolvingDecoder.java:229) ~[avro-1.7.4.jar:1.7.4]
at org.apache.avro.io.parsing.Parser.advance(Parser.java:88) ~[avro-1.7.4.jar:1.7.4]
at org.apache.avro.io.ResolvingDecoder.readIndex(ResolvingDecoder.java:206) ~[avro-1.7.4.jar:1.7.4]
at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:152) ~[avro-1.7.4.jar:1.7.4]
at org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:177) ~[avro-1.7.4.jar:1.7.4]
at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:148) ~[avro-1.7.4.jar:1.7.4]
at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:139) ~[avro-1.7.4.jar:1.7.4]
at com.appdynamics.blitz.shared.util.XXXXXXXXXXXXX.parseBinaryPayload(BlitzAvroSharedUtil.java:38) ~[blitz-shared.jar:na]

我从this了解到的记录这应该是向后兼容的,但不知何故似乎并非如此。知道我做错了什么吗?

最佳答案

终于我得到了这个工作。我需要在 SpecificDatumReader 中给出这两个模式所以我修改了这样的解析,我在阅读器中同时传递了旧模式和新模式,它就像一个魅力

public static final MyPayLoad parseBinaryPayload(byte[] payload) {
DatumReader<MyPayLoad> payloadReader = new SpecificDatumReader<>(SCHEMA_V1, SCHEMA_V2);
Decoder decoder = DecoderFactory.get().binaryDecoder(payload, null);
MyPayLoad myPayLoad = null;
try {
myPayLoad = payloadReader.read(null, decoder);
} catch (IOException e) {
logger.log(Level.SEVERE, e.getMessage(), e);
}

return myPayLoad;
}

关于java - Avro 模式不支持向后兼容性,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/34733604/

41 4 0
Copyright 2021 - 2024 cfsdn All Rights Reserved 蜀ICP备2022000587号
广告合作:1813099741@qq.com 6ren.com