apache-kafka - ElasticsearchSinkConnector 无法将数据反序列化为 Avro

Question

我创建了最简单的 kafka sink 连接器配置，并且我使用的是 confluent 4.1.0:

{
  "connector.class": 
  "io.confluent.connect.elasticsearch.ElasticsearchSinkConnector",
  "type.name": "test-type",
  "tasks.max": "1",
  "topics": "dialogs",
  "name": "elasticsearch-sink",
  "key.ignore": "true",
  "connection.url": "http://localhost:9200",
  "schema.ignore": "true"
}

在主题中，我将消息保存在JSON

{ "topics": "resd"}

但结果我得到一个错误:

Caused by: org.apache.kafka.common.errors.SerializationException: Error deserializing Avro message for id -1 Caused by: org.apache.kafka.common.errors.SerializationException: Unknown magic byte!

Answer 1

正如 cricket_007 所说，您需要告诉 Connect 使用 Json 反序列化器，如果这是您的数据所采用的格式。将其添加到您的连接器配置中:

"value.converter": "org.apache.kafka.connect.json.JsonConverter",
"value.converter.schemas.enable": "false",
"key.converter": "org.apache.kafka.connect.json.JsonConverter",
"key.converter.schemas.enable": "false"