I don't use confluent, just use apache kafka, already add mongo plugin, get ... / connector-plugins response, list the MongoSinkConnector
我不使用confluent,只使用apachekafka,已经添加了mongo插件,get…/连接器插件响应,列出MongoSinkConnector
[
{
"class": "com.mongodb.kafka.connect.MongoSinkConnector",
"type": "sink",
"version": "1.5.0"
},
{
"class": "com.mongodb.kafka.connect.MongoSourceConnector",
"type": "source",
"version": "1.5.0"
}...
]
]
when I use the following post I get the error:
当我使用以下帖子时,我得到了错误:
Request:
请求:
{
"name": "mongodb-sink",
"config": {
"connector.class": "com.mongodb.kafka.connect.MongoSinkConnector",
"tasks.max": "1",
"topics": "portalesMCor.db_portales.user",
"connection.uri": "mongodb://tiastest.mti:27017",
"database": "Big_Data",
"collection": "users",
"delete.on.null.values": false
}
}
}
Resposnse:
响应:
{
"error_code": 500,
"message": "Failed to find any class that implements Connector and which name matches com.mongodb.kafka.connect.MongoSinkConnector, available connectors are: PluginDesc{klass=class io.debezium.connector.db2.Db2Connector, name='io.debezium.connector.db2.Db2Connector', version='1.5.0.Final', encodedVersion=1.5.0.Final, type=source, typeName='source', location='file:/kafka/connect/debezium-connector-db2/'}, PluginDesc{klass=class io.debezium.connector.mongodb.MongoDbConnector, name='io.debezium.connector.mongodb.MongoDbConnector', version='1.5.0.Final', encodedVersion=1.5.0.Final, type=source, typeName='source', location='file:/kafka/connect/debezium-connector-mongodb/'}, PluginDesc{klass=class io.debezium.connector.mysql.MySqlConnector, name='io.debezium.connector.mysql.MySqlConnector', version='1.5.0.Final', encodedVersion=1.5.0.Final, type=source, typeName='source', location='file:/kafka/connect/debezium-connector-mysql/'}, PluginDesc{klass=class io.debezium.connector.oracle.OracleConnector, name='io.debezium.connector.oracle.OracleConnector', version='1.5.0.Final', encodedVersion=1.5.0.Final, type=source, typeName='source', location='file:/kafka/connect/debezium-connector-oracle/'}, PluginDesc{klass=class io.debezium.connector.postgresql.PostgresConnector, name='io.debezium.connector.postgresql.PostgresConnector', version='1.5.0.Final', encodedVersion=1.5.0.Final, type=source, typeName='source', location='file:/kafka/connect/debezium-connector-postgres/'}, PluginDesc{klass=class io.debezium.connector.sqlserver.SqlServerConnector, name='io.debezium.connector.sqlserver.SqlServerConnector', version='1.5.0.Final', encodedVersion=1.5.0.Final, type=source, typeName='source', location='file:/kafka/connect/debezium-connector-sqlserver/'}, PluginDesc{klass=class io.debezium.connector.vitess.VitessConnector, name='io.debezium.connector.vitess.VitessConnector', version='1.5.0.Final', encodedVersion=1.5.0.Final, type=source, typeName='source', location='file:/kafka/connect/debezium-connector-vitess/'}, PluginDesc{klass=class org.apache.kafka.connect.file.FileStreamSinkConnector, name='org.apache.kafka.connect.file.FileStreamSinkConnector', version='2.7.0', encodedVersion=2.7.0, type=sink, typeName='sink', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.file.FileStreamSourceConnector, name='org.apache.kafka.connect.file.FileStreamSourceConnector', version='2.7.0', encodedVersion=2.7.0, type=source, typeName='source', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.mirror.MirrorCheckpointConnector, name='org.apache.kafka.connect.mirror.MirrorCheckpointConnector', version='1', encodedVersion=1, type=source, typeName='source', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.mirror.MirrorHeartbeatConnector, name='org.apache.kafka.connect.mirror.MirrorHeartbeatConnector', version='1', encodedVersion=1, type=source, typeName='source', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.mirror.MirrorSourceConnector, name='org.apache.kafka.connect.mirror.MirrorSourceConnector', version='1', encodedVersion=1, type=source, typeName='source', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.tools.MockConnector, name='org.apache.kafka.connect.tools.MockConnector', version='2.7.0', encodedVersion=2.7.0, type=connector, typeName='connector', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.tools.MockSinkConnector, name='org.apache.kafka.connect.tools.MockSinkConnector', version='2.7.0', encodedVersion=2.7.0, type=sink, typeName='sink', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.tools.MockSourceConnector, name='org.apache.kafka.connect.tools.MockSourceConnector', version='2.7.0', encodedVersion=2.7.0, type=source, typeName='source', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.tools.SchemaSourceConnector, name='org.apache.kafka.connect.tools.SchemaSourceConnector', version='2.7.0', encodedVersion=2.7.0, type=source, typeName='source', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.tools.VerifiableSinkConnector, name='org.apache.kafka.connect.tools.VerifiableSinkConnector', version='2.7.0', encodedVersion=2.7.0, type=source, typeName='source', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.tools.VerifiableSourceConnector, name='org.apache.kafka.connect.tools.VerifiableSourceConnector', version='2.7.0', encodedVersion=2.7.0, type=source, typeName='source', location='classpath'}"
}
}
更多回答
I had a similar error when trying to use confluent's S3SinkConnector
. Am posting my problem and solution here because this page shows up in the search results and there's a dearth of useful information out there.
我在尝试使用confluent的S3SinkConnector时也出现了类似的错误。我在这里发布我的问题和解决方案,因为这个页面出现在搜索结果中,而且缺乏有用的信息。
Actually what happened is that I recompiled the connector after modifying its source code slightly because the original version didn't do exactly what I want. But then when I tried to start my connector I got the error message Failed to find any class that implements Connector and which name matches io.confluent.connect.s3.S3SinkConnector
. This is even though the relevant .jar
is in my plugin path and I could see with jar -tvf
that the .jar contains io/confluent/connect/s3/S3SinkConnector.class
.
事实上,我在稍微修改了连接器的源代码后重新编译了连接器,因为原始版本并没有完全按照我的意愿。但是,当我试图启动连接器时,我收到了错误消息Failed to find any class that implement connector and which name match io.confluent.connect.s3.S3SinkConnector。尽管相关的.jar在我的插件路径中,我可以用jar-tvf看到.jar包含io/confluent/connect/s3/S3SinkConnector.class。
I was running out of ideas so I tried moving the .jar
file into the libs
directory of my kafka installation, i.e. into /path/to/kafka_2.12-2.6.2/libs
. Then I started getting a different error message:
我没有什么想法了,所以我尝试将.jar文件移到我安装的kafka的libs目录中,即/path/to/kafka_2.12-2.6.2/libs中。然后我开始收到不同的错误消息:
java.lang.UnsupportedClassVersionError: io/confluent/connect/s3/S3SinkConnector has been compiled by a more recent version of the Java Runtime (class file version 55.0), this version of the Java Runtime only recognizes class file versions up to 52.0
So the solution was to recompile the file with an older version of Java (specifically Java 8 using javac -release 8
, as per these instructions.) Now it works!
因此,解决方案是使用旧版本的Java(特别是Java 8,根据这些说明使用javac-release 8)重新编译该文件。现在它可以工作了!
Based on the error returned, you have the following connectors on your worker:
根据返回的错误,您的工作人员上有以下连接器:
io.debezium.connector.db2.Db2Connector
io.debezium.connector.mongodb.MongoDbConnector
io.debezium.connector.mysql.MySqlConnector
io.debezium.connector.oracle.OracleConnector
io.debezium.connector.postgresql.PostgresConnector
io.debezium.connector.sqlserver.SqlServerConnector
io.debezium.connector.vitess.VitessConnector
org.apache.kafka.connect.file.FileStreamSinkConnector
org.apache.kafka.connect.file.FileStreamSourceConnector
org.apache.kafka.connect.mirror.MirrorCheckpointConnector
org.apache.kafka.connect.mirror.MirrorHeartbeatConnector
org.apache.kafka.connect.mirror.MirrorSourceConnector
org.apache.kafka.connect.tools.MockConnector
org.apache.kafka.connect.tools.MockSinkConnector
org.apache.kafka.connect.tools.MockSourceConnector
org.apache.kafka.connect.tools.SchemaSourceConnector
org.apache.kafka.connect.tools.VerifiableSinkConnector
org.apache.kafka.connect.tools.VerifiableSourceConnector
So perhaps you're running the /connector-plugins
REST call against a different worker?
Or do you have a cluster of distributed workers in which the MongoDB plugin has not been installed on all of them?
那么,也许您正在针对不同的工作程序运行/connector插件REST调用?或者你有一个分布式工作集群,其中没有在所有工作集群上安装MongoDB插件?
In you Dockerfile you need to install mongodb sink connector. For example like this:
在Dockerfile中,您需要安装mongodb接收器连接器。例如:
RUN confluent-hub install--no-prompt mongodb/kafka-connect-mongodb:1.6.1
更多回答
You are life saver for me. Thanks for describing your debugging procedures as well. I tried to use a new Java17 JRE to run a old connector jar compliled by a old version. Your solution explain it well.
你是我的救命稻草。谢谢你描述你的调试过程。我尝试使用一个新的Java17JRE来运行一个由旧版本编译的旧连接器jar。你的解决方案解释得很好。
Exactly Thanks .. make sure that connector is compiled with exact version of java where connect service is running.
没错,谢谢。。请确保连接器是使用运行connect服务的确切版本的java编译的。
Good idea to put the jar file in your kafka libs. The loading of plugins from the ${plugin.path} must have raised the same error. It would have been nice had the confluence developers logged it, because in the current version the jar is just silently ignored.
把jar文件放在你的kafka-libs中是个好主意。从${plugin.path}加载插件一定引发了相同的错误。如果汇流开发人员将其记录下来,那就太好了,因为在当前版本中,jar只是被默默地忽略了。
But when you sent the class wrong: "connector.class": "com.mongodb.kafka.connect.MongoConnector" The answer is: "Failed to find any class that implements Connector and which name matches com.mongodb.kafka.connect.MongoConnector, available connectors are: PluginDesc {klass = class com.mongodb.kafka.connect.MongoSinkConnector, name = 'com.mongodb.kafka .connect.MongoSinkConnector ', version =' 1.5.0 ', encodedVersion = 1.5.0, type = sink, typeName =' sink ', location =' file: / kafka / connect / mongo-connector / '} .....
但是,当您发送错误的类时:“connector.class”:“com.mongodb.kafka.connect.MongoConnector”答案是:“找不到任何实现连接器且名称与com.mongoodb.kafka.connect.MongoConnector匹配的类,可用的连接器有:PluginDesc{klass=class com.mongodb.kafka.connect.MongoSinkConnector,name='com.mongob.kafka.connect.MongoSinkConnector',version='1.5.0',encodedVersion=1.5.0,type=sink,typeName='sink',location='file:/kafka/connect/mongo-connector/'}。。。。。
@Alfredo As answered, it seems one worker has that installed and available (the server from that comment), but another is not (the server in your question). You'll have to restart the workers to pickup new plugins
@Alfredo回答说,似乎有一个工作人员已经安装并可用(该评论中的服务器),但另一个没有(您问题中的服务器。)。你必须重新启动工作程序才能获得新插件
我是一名优秀的程序员,十分优秀!