java - Apache Flink Google Pub/Sub 连接器在 intellij idea 中运行抛出 NoClassDefFoundError

Question

我正在尝试将 Google Pub/Sub 连接器与 Flink 结合使用，请参阅此处。

但是，我得到

Caused by: java.lang.NoClassDefFoundError: Could not initialize class com.google.cloud.pubsub.v1.stub.GrpcPublisherStub

我的连接器的依赖版本是 1.9.0

<dependency>
            <groupId>org.apache.flink</groupId>
            <artifactId>flink-connector-gcp-pubsub_2.12</artifactId>
            <version>1.9.0</version>
        </dependency>

完整的异常堆栈

Exception in thread "main" org.apache.flink.runtime.client.JobExecutionException: Job execution failed.
    at org.apache.flink.runtime.jobmaster.JobResult.toJobExecutionResult(JobResult.java:146)
    at org.apache.flink.runtime.minicluster.MiniCluster.executeJobBlocking(MiniCluster.java:638)
    at org.apache.flink.streaming.api.environment.LocalStreamEnvironment.execute(LocalStreamEnvironment.java:123)
    at btc.da.StreamingJob.main(StreamingJob.java:87)
Caused by: java.lang.NoClassDefFoundError: Could not initialize class com.google.cloud.pubsub.v1.stub.GrpcPublisherStub
    at com.google.cloud.pubsub.v1.Publisher.<init>(Publisher.java:149)
    at com.google.cloud.pubsub.v1.Publisher.<init>(Publisher.java:78)
    at com.google.cloud.pubsub.v1.Publisher$Builder.build(Publisher.java:614)
    at org.apache.flink.streaming.connectors.gcp.pubsub.PubSubSink.open(PubSubSink.java:111)
    at org.apache.flink.api.common.functions.util.FunctionUtils.openFunction(FunctionUtils.java:36)
    at org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.open(AbstractUdfStreamOperator.java:102)
    at org.apache.flink.streaming.api.operators.StreamSink.open(StreamSink.java:48)
    at org.apache.flink.streaming.runtime.tasks.StreamTask.openAllOperators(StreamTask.java:424)
    at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:290)
    at org.apache.flink.runtime.taskmanager.Task.run(Task.java:711)
    at java.base/java.lang.Thread.run(Thread.java:835)

Flink 生成的项目也使用了 Maven Shade 插件

<plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-shade-plugin</artifactId>
                <version>3.0.0</version>
                <executions>
                    <!-- Run shade goal on package phase -->
                    <execution>
                        <phase>package</phase>
                        <goals>
                            <goal>shade</goal>
                        </goals>
                        <configuration>
                            <artifactSet>
                                <excludes>
                                    <exclude>org.apache.flink:force-shading</exclude>
                                    <exclude>com.google.code.findbugs:jsr305</exclude>
                                    <exclude>org.slf4j:*</exclude>
                                    <exclude>log4j:*</exclude>
                                </excludes>
                            </artifactSet>
                            <filters>
                                <filter>
                                    <!-- Do not copy the signatures in the META-INF folder.
                                    Otherwise, this might cause SecurityExceptions when using the JAR. -->
                                    <artifact>*:*</artifact>
                                    <excludes>
                                        <exclude>META-INF/*.SF</exclude>
                                        <exclude>META-INF/*.DSA</exclude>
                                        <exclude>META-INF/*.RSA</exclude>
                                    </excludes>
                                </filter>
                            </filters>
                            <transformers>
                                <transformer
                                        implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
                                    <mainClass>btc.da.StreamingJob</mainClass>
                                </transformer>
                            </transformers>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
        </plugins> 

jar 里似乎有

macbookOOO:btc-utx-da aghan$ jar tf target/btc-utx-da-0.1.jar | grep GrpcPublisherStub
com/google/cloud/pubsub/v1/stub/GrpcPublisherStub.class
macbook5434:btc-utx-da aghan$ 

Answer 1

好吧，经过一些调试并使用强大的 Intellij Idea 我能够意识到问题是缺少对以下内容的依赖:

 <dependency>
        <groupId>com.google.protobuf</groupId>
        <artifactId>protobuf-java</artifactId>
        <version>3.9.1</version>
    </dependency>