java - 使用 BigQuery 时出现 HttpTransportOptions 的 ClassNotFoundException

Question

我正在尝试使用 BigQuery Java API 作为涉及 Dataflow 和其他项目的大型项目的一部分。但是，运行 mvncompile exec:java -Dexec.mainClass=com.xpw.bq_test 时出现以下错误:

java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:293)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NoClassDefFoundError: com/google/cloud/HttpTransportOptions
    at com.google.cloud.bigquery.BigQueryOptions.getDefaultHttpTransportOptions(BigQueryOptions.java:107)
    at com.google.cloud.bigquery.BigQueryOptions$BigQueryDefaults.getDefaultTransportOptions(BigQueryOptions.java:102)
    at com.google.cloud.ServiceOptions.<init>(ServiceOptions.java:255)
    at com.google.cloud.bigquery.BigQueryOptions.<init>(BigQueryOptions.java:84)
    at com.google.cloud.bigquery.BigQueryOptions.<init>(BigQueryOptions.java:31)
    at com.google.cloud.bigquery.BigQueryOptions$Builder.build(BigQueryOptions.java:79)
    at com.google.cloud.bigquery.BigQueryOptions.getDefaultInstance(BigQueryOptions.java:141)
    at com.xpw.bq_test.main(bq_test.java:12)
    ... 6 more
Caused by: java.lang.ClassNotFoundException: com.google.cloud.HttpTransportOptions
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)

bq_test.java 看起来像:

package com.xpw;

import com.google.cloud.bigquery.*;

import java.io.IOException;

public class bq_test {

    public static void main(String[] arg) throws IOException {

        // Instantiates a client
        BigQuery bigquery = BigQueryOptions.getDefaultInstance().getService();

    }
}

我的 pom 文件依赖项，以防冲突可能导致错误:

    <dependencies>

        <dependency>
            <groupId>com.google.cloud</groupId>
            <artifactId>google-cloud-storage</artifactId>
            <version>1.0.0-rc2</version>
        </dependency>

        <dependency>
            <groupId>org.apache.beam</groupId>
            <artifactId>beam-sdks-java-core</artifactId>
            <version>${beam.version}</version>
        </dependency>

        <dependency>
            <groupId>org.apache.beam</groupId>
            <artifactId>beam-sdks-java-io-google-cloud-platform</artifactId>
            <version>${beam.version}</version>
        </dependency>

        <dependency>
            <groupId>org.apache.beam</groupId>
            <artifactId>beam-runners-google-cloud-dataflow-java</artifactId>
            <version>0.6.0</version>
        </dependency>

        <dependency>
            <groupId>com.google.cloud</groupId>
            <artifactId>google-cloud-bigquery</artifactId>
            <version>0.11.0-beta</version>
        </dependency>

    </dependencies>

到目前为止我已经尝试过:

1. 运行mvn clean并重建。
2. 删除~/.m2/repositories并重建。
3. 将 google-cloud-core 直接添加到我的 pom 中。

Answer 1

我之前也遇到过同样的问题。我做了一些调整，下面的代码片段对我有用。

TransportOptions options = HttpTransportOptions.newBuilder().setHttpTransportFactory(new HttpTransportOptions.DefaultHttpTransportFactory()).build();
GoogleCredentials credential = GoogleCredentials.getApplicationDefault();
BigQuery service = BigQueryOptions.newBuilder().setProjectId(YOUR_PROJECT_ID)
                        .setTransportOptions(options)
                        .setCredentials(credential)
                        .build()
                        .getService();

请同时包含以下 GoogleCredentials 依赖项。

<dependency>
  <groupId>com.google.auth</groupId>
  <artifactId>google-auth-library-oauth2-http</artifactId>
  <version>0.6.0</version>
  <type>jar</type>                
</dependency>

试试这个并告诉我。

注意:使用这些库时请务必注意，确保使用相同的构建版本来使用它们。这是安全且无错误的唯一方法。这种情况在 Maven 中经常发生，需要升级和降级依赖项来匹配。在您的情况下，组 ID“com.google.cloud”在造成这些冲突的依赖项上有两个版本。使用 0.11.0-beta 进行 google-cloud-storage，它也应该可以解决所有 future 的问题