java - 如何修复 Google DataFlow Pipeline (args) 空指针异常？

Question

我正在尝试运行一个非常简单的数据流作业，只需在 BigQuery 中获取一些数据，对其进行一些处理并将其放入新的 bigquery 表中

管道 p = Pipeline.create(
PipelineOptionsFactory.fromArgs(args).withValidation().create());
p.apply(BigQueryIO.Read.fromQuery("SELECT * FROM realtime.status_6_output_11"));
p.run();

但是，每当我运行它时，我都会收到以下非常难以描述的 NullPointerException:

Exception in thread "main" java.lang.NullPointerException
    at java.util.regex.Matcher.getTextLength(Matcher.java:1283)
    at java.util.regex.Matcher.reset(Matcher.java:309)
    at java.util.regex.Matcher.<init>(Matcher.java:229)
    at java.util.regex.Pattern.matcher(Pattern.java:1093)
    at com.google.cloud.dataflow.sdk.util.IOChannelUtils.getFactory(IOChannelUtils.java:174)
    at com.google.cloud.dataflow.sdk.io.BigQueryIO$Read$Bound.apply(BigQueryIO.java:553)
    at com.google.cloud.dataflow.sdk.io.BigQueryIO$Read$Bound.apply(BigQueryIO.java:387)
    at com.google.cloud.dataflow.sdk.runners.PipelineRunner.apply(PipelineRunner.java:74)
    at com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.apply(DirectPipelineRunner.java:247)
    at com.google.cloud.dataflow.sdk.Pipeline.applyInternal(Pipeline.java:367)
    at com.google.cloud.dataflow.sdk.Pipeline.applyTransform(Pipeline.java:274)
    at com.google.cloud.dataflow.sdk.values.PBegin.apply(PBegin.java:47)
    at com.google.cloud.dataflow.sdk.Pipeline.apply(Pipeline.java:156)
    at com.noraway.conductor.NormalizedPipeline.main(NormalizedPipeline.java:42)

I think there's a problem with my command line arguments (don't have any right now) but I'm not sure what that would be.

Answer 1

似乎缺少可供 BigQuery 使用的 --tempLocation。这个晦涩的错误消息已作为 https://github.com/GoogleCloudPlatform/DataflowJavaSDK/issues/313 的一部分修复。 .