gpt4 book ai didi

org.apache.beam.sdk.io.WriteFilesResult.expand()方法的使用及代码示例

转载 作者:知者 更新时间:2024-03-27 06:31:05 27 4
gpt4 key购买 nike

本文整理了Java中org.apache.beam.sdk.io.WriteFilesResult.expand()方法的一些代码示例,展示了WriteFilesResult.expand()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。WriteFilesResult.expand()方法的具体详情如下:
包路径:org.apache.beam.sdk.io.WriteFilesResult
类名称:WriteFilesResult
方法名:expand

WriteFilesResult.expand介绍

暂无

代码示例

代码示例来源:origin: org.apache.beam/beam-runners-google-cloud-dataflow-java

private void testStreamingWriteOverride(PipelineOptions options, int expectedNumShards) {
 TestPipeline p = TestPipeline.fromOptions(options);
 StreamingShardedWriteFactory<Object, Void, Object> factory =
   new StreamingShardedWriteFactory<>(p.getOptions());
 WriteFiles<Object, Void, Object> original = WriteFiles.to(new TestSink(tmpFolder.toString()));
 PCollection<Object> objs = (PCollection) p.apply(Create.empty(VoidCoder.of()));
 AppliedPTransform<PCollection<Object>, WriteFilesResult<Void>, WriteFiles<Object, Void, Object>>
   originalApplication =
     AppliedPTransform.of("writefiles", objs.expand(), Collections.emptyMap(), original, p);
 WriteFiles<Object, Void, Object> replacement =
   (WriteFiles<Object, Void, Object>)
     factory.getReplacementTransform(originalApplication).getTransform();
 assertThat(replacement, not(equalTo((Object) original)));
 assertThat(replacement.getNumShardsProvider().get(), equalTo(expectedNumShards));
 WriteFilesResult<Void> originalResult = objs.apply(original);
 WriteFilesResult<Void> replacementResult = objs.apply(replacement);
 Map<PValue, ReplacementOutput> res =
   factory.mapOutputs(originalResult.expand(), replacementResult);
 assertEquals(1, res.size());
 assertEquals(
   originalResult.getPerDestinationOutputFilenames(),
   res.get(replacementResult.getPerDestinationOutputFilenames()).getOriginal().getValue());
}

代码示例来源:origin: org.apache.beam/beam-runners-core-construction-java

@Test
public void testExtractionDirectFromTransform() throws Exception {
 PCollection<String> input = p.apply(Create.of("hello"));
 WriteFilesResult<Void> output = input.apply(writeFiles);
 AppliedPTransform<PCollection<String>, WriteFilesResult<Void>, WriteFiles<String, Void, String>>
   appliedPTransform =
     AppliedPTransform.of("foo", input.expand(), output.expand(), writeFiles, p);
 assertThat(
   WriteFilesTranslation.isRunnerDeterminedSharding(appliedPTransform),
   equalTo(
     writeFiles.getNumShardsProvider() == null && writeFiles.getComputeNumShards() == null));
 assertThat(
   WriteFilesTranslation.isWindowedWrites(appliedPTransform),
   equalTo(writeFiles.getWindowedWrites()));
 assertThat(
   WriteFilesTranslation.<String, Void, String>getSink(appliedPTransform),
   equalTo(writeFiles.getSink()));
}

27 4 0
Copyright 2021 - 2024 cfsdn All Rights Reserved 蜀ICP备2022000587号
广告合作:1813099741@qq.com 6ren.com