gpt4 book ai didi

org.apache.gobblin.writer.WatermarkAwareWriter.isWatermarkCapable()方法的使用及代码示例

转载 作者:知者 更新时间:2024-03-26 09:53:05 25 4
gpt4 key购买 nike

本文整理了Java中org.apache.gobblin.writer.WatermarkAwareWriter.isWatermarkCapable()方法的一些代码示例,展示了WatermarkAwareWriter.isWatermarkCapable()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。WatermarkAwareWriter.isWatermarkCapable()方法的具体详情如下:
包路径:org.apache.gobblin.writer.WatermarkAwareWriter
类名称:WatermarkAwareWriter
方法名:isWatermarkCapable

WatermarkAwareWriter.isWatermarkCapable介绍

暂无

代码示例

代码示例来源:origin: apache/incubator-gobblin

private boolean isDataWriterWatermarkCapable(DataWriter<D> dataWriter) {
 return (dataWriter instanceof WatermarkAwareWriter) && (((WatermarkAwareWriter) dataWriter).isWatermarkCapable());
}

代码示例来源:origin: apache/incubator-gobblin

public final boolean isWatermarkCapable() {
 return watermarkAwareWriter.get().isWatermarkCapable();
}

代码示例来源:origin: apache/incubator-gobblin

@Override
public boolean isWatermarkCapable() {
 return watermarkAwareWriter.isPresent() && watermarkAwareWriter.get().isWatermarkCapable();
}

代码示例来源:origin: apache/incubator-gobblin

public void testWatermarkComputation(Long committed, Long unacknowledged, Long expected) throws IOException {
 State state = new State();
 state.setProp(ConfigurationKeys.WRITER_PARTITIONER_CLASS, TestPartitioner.class.getCanonicalName());
 String defaultSource = "default";
 WatermarkAwareWriter mockDataWriter = mock(WatermarkAwareWriter.class);
 when(mockDataWriter.isWatermarkCapable()).thenReturn(true);
 when(mockDataWriter.getCommittableWatermark()).thenReturn(Collections.singletonMap(defaultSource,
   new DefaultCheckpointableWatermark(defaultSource, new LongWatermark(committed))));
 when(mockDataWriter.getUnacknowledgedWatermark()).thenReturn(Collections.singletonMap(defaultSource,
   new DefaultCheckpointableWatermark(defaultSource, new LongWatermark(unacknowledged))));
 PartitionAwareDataWriterBuilder builder = mock(PartitionAwareDataWriterBuilder.class);
 when(builder.validatePartitionSchema(any(Schema.class))).thenReturn(true);
 when(builder.forPartition(any(GenericRecord.class))).thenReturn(builder);
 when(builder.withWriterId(any(String.class))).thenReturn(builder);
 when(builder.build()).thenReturn(mockDataWriter);
 PartitionedDataWriter writer = new PartitionedDataWriter<String, String>(builder, state);
 RecordEnvelope<String> recordEnvelope = new RecordEnvelope<String>("0");
 recordEnvelope.addCallBack(
   new AcknowledgableWatermark(new DefaultCheckpointableWatermark(defaultSource, new LongWatermark(0))));
 writer.writeEnvelope(recordEnvelope);
 Map<String, CheckpointableWatermark> watermark = writer.getCommittableWatermark();
 System.out.println(watermark.toString());
 if (expected == null) {
  Assert.assertTrue(watermark.isEmpty(), "Expected watermark to be absent");
 } else {
  Assert.assertTrue(watermark.size() == 1);
  Assert.assertEquals((long) expected, ((LongWatermark) watermark.values().iterator().next().getWatermark()).getValue());
 }
}

代码示例来源:origin: org.apache.gobblin/gobblin-core

private boolean isDataWriterWatermarkCapable(DataWriter<D> dataWriter) {
 return (dataWriter instanceof WatermarkAwareWriter) && (((WatermarkAwareWriter) dataWriter).isWatermarkCapable());
}

代码示例来源:origin: org.apache.gobblin/gobblin-core-base

public final boolean isWatermarkCapable() {
 return watermarkAwareWriter.get().isWatermarkCapable();
}

代码示例来源:origin: org.apache.gobblin/gobblin-core-base

@Override
public boolean isWatermarkCapable() {
 return watermarkAwareWriter.isPresent() && watermarkAwareWriter.get().isWatermarkCapable();
}

25 4 0
Copyright 2021 - 2024 cfsdn All Rights Reserved 蜀ICP备2022000587号
广告合作:1813099741@qq.com 6ren.com