- 使用 Spring Initializr 创建 Spring Boot 应用程序
- 在Spring Boot中配置Cassandra
- 在 Spring Boot 上配置 Tomcat 连接池
- 将Camel消息路由到嵌入WildFly的Artemis上
本文整理了Java中org.apache.gobblin.writer.WatermarkAwareWriter.isWatermarkCapable()
方法的一些代码示例,展示了WatermarkAwareWriter.isWatermarkCapable()
的具体用法。这些代码示例主要来源于Github
/Stackoverflow
/Maven
等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。WatermarkAwareWriter.isWatermarkCapable()
方法的具体详情如下:
包路径:org.apache.gobblin.writer.WatermarkAwareWriter
类名称:WatermarkAwareWriter
方法名:isWatermarkCapable
暂无
代码示例来源:origin: apache/incubator-gobblin
private boolean isDataWriterWatermarkCapable(DataWriter<D> dataWriter) {
return (dataWriter instanceof WatermarkAwareWriter) && (((WatermarkAwareWriter) dataWriter).isWatermarkCapable());
}
代码示例来源:origin: apache/incubator-gobblin
public final boolean isWatermarkCapable() {
return watermarkAwareWriter.get().isWatermarkCapable();
}
代码示例来源:origin: apache/incubator-gobblin
@Override
public boolean isWatermarkCapable() {
return watermarkAwareWriter.isPresent() && watermarkAwareWriter.get().isWatermarkCapable();
}
代码示例来源:origin: apache/incubator-gobblin
public void testWatermarkComputation(Long committed, Long unacknowledged, Long expected) throws IOException {
State state = new State();
state.setProp(ConfigurationKeys.WRITER_PARTITIONER_CLASS, TestPartitioner.class.getCanonicalName());
String defaultSource = "default";
WatermarkAwareWriter mockDataWriter = mock(WatermarkAwareWriter.class);
when(mockDataWriter.isWatermarkCapable()).thenReturn(true);
when(mockDataWriter.getCommittableWatermark()).thenReturn(Collections.singletonMap(defaultSource,
new DefaultCheckpointableWatermark(defaultSource, new LongWatermark(committed))));
when(mockDataWriter.getUnacknowledgedWatermark()).thenReturn(Collections.singletonMap(defaultSource,
new DefaultCheckpointableWatermark(defaultSource, new LongWatermark(unacknowledged))));
PartitionAwareDataWriterBuilder builder = mock(PartitionAwareDataWriterBuilder.class);
when(builder.validatePartitionSchema(any(Schema.class))).thenReturn(true);
when(builder.forPartition(any(GenericRecord.class))).thenReturn(builder);
when(builder.withWriterId(any(String.class))).thenReturn(builder);
when(builder.build()).thenReturn(mockDataWriter);
PartitionedDataWriter writer = new PartitionedDataWriter<String, String>(builder, state);
RecordEnvelope<String> recordEnvelope = new RecordEnvelope<String>("0");
recordEnvelope.addCallBack(
new AcknowledgableWatermark(new DefaultCheckpointableWatermark(defaultSource, new LongWatermark(0))));
writer.writeEnvelope(recordEnvelope);
Map<String, CheckpointableWatermark> watermark = writer.getCommittableWatermark();
System.out.println(watermark.toString());
if (expected == null) {
Assert.assertTrue(watermark.isEmpty(), "Expected watermark to be absent");
} else {
Assert.assertTrue(watermark.size() == 1);
Assert.assertEquals((long) expected, ((LongWatermark) watermark.values().iterator().next().getWatermark()).getValue());
}
}
代码示例来源:origin: org.apache.gobblin/gobblin-core
private boolean isDataWriterWatermarkCapable(DataWriter<D> dataWriter) {
return (dataWriter instanceof WatermarkAwareWriter) && (((WatermarkAwareWriter) dataWriter).isWatermarkCapable());
}
代码示例来源:origin: org.apache.gobblin/gobblin-core-base
public final boolean isWatermarkCapable() {
return watermarkAwareWriter.get().isWatermarkCapable();
}
代码示例来源:origin: org.apache.gobblin/gobblin-core-base
@Override
public boolean isWatermarkCapable() {
return watermarkAwareWriter.isPresent() && watermarkAwareWriter.get().isWatermarkCapable();
}
我正在尝试使用 gobblin 从 mysql 到 hdfs 数据摄取。使用以下步骤运行 mysql-to-gobblin.pull 时: 1)启动hadoop: sbin\start-all.cmd
本文整理了Java中gobblin.source.workunit.WorkUnit类的一些代码示例,展示了WorkUnit类的具体用法。这些代码示例主要来源于Github/Stackoverflow
我想在MacOS X上安装Apache Gobblin。为此,我下载了版本0.14.0,并按照此处的步骤进行操作。 Install Gobblin 我做的第一件事是: tar -xvf incubat
尝试从 git clone 下载和构建 Gobblin 时。通过关注,在本地下载并构建 Gobblin在您的本地计算机上,克隆 Gobblin 存储库: git clone git@github.co
本文整理了Java中org.apache.gobblin.writer.WriterOutputFormat类的一些代码示例,展示了WriterOutputFormat类的具体用法。这些代码示例主要来
本文整理了Java中org.apache.gobblin.writer.WatermarkAwareWriter类的一些代码示例,展示了WatermarkAwareWriter类的具体用法。这些代码示
本文整理了Java中org.apache.gobblin.writer.WriteResponseMapper类的一些代码示例,展示了WriteResponseMapper类的具体用法。这些代码示例主
本文整理了Java中org.apache.gobblin.util.WritableShimSerialization类的一些代码示例,展示了WritableShimSerialization类的具体
本文整理了Java中org.apache.gobblin.yarn.YarnContainerSecurityManager类的一些代码示例,展示了YarnContainerSecurityManag
本文整理了Java中org.apache.gobblin.yarn.YarnAppSecurityManager类的一些代码示例,展示了YarnAppSecurityManager类的具体用法。这些代
本文整理了Java中org.apache.gobblin.yarn.YarnHelixUtils类的一些代码示例,展示了YarnHelixUtils类的具体用法。这些代码示例主要来源于Github/S
本文整理了Java中org.apache.gobblin.yarn.YarnService类的一些代码示例,展示了YarnService类的具体用法。这些代码示例主要来源于Github/Stackov
我正在评估一个大数据项目,我们需要从各种互联网来源(ftp、api 等)提取大量大数据集,进行轻量级转换和轻量级数据质量/健全性检查(例如:行和列检查),并将其推向下游。直接关注点是批量的,但预计会支
我正在运行gobblin,以使用3节点EMR集群将数据从kafka移至s3。我在hadoop 2.6.0上运行,并且还针对2.6.0构建了gobblin。 似乎map-reduce作业成功运行。在我的
我研究 Gobblin 一段时间了,目前我在使用 Gobblin 从 Facebook 获取帖子时遇到了困难。我在互联网上找不到任何连接示例,或者我可能搜索错误。 我正在考虑将restfb集成到Gob
本文整理了Java中org.apache.gobblin.source.workunit.WorkUnitStream类的一些代码示例,展示了WorkUnitStream类的具体用法。这些代码示例主要
本文整理了Java中gobblin.source.workunit.WorkUnit.readFields()方法的一些代码示例,展示了WorkUnit.readFields()的具体用法。这些代码示
本文整理了Java中gobblin.source.workunit.WorkUnit.createEmpty()方法的一些代码示例,展示了WorkUnit.createEmpty()的具体用法。这些代
本文整理了Java中gobblin.source.workunit.WorkUnit.getExtract()方法的一些代码示例,展示了WorkUnit.getExtract()的具体用法。这些代码示
本文整理了Java中gobblin.source.workunit.WorkUnit.getLowWatermark()方法的一些代码示例,展示了WorkUnit.getLowWatermark()的
我是一名优秀的程序员,十分优秀!