gpt4 book ai didi

org.apache.gobblin.source.workunit.WorkUnitStream.getMaterializedWorkUnitCollection()方法的使用及代码示例

转载 作者:知者 更新时间:2024-03-21 04:41:05 25 4
gpt4 key购买 nike

本文整理了Java中org.apache.gobblin.source.workunit.WorkUnitStream.getMaterializedWorkUnitCollection()方法的一些代码示例,展示了WorkUnitStream.getMaterializedWorkUnitCollection()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。WorkUnitStream.getMaterializedWorkUnitCollection()方法的具体详情如下:
包路径:org.apache.gobblin.source.workunit.WorkUnitStream
类名称:WorkUnitStream
方法名:getMaterializedWorkUnitCollection

WorkUnitStream.getMaterializedWorkUnitCollection介绍

[英]Get a materialized collection of the WorkUnits in this stream. Note this call will fail if #isSafeToMaterialize() is false. This method should be avoided unless strictly necessary, as it introduces a synchronization point for all work units, removing the speed and memory-efficiency benefits.
[中]获取此流中工作单元的物化集合。请注意,如果#isSafeToMaterialize()为false,则此调用将失败。除非严格必要,否则应避免使用这种方法,因为它为所有工作单元引入了一个同步点,消除了速度和内存效率方面的优势。

代码示例

代码示例来源:origin: apache/incubator-gobblin

@Override
public List<WorkUnit> getWorkunits(SourceState state) {
 return Lists.newArrayList(getWorkunitStream(state).getMaterializedWorkUnitCollection());
}

代码示例来源:origin: apache/incubator-gobblin

public WriterInitializer getInitializer(State state, WorkUnitStream workUnits, int branches, int branchId) {
  JdbcWriterCommandsFactory factory = new JdbcWriterCommandsFactory();
  if (workUnits.isSafeToMaterialize()) {
   return new JdbcWriterInitializer(state, workUnits.getMaterializedWorkUnitCollection(),
     factory, branches, branchId);
  } else {
   throw new RuntimeException(JdbcWriterBuilder.class.getName() + " does not support work unit streams.");
  }
 }
}

代码示例来源:origin: apache/incubator-gobblin

@Override
 public ConverterInitializer getInitializer(State state, WorkUnitStream workUnits, int branches, int branchId) {
  JdbcWriterCommandsFactory factory = new JdbcWriterCommandsFactory();
  if (workUnits.isSafeToMaterialize()) {
   return new AvroToJdbcEntryConverterInitializer(state, workUnits.getMaterializedWorkUnitCollection(),
     factory, branches, branchId);
  } else {
   throw new RuntimeException(AvroToJdbcEntryConverter.class.getName() + " does not support work unit streams.");
  }
 }
}

代码示例来源:origin: apache/incubator-gobblin

Map<String, ParallelRunner> parallelRunners = Maps.newHashMap();
try {
 for (WorkUnit workUnit : JobLauncherUtils.flattenWorkUnits(workUnits.getMaterializedWorkUnitCollection())) {
  JobLauncherUtils.cleanTaskStagingData(new WorkUnitState(workUnit, jobState), LOG, closer, parallelRunners);

代码示例来源:origin: org.apache.gobblin/gobblin-data-management

@Override
public List<WorkUnit> getWorkunits(SourceState state) {
 return Lists.newArrayList(getWorkunitStream(state).getMaterializedWorkUnitCollection());
}

代码示例来源:origin: org.apache.gobblin/gobblin-sql

@Override
 public ConverterInitializer getInitializer(State state, WorkUnitStream workUnits, int branches, int branchId) {
  JdbcWriterCommandsFactory factory = new JdbcWriterCommandsFactory();
  if (workUnits.isSafeToMaterialize()) {
   return new AvroToJdbcEntryConverterInitializer(state, workUnits.getMaterializedWorkUnitCollection(),
     factory, branches, branchId);
  } else {
   throw new RuntimeException(AvroToJdbcEntryConverter.class.getName() + " does not support work unit streams.");
  }
 }
}

代码示例来源:origin: org.apache.gobblin/gobblin-sql

public WriterInitializer getInitializer(State state, WorkUnitStream workUnits, int branches, int branchId) {
  JdbcWriterCommandsFactory factory = new JdbcWriterCommandsFactory();
  if (workUnits.isSafeToMaterialize()) {
   return new JdbcWriterInitializer(state, workUnits.getMaterializedWorkUnitCollection(),
     factory, branches, branchId);
  } else {
   throw new RuntimeException(JdbcWriterBuilder.class.getName() + " does not support work unit streams.");
  }
 }
}

代码示例来源:origin: org.apache.gobblin/gobblin-runtime

Map<String, ParallelRunner> parallelRunners = Maps.newHashMap();
try {
 for (WorkUnit workUnit : JobLauncherUtils.flattenWorkUnits(workUnits.getMaterializedWorkUnitCollection())) {
  JobLauncherUtils.cleanTaskStagingData(new WorkUnitState(workUnit, jobState), LOG, closer, parallelRunners);

25 4 0
Copyright 2021 - 2024 cfsdn All Rights Reserved 蜀ICP备2022000587号
广告合作:1813099741@qq.com 6ren.com