gpt4 book ai didi

gobblin.source.workunit.WorkUnit.setId()方法的使用及代码示例

转载 作者:知者 更新时间:2024-03-21 08:11:05 29 4
gpt4 key购买 nike

本文整理了Java中gobblin.source.workunit.WorkUnit.setId()方法的一些代码示例,展示了WorkUnit.setId()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。WorkUnit.setId()方法的具体详情如下:
包路径:gobblin.source.workunit.WorkUnit
类名称:WorkUnit
方法名:setId

WorkUnit.setId介绍

暂无

代码示例

代码示例来源:origin: com.linkedin.gobblin/gobblin-runtime

@Override
 protected void forWorkUnit(WorkUnit workUnit) {
  workUnit.setProp(ConfigurationKeys.JOB_ID_KEY, this.jobId);
  String taskId = JobLauncherUtils.newTaskId(this.jobId, this.taskIdSequence++);
  workUnit.setId(taskId);
  workUnit.setProp(ConfigurationKeys.TASK_ID_KEY, taskId);
  workUnit.setProp(ConfigurationKeys.TASK_KEY_KEY, Long.toString(Id.Task.parse(taskId).getSequence()));
 }
}

代码示例来源:origin: com.linkedin.gobblin/gobblin-cluster

/**
 * Create a job from a given batch of {@link WorkUnit}s.
 */
private JobConfig.Builder createJob(List<WorkUnit> workUnits) throws IOException {
 Map<String, TaskConfig> taskConfigMap = Maps.newHashMap();
 try (ParallelRunner stateSerDeRunner = new ParallelRunner(this.stateSerDeRunnerThreads, this.fs)) {
  int multiTaskIdSequence = 0;
  for (WorkUnit workUnit : workUnits) {
   if (workUnit instanceof MultiWorkUnit) {
    workUnit.setId(JobLauncherUtils.newMultiTaskId(this.jobContext.getJobId(), multiTaskIdSequence++));
   }
   addWorkUnit(workUnit, stateSerDeRunner, taskConfigMap);
  }
  Path jobStateFilePath = new Path(this.appWorkDir, this.jobContext.getJobId() + "." + JOB_STATE_FILE_NAME);
  SerializationUtils.serializeState(this.fs, jobStateFilePath, this.jobContext.getJobState());
 }
 JobConfig.Builder jobConfigBuilder = new JobConfig.Builder();
 jobConfigBuilder.setMaxAttemptsPerTask(this.jobContext.getJobState().getPropAsInt(
   ConfigurationKeys.MAX_TASK_RETRIES_KEY, ConfigurationKeys.DEFAULT_MAX_TASK_RETRIES));
 jobConfigBuilder.setFailureThreshold(workUnits.size());
 jobConfigBuilder.addTaskConfigMap(taskConfigMap).setCommand(GobblinTaskRunner.GOBBLIN_TASK_FACTORY_NAME);
 jobConfigBuilder.setNumConcurrentTasksPerInstance(ConfigUtils.getInt(jobConfig,
   GobblinClusterConfigurationKeys.HELIX_CLUSTER_TASK_CONCURRENCY,
   GobblinClusterConfigurationKeys.HELIX_CLUSTER_TASK_CONCURRENCY_DEFAULT));
 return jobConfigBuilder;
}

29 4 0
Copyright 2021 - 2024 cfsdn All Rights Reserved 蜀ICP备2022000587号
广告合作:1813099741@qq.com 6ren.com