gpt4 book ai didi

org.apache.samza.validation.YarnJobValidationTool类的使用及代码示例

转载 作者:知者 更新时间:2024-03-19 01:19:31 25 4
gpt4 key购买 nike

本文整理了Java中org.apache.samza.validation.YarnJobValidationTool类的一些代码示例,展示了YarnJobValidationTool类的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。YarnJobValidationTool类的具体详情如下:
包路径:org.apache.samza.validation.YarnJobValidationTool
类名称:YarnJobValidationTool

YarnJobValidationTool介绍

[英]Command-line tool for validating the status of a Yarn job. It checks the job has been successfully submitted to the Yarn cluster, the status of the application attempt is running and the running container count matches the expectation. It also supports an optional MetricsValidator plugin through arguments so job metrics can be validated too using JMX. This tool can be used, for example, as an automated validation step after starting a job. When running this tool, please provide the configuration URI of job. For example: deploy/samza/bin/validate-yarn-job.sh --config-factory=org.apache.samza.config.factories.PropertiesConfigFactory --config-path=file://$PWD/deploy/samza/config/wikipedia-feed.properties [--metrics-validator=com.foo.bar.SomeMetricsValidator] The tool prints out the validation result in each step and throws an exception when the validation fails.
[中]用于验证纱线作业状态的命令行工具。它检查作业是否已成功提交到纱线群集,应用程序尝试的状态是否为正在运行,以及正在运行的容器计数是否符合预期。它还通过参数支持可选的MetricsValidator插件,因此也可以使用JMX验证作业指标。例如,该工具可以在开始作业后用作自动验证步骤。运行此工具时,请提供作业的配置URI。例如:deploy/samza/bin/validate-job。sh--config factory=org。阿帕奇。萨姆萨。配置。工厂。PropertiesConfigFactory--config path=file://$PWD/deploy/samza/config/wikipedia提要。属性[--metrics validator=com.foo.bar.SomeMetricsValidator]该工具在每个步骤中打印出验证结果,并在验证失败时抛出异常。

代码示例

代码示例来源:origin: org.apache.samza/samza-yarn

public void run() {
 ApplicationId appId;
 ApplicationAttemptId attemptId;
 try {
  log.info("Start validating job " + this.jobName);
  appId = validateAppId();
  attemptId = validateRunningAttemptId(appId);
  validateContainerCount(attemptId);
  if(validator != null) {
   validateJmxMetrics();
  }
  log.info("End of validation");
 } catch (Exception e) {
  log.error(e.getMessage(), e);
  System.exit(1);
 }
}

代码示例来源:origin: org.apache.samza/samza-yarn

public static void main(String [] args) throws Exception {
  CommandLine cmdline = new CommandLine();
  OptionParser parser = cmdline.parser();
  OptionSpec<String> validatorOpt = parser.accepts("metrics-validator", "The metrics validator class.")
                      .withOptionalArg()
                      .ofType(String.class).describedAs("com.foo.bar.ClassName");
  OptionSet options = cmdline.parser().parse(args);
  Config config = cmdline.loadConfig(options);
  MetricsValidator validator = null;
  if (options.has(validatorOpt)) {
   String validatorClass = options.valueOf(validatorOpt);
   validator = Util.getObj(validatorClass, MetricsValidator.class);
  }

  YarnConfiguration hadoopConfig = new YarnConfiguration();
  hadoopConfig.set("fs.http.impl", HttpFileSystem.class.getName());
  hadoopConfig.set("fs.https.impl", HttpFileSystem.class.getName());
  ClientHelper clientHelper = new ClientHelper(hadoopConfig);

  new YarnJobValidationTool(new JobConfig(config), clientHelper.yarnClient(), validator).run();
 }
}

代码示例来源:origin: org.apache.samza/samza-yarn_2.11

public static void main(String [] args) throws Exception {
  CommandLine cmdline = new CommandLine();
  OptionParser parser = cmdline.parser();
  OptionSpec<String> validatorOpt = parser.accepts("metrics-validator", "The metrics validator class.")
                      .withOptionalArg()
                      .ofType(String.class).describedAs("com.foo.bar.ClassName");
  OptionSet options = cmdline.parser().parse(args);
  Config config = cmdline.loadConfig(options);
  MetricsValidator validator = null;
  if (options.has(validatorOpt)) {
   String validatorClass = options.valueOf(validatorOpt);
   validator = Util.getObj(validatorClass, MetricsValidator.class);
  }

  YarnConfiguration hadoopConfig = new YarnConfiguration();
  hadoopConfig.set("fs.http.impl", HttpFileSystem.class.getName());
  hadoopConfig.set("fs.https.impl", HttpFileSystem.class.getName());
  ClientHelper clientHelper = new ClientHelper(hadoopConfig);

  new YarnJobValidationTool(new JobConfig(config), clientHelper.yarnClient(), validator).run();
 }
}

代码示例来源:origin: org.apache.samza/samza-yarn_2.11

public void run() {
 ApplicationId appId;
 ApplicationAttemptId attemptId;
 try {
  log.info("Start validating job " + this.jobName);
  appId = validateAppId();
  attemptId = validateRunningAttemptId(appId);
  validateContainerCount(attemptId);
  if(validator != null) {
   validateJmxMetrics();
  }
  log.info("End of validation");
 } catch (Exception e) {
  log.error(e.getMessage(), e);
  System.exit(1);
 }
}

25 4 0
Copyright 2021 - 2024 cfsdn All Rights Reserved 蜀ICP备2022000587号
广告合作:1813099741@qq.com 6ren.com