gpt4 book ai didi

hadoop - 有没有一种方法可以激活为 yarn 制造的giraph中的Giraph Stats?

转载 作者:行者123 更新时间:2023-12-02 20:33:26 27 4
gpt4 key购买 nike

看来Giraph Stas仅使用map-reduce(使用-Phadoop2 mvn配置文件构建的giraph-1.3.0-snapshot)写入日志中。

是否也有一种方法可以使用yarn激活日志中的Giraph Stats(使用-Phadoop_yarn mvn配置文件构建的giraph-1.3.0-snapshot)?

遵循Giraph统计资料的示例。

2018-09-08 16:03:01,604 INFO org.apache.hadoop.mapreduce.Job: Counters:   42
File System Counters
FILE: Number of bytes read=74623213
FILE: Number of bytes written=75518013
FILE: Number of read operations=0
FILE: Number of large read operations=0
FILE: Number of write operations=0
Map-Reduce Framework
Map input records=1
Map output records=0
Input split bytes=44
Spilled Records=0
Failed Shuffles=0
Merged Map outputs=0
GC time elapsed (ms)=100
CPU time spent (ms)=0
Physical memory (bytes) snapshot=0
Virtual memory (bytes) snapshot=0
Total committed heap usage (bytes)=3087007744
Giraph Stats
Aggregate bytes loaded from local disks (out-of-core)=0
Aggregate bytes stored to local disks (out-of-core)=0
Aggregate edges=12
Aggregate finished vertices=0
Aggregate sent message bytes=3154
Aggregate sent messages=24
Aggregate vertices=5
Current master task partition=0
Current workers=1
Last checkpointed superstep=0
Lowest percentage of graph in memory so far (out-of-core)=100
Sent message bytes=1701
Sent messages=12
Superstep=2
Giraph Timers
Initialize (ms)=653
Input superstep (ms)=599
Setup (ms)=58
Shutdown (ms)=8967
Superstep 0 Worker_Superstep0 (ms)=295
Superstep 1 Worker_Superstep1 (ms)=605
Total (ms)=10538
Zookeeper base path
/_hadoopBsp/job_local881103217_0001=0
Zookeeper halt node
/_hadoopBsp/job_local881103217_0001/_haltComputation=0
Zookeeper server:port
cri:22181=0
File Input Format Counters
Bytes Read=0
File Output Format Counters
Bytes Written=0

最佳答案

使用

-ca giraph.metrics.enable=true

http://giraph.apache.org/options.html中所述

关于hadoop - 有没有一种方法可以激活为 yarn 制造的giraph中的Giraph Stats?,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/52236943/

27 4 0
Copyright 2021 - 2024 cfsdn All Rights Reserved 蜀ICP备2022000587号
广告合作:1813099741@qq.com 6ren.com