hadoop - 无法删除 HDFS 损坏的文件

Question

我无法删除我的 HDFS 中存在的损坏文件。 Namenode 已进入安全模式。区 block 总数为 980，其中 978 已报告。当我运行以下命令时，

sudo -u hdfs hdfs dfsadmin -report

生成的报告是，

Safe mode is ON
Configured Capacity: 58531520512 (54.51 GB)
Present Capacity: 35774078976 (33.32 GB)
DFS Remaining: 32374509568 (30.15 GB)
DFS Used: 3399569408 (3.17 GB)
DFS Used%: 9.50%
Under replicated blocks: 0
Blocks with corrupt replicas: 0
Missing blocks: 0
Missing blocks (with replication factor 1): 0
-------------------------------------------------
Live datanodes (1):

Name: 10.0.2.15:50010 (quickstart.cloudera)
Hostname: quickstart.cloudera
Decommission Status : Normal
Configured Capacity: 58531520512 (54.51 GB)
DFS Used: 3399569408 (3.17 GB)
Non DFS Used: 19777388544 (18.42 GB)
DFS Remaining: 32374509568 (30.15 GB)
DFS Used%: 5.81%
DFS Remaining%: 55.31%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 2
Last contact: Tue Nov 14 10:39:58 IST 2017

而对于下面的命令执行时，

sudo -u hdfs hdfs fsck /

输出是，

Connecting to namenode via http://quickstart.cloudera:50070/fsck?ugi=hdfs&path=%2F
FSCK started by hdfs (auth:SIMPLE) from /10.0.2.15 for path / at Tue Nov 14 10:41:25 IST 2017
/hbase/oldWALs/quickstart.cloudera%2C60020%2C1509698296866.default.1509701903728: CORRUPT blockpool BP-1914853243-127.0.0.1-1500467607052 block blk_1073743141

/hbase/oldWALs/quickstart.cloudera%2C60020%2C1509698296866.default.1509701903728: MISSING 1 blocks of total size 83 B..
/hbase/oldWALs/quickstart.cloudera%2C60020%2C1509698296866.meta.1509701932269.meta: CORRUPT blockpool BP-1914853243-127.0.0.1-1500467607052 block blk_1073743142

/hbase/oldWALs/quickstart.cloudera%2C60020%2C1509698296866.meta.1509701932269.meta: MISSING 1 blocks of total size 83 B
Status: CORRUPT
Total size: 3368384392 B (Total open files size: 166 B)
Total dirs: 286
Total files:    966
Total symlinks:     0 (Files currently being written: 3)
Total blocks (validated):   980 (avg. block size 3437126 B) (Total open file blocks (not validated): 2)
********************************
UNDER MIN REPL'D BLOCKS:    2 (0.20408164 %)
dfs.namenode.replication.min:   1
CORRUPT FILES:  2
MISSING BLOCKS: 2
MISSING SIZE:       166 B
CORRUPT BLOCKS:     2
********************************
Minimally replicated blocks:    978 (99.79592 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks:    0 (0.0 %)
Mis-replicated blocks:      0 (0.0 %)
Default replication factor: 1
Average block replication:  0.9979592
Corrupt blocks:     2
Missing replicas:       0 (0.0 %)
Number of data-nodes:       1
Number of racks:        1
FSCK ended at Tue Nov 14 10:41:26 IST 2017 in 774 milliseconds
The filesystem under path '/' is CORRUPT

任何人都可以帮助纠正损坏的 block ，(或)删除它们吗？提前致谢。

Answer 1

据说Namenode处于安全模式，先把它关掉。

hdfs dfsadmin -safemode leave

然后执行其中一个命令

hdfs fsck / | egrep -v '^\.+$' | grep -v replica | grep -v Replica

或

hdfs fsck hdfs://quickstart.cloudera:50070/ | egrep -v '^\.+$' | grep -v replica | grep -v Replica

输出会有点类似于

/path/to/filename.fileextension: CORRUPT blockpool BP-1016133662-10.29.100.41-1415825958975 block blk_1073904305

/path/to/filename.fileextension: MISSING 1 blocks of total size 15620361 B

在您的情况下，已列出损坏的文件。所以执行下面的命令

hdfs dfs -rm /hbase/oldWALs/quickstart.cloudera%2C60020%2C1509698296866.default.1509701903728
hdfs dfs -rm /hbase/oldWALs/quickstart.cloudera%2C60020%2C1509698296866.meta.1509701932269.meta

并且不要进入安全模式。继续工作。Yippee!!