hadoop - 使用 FILE channel 配置 flume 的多个源时出现 channel 锁定错误

Question

为代理配置多个源，使用 FILE channel 抛出锁定错误。下面是我的配置文件。

a1.sources = r1 r2
a1.sinks = k1 k2
a1.channels = c1 c3

#sources
a1.sources.r1.type=netcat
a1.sources.r1.bind=localhost
a1.sources.r1.port=4444

a1.sources.r2.type=exec
a1.sources.r2.command=tail -f /opt/gen_logs/logs/access.log

#sinks
a1.sinks.k1.type=hdfs
a1.sinks.k1.hdfs.path=/flume201
a1.sinks.k1.hdfs.filePrefix=netcat-
a1.sinks.k1.rollInterval=100
a1.sinks.k1.hdfs.fileType=DataStream
a1.sinks.k1.hdfs.callTimeout=100000


a1.sinks.k2.type=hdfs
a1.sinks.k2.hdfs.path=/flume202
a1.sinks.k2.hdfs.filePefix=execCommand-
a1.sinks.k2.rollInterval=100
a1.sinks.k2.hdfs.fileType=DataStream
a1.sinks.k2.hdfs.callTimeOut=100000

#channels
a1.channels.c1.type=file
a1.channels.c1.checkpointDir=/home/cloudera/alpha/001
a1.channels.c3.type=file
a1.channels.c3.checkpointDir=/home/cloudera/beta/001

#bind r1 c1 k1
a1.sources.r1.channels=c1
a1.sinks.k1.channel=c1

a1.sources.r2.channels=c3
a1.sinks.k2.channel=c3

我遇到了错误

Channel closed [channel=c3]. Due to java.io.IOException: Cannot lock /home/cloudera/.flume/file-channel/data. The directory is already locked. [channel=c3]

但是当我使用内存 channel 时。它工作正常。

Answer 1

我的猜测是您需要为每个 channel 指定不同的 dataDirs 属性，因为它们都使用默认值