python - FileNotFoundException :File file:/path/to/file/in. txt不存在或者运行Flink的用户没有足够的权限访问它

Question

我正在尝试使用 flink 和 python 批处理 api 测试 Wordcount 经典示例。我的问题是，将数据源从 env.from_elements() 修改为 env.read_text() (对于更大的测试用例)后，会发生错误。以下代码描述了我的实现。

[...]
if __name__ == "__main__":
env = get_environment()
input_file = 'file:///workfile.txt/'

if len(sys.argv) != 1 and len(sys.argv) != 3:
    sys.exit("Usage: ./bin/pyflink.sh WordCount[ - <text path> <result path>]")

if len(sys.argv) == 3:
    data = env.read_text(sys.argv[1])
else:
    #data = env.from_elements("hello","world","hello","car","tree","data","hello","car","tree","data","hello","car","tree","data","hello","car","tree","data","hello","car","tree","data","hello","car","tree","data","hello","car","tree","data","hello","car","tree","data","hello","car","tree","data","hello","car","tree","data","hello","car","tree","data","hello","car","tree","data","hello","car","tree","data","hello","car","tree","data","hello","car","tree","data","hello","car","tree","data","hello","car","tree","data","hello","car","tree","data","hello","car","tree","data","hello","car","tree","data","hello","car","tree","data","hello","car","tree","data","hello","car","tree","data","hello","car","tree","data","hello","car","tree","data","hello","car","tree","data","hello","car","tree","data","hello","car","tree","data","hello","car","tree","data","hello","car","tree","data","hello")
    data = env.read_text(input_file)

result = data \
    .flat_map(Tokenizer()) \
    .group_by(1) \
    .reduce_group(Adder(), combinable=True) \

if len(sys.argv) == 3:
    result.write_csv(sys.argv[2])
else:
    result.output()
[...]

执行上面的代码，抛出文件权限错误。更具体地说，以下消息

原因:org.apache.flink.runtime.JobException:创建输入拆分导致错误:文件 file:/workfile.txt 不存在或运行 Flink 的用户('user')没有足够的权限访问它.

PS:寻找解决方案，但找不到任何东西。如果这个问题已经解决，我将不胜感激重定向。

Answer 1

我假设“workfile.txt”应该是相对路径。但是，您不能拥有具有方案(“file:///”)的相关文件。

请提供完整的绝对路径，它应该可以工作。

请注意，当我们在临时位置执行脚本时，相对路径通常不适用于 Python API。