gpt4 book ai didi

hadoop - pig 脚本不存在错误,即使我可以在hdfs中看到它

转载 作者:行者123 更新时间:2023-12-02 21:24:51 25 4
gpt4 key购买 nike

我正在尝试使用-f usecatalog选项运行Pig脚本,但这给了我一个问题。
它说脚本不存在,虽然我可以看到该文件存在于hdfs文件系统中。见下文。

[hdfs@ip-xx-xx-xx-x-xx ec2-user]$ pig -useHCatalog -f   /user/admin/pig/scripts/hcat1.pig  
WARNING: Use "yarn jar" to launch YARN applications.
16/04/01 13:44:13 INFO pig.ExecTypeProvider: Trying ExecType : LOCAL
16/04/01 13:44:13 INFO pig.ExecTypeProvider: Trying ExecType : MAPREDUCE
16/04/01 13:44:13 INFO pig.ExecTypeProvider: Picked MAPREDUCE as the ExecType
2016-04-01 13:44:13,645 [main] INFO org.apache.pig.Main - Apache Pig version 0.15.0.2.3.4.0-3485 (rexported) compiled Dec 16 20 15, 04:30:33
2016-04-01 13:44:13,645 [main] INFO org.apache.pig.Main - Logging error messages to: /tmp/hsperfdata_hdfs/pig_1459532653643.log
2016-04-01 13:44:14,184 [main] ERROR org.apache.pig.Main - ERROR 2997: Encountered IOException. File /user/admin/pig/scripts/hca t1.pig does not exist
Details at logfile: /tmp/hsperfdata_hdfs/pig_1459532653643.log
2016-04-01 13:44:14,203 [main] INFO org.apache.pig.Main - Pig script completed in 753 milliseconds (753 ms)

[hdfs@ip-xxx-xx-xx-xx ec2-user]$ hadoop fs -cat /user/admin/pig/scripts/hcat1.pig
a = load 'trucks' using org.apache.hive.hcatalog.pig.HCatLoader();
b = filter a by truckid == 'A1';
store b INTO '/user/admin/pig/scritps/outputb1';

最佳答案

您需要指定完整的HDFS URI才能运行HDFS中存储的脚本。

这是您需要的:

$pig -useHCatalog hdfs://namenode_hostname:port/user/admin/pig/scripts/hcat1.pig 

关于hadoop - pig 脚本不存在错误,即使我可以在hdfs中看到它,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/36362840/

25 4 0
Copyright 2021 - 2024 cfsdn All Rights Reserved 蜀ICP备2022000587号
广告合作:1813099741@qq.com 6ren.com