gpt4 book ai didi

hadoop - Pig UDF 计算两个数的幂

转载 作者:可可西里 更新时间:2023-11-01 16:57:10 24 4
gpt4 key购买 nike

我有一个 pig 脚本。

Script.pig:

register /home/cloudera/Desktop/Pow.jar # registering the jar file
A = LOAD '/input.txt' using PigStorage(',') as (a1:int,a2:int,name:chararray); # loading the relation
B = foreach A generate A.a1,A.a2,Pow(A.a1,A.a2); # just generating field1,field2
dump B;# dumping the result

用于计算幂函数的 java UDF。

import java.io.IOException;

import org.apache.pig.EvalFunc;
import org.apache.pig.PigWarning;
import org.apache.pig.data.Tuple;

// Pow function to calculate the power of two numbers
public class Pow extends EvalFunc<Long> {

public Long exec(Tuple input) throws IOException {
try {

int base = (Integer)input.get(0);# Getting the base value from tuple.
int exponent = (Integer)input.get(1);# Getting the second value from tuple.
long result = 1;

/* Probably not the most efficient method...*/
for (int i = 0; i < exponent; i++) {
long preresult = result;
result *= base;
if (preresult > result) {
// We overflowed. Give a warning, but do not throw an
// exception.
warn("Overflow!", PigWarning.TOO_LARGE_FOR_INT);
// Returning null will indicate to Pig that we failed but
// we want to continue execution.
return null;
}
}
return result;
} catch (Exception e) {
// Throwing an exception will cause the task to fail.
throw new IOException("Something bad happened!", e);
}
}


}

输入文件

Input.txt
1,2,Vijay
3,4,Ram

运行脚本时出现以下错误

 ERROR 1066: Unable to open iterator for alias 

B. Backend error : Scalar has more than one row in the output.

1st : (1,2,Vijay), 2nd :(3,4,Ram)# Error at this point

请帮助我解决这个问题,我也是 Apache pig 的新手。

最佳答案

B = foreach A generate A.a1,A.a2,Pow(A.a1,A.a2);

应该是

B = foreach A generate a1,a2,Pow(a1,a2);

关于hadoop - Pig UDF 计算两个数的幂,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/28507278/

24 4 0
Copyright 2021 - 2024 cfsdn All Rights Reserved 蜀ICP备2022000587号
广告合作:1813099741@qq.com 6ren.com