hadoop - hadoop中具有复杂类型的ReadField

Question

我有这个类(class):

public class Stripe implements WritableComparable<Stripe>{
    private List<Term> occorrenze = new ArrayList<Term>();

    public Stripe(){}

    @Override
    public void readFields(DataInput in) throws IOException {

    }
}


public class Term implements WritableComparable<Term> {

    private Text key;
    private IntWritable frequency;

    @Override
    public void readFields(DataInput in) throws IOException {
        this.key.readFields(in);
        this.frequency.readFields(in);
    }

Stripe是术语列表(一对文本和intWritable)。
如何设置方法“readField”以从DataInput读取复杂类型Stripe？

Answer 1

要序列化列表，您需要写出列表的长度，然后写出元素本身。 Stripe的简单readFields / write方法对可以是:

@Override
public void readFields(DataInput in) throws IOException {
    occorrenze.clear();
    int cnt = in.readInt();
    for (int x = 0; x < cnt; x++) {
        Term term = new Term();
        term.readFields(in);
        occorrence.add(term);
    }
}

@Override
public void write(DataOutput out) throws IOException {
    out.writeInt(occorrenze.size());
    for (Term term : occorrenze) {
        term.write(out);
    }
}

通过使用VInt而不是int，以及可以在 readFields方法中重新使用以节省对象创建/垃圾收集的术语池，可以使此方法更加有效。