java - 我如何阅读和打印 Lucene 索引 4.0

Question

我想从我的索引器文件中读取索引。

所以我想要的结果是每个文档的所有术语和 TF-IDF 的数量。

请为我推荐一些示例代码。谢谢 :)

Answer 1

首先是获取文档列表。另一种方法可能是遍历索引项，但方法 IndexReader.terms() 似乎已从 4.0 中删除(尽管它存在于 AtomicReader 中，这可能值得一看在)。我知道获取所有文档的最佳方法是简单地按文档 ID 遍历文档:

//where reader is your IndexReader, however you go about opening/managing it
for (int i=0; i<reader.maxDoc(); i++) {
    if (reader.isDeleted(i))
        continue;
    //operate on the document with id = i ...
}

然后您需要一个包含所有索引词的列表。我假设我们对存储字段不感兴趣，因为您想要的数据对它们没有意义。要检索术语，您可以使用 IndexReader.getTermVectors(int)。请注意，我实际上并没有检索文档，因为我们不需要直接访问它。从我们离开的地方继续:

String field;
FieldsEnum fieldsiterator;
TermsEnum termsiterator;
//To Simplify, you can rely on DefaultSimilarity to calculate tf and idf for you.
DefaultSimilarity freqcalculator = new DefaultSimilarity()
//numDocs and maxDoc are not the same thing:
int numDocs = reader.numDocs();
int maxDoc = reader.maxDoc();

for (int i=0; i<maxDoc; i++) {
    if (reader.isDeleted(i))
        continue;
    fieldsiterator = reader.getTermVectors(i).iterator();
    while (field = fieldsiterator.next()) {
        termsiterator = fieldsiterator.terms().iterator();
        while (terms.next()) {
            //id = document id, field = field name
            //String representations of the current term
            String termtext = termsiterator.term().utf8ToString();
            //Get idf, using docfreq from the reader.
            //I haven't tested this, and I'm not quite 100% sure of the context of this method.
            //If it doesn't work, idfalternate below should.
            int idf = termsiterator.docfreq();
            int idfalternate = freqcalculator.idf(reader.docFreq(field, termsiterator.term()), numDocs);
        }
    }
}