java - 生成 Lucene segments_N 文件

Question

在将 Lucene 索引文件从一个服务器移动到另一个服务器时，我忘记移动 segments_N 文件(因为我使用了模式 *.*)

不幸的是，我已经删除了原始文件夹，现在我的目录中只有这些文件:

_1rpt.fdt
_1rpt.fdx
_1rpt.fnm
_1rpt.nvd
_1rpt.nvm
_1rpt.si
_1rpt_Lucene50_0.doc
_1rpt_Lucene50_0.dvd
_1rpt_Lucene50_0.dvm
_1rpt_Lucene50_0.pos
_1rpt_Lucene50_0.tim
_1rpt_Lucene50_0.tip
write.lock

我缺少 segments_42u 文件，没有它我什至无法执行 org.apache.lucene.index.CheckIndex :

Exception in thread "main" org.apache.lucene.index.IndexNotFoundException: no segments* file found in MMapDirectory@/solr-5.3.1/nodes/node1/core/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@119d7047: files: [write.lock, _1rpt.fdt, _1rpt.fdx, _1rpt.fnm, _1rpt.nvd, _1rpt.nvm, _1rpt.si, _1rpt_Lucene50_0.doc, _1rpt_Lucene50_0.dvd, _1rpt_Lucene50_0.dvm, _1rpt_Lucene50_0.pos, _1rpt_Lucene50_0.tim, _1rpt_Lucene50_0.tip]
at org.apache.lucene.index.CheckIndex.checkIndex(CheckIndex.java:483)
at org.apache.lucene.index.CheckIndex.doMain(CheckIndex.java:2354)
at org.apache.lucene.index.CheckIndex.main(CheckIndex.java:2237)

索引非常大(> 800GB)，重建它需要数周时间。

有没有办法生成这个丢失的段信息文件？

非常感谢您的帮助。

Answer 1

添加了无需调试即可在 Lucene62 中查找 segmentID 的自动化:

package org.apache.lucene.index;

import java.io.IOException;
import java.nio.file.Paths;

import org.apache.lucene.codecs.Codec;
import org.apache.lucene.codecs.lucene62.Lucene62SegmentInfoFormat;
import org.apache.lucene.store.BufferedChecksumIndexInput;
import org.apache.lucene.store.ChecksumIndexInput;
import org.apache.lucene.store.DataInput;
import org.apache.lucene.store.Directory;
import org.apache.lucene.store.IOContext;
import org.apache.lucene.store.SimpleFSDirectory;
import org.apache.lucene.util.StringHelper;

public class GenSegmentInfo {
    public static void main(String[] args) throws IOException {
        if (args.length < 2) {
            help();
            System.exit(1);
        }
        Codec codec = Codec.getDefault();
        Directory directory = new SimpleFSDirectory(Paths.get(args[0]));

        SegmentInfos infos = new SegmentInfos();
        for (int i = 1; i < args.length; i++) {
            infos.add(getSegmentCommitInfo6(codec, directory, args[i]));
        }
        infos.commit(directory);
    }

    private static SegmentCommitInfo getSegmentCommitInfo(Codec codec, Directory directory, String segmentName) throws IOException {
        byte[] segmentID = new byte[StringHelper.ID_LENGTH];
        final String fileName = IndexFileNames.segmentFileName(segmentName, "", Lucene62SegmentInfoFormat.SI_EXTENSION);
        ChecksumIndexInput input = directory.openChecksumInput(fileName, IOContext.READ);
        DataInput in = new BufferedChecksumIndexInput(input);

        final int actualHeader = in.readInt();
        final String actualCodec = in.readString();
        final int actualVersion = in.readInt();
        in.readBytes(segmentID, 0, segmentID.length);

        SegmentInfo info = codec.segmentInfoFormat().read(directory, segmentName, segmentID, IOContext.READ);

        info.setCodec(codec);
        return new SegmentCommitInfo(info, 1, -1, -1, -1);
    }

    private static void help() {
        System.out.println("Not enough arguments");
        System.out.println("Usage: java -cp lucene-core-6.6.0.jar GenSegmentInfo <path to index> [segment1 [segment2 ...] ]");
    }
}

要使其在 Lucene410 库下工作，必须调整以下部分代码，因为库的工作方式不同:

• SimpleFSDirectory 需要文件而不是路径
• 没有checkIndexHeaderID函数，不需要segmentID
• SegmentInfo 可从 codec.segmentInfoFormat().getSegmentInfoReader().read(directory, segmentName, IOContext.READ) 获得