gpt4 book ai didi

lucene.net - 使用 lucene.net 搜索时如何启用词干提取?

转载 作者:行者123 更新时间:2023-12-03 20:07:53 24 4
gpt4 key购买 nike

使用 lucene.net 搜索时如何启用词干提取?

最佳答案

为此,您需要编写自己的分析器类。这是相对简单的。这是我正在使用的一种。它结合了停用词过滤。 Porter 词干和(这可能对您的需求来说太多了)从字符中剥离重音。

/// <summary>
/// An analyzer that implements a number of filters. Including porter stemming,
/// Diacritic stripping, and stop word filtering.
/// </summary>
public class CustomAnalyzer : Analyzer
{
/// <summary>
/// A rather short list of stop words that is fine for basic search use.
/// </summary>
private static readonly string[] stopWords = new[]
{
"0", "1", "2", "3", "4", "5", "6", "7", "8",
"9", "000", "$", "£",
"about", "after", "all", "also", "an", "and",
"another", "any", "are", "as", "at", "be",
"because", "been", "before", "being", "between",
"both", "but", "by", "came", "can", "come",
"could", "did", "do", "does", "each", "else",
"for", "from", "get", "got", "has", "had",
"he", "have", "her", "here", "him", "himself",
"his", "how","if", "in", "into", "is", "it",
"its", "just", "like", "make", "many", "me",
"might", "more", "most", "much", "must", "my",
"never", "now", "of", "on", "only", "or",
"other", "our", "out", "over", "re", "said",
"same", "see", "should", "since", "so", "some",
"still", "such", "take", "than", "that", "the",
"their", "them", "then", "there", "these",
"they", "this", "those", "through", "to", "too",
"under", "up", "use", "very", "want", "was",
"way", "we", "well", "were", "what", "when",
"where", "which", "while", "who", "will",
"with", "would", "you", "your",
"a", "b", "c", "d", "e", "f", "g", "h", "i",
"j", "k", "l", "m", "n", "o", "p", "q", "r",
"s", "t", "u", "v", "w", "x", "y", "z"
};

private Hashtable stopTable;

/// <summary>
/// Creates an analyzer with the default stop word list.
/// </summary>
public CustomAnalyzer() : this(stopWords) {}

/// <summary>
/// Creates an analyzer with the passed in stop words list.
/// </summary>
public CustomAnalyzer(string[] stopWords)
{
stopTable = StopFilter.MakeStopSet(stopWords);
}

public override TokenStream TokenStream(string fieldName, System.IO.TextReader reader)
{
return new PorterStemFilter(new ISOLatin1AccentFilter(new StopFilter(new LowerCaseTokenizer(reader), stopWords)));
}
}

关于lucene.net - 使用 lucene.net 搜索时如何启用词干提取?,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/1192671/

24 4 0
Copyright 2021 - 2024 cfsdn All Rights Reserved 蜀ICP备2022000587号
广告合作:1813099741@qq.com 6ren.com