Solr 搜索主题标签或提及

Question

我们使用 solr 3.5 版来搜索推文，我使用的是 WordDelimiterFactory使用以下设置，可以搜索@username或 #hashtags :

<filter class="solr.WordDelimiterFilterFactory" generateWordParts="1" generateNumberParts="1" catenateWords="1" catenateNumbers="1" catenateAll="0" splitOnCaseChange="0" splitOnNumerics="0" preserveOriginal="1" handleAsChar="@#"/>

我看到了以下补丁，但这似乎不像我预期的那样工作，我错过了什么吗？

https://issues.apache.org/jira/browse/SOLR-2059

但是搜索 @username也只返回用户名或 #hashtag 的结果只是返回hastag的结果。我怎样才能做到这一点？

整个字段类型:

<fieldType name="textnostem" class="solr.TextField" positionIncrementGap="100" autoGeneratePhraseQueries="true">
  <analyzer type="index">
    <charFilter class="solr.HTMLStripCharFilterFactory"/>
    <tokenizer class="solr.WhitespaceTokenizerFactory"/>
    <filter class="solr.StopFilterFactory"
            ignoreCase="true"
            words="stopwords.txt"
            enablePositionIncrements="true"
            />
    <filter class="solr.WordDelimiterFilterFactory" 
            generateWordParts="1" 
            generateNumberParts="1" 
            catenateWords="1" 
            catenateNumbers="1" 
            catenateAll="0" 
            splitOnCaseChange="0" 
            splitOnNumerics="0"
            preserveOriginal="1"
            />
    <filter class="solr.LowerCaseFilterFactory"/>
  </analyzer>
  <analyzer type="query">
    <tokenizer class="solr.WhitespaceTokenizerFactory"/>
    <filter class="solr.StopFilterFactory"
            ignoreCase="true"
            words="stopwords.txt"
            enablePositionIncrements="true"
            />
    <filter class="solr.WordDelimiterFilterFactory" 
            generateWordParts="1" 
            generateNumberParts="1" 
            catenateWords="1" 
            catenateNumbers="1" 
            catenateAll="0" 
            splitOnCaseChange="0" 
            splitOnNumerics="0"
            preserveOriginal="1"/>
    <filter class="solr.LowerCaseFilterFactory"/>
  </analyzer>

<fieldType name="textnostem" class="solr.TextField" positionIncrementGap="100" autoGeneratePhraseQueries="true">
  <analyzer type="index">
    <charFilter class="solr.HTMLStripCharFilterFactory"/>
    <tokenizer class="solr.WhitespaceTokenizerFactory"/>
    <filter class="solr.StopFilterFactory"
            ignoreCase="true"
            words="stopwords.txt"
            enablePositionIncrements="true"
            />
    <filter class="solr.WordDelimiterFilterFactory" 
            generateWordParts="1" 
            generateNumberParts="1" 
            catenateWords="1" 
            catenateNumbers="1" 
            catenateAll="0" 
            splitOnCaseChange="0" 
            splitOnNumerics="0"
            preserveOriginal="1"
            />
    <filter class="solr.LowerCaseFilterFactory"/>
  </analyzer>
  <analyzer type="query">
    <tokenizer class="solr.WhitespaceTokenizerFactory"/>
    <filter class="solr.StopFilterFactory"
            ignoreCase="true"
            words="stopwords.txt"
            enablePositionIncrements="true"
            />
    <filter class="solr.WordDelimiterFilterFactory" 
            generateWordParts="1" 
            generateNumberParts="1" 
            catenateWords="1" 
            catenateNumbers="1" 
            catenateAll="0" 
            splitOnCaseChange="0" 
            splitOnNumerics="0"
            preserveOriginal="1"/>      
    <filter class="solr.LowerCaseFilterFactory"/>
  </analyzer>
</fieldType>

<fieldType name="textnostem" class="solr.TextField" positionIncrementGap="100" autoGeneratePhraseQueries="true">
  <analyzer type="index">
    <charFilter class="solr.HTMLStripCharFilterFactory"/>
    <tokenizer class="solr.WhitespaceTokenizerFactory"/>
    <filter class="solr.StopFilterFactory"
            ignoreCase="true"
            words="stopwords.txt"
            enablePositionIncrements="true"
            />
    <filter class="solr.WordDelimiterFilterFactory" 
            generateWordParts="1" 
            generateNumberParts="1" 
            catenateWords="1" 
            catenateNumbers="1" 
            catenateAll="0" 
            splitOnCaseChange="0" 
            splitOnNumerics="0"
            preserveOriginal="1"
            handleAsChar="@#"
            />
    <filter class="solr.LowerCaseFilterFactory"/>
  </analyzer>
  <analyzer type="query">
    <tokenizer class="solr.WhitespaceTokenizerFactory"/>
    <filter class="solr.StopFilterFactory"
            ignoreCase="true"
            words="stopwords.txt"
            enablePositionIncrements="true"
            />
    <filter class="solr.WordDelimiterFilterFactory" 
            generateWordParts="1" 
            generateNumberParts="1" 
            catenateWords="1" 
            catenateNumbers="1" 
            catenateAll="0" 
            splitOnCaseChange="0" 
            splitOnNumerics="0"
            preserveOriginal="1"
            handleAsChar="@#"
            />      
    <filter class="solr.LowerCaseFilterFactory"/>
  </analyzer>
</fieldType>

Answer 1

好的，所以通读你提到的 SOLR-2059 补丁，看起来他们已经取代了 handleAsChar WordDelimiterFactory 上的属性与 types属性。以下是来自 Analyzers、Tokenizers 和 Token Filters Solr Wiki 页面的该属性的规范:

types="wdfftypes.txt" allows customized tokenization for this filter. The file should exist in the solr/conf directory, and entries are of the form (without quotes) "% => ALPHA" or "\u002C => DIGIT". Allowable types are: LOWER, UPPER, ALPHA, DIGIT, ALPHANUM, SUBWORD_DELIM.

因此，如果我们采用此文档，再加上 SOLR-2059 中的文件示例，我会推荐以下内容:
<filter class="solr.WordDelimiterFilterFactory" generateWordParts="1" generateNumberParts="1" catenateWords="1" catenateNumbers="1" catenateAll="0" splitOnCaseChange="0" splitOnNumerics="0" preserveOriginal="1" types="twittertypes.txt"/>
然后按如下方式定义 twittertypes.txt 文件，并将其放置在 Solr 实例(可能是 conf 文件夹)中与 schema.xml 文件相同的文件夹中。

 # A customized type mapping for WordDelimiterFilterFactory
 # the allowable types are: LOWER, UPPER, ALPHA, DIGIT, ALPHANUM, SUBWORD_DELIM
 #    
 # the default for any character without a mapping is always computed from 
 # Unicode character properties

 # Map the $, %, '.', and ',' characters to DIGIT 
 # This might be useful for financial data.
 @ => ALPHA
 \u0023 => ALPHA

请注意，您需要使用 Unicode 字符 (UTF-8) 作为哈希符号，因为它在文本文件中被视为注释。

根据所有文档，这应该可以解决您的问题并将 # 和 @ 符号视为字母字符，这将提供您正在寻找的行为。