asp.net - 在ElasticSearch NEST中创建自定义标记生成器

Question

我在以下ES 2.5中有一个自定义类:

Title
DataSources
Content

运行搜索很好，除了中间字段外-使用定界符“|”构建/索引。

ex: "|4|7|8|9|10|12|14|19|20|21|22|23|29|30"

我需要构建一个查询，该查询与所有字段中的某些匹配，并与DataSource字段中的至少一个数字匹配。

因此，总结一下我目前所拥有的:

    QueryBase query = new SimpleQueryStringQuery
    {
        //DefaultOperator = !operatorOR ? Operator.And : Operator.Or,
        Fields = LearnAboutFields.FULLTEXT,
        Analyzer = "standard",
        Query = searchWords.ToLower()
    };
    _boolQuery.Must = new QueryContainer[] {query};

那就是搜索词查询。

    foreach (var datasource in dataSources)
    {
        // Add DataSources with an OR
        queryContainer |= new WildcardQuery { Field = LearnAboutFields.DATASOURCE, Value = string.Format("*{0}*", datasource) };
    }
    // Add this Boolean Clause to our outer clause with an AND
    _boolQuery.Filter = new QueryContainer[] {queryContainer};
}

这是针对数据源查询的。可以有多个数据源。

它不起作用，并返回添加了过滤器查询的结果。我认为我需要对 token 生成器/分析器进行一些工作，但是我对ES的了解还不足以解决这一问题。

编辑:Per Val下面的评论我试图这样重新编码索引器:

        _elasticClientWrapper.CreateIndex(_DataSource, i => i
            .Mappings(ms => ms
                .Map<LearnAboutContent>(m => m
                    .Properties(p => p
                        .String(s => s.Name(lac => lac.DataSources)
                            .Analyzer("classic_tokenizer")
                            .SearchAnalyzer("standard")))))
            .Settings(s => s
                .Analysis(an => an.Analyzers(a => a.Custom("classic_tokenizer", ca => ca.Tokenizer("classic"))))));
        var indexResponse = _elasticClientWrapper.IndexMany(contentList);

它使用数据成功构建。但是查询仍然无法正常工作。

数据源的新查询:

        foreach (var datasource in dataSources)
        {
            // Add DataSources with an OR
            queryContainer |= new TermQuery {Field = LearnAboutFields.DATASOURCE, Value = datasource};
        }
        // Add this Boolean Clause to our outer clause with an AND
        _boolQuery.Must = new QueryContainer[] {queryContainer};

和JSON:

{"learnabout_index":{"aliases":{},"mappings":{"learnaboutcontent":{"properties":{"articleID":{"type":"string"},"content":{"type":"string"},"dataSources":{"type":"string","analyzer":"classic_tokenizer","search_analyzer":"standard"},"description":{"type":"string"},"fileName":{"type":"string"},"keywords":{"type":"string"},"linkURL":{"type":"string"},"title":{"type":"string"}}}},"settings":{"index":{"creation_date":"1483992041623","analysis":{"analyzer":{"classic_tokenizer":{"type":"custom","tokenizer":"classic"}}},"number_of_shards":"5","number_of_replicas":"1","uuid":"iZakEjBlRiGfNvaFn-yG-w","version":{"created":"2040099"}}},"warmers":{}}}

查询JSON请求:

{
  "size": 10000,
  "query": {
    "bool": {
      "must": [
        {
          "simple_query_string": {
            "fields": [
              "_all"
            ],
            "query": "\"housing\"",
            "analyzer": "standard"
          }
        }
      ],
      "filter": [
        {
          "terms": {
            "DataSources": [
              "1"
            ]
          }
        }
      ]
    }
  }
}

Answer 1

实现此目的的一种方法是创建一个带有classic tokenizer的自定义分析器，该分析器会将您的DataSources字段分解为组成它的数字，即，它将标记每个|字符上的字段。

因此，在创建索引时，需要添加此自定义分析器，然后在DataSources字段中使用它:

PUT my_index
{
  "settings": {
    "analysis": {
      "analyzer": {
        "number_analyzer": {
          "type": "custom",
          "tokenizer": "number_tokenizer"
        }
      },
      "tokenizer": {
        "number_tokenizer": {
          "type": "classic"
        }
      }
    }
  },
  "mappings": { 
    "my_type": {
      "properties": {
        "DataSources": {
          "type": "string",
          "analyzer": "number_analyzer",
          "search_analyzer": "standard"
        }
      }
    }
  }
}

结果，如果您为字符串 "|4|7|8|9|10|12|14|19|20|21|22|23|29|30"编制索引，则 DataSources字段将有效地包含以下 token 数组: [4, 7, 8, 9, 10, 12, 14, 191, 20, 21, 22, 23, 29, 30]
然后，您可以摆脱 WildcardQuery，而只需使用 TermsQuery即可:

terms = new TermsQuery {Field = LearnAboutFields.DATASOURCE, Terms = dataSources }
// Add this Boolean Clause to our outer clause with an AND
_boolQuery.Filter = new QueryContainer[] { terms };