elasticsearch - Elasticsearch发布荧光笔返回太多句子

Question

我的帖子荧光笔有问题。根据文档:
“...张贴荧光笔...输出句子，无论其长度如何。”

因此，通过设置:"number_of_fragments" : 1我应该只还一句话。这是90％的情况，但是有时我会得到很长的文本，显然超过一句话。例如:(突出显示的单词是河和污染)

It is a collegiate body with an advisory and deliberative of the Integrated Water Resources Management - working on Unit Water Resources Management 10, built by the state, municipalities and civil society, equally. [ 2 ] This committee took the initiative of civil society and currently includes 34 municipalities, 18 were located in Sorocaba River basin and 16 situated in the sub-basin of the upper Middle Tietê. [ 3 ] It has been a very polluted river due to industrial activities, mining, sewage without treatment, etc.

共有3个句子，前两个甚至没有加亮的单词。
我认为这里存在一个错误，导致帖子荧光笔忽略“。”。当后跟'['时。我注意到在所有不良的突出显示结果中都是这种情况。

这是一个已知的错误？还是我错过了什么？
谢谢

Answer 1

本质上，我不确定我是否认为这是一个错误。句子的边界并不像分割句那么简单(您不想破坏“3.14”或“史密斯先生”)，而且常常是模棱两可的。
PostingsHighlighter使用java.text.BreakIterator检测在哪里分解句子。我以为BreakIterator的行为是基于UAX #29的，但是这种行为与you can try it here不太一致。

因此，很可能是java.text.BreakIterator中的错误，或者这可能只是其算法的工作方式。