elasticsearch - 如何在Elasticsearch提示器中使用Logstash导入CSV数据以完成字段类型

Question

ElasticSearch索引创建

curl -XPOST 'http://localhost:9200/music/' -d '{}'

场图

curl -XPUT 'http://localhost:9200/music/_mapping/song' -d '
{
  "properties": {
    "name" : {
      "type" : "string"
    },
    "suggest": {
      "type" : "completion"
    }
  }
}'

LogStash配置文件musicStash.config

input {
    file {
        path => "pathToCsv"
        start_position => beginning
    }
}

filter {  
    csv {
        columns => ["id", "name", "suggest"]
        separator => ","
    }
}

output {
    elasticsearch {
        hosts => "localhost"
        index => "music"
        document_id => "%{id}"
    }
}

现在在执行logstash配置文件时，在elasticsearch控制台中收到以下异常

failed to put mappings on indices [[music]], type [logs]
java.lang.IllegalArgumentException: Mapper for [suggest] conflicts with existing mapping in other types:
[mapper [suggest] cannot be changed from type [completion] to [string]]
at org.elasticsearch.index.mapper.FieldTypeLookup.checkCompatibility(FieldTypeLookup.java:117)

在Logstash控制台中收到错误，

response=>{"index"=>{"_index"=>"music", "_type"=>"logs", "_id"=>"5", "status"=>400, 
"error"=>{"type"=>"illegal_argument_exception", 
"reason"=>"Mapper for [suggest] conflicts with existing mapping in other types:\n[mapper [suggest] cannot be changed from type [completion] to [string]]"}}}, :level=>:warn}

那么如何通过Logstash导入csv文件来实现elasticsearch自动完成功能。

Answer 1

您在elasticsearch输出中缺少以下设置:

document_type => "song"

发生的事情是logstash正在创建一种名为 logs( by default)的新类型，并且由于从ES 2.0开始，它被禁止在相同的索引中包含两个具有相同名称但不同类型( string与 completion)的字段，这是错误的。

只需像这样修改您的输出，它将起作用:

output {
    elasticsearch {
        hosts => "localhost"
        index => "music"
        document_type => "song"
        document_id => "%{id}"
    }
}