c# - ElasticSearch 索引通过 REST API 工作，但不是 C# 代码

Question

我正在尝试在 Elastic Search 中索引包含地理点的数据。当我通过代码索引时，它失败了。当我通过 REST 端点建立索引时，它成功了。但是我找不到通过 REST 端点发送的 JSON 和使用代码发送的 JSON 之间的区别。

这是配置索引的代码(作为 LINQPad 程序):

async Task Main()
{
    var pool = new SingleNodeConnectionPool(new Uri("http://localhost:9200"));
    var connectionSettings = new ConnectionSettings(pool)
        .DefaultMappingFor<DataEntity>(m => m.IndexName("data").TypeName("_doc"));

    var client = new ElasticClient(connectionSettings);

    await client.CreateIndexAsync(
        "data",
        index => index.Mappings(mappings => mappings.Map<DataEntity>(mapping => mapping.AutoMap().Properties(
            properties => properties.GeoPoint(field => field.Name(x => x.Location))))));

//    var data = new DataEntity(new GeoLocationEntity(50, 30));
//            
//    var json = client.RequestResponseSerializer.SerializeToString(data);
//    json.Dump("JSON");
//            
//    var indexResult = await client.IndexDocumentAsync(data);
//    indexResult.DebugInformation.Dump("Debug Information");
}

public sealed class GeoLocationEntity
{
    [JsonConstructor]
    public GeoLocationEntity(
        double latitude,
        double longitude)
    {
        this.Latitude = latitude;
        this.Longitude = longitude;
    }

    [JsonProperty("lat")]
    public double Latitude { get; }

    [JsonProperty("lon")]
    public double Longitude { get; }
}

public sealed class DataEntity
{
    [JsonConstructor]
    public DataEntity(
        GeoLocationEntity location)
    {
        this.Location = location;
    }

    [JsonProperty("location")]
    public GeoLocationEntity Location { get; }
}

运行后，我的映射看起来是正确的，因为 GET/data/_doc/_mapping 返回:

{
  "data" : {
    "mappings" : {
      "_doc" : {
        "properties" : {
          "location" : {
            "type" : "geo_point"
          }
        }
      }
    }
  }
}

我可以通过开发控制台成功地将文档添加到索引中:

POST /data/_doc
{
  "location": {
    "lat": 88.59,
    "lon": -98.87
  }
}

结果:

{
  "_index" : "data",
  "_type" : "_doc",
  "_id" : "RqpyjGgBZ27KOduFRIxL",
  "_version" : 1,
  "result" : "created",
  "_shards" : {
    "total" : 2,
    "successful" : 1,
    "failed" : 0
  },
  "_seq_no" : 0,
  "_primary_term" : 1
}

但是当我取消注释上面的LINQPad程序中的代码并执行时，在索引时出现这个错误:

Invalid NEST response built from a unsuccessful low level call on POST: /data/_doc
# Audit trail of this API call:
 - [1] BadResponse: Node: http://localhost:9200/ Took: 00:00:00.0159927
# OriginalException: Elasticsearch.Net.ElasticsearchClientException: The remote server returned an error: (400) Bad Request.. Call: Status code 400 from: POST /data/_doc. ServerError: Type: mapper_parsing_exception Reason: "failed to parse" CausedBy: "Type: parse_exception Reason: "field must be either [lat], [lon] or [geohash]"" ---> System.Net.WebException: The remote server returned an error: (400) Bad Request.
   at System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult)
   at Elasticsearch.Net.HttpWebRequestConnection.<>c__DisplayClass5_0`1.<RequestAsync>b__1(IAsyncResult r)
   at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd(Task task)
   at Elasticsearch.Net.HttpWebRequestConnection.<RequestAsync>d__5`1.MoveNext()
   --- End of inner exception stack trace ---
# Request:
<Request stream not captured or already read to completion by serializer. Set DisableDirectStreaming() on ConnectionSettings to force it to be set on the response.>
# Response:
<Response stream not captured or already read to completion by serializer. Set DisableDirectStreaming() on ConnectionSettings to force it to be set on the response.>

转储的 JSON 如下所示:

{
  "location": {
    "latitude": 50.0,
    "longitude": 30.0
  }
}

因此它与开发控制台有效的 JSON 结构相匹配。

为了解决这个问题，我编写了一个自定义 JsonConverter，它以 {lat},{lon} 格式序列化我的 GeoLocationEntity 对象:

public sealed class GeoLocationConverter : JsonConverter
{
    public override bool CanConvert(Type objectType) =>
        objectType == typeof(GeoLocationEntity);

    public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
    {
        var token = JToken.Load(reader);

        if (!(token is JValue))
        {
            throw new JsonSerializationException("Token was not a primitive.");
        }

        var stringValue = (string)token;
        var split = stringValue.Split(',');
        var latitude = double.Parse(split[0]);
        var longitude = double.Parse(split[1]);

        return new GeoLocationEntity(latitude, longitude);
    }

    public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
    {
        var geoLocation = (GeoLocationEntity)value;

        if (geoLocation == null)
        {
            writer.WriteNull();
            return;
        }

        var geoLocationValue = $"{geoLocation.Latitude},{geoLocation.Longitude}";
        writer.WriteValue(geoLocationValue);
    }
}

将此 JsonConverter 应用于序列化程序设置让我解决了这个问题。但是，我不想像这样解决这个问题。

谁能告诉我如何解决这个问题？

Answer 1

6.x Elasticsearch 高级客户端 NEST 通过以下方式内部化了 Json.NET 依赖项

• IL 合并 Json.NET 程序集
• 将所有类型转换为内部
• 在 Nest.* 下重命名它们

这实际上意味着客户端不直接依赖于 Json.NET(阅读 release blog post 以了解我们这样做的原因)并且不知道 Json.NET 类型，包括 JsonPropertyAttribute 或 JsonConverter。

有几种方法可以解决这个问题。首先，以下设置在开发过程中可能会有帮助

var defaultIndex = "default-index";
var pool = new SingleNodeConnectionPool(new Uri("http://localhost:9200"));

var settings = new ConnectionSettings(pool)
    .DefaultMappingFor<DataEntity>(m => m
        .IndexName(defaultIndex)
        .TypeName("_doc")
    )
    .DisableDirectStreaming()
    .PrettyJson()
    .OnRequestCompleted(callDetails =>
    {
        if (callDetails.RequestBodyInBytes != null)
        {
            Console.WriteLine(
                $"{callDetails.HttpMethod} {callDetails.Uri} \n" +
                $"{Encoding.UTF8.GetString(callDetails.RequestBodyInBytes)}");
        }
        else
        {
            Console.WriteLine($"{callDetails.HttpMethod} {callDetails.Uri}");
        }

        Console.WriteLine();

        if (callDetails.ResponseBodyInBytes != null)
        {
            Console.WriteLine($"Status: {callDetails.HttpStatusCode}\n" +
                     $"{Encoding.UTF8.GetString(callDetails.ResponseBodyInBytes)}\n" +
                     $"{new string('-', 30)}\n");
        }
        else
        {
            Console.WriteLine($"Status: {callDetails.HttpStatusCode}\n" +
                     $"{new string('-', 30)}\n");
        }
    });

var client = new ElasticClient(settings);

这会将所有请求和响应写入控制台，因此您可以看到客户端从 Elasticsearch 发送和接收的内容。 .DisableDirectStreaming() 在内存中缓冲请求和响应字节，使它们可供传递给 .OnRequestCompleted() 的委托(delegate)使用，因此它对开发很有用，但您将可能不希望在生产中使用它，因为它会带来性能成本。

现在，解决方案:

1。使用 PropertyNameAttribute

您可以使用PropertyNameAttribute 来命名序列化属性，而不是使用JsonPropertyAttribute

public sealed class GeoLocationEntity
{
    public GeoLocationEntity(
        double latitude,
        double longitude)
    {
        this.Latitude = latitude;
        this.Longitude = longitude;
    }

    [PropertyName("lat")]
    public double Latitude { get; }

    [PropertyName("lon")]
    public double Longitude { get; }
}

public sealed class DataEntity
{
    public DataEntity(
        GeoLocationEntity location)
    {
        this.Location = location;
    }

    [PropertyName("location")]
    public GeoLocationEntity Location { get; }
}

和使用

if (client.IndexExists(defaultIndex).Exists)
    client.DeleteIndex(defaultIndex);


var createIndexResponse = client.CreateIndex(defaultIndex, c => c 
    .Mappings(m => m
        .Map<DataEntity>(mm => mm
            .AutoMap()
            .Properties(p => p
                .GeoPoint(g => g
                    .Name(n => n.Location)
                )
            )
        )
    )
);

var indexResponse = client.Index(
    new DataEntity(new GeoLocationEntity(88.59, -98.87)), 
    i => i.Refresh(Refresh.WaitFor)
);

var searchResponse = client.Search<DataEntity>(s => s
    .Query(q => q
        .MatchAll()
    )
);

PropertyNameAttribute 的行为类似于您通常将 JsonPropertAttribute 与 Json.NET 一起使用的方式。

2。使用 DataMemberAttribute

在这种情况下，这将与 PropertyNameAttribute 相同，如果您不希望您的 POCO 归因于 NEST 类型(尽管我认为 POCO 与 Elasticsearch 相关联，所以将它们绑定(bind)到 .NET Elasticsearch 类型可能不是问题)。

3。使用Geolocation类型

您可以将 GeoLocationEntity 类型替换为 Nest 的 GeoLocation 类型，该类型映射到 geo_point 字段数据类型映射。使用这个，少了一个POCO，可以从属性类型推断出正确的映射

public sealed class DataEntity
{
    public DataEntity(
        GeoLocation location)
    {
        this.Location = location;
    }

    [DataMember(Name = "location")]
    public GeoLocation Location { get; }
}

// ---

if (client.IndexExists(defaultIndex).Exists)
    client.DeleteIndex(defaultIndex);

var createIndexResponse = client.CreateIndex(defaultIndex, c => c 
    .Mappings(m => m
        .Map<DataEntity>(mm => mm
            .AutoMap()
        )
    )
);

var indexResponse = client.Index(
    new DataEntity(new GeoLocation(88.59, -98.87)), 
    i => i.Refresh(Refresh.WaitFor)
);

var searchResponse = client.Search<DataEntity>(s => s
    .Query(q => q
        .MatchAll()
    )
);

4。连接 JsonNetSerializer

NEST 允许 custom serializer to be hooked up , 负责序列化您的类型。一个单独的 nuget 包，NEST.JsonNetSerializer ，允许您使用 Json.NET 序列化您的类型，序列化程序将 NEST 类型的属性委托(delegate)回内部序列化程序。

首先，您需要将 JsonNetSerializer 传递给 ConnectionSettings 构造函数

var settings = new ConnectionSettings(pool, JsonNetSerializer.Default)

然后您的原始代码将按预期工作，无需自定义 JsonConverter

public sealed class GeoLocationEntity
{
    public GeoLocationEntity(
        double latitude,
        double longitude)
    {
        this.Latitude = latitude;
        this.Longitude = longitude;
    }

    [JsonProperty("lat")]
    public double Latitude { get; }

    [JsonProperty("lon")]
    public double Longitude { get; }
}

public sealed class DataEntity
{
    public DataEntity(
        GeoLocationEntity location)
    {
        this.Location = location;
    }

    [JsonProperty("location")]
    public GeoLocationEntity Location { get; }
}


// ---

if (client.IndexExists(defaultIndex).Exists)
    client.DeleteIndex(defaultIndex);


var createIndexResponse = client.CreateIndex(defaultIndex, c => c 
    .Mappings(m => m
        .Map<DataEntity>(mm => mm
            .AutoMap()
            .Properties(p => p
                .GeoPoint(g => g
                    .Name(n => n.Location)
                )
            )
        )
    )
);

var indexResponse = client.Index(
    new DataEntity(new GeoLocationEntity(88.59, -98.87)), 
    i => i.Refresh(Refresh.WaitFor)
);

var searchResponse = client.Search<DataEntity>(s => s
    .Query(q => q
        .MatchAll()
    )
);

我最后列出此选项是因为在内部，以这种方式将序列化移交给 Json.NET 会产生性能和分配开销。包含它是为了提供灵 active ，但我建议仅在您确实需要时才使用它，例如，在序列化结构不是常规的情况下完成 POCO 的自定义序列化。我们正在努力实现更快的序列化，将来会减少这种开销。