elasticsearch - ElasticSearch.Net无法识别ElasticSearch返回的时间戳

Question

我正在尝试从包含时间戳的kibana_sample_data_flights索引获取一组数据。运行查询时，ElasticSearch可以正确识别它并返回结果，但是会崩溃

invalid datetime format. value:2019-06-03T

上

Elasticsearch.Net.Utf8Json.Formatters.ISO8601DateTimeFormatter.Deserialize(JsonReader& reader, IJsonFormatterResolver formatterResolver)

第一次，我以为是我的对象被错误地索引了，只是排除了结果，但在另一个条目上却遇到了相同的错误。使用Kibana进行搜索可以正常工作，时间戳为2019年6月2日@ 20:00:00.000，这很有意义，因为我位于UTC-5区域。

如何让NEST / ElasticSearch.net正确识别日期？

-编辑1:按要求提供 flex 版本和示例代码。

我正在使用NEST和ElasticSearch.Net v7.0.0。 ElasticSearch版本请求返回

{
  "name" : "M79539",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "CxG9dTeSTsudlhJBGiVmJQ",
  "version" : {
    "number" : "7.1.0",
    "build_flavor" : "default",
    "build_type" : "zip",
    "build_hash" : "606a173",
    "build_date" : "2019-05-16T00:43:15.323135Z",
    "build_snapshot" : false,
    "lucene_version" : "8.0.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

如果有帮助，这里是程序

class Program
    {
        static void Main(string[] args)
        {
            //Kibana: http://m79539:5601/app/kibana#/home?_g=()
            // NEST
            var settings = new ConnectionSettings(new Uri("http://m79539:9200/"))
                .DefaultIndex("kibana_sample_data_flights")
                .DefaultFieldNameInferrer(p => p);

            var client = new ElasticClient(settings);

            var scrollTimeout = "5m";
            var scrollSize = 1000;
            var initialResponse = client.Search<FlightDetail>(s => s
                        .From(0).Take(scrollSize)
                        .Query(q => +q
                             .DateRange(r => r
                                    .Field(f => f.Timestamp)
                                    .GreaterThan(DateMath.Anchored(new DateTime(2019,06,01)))
                                    .LessThan(DateMath.Now)) && +!q
                              .Term(p => p.FlightNum, "59H86EL")
                          )
                        .Scroll(scrollTimeout)
                        //scroll is optimized for this sort and we will sort in our own code
                        .Sort(ss => ss.Ascending(SortSpecialField.DocumentIndexOrder))
                      );

            var results = new List<FlightDetail>();

            if (!initialResponse.IsValid || string.IsNullOrEmpty(initialResponse.ScrollId))
                throw new Exception(initialResponse.ServerError.Error.Reason);

            if (initialResponse.Documents.Any())
                results.AddRange(initialResponse.Documents);

            string scrollid = initialResponse.ScrollId;
            bool isScrollSetHasData = true;
            while (isScrollSetHasData)
            {
                ISearchResponse<FlightDetail> loopingResponse = client.Scroll<FlightDetail>(scrollTimeout, scrollid);
                if (loopingResponse.IsValid)
                {
                    results.AddRange(loopingResponse.Documents);
                    scrollid = loopingResponse.ScrollId;
                }
                isScrollSetHasData = loopingResponse.Documents.Any();
            }

            client.ClearScroll(new ClearScrollRequest(scrollid));


            foreach (var result in results){
                Console.WriteLine($"{result.Timestamp} | {result.FlightNum} | {result.FlightDelayMin}");
            }
        }
    }

    public class FlightDetail
    {
        public string FlightNum { get; set; }
        public int FlightDelayMin { get; set; }
        public object Source { get; set; }

        [Nest.Text(Name = "timestamp")]
        public DateTime Timestamp { get; set; }
    }

有问题的结果(在我的安装中)为548

{
    "_scroll_id": "DXF1ZXJ5QW5kRmV0Y2gBAAAAAAANizsWaTdsY3ViS2hUR2FyRHZESmdKeVRWZw==",
    "took": 1,
    "timed_out": false,
    "terminated_early": true,
    "_shards": {
        "total": 1,
        "successful": 1,
        "skipped": 0,
        "failed": 0
    },
    "hits": {
        "total": {
            "value": 9227,
            "relation": "eq"
        },
        "max_score": null,
        "hits": [{
            "_index": "kibana_sample_data_flights",
            "_type": "_doc",
            "_id": "1Lr3DmsBoDCXrrhScXCk",
            "_score": null,
            "_source": {
                "FlightNum": "86C3GQY",
                "DestCountry": "MX",
                "OriginWeather": "Sunny",
                "OriginCityName": "Rome",
                "AvgTicketPrice": 395.9230418749486,
                "DistanceMiles": 6362.054220749258,
                "FlightDelay": false,
                "DestWeather": "Damaging Wind",
                "Dest": "Licenciado Benito Juarez International Airport",
                "FlightDelayType": "No Delay",
                "OriginCountry": "IT",
                "dayOfWeek": 0,
                "DistanceKilometers": 10238.733787837495,
                "timestamp": "2019-06-03T",
                "DestLocation": {
                    "lat": "19.4363",
                    "lon": "-99.072098"
                },
                "DestAirportID": "AICM",
                "Carrier": "JetBeats",
                "Cancelled": false,
                "FlightTimeMin": 602.2784581080879,
                "Origin": "Leonardo da Vinci - Fiumicino Airport",
                "OriginLocation": {
                    "lat": "41.8002778",
                    "lon": "12.2388889"
                },
                "DestRegion": "MX-DIF",
                "OriginAirportID": "FCO",
                "OriginRegion": "SE-BD",
                "DestCityName": "Mexico City",
                "FlightTimeHour": 10.037974301801464,
                "FlightDelayMin": 0
            },
            "sort": [4379]
        }]
    }
}

---编辑2:基于@Rob命题的简单代码示例。以相同的问题结束。我添加了try / catch以便能够解决错误，然后查看Fiddler中遇到的情况。

        private static void SimpleSample()
        {
            var client = new ElasticClient(new Uri("http://m79539:9200"));

            for (var i = 0; ; i++)
            {
                try
                {
                    var results = client.Search<SampleData>(s => s
                        .Index("kibana_sample_data_flights")
                        .From(i * 10)
                        .Query(q => q.MatchAll()));

                    foreach (var document in results.Documents)
                    {
                        Console.WriteLine(document.Timestamp.ToString("f") + "|" + document.FlightNum);
                    }
                }
                catch (Elasticsearch.Net.UnexpectedElasticsearchClientException ex)
                {
                    Console.WriteLine(ex.Message);
                }
            }
        }

        private class SampleData
        {
            public DateTime Timestamp { get; set; }
            [Nest.Text(Name = "FlightNum")]
            public string FlightNum { get; set; }
        }

Answer 1

这是示例应用程序，可以从kibana_sample_data_flights索引中检索时间戳数据。

class Program
{
    static void Main(string[] args)
    {
        var client = new ElasticClient(new Uri("http://localhost:9200"));

        var results = client.Search<SampleData>(s => s
            .Index("kibana_sample_data_flights").Query(q => q.MatchAll()));

        foreach (var document in results.Documents)
        {
            Console.WriteLine(document.Timestamp.ToString("f"));
        }
    }
}

internal class SampleData
{
    public DateTime Timestamp { get; set; }
}

产生以下输出:

Monday, 17 June 2019 00:00
Monday, 17 June 2019 18:27
Monday, 17 June 2019 17:11
Monday, 17 June 2019 10:33
Monday, 17 June 2019 05:13
Monday, 17 June 2019 01:43
Monday, 17 June 2019 13:49
Monday, 17 June 2019 04:54
Monday, 17 June 2019 12:09
Monday, 17 June 2019 12:09

在此示例中，由于 default field name convention being used，NEST能够正确地将SampleData属性 Timestamp映射到索引的字段 timestamp。

更新

确实存在解析ID为 1Lr3DmsBoDCXrrhScXCk z和值为“timestamp”的文档的不正确日期值的问题:“2019-06-03T”，`。

我们可以在定义 ConnectionSettings对象时通过自定义序列化来处理错误，并在遇到错误时继续处理:

var pool = new SingleNodeConnectionPool(new Uri("http://localhost:9200"));
var connectionSettings =
    new ConnectionSettings(pool,
        sourceSerializer: (builtin, settings) => new JsonNetSerializer(
            builtin, settings,
            () => new JsonSerializerSettings {Error = HandlerError}
        ));
var client = new ElasticClient(connectionSettings);

private static void HandlerError(object sender, ErrorEventArgs e)
{
    //handle error the way you like
    e.ErrorContext.Handled = true;
}

现在我们得到以下输出:

..
Wednesday, 19 June 2019 21:39

Thursday, 20 June 2019 04:47
Thursday, 20 June 2019 21:51
Thursday, 20 June 2019 02:27
..

您可以在NEST here中阅读有关配置序列化过程的更多信息。

希望能有所帮助。