c# - 无法从弹性客户端搜索响应中获取_source字典键值

Question

我正在尝试访问hits _source词典以加载到数据库中。
hits返回null，我在做什么错？

笔记:
searchResponse与JSON数据一起返回，并且调试信息对其进行确认。

但是，不可访问Hit和_Source类以及基础数据变量，并且变量hit将返回null。

下面的 Debug模式下的局部变量代码显示了数据。
如果需要的话，我可以提供更多数据或局部变量的图像或调试信息窗口(如果这有助于解决问题)。

提前致谢。

尝试使用searchResponse.Documents和foreach语句访问_source键值对以访问匹配中的元素。但是无法访问_source键值对。

/*Declared classes in visual studio console application for c#:
.NET framework 4.5*/

class Program
{

    public class Doc
    {
        public int took { get; set; }
        public bool timed_out { get; set; }
        public _Shards _shards { get; set; }
        public Hits hits { get; set; }
    }

    public class _Shards
    {
        public int total { get; set; }
        public int successful { get; set; }
        public int skipped { get; set; }
        public int failed { get; set; }
    }

    public class Hits
    {
        public int total { get; set; }
        public float max_score { get; set; }
        public Hit[] hits { get; set; }
    }

    public class Hit
    {
        public string _index { get; set; }
        public string _type { get; set; }
        public string _id { get; set; }
        public float _score { get; set; }
        public _Source _source { get; set; }
    }

    public class _Source
    {
        public int duration { get; set; }
        public string group_id { get; set; }
        public DateTime var_time { get; set; }
        public string var_name { get; set; }
    }

    static void Main(string[] args)
    {
        var uri = new Uri("http://domain_name.val.url:9203/");
        var pool = new SingleNodeConnectionPool(uri);
        var connectionSettings = new ConnectionSettings(pool)
                                .DisableDirectStreaming();
        var resolver = new IndexNameResolver(connectionSettings);
        var client = new ElasticClient(connectionSettings);

        if (!client.IndexExists("test_index").Exists)
        {
            client.CreateIndex("test_index");
        }

        var searchResponse = client.Search<Doc>(s => s
        .Index("test_index")
        .AllTypes()
        .Size(1)
        .Query(q => q
        .MatchAll())
        .TypedKeys(null)
        .SearchType(Elasticsearch.Net.SearchType.DfsQueryThenFetch)
        .Scroll("30s")
      );
    MessageBox.Show("searchResponse.DebugInformation=" + searchResponse.DebugInformation);
    }
}

Elastic Search示例URL数据:

{
  "took" : 12,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : 2700881,
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "test_index",
        "_type" : "doc",
        "_id" : "R22224!!5333e7e4-9ee3-45f4-9dc3-2a8b8d8cdcf8",
        "_score" : 1.0,
        "_source" : {
          "duration" : 14986283,
          "group_id" : "com",
          "var_time" : "2018-04-24T17:05:13.082+02:00",
          "var_name" : "2",
        }
      }
    ]
  }
}

更新:
办公室内部的某人建议使用以下代码解决方案，然后遍历键值对。

        var searchResponse = client.Search<Doc>(s => s
            .Index("test_index")
            .AllTypes()
            .Size(10)
            .Query(q => q
            .MatchAll())
            .TypedKeys(null)
            .SearchType(Elasticsearch.Net.SearchType.DfsQueryThenFetch)
            .Scroll("30s")
            .RequestConfiguration(r=>r
            .DisableDirectStreaming()
            )
            );
        var raw = Encoding.UTF8.GetString(searchResponse.ApiCall.ResponseBodyInBytes);  
        JavaScriptSerializer jss = new JavaScriptSerializer();
        jss.MaxJsonLength = Int32.MaxValue;
        var pairs = jss.Deserialize<Dictionary<string, dynamic>>(raw); 

Answer 1

看来您误解了客户端的API。您不需要声明_Shards，Hit，Hits，_Source等。客户端会为您代劳反序列化Elasticsearch API的这些部分。

您需要定义的唯一部分是POCO，它将映射到响应中每个"_source"字段中的JSON对象，即

{
  "duration" : 14986283,
  "group_id" : "com",
  "var_time" : "2018-04-24T17:05:13.082+02:00",
  "var_name" : "2",
}

看起来就像 _Source POCO一样(尽管我倾向于给它起一个更有意义的名字!)。现在暂时将其称为 MyDocument。

使用 MyDocument定义为

public class MyDocument
{
    [PropertyName("duration")]
    public int Duration { get; set; }

    [PropertyName("group_id")]
    public string GroupId { get; set; }

    [PropertyName("var_time")]
    public DateTime Time { get; set; }

    [PropertyName("var_name")]
    public string Name { get; set; }
}

一个简单的搜索将是

var pool = new SingleNodeConnectionPool(new Uri("http://localhost:9200"));

var settings = new ConnectionSettings(pool)
    .DefaultMappingFor<MyDocument>(m => m
        .IndexName("test_index")
        .TypeName("doc")
    );

var client = new ElasticClient(settings);

var searchResponse = client.Search<MyDocument>();

// A collection of the top 10 matching documents
var documents = searchResponse.Documents;

每当文档的通用类型为 DefaultMappingFor<MyDocument>(...)且未在请求中明确定义它们时， "test_index"将使用索引名称 doc和类型名称“ MyDocument”。

上面的搜索对Elasticsearch产生以下查询

POST http://localhost:9200/test_index/doc/_search
{}

现在，您似乎想使用Scroll API返回所有匹配的文档。要使用Scroll API做到这一点，您将编写一个循环，以便只要返回文档，就不断发出滚动请求

var searchResponse = client.Search<MyDocument>(s => s
    .Size(1000)
    .Scroll("30s")
);

while (searchResponse.Documents.Any())
{
    foreach (var document in searchResponse.Documents)
    {
        // do something with this set of 1000 documents
    }

    // make an additional request
    searchResponse = client.Scroll<MyDocument>("30s", searchResponse.ScrollId);
}

// clear scroll id at the end
var clearScrollResponse = client.ClearScroll(c => c.ScrollId(searchResponse.ScrollId));

有一个 ScrollAll可观察的助手，您可以使用它来简化编写工作，并使用 parallelizes the operation using sliced_scroll 。与上述相同的操作，但使用 ScrollAll

// set to number of shards in targeted indices
var numberOfSlices = 4;

var scrollAllObservable = client.ScrollAll<MyDocument>("30s", numberOfSlices);

Exception exception = null;
var manualResetEvent = new ManualResetEvent(false);

var scrollAllObserver = new ScrollAllObserver<MyDocument>(
    onNext: s => 
    {
        var documents = s.SearchResponse.Documents;

        foreach (var document in documents)
        {
            // do something with this set of documents
        }
    },
    onError: e =>
    {
        exception = e;
        manualResetEvent.Set();
    },
    onCompleted: () => manualResetEvent.Set()
);

scrollAllObservable.Subscribe(scrollAllObserver);

manualResetEvent.WaitOne();

if (exception != null)
    throw exception;

如果不需要对观察者的所有控制，则可以使用简化版本。这样，您确实需要为整个操作指定最大运行时间

var numberOfSlices = 4;

var scrollAllObservable = client.ScrollAll<MyDocument>("30s", numberOfSlices)
    .Wait(TimeSpan.FromHours(2), onNext: s =>
        {
           var documents = s.SearchResponse.Documents;

           foreach (var document in documents)
           {
                // do something with this set of documents
            }
        });