c# - 使用 GET 从多个页面获取 JSON 数据

Question

我需要从 URL 获取数据。

1. 下面的代码给出了错误“读取超过文件末尾”。然而 URL 是正确的，可以粘贴到浏览器中查看结果。有几次我收到“超出时间限制”错误，不确定我是否可以做些什么。

2. 问题是数据可以在多个页面上。我是否必须从 page1 中选择 total_pages 并进行循环？有没有更好的解决方案？

代码如下:

string url="https://jsonmock.hackerrank.com/api/movies/search/?Title=spiderman&page=1";
string res=MakeRequest(url);

制作请求:

static public string MakeRequest(string url)
{

 HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
    request.ContentType = "application/json; charset=utf-8";
    request.PreAuthenticate = true;

 HttpWebResponse response = request.GetResponse() as HttpWebResponse;
    using (Stream responseStream = response.GetResponseStream())
    {
        StreamReader reader = new StreamReader(responseStream, Encoding.UTF8);
        return (reader.ReadToEnd());

    }

}

这是数据(第 2 页)的样子

{"page":"2","per_page":10,"total":13,"total_pages":2,"data":[{"Poster":"N/A","Title":"They Call Me Spiderman","Type":"movie","Year":2016,"imdbID":"tt5861236"},{"Poster":"N/A","Title":"The Death of Spiderman","Type":"movie","Year":2015,"imdbID":"tt5921428"},{"Poster":"https://images-na.ssl-images-amazon.com/images/M/MV5BZDlmMGQwYmItNTNmOS00OTNkLTkxNTYtNDM3ZWVlMWUyZDIzXkEyXkFqcGdeQXVyMTA5Mzk5Mw@@._V1_SX300.jpg","Title":"Spiderman in Cannes","Type":"movie","Year":2016,"imdbID":"tt5978586"}]}

Answer 1

使用 HttpClient 的完整示例和 Newtonsoft.Json。

首先，让我们为我们的数据传输对象定义类，以匹配它们在 API 中的 json 对应物:

public class PageResponse
{
    public string Page { get; set;}

    [JsonProperty("per_page")]
    public int PerPage { get; set;}

    public int Total { get; set; }

    [JsonProperty("total_pages")]
    public int TotalPages { get; set; }

    public IEnumerable<Movie> Data { get; set; }
}

public class Movie
{
    public string Poster { get; set; }
    public string Title { get; set; }
    public string Type { get; set; }
    public int Year { get; set; }

    [JsonProperty("imdbID")]
    public string ImdbId { get; set; }
}

然后让我们创建 API 调用方法，它使用 do-while 循环并执行请求，直到我们检索到所有页面或失败(由于某种原因):

public static async Task<IEnumerable<Movie>> GetMoviesAsync()
{
    var movies = new List<Movie>();
    var url = "http://jsonmock.hackerrank.com/api/movies/search/?Title=spiderman";
    int currentPage = 1;
    int totalPages = 0;
    var nextUrl = $"{url}&page={currentPage}";

    using (var httpClient = new HttpClient())
    { 
        do
        {
            HttpResponseMessage response = await httpClient.GetAsync(nextUrl);

            if (response.IsSuccessStatusCode)
            {
                string json = await response.Content.ReadAsStringAsync();
                var pageResponse = JsonConvert.DeserializeObject<PageResponse>(json);

                if (pageResponse != null && pageResponse.Data.Any())
                {
                    movies.AddRange(pageResponse.Data);
                    totalPages = pageResponse.TotalPages;

                    currentPage++;
                    nextUrl = $"{url}&page={currentPage}";
                }
                else
                {
                    break; // or throw exception
                }
            }
            else
            {
                break; // or throw exception
            }
        } while (currentPage < totalPages);
    }

    return movies;
}

最后从我们的 Main 方法调用 GetMoviesAsync:

static void Main(string[] args)
{
    IEnumerable<Movie> movies = GetMoviesAsync().GetAwaiter().GetResult();
    Console.WriteLine($"Retrieved {movies.Count()} movies.");
}

...在进行此编辑时，总共检索到 13 部电影:

Retrieved 13 movies.

另请注意，最佳做法是访问修饰符，例如public 总是写在 static 关键字之前。即

public static ...

不是:

static public ...

编辑:按照下面评论中的建议更新更多详细信息

EDIT2:通过省略 ContinueWith 调用改进 GetMoviesAsync()