java - 如何使用jsoup重定向到另一个页面并继续打印ListView中的内容？

Question

一般来说，我的网站主要内容是带有文本的帖子列表。所以我解析了这个 HTML 代码块中的每个帖子。

<div class="col-xs-12" style="margin:0.5em 0;line-height:1.785em">Some text</div>

为此，我创建了这个 AsyncTask。

class NewPostsAsyncTask extends AsyncTask<String, Void, String> {

    @Override
    protected void onPreExecute() {
        super.onPreExecute();

        progressDialog = new ProgressDialog(MainActivity.this);
        progressDialog.setTitle("Новые");
        progressDialog.setMessage("Загрузка...");
        progressDialog.setIndeterminate(false);
        progressDialog.show();
    }

    @Override
    protected String doInBackground(String... params) {
        Document doc;

        try {
            doc = Jsoup.connect(URL).get(); 

            content = doc.select("[style=margin:0.5em 0;line-height:1.785em]");
            titleList.clear();

            for (Element contents : content) {
                if (!contents.text().contains("18+")) {
                    titleList.add(contents.text());
                }
            }

        } catch (IOException e) {
            e.printStackTrace(); 
        }

        return null;
    }

    @Override
    protected void onPostExecute(String s) {
        super.onPostExecute(s);
        listView.setAdapter(adapter);
        progressDialog.dismiss();
    }
}

但是我有一些问题。所有帖子都不会存储在一个网页上。您必须单击所有帖子末尾的链接才能重定向到包含帖子的另一个页面。

这个 block 有这个 HTML 代码。

    <div class="row"><div class="col-xs-12">
        <div class="paginator">

                <span class="pagina">1683</span> " | " 

                <span class="pagina"><a href="/page/1682">1682</a></span> " | " 

                <span class="pagina"><a href="/page/1681">1681</a></span> " | " 

                <span class="pagina"><a href="/page/1680">1680</a></span> " | " 

                <span class="pagina"><a href="/page/1679">1679</a></span> " | " 

                <span class="pagina"><a href="/page/3">3</a></span> " | "

                <span class="pagina"><a href="/page/2">2</a></span> " | " 

                <span class="pagina"><a href="/page/1">1</a></span>

        </div>
    </div>
</div>

我怎样才能转到另一个页面，解析其他帖子并在之前的帖子之后在 ListView 中打印它们？结果我想把这个网站的所有帖子都放在一个 ListView 中。你能告诉我应该怎么做吗？

Answer 1

下面是我的做法:

示例代码

@Override
protected String doInBackground(String... params) {
    Document doc;

    // I supposed URL variable is initialized like this: URL="killpls.me";
    try {
        do {
            doc = Jsoup.connect(URL).get();

            content = doc.select("[style=margin:0.5em 0;line-height:1.785em]");
            titleList.clear();

            for (Element contents : content) {
                if (!contents.text().contains("18+")) {
                    titleList.add(contents.text());
                }
            }

            Element anchor = doc.select( //
                "#stories > div:nth-child(3) > div:nth-child(1) > div:nth-child(1) > span.pagina:not(:has(a)) + span > a" //
            ).first();
            if (anchor==null) {
                break;
            } else {
                doc = null;
                URL = anchor.absUrl("href"); 
            }
        } while(canContinue());
    } catch (IOException e) {
        e.printStackTrace(); 
    }

    return null;
}

private boolean canContinue() {
     // Implement custom logic here ...
     // Return true if additionnal posts should be downloaded false otherwise.
     return true;
}

一些细节

该方法的核心在于以下一行:

Element anchor = doc.select( //
   "#stories > div:nth-child(3) > div:nth-child(1) > div:nth-child(1) > span.pagina:not(:has(a)) + span > a" //
).first();

只要下一页存在，first() 方法将返回一个非 null 引用。当到达第一页时，first() 返回 null 并且没有更多的页面可以获取。

#stories           /* Find an element with id `stories`*/
> div:nth-child(3) /* Select its third div child */
> div:nth-child(1) /* Select first div child of the previous div */
> div:nth-child(1) /* Select first div (DIV-a) child of the previous div */
> span.pagina:not(:has(a)) /* Select a span with class `pagina` without any anchor as child */
+ span /* Select closest span next to previous span and child of `DIV-a` */
> a    /* Here is the next page to fetch */