java - 在谷歌学术中点击使用 HtmlUnit 的链接

Question

我正在使用 HtmlUnit 并尝试在谷歌学术中搜索，然后通过执行以下操作来获取 bibtex 项目:

1.转到谷歌学术。

2.输入我要搜索的文件的名称。

3.点击“引用”链接，然后会出现一个小框。

4.在小框中我想按“导入到bibtex”并获取文本。

例如，您可以查看此页面并尝试:https://scholar.google.com/scholar?q=internet+of+things+for+smart+cities&btnG=&hl=en&as_sdt=0%2C5

我能够访问搜索页面，但无法完成其他步骤。这是我的代码

WebClient webClient = new WebClient(BrowserVersion.CHROME);
    HtmlPage page = webClient.getPage("https://scholar.google.com/");

    HtmlInput searchBox = page.getElementByName("q");
    searchBox.setValueAttribute("internet of things for smart cities");


    HtmlButton googleSearchSubmitButton = page.getElementByName("btnG");
    page = googleSearchSubmitButton.click();

    HtmlAnchor anchor = page.getAnchorByName("Cite");
    page = anchor.click();

    System.out.println(page.asText());

    webClient.close();

有什么帮助吗？

Answer 1

这是您想要做的事情的开始:

    WebClient webClient = new WebClient(BrowserVersion.CHROME);
    webClient.getOptions().setJavaScriptEnabled(true);
    webClient.getOptions().setThrowExceptionOnScriptError(false);
    webClient.getOptions().setThrowExceptionOnFailingStatusCode(false);

    HtmlPage page = webClient.getPage("https://scholar.google.com/");

    HtmlInput searchBox = (HtmlInput) page.getElementById("gs_hp_tsi");
    searchBox.setValueAttribute("internet of things for smart cities");

    HtmlButton googleSearchSubmitButton = page.getElementByName("btnG");
    page = googleSearchSubmitButton.click();

    HtmlAnchor anchor = page.getAnchorByText("Cite");
    anchor.click();

    webClient.waitForBackgroundJavaScript(5000);

    HtmlAnchor linkBibTex = page.getAnchorByText("BibTeX");

    TextPage neededPage = linkBibTex.click();

    System.out.println(neededPage.getContent());

    webClient.close();