web-crawler - 使用nightmare.js拒绝了如何解决自动访问此页面的问题

Question

我正在使用nightmare.js抓取网站，当我转到url时出现错误:

"Unfortunately, automated access to this page was denied."

await nightmare
      .goto(url)
      .wait(10*1000)
      .useragent('Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36');


html = await nightmare
       .evaluate(() => {
            return document.body.innerHTML;
        })
        .end()
        .then((html: any) => {
            return html;
        })
        .catch((error: any) => {
            throws(error.toString);
        });

Answer 1

此代码解决了问题。

   await nightmare.useragent('Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36')
        .goto(url);