web-scraping - 是否允许网页抓取？

Question

关闭。这个问题需要更多focused .它目前不接受答案。












想改善这个问题吗？更新问题，使其仅关注一个问题 editing this post .

3年前关闭。




Improve this question




我正在处理一个需要来自另一个网站的某些统计数据的项目，我创建了一个 HTML 抓取工具，每 15 分钟自动获取一次这些数据。但是，我现在停止了机器人，因为在他们的使用条款中，他们提到他们不允许这样做。

我真的很想尊重这一点，尤其是如果有法律禁止我获取这些数据，但我已经多次通过电子邮件与他们联系而没有得到任何答复，所以现在我得出的结论是，我只会获取数据，如果它是合法的。

在某些论坛上，我读到它是合法的，但我更愿意在 StackOverflow 上得到更“精确”的答案。

假设这实际上并不违法，他们是否有任何软件可以发现我的机器人每 15 分钟建立几次连接？

此外，在谈论获取他们的数据时，我们谈论的是每个“团队”的一个号码，我将把这个号码转入我们自己的号码。

Answer 1

我将引用 Pablo Hoffman(Scrapinghub 联合创始人)对“网络抓取的合法性是什么？”的回答，我在其他网站上找到:

First things first: I am not a lawyer and these comments are solely based on my experience working at Scrapinghub, please seek legal assistance accordingly.

Here are a few things to consider when scraping public data from websites (note that the following addresses only US law):

• As long as they don't crawl at a disruptive rate, scrapers do not breach any contract (in the form of terms of use) or commit a crime (as defined in the Computer Fraud and Abuse Act).
• Website's user agreement is not enforceable as a browsewrap agreement because companies do not provide sufficient notice of the terms to site visitors.
• Scrapers accesses website data as a visitor, and by following paths similar to a search engine. This can be done without registering as a user (and explicitly accepting any terms).
• In Nguyen v. Barnes & Noble, Inc. the courts ruled that simply placing a link to a terms of use at the bottom of webpage is not sufficient to "give rise to constructive notice." In other words, there is nothing on a public page that would imply that merely accessing the information is subject to any contractual terms. Scrapers gives neither explicit nor implicit assent to any agreement, therefore breaches no contract.
• Social networks, for example, assign the value of becoming a user (based on call-to-action on public page), as the ability to: i) Gain access to full profiles, ii) Identify common friends/connections, iii) Get introduced to others, and iv) Contact members directly. As long as scrapers makes no attempt to perform any of these actions they do not gain "unauthorized access" to their services and thus does not violate CFAA
• A thorough evaluation of the legal issues involved can be seen here: http://www.bna.com/legal-issues-raised-by-the-use-of-web-crawling-and-scraping-tools-for-analytics-purposes