python-3.x - SyntaxError;名称 'x'是参数且非本地

Question

我最近在抓取HackerNews网站时遇到了这个SyntaxError:“标题”是参数，并且是非本地的，这是相同代码。

import requests
from bs4 import BeautifulSoup
import time


# Url Generator


def urls_gen(base_url, pages=1):
    try:
        for page in range(1, int(pages) + 1):
            yield f'{base_url}news?p={str(page)}'
    except ValueError as e:
        print(e)

# Url Crawler
# max pages = 15


def urls_crawler(urls):
    for url in urls:
        try:
            res = requests.get(url)
            soup = BeautifulSoup(res.text, 'html.parser')
            soup.prettify()
            titles = soup.select('.storylink')
            subtext = soup.select('.subtext')
        except IndexError:

            return('Max Pages Limit Reached!')

    def custom_hn(titles, subtext):
        data = []
        for i, item in enumerate(titles):
            nonlocal titles
            nonlocal subtext
            title = titles[i].getText()
            link = titles[i].get('href', None)
            print(title, link)
            time.sleep(1)

    return custom_hn(titles, subtext)


urls = urls_gen('https://news.ycombinator.com/', 'a')
links = urls_crawler(urls)

我还注意到，当我为urls_gen的第二个参数提供int参数时，它可以正常工作，但是一旦我将其更改回str，它就会给我发出SyntaxError而不是上面在try-except块中定义的ValueError。它还给了Unbound局部变量错误。
任何帮助将不胜感激!

Answer 1

我对您的脚本进行了一些修改，并删除了try..except语句。另外，您可以测试是否在最后一页上，如果没有任何链接需要刮一下。
例如:

import requests
from bs4 import BeautifulSoup


def urls_gen(base_url, pages=1):
    for page in range(1, pages + 1):      # <-- remove int()
        yield f'{base_url}news?p={page}'  # <-- remove str()

def urls_crawler(urls):
    for url in urls:
        res = requests.get(url)
        soup = BeautifulSoup(res.text, 'html.parser')

        all_titles = soup.select('.storylink')
        if not all_titles:  # <-- if there arent't any link, break the loop
            break

        for title in all_titles:
            yield title.get_text(), title['href']


urls = urls_gen('https://news.ycombinator.com/', 2) # <-- put number here instead of 'a'
for text, link in urls_crawler(urls):
    print('{:<80} {}'.format(text, link))

打印品:

Show HN: A weirdly detailed graphical analysis of women's tops sold by Goodwill  https://goodwill.awardwinninghuman.com/
EasyOCR: Ready-to-use OCR with 40 languages                                      https://github.com/JaidedAI/EasyOCR
Tauri – toolchain for building highly secure native apps that have tiny binaries https://github.com/tauri-apps/tauri
Foreign data wrappers: PostgreSQL's secret weapon?                               https://www.splitgraph.com/blog/foreign-data-wrappers
Why Developers Stop Learning: Rise of the Expert Beginner (2012)                 https://daedtech.com/how-developers-stop-learning-rise-of-the-expert-beginner/
The EU General Data Protection Regulation Explained by Americans                 https://hroy.eu/posts/gdprExplainedByUS/
Git commit accepts several message flags (-m) to allow multiline commits         https://www.stefanjudis.com/today-i-learned/git-commit-accepts-several-message-flags-m-to-allow-multiline-commits/
Ariane RISC-V CPU – An open source CPU capable of booting Linux                  https://github.com/openhwgroup/cva6
SUSE to Acquire Rancher Labs                                                     https://rancher.com/press/suse-to-acquire-rancher/
Yoloface-500k: ultra-light real-time face detection model, 500kb                 https://github.com/dog-qiuqiu/MobileNetv2-YOLOV3#500kb%E7%9A%84yolo-face-detection
Google drops blogspot.in breaking hundred thousands of permalinks                item?id=23767781
Symbolic execution with SymCC: Don't interpret, compile                          http://www.s3.eurecom.fr/tools/symbolic_execution/symcc.html
The More Senior Your Job Title, the More You Need to Keep a Journal              https://hbr.org/2017/07/the-more-senior-your-job-title-the-more-you-need-to-keep-a-journal
SUSE to Acquire Rancher Labs                                                     https://www.suse.com/c/news/suse-acquires-rancher/
Mcfly – neural-network powered directory and context-aware shell history search  https://github.com/cantino/mcfly
Ron Graham has died                                                              https://www.ams.org/news?news_id=6244
SUSE acquires Kubernetes management platform Rancher Labs                        https://techcrunch.com/2020/07/08/suse-acquires-kubernetes-management-platform-rancher-labs/
KeePassXC 2.6.0 Released                                                         https://keepassxc.org/blog/2020-07-07-2.6.0-released/
SymPy - a Python library for symbolic mathematics                                https://www.sympy.org/en/index.html
Curry Before Columbus                                                            https://contingentmagazine.org/2020/06/25/curry-before-columbus/

...and so on.