python - Selenium 无法从 workera.ai 中找到元素

Question

我试图从 workera.ai 中抓取问题答案，但我被卡住了，因为 Selenium 找不到我使用类搜索的任何元素。当我检查页面源时，该元素可用但 Selenium 找不到它。这就是我正在做的事情。

注册使用:https://workera.ai/candidates/signup

from selenium import webdriver
from selenium.webdriver.chrome import service
from selenium.webdriver.common.by import By
from selenium.webdriver.chrome.service import Service
from webdriver_manager.chrome import ChromeDriverManager
import time, os

option = webdriver.ChromeOptions()
option.add_argument("start-maximized")
option.add_experimental_option("excludeSwitches", ["enable-automation"])
option.add_experimental_option('useAutomationExtension', False)
option.add_argument("--disable-blink-features")
option.add_argument("--disable-gpu")
option.add_argument(r"--user-data-dir=C:\Users\user_name\AppData\Local\Google\Chrome\User Data") #e.g. C:\Users\You\AppData\Local\Google\Chrome\User Data
option.add_argument(r'--profile-directory=Profile 2') # using profile which is logged into the website
#option.add_argument("--headless")
option.add_argument('--disable-blink-features=AutomationControlled')
wd = webdriver.Chrome(service=Service(ChromeDriverManager().install()), options=option)
skill_sets = ['https://workera.ai/app/learner/skillset/82746bf6-4eb2-4065-b2fb-740bc3207d14','https://workera.ai/app/learner/skillset/7553e8f8-52bf-4136-a4ea-6aa63eb963d9','https://workera.ai/app/learner/skillset/e11cb698-38c1-4a4f-aa7b-43b85bdf5a51','https://workera.ai/app/learner/skillset/a999048c-ab99-4576-b849-4e72c9455418','https://workera.ai/app/learner/skillset/7df84ad9-ae67-4faf-a981-a95c1c02adbb', 'https://workera.ai/app/learner/skillset/737fa250-8c66-4ea0-810b-6847c304aa5b','https://workera.ai/app/learner/skillset/ed4f2f1f-2333-4b28-b36a-c7f736da9647','https://workera.ai/app/learner/skillset/323ba5d9-fffe-48c0-b7b4-966d1ebca99a','https://workera.ai/app/learner/skillset/488492e9-53c4-4600-b336-6dfe44340402']
#                                             AI fluent                                                                       AI literate                                                                       DATA ANAlyst                                                           DATA Engineer                                                                        DATA scientist                                                                        Deep learn                                                                  ML                                                                           Responsible AI                                                                   Software Engineer
for skill in skill_sets:
    wd.get(skill)
    time.sleep(20)
    num = wd.find_element(By.CLASS_NAME, "sc-jNHgKk hrMhpT")# class name is different for every account
    num = num.split('of')[1]
    num = int(num)
    print(num)
    button = wd.find_elements(By.CLASS_NAME, "styled__SBase-sc-cmjz60-0 styled__SPrimary-sc-cmjz60-1 kSmXiJ hwoYMb sc-fKVqWL eOjNfz")
    print(len(button))
wd.close()

我不知道为什么会这样。该站点是否阻止了 Selenium Web 驱动程序，还是其他东西？

编辑

我尝试从 Selenium 获取页面源代码，然后使用 bs4 访问元素，它正在运行。所以我认为该网站以某种方式阻止了 Selenium。

Answer 1

selenium 的问题是您不能像这样选择具有多个类的元素。

为了选择它们，您可以在值中提及一个类别，或使用“.”。例如:

wd.find_element(By.CLASS_NAME,"class1.class2") 

您还可以选择所有答案都存在的类，我相信是这个“sc-jNHgKk”，这样您就不会为每个帐户选择一个类，或者您可以只使用 XPATH相反。

num = int(wd.find_element(By.CLASS_NAME, "sc-jNHgKk").text.split("of ")[1])
button = wd.find_elements(By.CLASS_NAME, "styled__SBase-sc-cmjz60-0")
print(len(button))