python - 使用 BeautifulSoup 解析 HTML 标签

Question

我正在解析一个包含许多超链接的网页。能够使用 beautifulsoup 通过查找 <a> 中的所有“href”来提取超链接标签。接下来，我试图获取所有这些链接的“标题”，这些链接显示为 <a> 的标签内容。标签。 [HTML 新手，我认为这个尾随描述称为标签内容，对吧？] 下面是一个示例。字符串“哪种教室环境适合您？”需要提取。感谢您的帮助。

<a href="https://www.nytimes.com/video/education/100000005108366/classroom-waldorf-montessori-public.html">
Which Classroom Setting Suits You?
</a>

代码

import requests
from bs4 import BeautifulSoup 
import sys 
import codecs
sys.stdout = codecs.getwriter('utf8')(sys.stdout)

url = 'https://www.nytimes.com'

r = requests.get(url)

html_as_string = r.text

soup = BeautifulSoup(html_as_string, 'html.parser')

for link in soup.find_all('a'):
    print soup.a.string     ##Prints "LEARN MORE"
    print soup.a.text       ##Prints "LEARN MORE"
    print (link.get('text'))##prints "None"

Answer 1

代码:

from bs4 import BeautifulSoup

data = '''
<a href="https://www.nytimes.com/video/education/100000005108366/classroom-waldorf-montessori-public.html">
Which Classroom Setting Suits You?
</a>
'''

bs = BeautifulSoup(data, 'html.parser')
href = bs.find('a').text
print(href)

输出:

Which Classroom Setting Suits You?

这是我对问题进行第二次编辑后的代码:目前，该脚本从网站中提取所有 href 链接并打印其文本。您需要优化脚本以专注于您想要解析的特定 href 链接。

这是我对问题进行第二次编辑后的代码:

import requests
from bs4 import BeautifulSoup

url = 'https://www.nytimes.com'
r = requests.get(url)
html_as_string = r.text
soup = BeautifulSoup(html_as_string, 'html.parser')
for link in soup.find_all('a'):
    print link.text

输出:

LEARN MORE »
Skip to content
Skip to navigation
中文 (Chinese)
Español





Today’s Paper
Video
World
U.S.
Politics
N.Y.
Business
Business
Opinion
Opinion
Tech
Science
Health
Sports
Sports
Arts
Arts
Style
Style
Food
Food
Travel
Magazine
T Magazine
Real Estate
Trump Allows for Cheaper, Less Regulated Health Plans
 Comments




1
Open the door to relaxing A.C.A. rules on coverage.





2
Stop sending department officials to enrollment events.





3
Shut down the A.C.A. website for periods of time.





4
Cut funding to groups that help people enroll.





5
Slash spending on advertising and promotion.





Here Are 11 Ways Trump Is Undermining Obamacare
Trump Warns Puerto Rico: Federal Help Can’t Stay ‘Forever’
 Comments


Obliterated: One of the Few Tropical Rain Forests in the U.S.
U.S. Will Withdraw From Unesco, Citing ‘Anti-Israel Bias’
What’s It Like to Cover Trump? Our Reporters Answer Questions at 6:30 P.M. Eastern
California Fires Burn ‘Faster Than Firefighters Can Run’
 Comments
‘Thank God We Left’: Californians Describe Escapes


Satellite Images Show 1,800 Buildings Destroyed by Fire in Santa Rosa
The Daily 360













1:32



                                    Sifting Through the Charred Ruins                            






















More in 
The Daily 360 »


Weinstein Company Has Been Aware of Payouts Since 2015

 Comments
Facing Down the Network That Protected Weinstein
 Comments
How Did NBC Miss Out on a Weinstein Exposé?
Palestinian Factions Sign Deal Paving Way for Joint Rule
 Comments
The Bronx’s Quiet, Brutal War With Opioids
Comments

California Today: A Rising Death Toll From Wildfires
 Comments

Your Friday Briefing



Audio

      Listen to ‘The Daily’

Apple Podcasts
Radio Public
Stitcher
Smarter Living
What to Know About Having a Microwedding

How Would That Couch Look at Home? Check Your Phone

 Comments

Journalism that matters. More essential than ever.
Subscribe to the Times

From the Magazine

Philip Pullman Returns to His Fantasy World
 Comments
U.S. Woman and Family Freed From Afghan Militants

 Comments
South Carolina Sidesteps the Law, and the Poor Go to Jail

 Comments
Protesters in N.F.L. May Have an Ally: Labor Law

 Comments
Different Ads, Different Ethnicities, Same Car

Genes for Skin Color Rebut Notions of Race, Study Finds

 Comments
Citing Supreme Court, Judge Questions Menendez Case

 Comments
Leader of Philippines Threatens to Expel E.U. Diplomats
Qatari Businessman Accused of Bribing Ex-FIFA Chief
Yankees Complete Comeback From 0-2 to Reach A.L.C.S.
Seychelles Reports a Case of Plague
Fed Still Puzzled by Inflation, but Rate Increase Is on Track

Morning Briefing: Australia Edition

Morning Briefing: Asia Edition

Morning Briefing: Europe Edition

California Today

Morning Briefing
Got a confidential news tip?
Learn more





Aussie Rules: Readers Respond





Vegemite, the Beloved Australian Spread, Goes Upmarket





Australia’s New Offer to Refugees: A Different Detention Center





She Is New Yorkier Than You





Olympic Tales From a 105-Year-Old Baron





Art Review: Collecting Strokes of Genius
Opinion
Mr. Trump, I Live in South Korea. You’re Scaring Me.

The Trumps, The Poodle, The Sex Scandal

 Comments
Blow: Trump’s War Games
Kristof: North Korea Fears
Leonhardt: Falling Behind
Rosenthal: Trump’s Latest Outrage Against Puerto Rico
Stephens: Weinstein Enablers
Texas Should Not Execute Robert Pruett Tonight

Editorial: Trump Alone Can Order a Nuclear Strike
Edsall: Democrats Play Checkers, Trump Plays Chess
Greenhouse: Church Over State on Contraception
Has America Forgotten the Virgin Islands?
Congress’s Un-American, Bad-for-Israel Agenda
From Amazon, a Change That Hurts Authors
Ben Shapiro’s Hollow Bravery
What Makes Eminem’s Anti-Trump Rap Different
Getting Help Where It’s Needed in Puerto Rico
New York Times Recipe Commenters (Politely) Spill Their Guts
Smelly and Dirty and, in Bombing Trial, Suddenly a Star





Times Insider »
New York Times Recipe Commenters (Politely) Spill Their Guts
The Crossword »
Play Today’s Puzzle 



The Crossword »
Play Today’s Puzzle 



Wordplay »
timesvideo
 explore all videos »
Television




‘Dynasty’ or ‘Dallas’? Squaring Off

Opinion




You Are Special! Now Stop Being Different

Science




Mountain Lions Sometimes Share Food

Food




Our Best Pizza Recipes

Opinion




Getting Help Where It’s Needed in Puerto Rico

Mind




High Blood Pressure Tied to Later Dementia

Travel




What Martha Stewart Can’t Travel Without

Opinion




Publishing’s Unfair Gray Market

Live




How to Stop Bleeding and Save a Life

Television




Review: Netflix’s ‘Mindhunter’

Opinion




Has America Forgotten the Virgin Islands?

N.Y. / Region




Lawsuits Seek Access to College Websites

World »





        For Dignity and Development, East Africa Curbs Used Clothes Imports        


            Tech and Real Estate Lift the Fortunes of China’s Richest        

            Palestinian Factions, Hamas and Fatah, Sign Unity Deal        
Business Day »





        Sheryl Sandberg Says Facebook Will Hand Over More Data on Russia-Linked Ads        


            Different Ads, Different Ethnicities, Same Car        

            Kobe Steel Problems May Be More Widespread, Raising Fears on High-Speed Rail        
Opinion »





        Linda Greenhouse: On Contraception, It’s Church Over State        


            Editorial: Mr. Trump Alone Can Order a Nuclear Strike. Congress Can Change That.        

            Charles M. Blow: Trump’s War Games        
U.S. »





        We’re Tracking Ways Trump Is Undermining Obamacare. Here Are 11.        


            Another Victim of Hurricane Maria: Puerto Rico’s Treasured Rainforest        

            California Wildfires Burn ‘Faster Than Firefighters Can Run’        
Technology »


        Sheryl Sandberg Says Facebook Will Hand Over More Data on Russia-Linked Ads        


            State of the Art: The Frightful Five Want to Rule Entertainment. They Are Hitting Limits.        

            The Shift: We Asked Facebook 12 Questions About the Election, and Got 5 Answers        
Arts »





        Critic’s Notebook: Harvey Weinstein Is Gone. But Hollywood Still Has a Problem.        


            Rose McGowan’s Twitter Account Locked After Posts About Weinstein        

            ‘Dynasty’ or ‘Dallas’? Squaring Off in the Lily Pond        
Politics »





        Foiled in Congress, Trump Signs Order to Undermine Obamacare        


            U.S. Will Withdraw From Unesco, Citing Its ‘Anti-Israel Bias’        

            Trump’s Warning on Limits to U.S. Aid in Puerto Rico Adds ‘Insult to Injury,’ Mayor Says        
Fashion & Style »





        Unbuttoned: Fashion’s Gossip Addiction        


            With...Tom Hanks: Hollywood’s Most Decent Fella on Weinstein, Trump and History        

            Social Q’s: What Happens When a Will is Unsigned? (It’s Not Fun)        
Movies »





         All the Films of Studio Ghibli, Ranked        


            Critic’s Notebook: Harvey Weinstein Is Gone. But Hollywood Still Has a Problem.        

            Rose McGowan’s Twitter Account Locked After Posts About Weinstein        
New York »





        The Bronx’s Quiet, Brutal War With Opioids        


            Cy Vance Defends Decision Not to Pursue Case Against Harvey Weinstein        

            College Websites Must Accommodate Disabled Students, Lawsuits Say        
Sports »





        Yankees 5, Indians 2 | Yankees win series, 3-2: Didi Gregorius Leads Yankees Past Indians and Into A.L.C.S.        


            P.S.G.’s Nasser al-Khelaifi Accused of Bribing Ex-FIFA Chief Valcke        

            N.F.L. Week 6 Picks: Chiefs Over Steelers; Packers Over Vikings        
Theater »





        ‘Brilliant,’ 41 and Lost to AIDS: The Theater World Asks Why        


            She Was About to Quit Acting. Then She Got the Role of a Lifetime.        

            Review: A Richard III Who Steps Straight Out of the Land of Id        
Science »





        Matter: Genes for Skin Color Rebut Dated Notions of Race, Researchers Say        


            ScienceTake: How Kangaroo Rats Escape Rattlesnakes        

            Trilobites: Solitary Pumas Turn Out to Be Mountain Lions Who Lunch        
Obituaries »





        Vincent La Selva, Dauntless Founder of a People’s Opera Troupe, Dies at 88        


            Michel Jouvet, Who Unlocked REM Sleep’s Secrets, Dies at 91        

            Y.A. Tittle, Quarterback Who Led Giants to 3 Title Games, Dies at 90        
Television »





        ‘Dynasty’ or ‘Dallas’? Squaring Off in the Lily Pond        


            Review: Netflix’s ‘Mindhunter’ Is More Chatter Than Splatter        

            Ask a Showrunner: ‘Mr. Robot’ Season 3 Premiere: Sam Esmail on Parallel Universes and President Trump        
Health »





        In New Test for Obamacare, Iowa Seeks to Abandon Marketplace        


            Seychelles Reports a Case of Plague        

            Phys Ed: What Half-Marathons Teach Us About Running a Marathon        
Travel »





        36 Hours: 36 Hours in Aarhus, Denmark        


            Personal Journeys: Seeking Solitude in Japan’s Mountain Monasteries        

            Carry-On: What Martha Stewart Can’t Travel Without        
Books »





        American Beauties: Riding Shotgun on Martha Gellhorn’s Brave and Comic Adventures        


            Books of The Times: In Ron Chernow’s ‘Grant,’ an American Giant’s Makeover Continues        

            Feature: Philip Pullman Returns to His Fantasy World        
Education »





        On Campus: Campus Hate Lives on the Internet. Administrators Need to Catch Up.        


            Which Classroom Setting Suits You?        

            Your Money: A Game to Help Students Pay the Right Price for College        
Food »





        The Pour: Five Common Wine Myths, Debunked        


            Hungry City: Traditional Ethiopian Food in a Cosmopolitan Setting        

            Mike Isabella, Washington’s Restless Restaurateur        
Magazine »





        Feature: Philip Pullman Returns to His Fantasy World        


            Feature: In Northern Minnesota, Two Economies Square Off: Mining vs. Wilderness        

            Letter of Recommendation: Letter of Recommendation: Dunking        
Real Estate »





        The Hunt: Seeking the Peace of the Suburbs        


            On the Market: Homes for Sale in New York and Connecticut        

            On the Market: Homes for Sale in New York City        
The Upshot »





        A Bump Stock Ban Is Popular With the Public. But Experts Have Their Doubts.        


            The N.F.L. Is Now One of the Most Divisive Brands in the U.S.        

            Quality of Mercy: The Word May Be Toxic, but Amnesty Is Everywhere        
Times Insider »





        Why Tech is Starting to Make Me Uneasy        


            Times Insider: New York Times Recipe Commenters (Politely) Spill Their Guts        

            Smelly and Dirty and, in Bombing Trial, Suddenly a Star        
Automobiles »





        Wheels: For Electric Car Owners, ‘Range Anxiety’ Gives Way to ‘Charging Time Trauma’        


            Kobe Steel’s Falsified Data Is Another Blow to Japan’s Reputation        

            Different Ads, Different Ethnicities, Same Car        
T Magazine »





        Home and Work: A Paris Boudoir Where Everything’s Meant to Be Touched        


            Slinky Dresses, Meet Slouchy Trousers        

            The Illustrated Interview: Gloria Vanderbilt        
Real Estate »
International Rents Per Square Foot

 Comments
Search for Homes for Sale or Rent
Mortgage Calculator
Most Emailed
Most Viewed
Trending
Recommended for you
Go to Home Page »

The New York Times

World
U.S.
Politics
N.Y.
Business
Tech
Science
Health
Sports
Education
Obituaries
Today's Paper
Corrections
Today's Opinion
Op-Ed Columnists
Editorials
Op-Ed Contributors
Letters
Sunday Review
Video: Opinion
Today's Arts
Art & Design
Books
Dance
Movies
Music
N.Y.C. Events Guide
Television
Theater
Video: Arts
Automobiles
Crossword
Food
Education
Fashion & Style
Health
Jobs
Magazine
N.Y.C. Events Guide
Real Estate
T Magazine
Travel
Weddings & Celebrations
Reader Center
Classifieds
Tools & Services
N.Y.C. Events Guide
Multimedia
Photography
Video
NYT Store
Times Journeys
Subscribe
Manage My Account
NYTCo
Home Delivery
Digital Subscriptions
Crossword
Email Newsletters
Alerts
Gift Subscriptions
Corporate Subscriptions
Education Rate
Mobile Applications
Replica Edition

                    © 2017 The New York Times Company

Contact Us
Work With Us
Advertise
Your Ad Choices
Privacy
Terms of Service
Terms of Sale
Site Map
Help
Site Feedback
Subscriptions
View Mobile Version