python - 如何使用 beautifulsoup 从嵌套表中获取值

Question

我需要获取下面示例 html 中每一行的名称和价格，但是当我使用 beatifulsoup 来 find_all('tr') 时，它会返回所有 tr 主表和嵌套表。仅提取每行的值(value)和价格的最佳方法是什么？

soup = BeautifulSoup(f, 'html.parser')
priceTable = soup.find('table', attrs={"class":"table table-hover table-responsive"})

上面是我所拥有的，它返回“所有”tr，包括嵌套表。

我需要的是获取前面每件商品的所有名称和价格，最后保存在csv文件中

 <table class="table table-hover table-responsive">
                                        <tbody><tr>
                    <td style="vertical-align: middle; width: 20%;" class="hidden-xs">
                                                    <img class="retailer-logo" data-placement="right" src="/images/20180813125BhYNMEK8lgOpXj3zxze53WmqeRWov7h.jpg" alt="Contact Energy" style="width:150px;" title="" data-original-title="" />
                                            </td>
                    <td style="vertical-align: middle; width: 75px;" class="hidden-xs">
                        <img src="/images/result-arrow.png" />
                    </td>
                    <td>
                        <table style="width: 100%;">
                            <tbody><tr class="visible-xs">
                                <td class="text-center" colspan="2">
  <img class="retailer-logo" data-placement="right" src="/images/20180813125BhYNMEK8lgOpXj3zxze53WmqeRWov7h.jpg" alt="Contact Energy" style="width:150px;" title="" data-original-title="" />
                                  </td>
                            </tr>
                            <tr>
                                <td colspan="3"><h4>Contact Energy Saver Plus</h4></td>
                            </tr>
                            <tr style="text-transform: uppercase">
                                <td width="150px">Electricity:</td>
                                <td>$242.85                                    <a class="plan-breakdown" data-placement="right" title="" data-original-title="&lt;table&gt;&lt;tr&gt;&lt;td&gt;Anytime&lt;/td&gt;&lt;td&gt;$0.334&lt;/td&gt;&lt;td&gt;per kWh&lt;/td&gt;&lt;tr&gt;&lt;td&gt;Daily&lt;/td&gt;&lt;td&gt;$0.333&lt;/td&gt;&lt;td&gt;per day&lt;/td&gt;&lt;tr&gt;&lt;td&gt;EA Levy&lt;/td&gt;&lt;td&gt;$0.0013&lt;/td&gt;&lt;td&gt;per kWh&lt;/td&gt;&lt;/table&gt;"><i class="glyphicon glyphicon-info-sign"> </i></a>
                                </td>
                            </tr>
                                                        <tr style="text-transform: uppercase">
                                <td>Discount:</td>
                                <td>$63.14                                    (26%)
                                                                    </td>
                            </tr>
                                                            <tr>
                                    <td colspan="3">
                                        <a class="plan-detail" data-placement="right" title="" data-original-title="&lt;ul&gt;&lt;li&gt;Provides fixed pricing until 31 June 2021 unless there are changes to taxes and levies.&lt;/li&gt;&lt;li&gt;24% Prompt Payment Discount when you pay on time. additional 1% discount for paying by direct debit (excl. credit card), and 1% discount for getting bills and correspondence by email. Up to 26% PPD available.&lt;/li&gt;&lt;li&gt;An early termination fee of $150 per contracted ICP if you terminate the contract before the end date�(31/06/2021). Fee may be waived if you are moving house and take Contact Energy to the new property.&lt;/li&gt;&lt;li&gt;Not available to prepay customers.&lt;/li&gt;&lt;/ul&gt;"><i class="glyphicon glyphicon-info-sign"> </i> What you need to know</a>
                                    </td>
                                </tr>
                                                        <tr class="visible-xs">
                                <td colspan="2">
                                    <h3 class="total">$179.71</h3>
                                                                            <div class="incentive">
                                            <b style="text-transform: uppercase">SPECIAL SwitchMe OFFER</b><br />
                                            Special PPD &amp; Fixed rates<br />
                                                                                            <a style="font-size: 0.9em;" class="incentive-info" title="" data-original-title="Receive�a special Prompt Payment Discount and fixed rates until 31 June 2021 unless there are changes to taxes and levies">More Info</a>
                                                                                    </div>
                                                                    </td>
                            </tr>
                            <tr class="visible-xs">
                                <td colspan="2">
                                                                            <form id="w0" action="/switch/" method="post">
<input type="hidden" name="_csrf" value="Hi21xBvkP6NpUl0UcaFwxn4U5-94Jj8KqEeprOfuG9tMfP2gStRY6RFrBGdF6gGvT0uM3CAQaVvOPpnq1IddtQ==" />                                        <input type="hidden" name="query_id" value="409884" />                                        <input type="hidden" name="plan_group_id" value="54" />                                        <input type="hidden" name="plan_stage_id" value="367" />                                        <button type="submit" class="btn btn-block btn-switch" style="max-width: 100%; margin-top: 10px">Switch Now!</button>                                        </form>                                        <div class="wannatalk" style="max-width: 100%">
                                            Want to talk?<br />
                                            Call our friendly team on<br />
                                            <b>0800 179 482</b>
                                        </div>
                                                                    </td>
                            </tr>
                        </tbody></table>
                    </td>
                    <td style="text-align: center" class="hidden-xs">
                        <h3 class="total">$179.71</h3>
                                                    <div class="incentive">
                                <b style="text-transform: uppercase">SPECIAL SwitchMe OFFER</b><br />
                                Special PPD &amp; Fixed rates<br />
                                                                    <a style="font-size: 0.9em;" class="incentive-info" title="" data-original-title="Receive�a special Prompt Payment Discount and fixed rates until 31 June 2021 unless there are changes to taxes and levies">More Info</a>
                                                            </div>
                                            </td>
                    <td class="hidden-xs">
                                                    <form id="w1" action="/switch/" method="post">
<input type="hidden" name="_csrf" value="Hi21xBvkP6NpUl0UcaFwxn4U5-94Jj8KqEeprOfuG9tMfP2gStRY6RFrBGdF6gGvT0uM3CAQaVvOPpnq1IddtQ==" />                            <input type="hidden" name="query_id" value="409884" />                            <input type="hidden" name="plan_group_id" value="54" />                            <input type="hidden" name="plan_stage_id" value="367" />                            <button type="submit" class="btn btn-block btn-switch">Switch Now!</button>                            </form>                            <div class="wannatalk">
                                Want to talk?<br />
                                Call our friendly team on<br />
                                <b>0800 179 482</b>
                            </div>
                                            </td>
                </tr>
                            <tr>
                    <td style="vertical-align: middle; width: 20%;" class="hidden-xs">
                                                    <img class="retailer-logo" data-placement="right" src="/images/20171013102LzWd_kdtQOk4yxxyZuCZBG6q7xIuClx.jpg" alt="Powershop" style="width:150px;" title="" data-original-title="" />
                                            </td>
                    <td style="vertical-align: middle; width: 75px;" class="hidden-xs">
                        <img src="/images/result-arrow.png" />
                    </td>
                    <td>
                        <table style="width: 100%;">
                            <tbody><tr class="visible-xs">
                                <td class="text-center" colspan="2">
                                                                            <img class="retailer-logo" data-placement="right" src="/images/20171013102LzWd_kdtQOk4yxxyZuCZBG6q7xIuClx.jpg" alt="Powershop" style="width:150px;" title="" data-original-title="" />
                                                                    </td>
                            </tr>
                            <tr>
                                <td colspan="3"><h4>Powershop Saver</h4></td>
                            </tr>
                            <tr style="text-transform: uppercase">
                                <td width="150px">Electricity:</td>
                                <td>$183.40                                    <a class="plan-breakdown" data-placement="right" title="" data-original-title="&lt;table&gt;&lt;tr&gt;&lt;td&gt;Anytime&lt;/td&gt;&lt;td&gt;$0.2508&lt;/td&gt;&lt;td&gt;per kWh&lt;/td&gt;&lt;tr&gt;&lt;td&gt;Daily&lt;/td&gt;&lt;td&gt;$0.30&lt;/td&gt;&lt;td&gt;per day&lt;/td&gt;&lt;tr&gt;&lt;td&gt;EA Levy&lt;/td&gt;&lt;td&gt;$0.00&lt;/td&gt;&lt;td&gt;per kWh&lt;/td&gt;&lt;/table&gt;"><i class="glyphicon glyphicon-info-sign"> </i></a>
                                </td>
                            </tr>
                                                        <tr style="text-transform: uppercase">
                                <td>Discount:</td>
                                <td>$0.00                                    (0%)
                                                                    </td>
                            </tr>
                                                            <tr>
                                    <td colspan="3">
                                        <a class="plan-detail" data-placement="right" title="" data-original-title="&lt;ul&gt;&lt;li&gt;The price estimate is based on forecast charges from Powershop for the next 12 months.&lt;/li&gt;&lt;li&gt;It assumes you purchase the Powershop Simple Saver powerpack once a month and special powerpacks that are made available from time to time.&lt;/li&gt;&lt;li&gt;This offer does not require a contract or a minimum supply period.&lt;/li&gt;&lt;li&gt;New customers will get a $150 power credit applied over their first 12 months ($25 straight away, $10 on the next 10�monthly account&#10;review periods, and a final credit of $25 in the final account review period of&#10;your first year as a Powershop customer).�&lt;/li&gt;&lt;/ul&gt;"><i class="glyphicon glyphicon-info-sign"> </i> What you need to know</a>
</td>
</tr>
<tr class="visible-xs">
<td colspan="2">
<h3 class="total">$183.40</h3>
<div class="incentive">
<b style="text-transform: uppercase">SPECIAL SwitchMe OFFER</b><br />
Get $150 off your bill over 12 months!<br />                                                                                            <a style="font-size: 0.9em;" class="incentive-info" title="" data-original-title="&lt;div&gt;&lt;div&gt;New customers will get a $150 power credit applied over their first 12 months ($25 straight away, then $10 for the next 10�monthly account&#10;review periods, and a final credit of $25 in the final account review period of&#10;your first year as a Powershop customer).&lt;/div&gt;&lt;div&gt;�&lt;/div&gt;&lt;/div&gt;&lt;div&gt;&lt;br&gt;&lt;/div&gt;&lt;div&gt;&lt;br&gt;&lt;/div&gt;">More Info</a>                                                                                    </div>
</td>
</tr>
<tr class="visible-xs">
<td colspan="2">
                                                                            <form id="w2" action="/switch/" method="post">
<input type="hidden" name="_csrf" value="Hi21xBvkP6NpUl0UcaFwxn4U5-94Jj8KqEeprOfuG9tMfP2gStRY6RFrBGdF6gGvT0uM3CAQaVvOPpnq1IddtQ==" />                                        <input type="hidden" name="query_id" value="409884" />                                        <input type="hidden" name="plan_group_id" value="53" />                                        <input type="hidden" name="plan_stage_id" value="273" />                                        <button type="submit" class="btn btn-block btn-switch" style="max-width: 100%; margin-top: 10px">Switch Now!</button>                                        </form><div class="wannatalk" style="max-width: 100%">
Want to talk?<br />
Call our friendly team on<br />
<b>0800 179 482</b>
</div>
</td>
</tr>
</tbody></table>
</td>

<小时/>

所以输出应该是:来自第一行的 td[3] 和 td[4]:

联系节能加号179.71 美元

然后是下一行:

Powershop 保护程序183.40 美元

依此类推，直到最后一行(主表)。

Answer 1

与评论中给出的过程类似，但选择器不同

from bs4 import BeautifulSoup as bs

html = '''yourhtml'''

soup = bs(html, 'lxml')
names = [item.text for item in soup.select('.table h4 ')]
prices = [item.text for item in soup.select('[colspan="2"] > .total')]
results = list(zip(names, prices))
print(results)