Python如何使用suds MessagePlugin和lxml仅解码xml中的特定部分

Question

我正在从端点获取产品信息。为了解析该信息，我使用了一个过滤器，即 suds MessagePlugin。
传入的数据如下:(即不包含空洞请求。它包含其中的一小部分)

<env:Envelope xmlns:env='http://schemas.xmlsoap.org/soap/envelope/'><env:Header></env:Header><env:Body><prod:getProductsResponse xmlns:prod='https://product.individual.ns.listinsgapi.aa.com'><return><ackCode>success</ackCode><responseTime>13/09/2021 09:47:34</responseTime><timeElapsed>211 ms</timeElapsed><productCount>199</productCount><products><product><productId>01201801947</productId><product><categoryCode>cn1g</categoryCode><storeCategoryId>0</storeCategoryId><title>Morphy Richards Sensörlü çöp kutusu, 30 litre, yuvarlak, siyah paslanmaz çelik</title><specs><spec required="false" value="Standart Çöp Kovası" name="Ürün Tipi"/><spec required="false" value="Montajsız" name="Montaj Tipi"/><spec required="false" value="Sensörlü Kapak" name="Kapak Tipi"/><spec required="false" value="26 lt-30 lt" name="İç Hacim"/><spec required="false" value="Çelik" name="Malzeme"/><spec required="false" value="Sıfır" name="Durum"/></specs><photos><photo photoId="0"><url>https://mcdn301.gi1ttigidliyor.net/622080/620801947_0.jpg</url></photo><photo photoId="1"><url>https://mcdn011.gittigidliyor.net/620380/62081101947_1.jpg</url></photo><photo photoId="2"><url>https://mcdn021.gittigidliyor.net/620180/6210801947_2.jpg</url></photo><photo photoId="3"><url>https://mcdn201.gittigidliyor.net/620850/6208013947_3.jpg</url></photo><photo photoId="4"><url>https://mcdn301.gittigidliyor.net/623080/6208101947_4.jpg</url></photo><photo photoId="5"><url>https://mcdn01.gittigidiyor.net/62080/620801947_5.jpg</url></photo><photo photoId="6"><url>https://mcdn01.gittigidiyor.net/62080/620801947_6.jpg</url></photo></photos><pageTemplate>4</pageTemplate><description>&lt;body&gt;
 &lt;ul class=&quot;a-unordered-list a-vertical a-spacing-mini&quot; style=&quot;padding-right: 0px; padding-left: 0px; box-sizing: border-box; margin: 0px 0px 0px 18px; color: rgb(17, 17, 17); font-family: &quot;&gt; 
  &lt;li style=&quot;box-sizing: border-box; list-style: disc; overflow-wrap: break-word; margin: 0px;&quot;&gt;&amp;nbsp; &lt;h2 style=&quot;box-sizing: border-box; padding: 0px 0px 4px; margin: 3px 0px 7px; text-rendering: optimizelegibility; line-height: 32px; font-family: &quot;&gt;Ürün Bilgileri&lt;/h2&gt; &lt;span style=&quot;background-color:rgb(255, 255, 255); box-sizing:border-box; color:rgb(15, 17, 17); font-family:amazon ember,arial,sans-serif; font-size:14px&quot;&gt;Renk:&lt;strong style=&quot;box-sizing:border-box; font-weight:700&quot;&gt;Paslanmaz Çelik&lt;/strong&gt;&lt;/span&gt; 
   &lt;div class=&quot;a-row a-spacing-top-base&quot; style=&quot;box-sizing: border-box; width: 1213px; color: rgb(15, 17, 17); font-family: &quot;&gt; 
    &lt;div class=&quot;a-column a-span6&quot; style=&quot;box-sizing: border-box; margin-right: 24.25px; float: left; min-height: 1px; overflow: visible; width: 593.734px;&quot;&gt; 
     &lt;div class=&quot;a-row a-spacing-base&quot; style=&quot;box-sizing: border-box; width: 593.734px; margin-bottom: 12px !important;&quot;&gt; 
      &lt;div class=&quot;a-row a-expander-container a-expander-extend-container&quot; style=&quot;box-sizing: border-box; width: 593.734px;&quot;&gt; 
       &lt;div class=&quot;a-row&quot; style=&quot;box-sizing: border-box; width: 593.734px;&quot;&gt;

我只想将 html 解码应用于信息的描述部分。因为某些产品的描述部分由于某种原因出现错误，因为传入信息中的html标签没有完全解析。
例如:

0979c08d37cd.CR0,0,2000,2000_PT0_SX220_.jpgstyle=-webkit-tap-highlight-color:transparent; border:none;box-sizing:border-box; display:block; margin:0px auto; max-width:100%;padding:0px; vertical-align:top/p /div /th /tr /tbody /table /div /div/div /div /div /div /div /div /div /body

就我解决这个问题而言，我尝试了两种不同的方法。
在深入研究方法之前:

context: The reply context.The I{reply} is the raw text.context.reply = incoming data type(context.reply) = Bytes

class UnicodeFilter(MessagePlugin):

    def received(self, context):
        from lxml import etree
        from io import BytesIO

        parser = etree.XMLParser(recover=True)
        request_string = context.reply.decode("utf-8")
        replaced_string = request_string.replace("&gt;", ">").replace("&lt;", "<")

        byte_rep_string = str.encode(replaced_string)
      
        doc = etree.parse(BytesIO(byte_rep_string), parser)
        byte_str_doc = etree.tostring(doc)
        context.reply = byte_str_doc

这种方法不起作用。它没有抛出错误，但描述正文中的 html 标签没有变化，产品描述部分中的 html 标签仍然损坏。
2.

class UnicodeFilter(MessagePlugin):

    def received(self, context):
        from lxml import etree
        from io import BytesIO
        import html

        parser = etree.XMLParser(recover=True) # Initialize the parser
        request_string = context.reply.decode("utf-8") # Converting incoming data byte to string
        html_decoded = html.unescape(request_string) # Html decoding to the data
        byte_rep_string = str.encode(html_decoded) # Converting the data from string to byte
      
        doc = etree.parse(BytesIO(byte_rep_string), parser)
        byte_str_doc = etree.tostring(doc)
        context.reply = byte_str_doc

在这种方法中，我得到了 TypeNotFound: Type not found: 'body' 错误。
总结一下我想做什么。我想使用 lxml 库解析传入的数据，因为数据中的某些字符可能会导致问题，并且出现“格式不正确(无效 token )”错误。 (我解决了这个)。其次，我只想对这些数据的描述部分进行 html 解码。 (修复html标签问题)
任何帮助都会很棒。

Answer 1

我不确定我是否可以重现您的特定错误，但我会使用 etree.fromstring() 来使用这种方法。一旦您获得了请求中的字符串。 (我已经尝试清理并关闭测试数据的标签，以便对其进行解析以演示解决方案。还有一个额外的 <product> 标签可以防止您可能需要处理的解析。)

In [104]: import lxml

In [105]: string = '''<env:Envelope xmlns:env='http://schemas.xmlsoap.org/soap/envelope/'><env:Header></env:Header><env:Body><pr
     ...: od:getProductsResponse xmlns:prod='https://product.individual.ns.listinsgapi.aa.com'><return><ackCode>success</ackCode
     ...: ><responseTime>13/09/2021 09:47:34</responseTime><timeElapsed>211 ms</timeElapsed><productCount>199</productCount><pro
     ...: ducts><product><productId>01201801947</productId><categoryCode>cn1g</categoryCode><storeCategoryId>0</storeCategoryId>
     ...: <title>Morphy Richards Sensörlü çöp kutusu, 30 litre, yuvarlak, siyah paslanmaz çelik</title><specs><spec required="fa
     ...: lse" value="Standart Çöp Kovası" name="Ürün Tipi"/><spec required="false" value="Montajsız" name="Montaj Tipi"/><spec 
     ...: required="false" value="Sensörlü Kapak" name="Kapak Tipi"/><spec required="false" value="26 lt-30 lt" name="İç Hacim"/
     ...: ><spec required="false" value="Çelik" name="Malzeme"/><spec required="false" value="Sıfır" name="Durum"/></specs><phot
     ...: os><photo photoId="0"><url>https://mcdn301.gi1ttigidliyor.net/622080/620801947_0.jpg</url></photo><photo photoId="1"><
     ...: url>https://mcdn011.gittigidliyor.net/620380/62081101947_1.jpg</url></photo><photo photoId="2"><url>https://mcdn021.gi
     ...: ttigidliyor.net/620180/6210801947_2.jpg</url></photo><photo photoId="3"><url>https://mcdn201.gittigidliyor.net/620850/
     ...: 6208013947_3.jpg</url></photo><photo photoId="4"><url>https://mcdn301.gittigidliyor.net/623080/6208101947_4.jpg</url><
     ...: /photo><photo photoId="5"><url>https://mcdn01.gittigidiyor.net/62080/620801947_5.jpg</url></photo><photo photoId="6"><
     ...: url>https://mcdn01.gittigidiyor.net/62080/620801947_6.jpg</url></photo></photos><pageTemplate>4</pageTemplate><descrip
     ...: tion>&lt;body&gt;
     ...:  &lt;ul class=&quot;a-unordered-list a-vertical a-spacing-mini&quot; style=&quot;padding-right: 0px; padding-left: 0px
     ...: ; box-sizing: border-box; margin: 0px 0px 0px 18px; color: rgb(17, 17, 17); font-family: &quot;&gt; 
     ...:   &lt;li style=&quot;box-sizing: border-box; list-style: disc; overflow-wrap: break-word; margin: 0px;&quot;&gt;&amp;n
     ...: bsp; &lt;h2 style=&quot;box-sizing: border-box; padding: 0px 0px 4px; margin: 3px 0px 7px; text-rendering: optimizeleg
     ...: ibility; line-height: 32px; font-family: &quot;&gt;Ürün Bilgileri&lt;/h2&gt; &lt;span style=&quot;background-color:rgb
     ...: (255, 255, 255); box-sizing:border-box; color:rgb(15, 17, 17); font-family:amazon ember,arial,sans-serif; font-size:14
     ...: px&quot;&gt;Renk:&lt;strong style=&quot;box-sizing:border-box; font-weight:700&quot;&gt;Paslanmaz Çelik&lt;/strong&gt;
     ...: &lt;/span&gt; 
     ...:    &lt;div class=&quot;a-row a-spacing-top-base&quot; style=&quot;box-sizing: border-box; width: 1213px; color: rgb(15
     ...: , 17, 17); font-family: &quot;&gt; 
     ...:     &lt;div class=&quot;a-column a-span6&quot; style=&quot;box-sizing: border-box; margin-right: 24.25px; float: left;
     ...:  min-height: 1px; overflow: visible; width: 593.734px;&quot;&gt; 
     ...:      &lt;div class=&quot;a-row a-spacing-base&quot; style=&quot;box-sizing: border-box; width: 593.734px; margin-botto
     ...: m: 12px !important;&quot;&gt; 
     ...:       &lt;div class=&quot;a-row a-expander-container a-expander-extend-container&quot; style=&quot;box-sizing: border-
     ...: box; width: 593.734px;&quot;&gt; 
     ...:        &lt;div class=&quot;a-row&quot; style=&quot;box-sizing: border-box; width: 593.734px;&quot;&gt;
     ...: </description>
     ...: </product>
     ...: </products>
     ...: </return>
     ...: </prod:getProductsResponse>
     ...: </env:Body>
     ...: </env:Envelope>'''

In [106]: root = lxml.etree.fromstring(string)

In [108]: descriptions = root.xpath('//description')

In [109]: description = descriptions[0]

In [110]: description.text
Out[110]: '<body>\n <ul class="a-unordered-list a-vertical a-spacing-mini" style="padding-right: 0px; padding-left: 0px; box-sizing: border-box; margin: 0px 0px 0px 18px; color: rgb(17, 17, 17); font-family: "> \n  <li style="box-sizing: border-box; list-style: disc; overflow-wrap: break-word; margin: 0px;">&nbsp; <h2 style="box-sizing: border-box; padding: 0px 0px 4px; margin: 3px 0px 7px; text-rendering: optimizelegibility; line-height: 32px; font-family: ">Ürün Bilgileri</h2> <span style="background-color:rgb(255, 255, 255); box-sizing:border-box; color:rgb(15, 17, 17); font-family:amazon ember,arial,sans-serif; font-size:14px">Renk:<strong style="box-sizing:border-box; font-weight:700">Paslanmaz Çelik</strong></span> \n   <div class="a-row a-spacing-top-base" style="box-sizing: border-box; width: 1213px; color: rgb(15, 17, 17); font-family: "> \n    <div class="a-column a-span6" style="box-sizing: border-box; margin-right: 24.25px; float: left; min-height: 1px; overflow: visible; width: 593.734px;"> \n     <div class="a-row a-spacing-base" style="box-sizing: border-box; width: 593.734px; margin-bottom: 12px !important;"> \n      <div class="a-row a-expander-container a-expander-extend-container" style="box-sizing: border-box; width: 593.734px;"> \n       <div class="a-row" style="box-sizing: border-box; width: 593.734px;">\n'

In [112]: html_root = lxml.etree.fromstring(description.text, lxml.etree.HTMLParser())

In [114]: lxml.etree.tostring(html_root)
Out[114]: b'<html><body>\n <ul class="a-unordered-list a-vertical a-spacing-mini" style="padding-right: 0px; padding-left: 0px; box-sizing: border-box; margin: 0px 0px 0px 18px; color: rgb(17, 17, 17); font-family: "> \n  <li style="box-sizing: border-box; list-style: disc; overflow-wrap: break-word; margin: 0px;">&#160; <h2 style="box-sizing: border-box; padding: 0px 0px 4px; margin: 3px 0px 7px; text-rendering: optimizelegibility; line-height: 32px; font-family: ">&#220;r&#252;n Bilgileri</h2> <span style="background-color:rgb(255, 255, 255); box-sizing:border-box; color:rgb(15, 17, 17); font-family:amazon ember,arial,sans-serif; font-size:14px">Renk:<strong style="box-sizing:border-box; font-weight:700">Paslanmaz &#199;elik</strong></span> \n   <div class="a-row a-spacing-top-base" style="box-sizing: border-box; width: 1213px; color: rgb(15, 17, 17); font-family: "> \n    <div class="a-column a-span6" style="box-sizing: border-box; margin-right: 24.25px; float: left; min-height: 1px; overflow: visible; width: 593.734px;"> \n     <div class="a-row a-spacing-base" style="box-sizing: border-box; width: 593.734px; margin-bottom: 12px !important;"> \n      <div class="a-row a-expander-container a-expander-extend-container" style="box-sizing: border-box; width: 593.734px;"> \n       <div class="a-row" style="box-sizing: border-box; width: 593.734px;">\n</div></div></div></div></div></li></ul></body></html>'

如果在此之后需要操作html，最好操作 html_root而不是试图操纵字符串。如果是这样，我可以根据需要扩展答案。