python - 如何将自定义解析器与 Python 的 Beautiful Soup 一起使用？

Question

我正在使用 Beautiful Soup 4解析和修改一组 HTML 文件。 HTML 文件是 Angular 模板，这意味着它们的标记在某种程度上与常规 HTML 文档中的标记不同(混合大小写属性、指令、输入/输出绑定(bind)等)。

Beautiful Soups documentation 中没有列出任何解析器(html.parser, lxml, html5lib) 完全符合我的需要。最接近它的是 Python 内置的 html.parser，但我不得不对其进行一些调整。

是否可以将自定义解析器类与 Beautiful Soup 一起使用？如果是，如何实现？

编辑: 下面是 Beautiful Soup 如何解析模板的示例。主要问题是结果中的所有属性都是小写的(*ngIf -> *ngif)并且指令(appAutoFocus、appKeepFocusInside 等)得到一个空字符串值(例如 appautofocus="")。

from bs4 import BeautifulSoup

test_html = """
<kendo-dialog *ngIf="dialogOpened" appAutoFocus appKeepFocusInside (close)="closeDialog()" width="1000">
   <kendo-dialog-titlebar>A title</kendo-dialog-titlebar>
   <div *ngIf="model.showValidationFailedMessage" class="alert alert-danger alert-dismissible">
      <button type="button" class="close" aria-label="Close" (click)="model.closeValidationAlert()"><span
         aria-hidden="true">&amp;times;</span></button>
      Validation failed
   </div>

   <kendo-tabstrip [keepTabContent]="true">
      <kendo-tabstrip-tab title="Tab title 1" [selected]="true">
         <ng-template kendoTabContent>
            <div>Tab content</div>
         </ng-template>
      </kendo-tabstrip-tab>
   </kendo-tabstrip>
</kendo-dialog>
"""

document = BeautifulSoup(test_html, "html.parser")
print(document.prettify())

结果:

<kendo-dialog (close)="closeDialog()" *ngif="dialogOpened" appautofocus="" appkeepfocusinside="" width="1000">
 <kendo-dialog-titlebar>
  A title
 </kendo-dialog-titlebar>
 <div *ngif="model.showValidationFailedMessage" class="alert alert-danger alert-dismissible">
  <button (click)="model.closeValidationAlert()" aria-label="Close" class="close" type="button">
   <span aria-hidden="true">
    &amp;times;
   </span>
  </button>
  Validation failed
 </div>
 <kendo-tabstrip [keeptabcontent]="true">
  <kendo-tabstrip-tab [selected]="true" title="Tab title 1">
   <ng-template kendotabcontent="">
    <div>
     Tab content
    </div>
   </ng-template>
  </kendo-tabstrip-tab>
 </kendo-tabstrip>
</kendo-dialog>

Answer 1

您可以使用 bs4.builder.register_treebuilders_from .内置构建器是 bs4.builder 中的模块包裹。例如，bs4.builder._lxml .

这是 register_treebuilders_from。

def register_treebuilders_from(module):
    """Copy TreeBuilders from the given module into this module."""
    # I'm fairly sure this is not the best way to do this.
    this_module = sys.modules['bs4.builder']
    for name in module.__all__:
        obj = getattr(module, name)

        if issubclass(obj, TreeBuilder):
            setattr(this_module, name, obj)
            this_module.__all__.append(name)
            # Register the builder while we're at it.
            this_module.builder_registry.register(obj)

和包的根模块的尾部。

# Builders are registered in reverse order of priority, so that custom
# builder registrations will take precedence. In general, we want lxml
# to take precedence over html5lib, because it's faster. And we only
# want to use HTMLParser as a last result.
from . import _htmlparser
register_treebuilders_from(_htmlparser)
try:
    from . import _html5lib
    register_treebuilders_from(_html5lib)
except ImportError:
    # They don't have html5lib installed.
    pass
try:
    from . import _lxml
    register_treebuilders_from(_lxml)
except ImportError:
    # They don't have lxml installed.
    pass

因此，您需要创建一个带有 bs4.builder.TreeBuilder 子类的模块，并将其注册到模块的 __all__ 中。然后将模块传递给 register_treebuilders_from。