python - 在 Python 3.5.1 文档中找不到方法 regex.scanner()，但解释器运行良好

Question

我正在通过 Python Cookbook，第 3 学习 Python。在第 67 页，这里有一个这样的示例代码

import re
NAME = r'(?P<NAME>[a-zA-Z_][a-zA-Z_0-9]*)'
NUM = r'(?P<NUM>\d+)'
PLUS = r'(?P<PLUS>\+)'
TIMES = r'(?P<TIMES>\*)'
EQ = r'(?P<EQ>=)'
WS = r'(?P<WS>\s+)'    
master_pat = re.compile('|'.join([NAME, NUM, PLUS, TIMES, EQ, WS]))
scanner = master_pat.scanner('foo = 42')
scanner.match()
 ......

我试图在 Python 标准文档中找到方法 regex.scanner() 的签名，但我失败了。没有关于 regex.scanner() 的内容。另一方面，示例代码在解释器中运行得很快.有谁知道这是什么情况？或者这只是CPython中缺少签名细节的常见情况？

Answer 1

这是一个hidden gem :-)

This is where things get interesting. For the last 15 years or so, there has been a completely undocumented feature in the regular expression engine: the scanner. The scanner is a property of the underlying SRE pattern object where the engine keeps matching after it found a match for the next one. There even exists an re.Scanner class (also undocumented) which is built on top of the SRE pattern scanner which gives this a slightly higher level interface.

The scanner as it exists in the re module is not very useful unfortunately for making the 'not matching' part faster, but looking at its sourcecode reveals how it's implemented: on top of the SRE primitives.