php - 正则表达式 - Unicode 属性引用和示例

Question

我对 RegexBuddy 提供的 Regex Unicode 属性感到迷茫，我无法区分任何数字属性，而数学符号属性似乎只匹配 +但不是 - , * , / , ^例如。



是否有任何文档/引用与正则表达式 Unicode 属性的示例？

Answer 1

Unicode 字符属性

您在示例中列出的那些实际上都是相同的 Unicode 字符属性，即 General Category 属性。一些正则表达式系统仅提供对这一属性的访问；其他包括访问 Block 属性(不是很有用)或 Script 属性( 更 更有用)。

在 🐪 Programming Perl, 4^th edition 的第 209 页的以下文本中给出了对 Perl 正则表达式中的 \p{Property Name} 和 \p{Property Name = Property Value} 语法的更完整的解释，在此经其作者的许可转载:😼

All standard Unicode properties are actually composed of two parts, as in \p{NAME=VALUE}. All one-part properties are therefore additions to official Unicode properties. Boolean properties whose values are true can always be abbreviated as one-part properties, which allows you to write \p{Lowercase} for \p{Lowercase=True}. Other types of properties besides Boolean properties take string, numeric, or enumerated values. Perl also provides one-part aliases for all general category, script, and block properties, plus the level-one recommendations from Unicode Technical Standard #18 on Regular Expressions (version 13, from 2008-08), such as \p{Any}.

For example, \p{Armenian}, \p{IsArmenian}, and \p{Script=Armenian} all represent the same property, as do \p{Lu}, \p{GC=Lu}, \p{Uppercase_Letter}, and \p{General_Category=Uppercase_Letter}. Other examples of binary properties (those whose values are implicitly true) include \p{Whitespace}, \p{Alphabetic}, \p{Math}, and \p{Dash}. Examples of properties that aren’t binary properties include \p{Bidi_Class=Right_to_Left}, \p{Word_Break=A_Letter}, and \p{Numeric_Value=10}. The perluniprops manpage lists all properties and their aliases that Perl supports, both standard Unicode properties and the Perl specials, too.

Unicode 字符属性及其含义的完整列表记录在 section 5 on Properties from UAX#44, the Unicode Character Database 中。必须支持 才能满足 UTS#18’s RL 1.2 的这 11 个属性是:

RL1.2 Properties

To meet this requirement, an implementation shall provide at least a minimal list of properties, consisting of the following:

• General_Category
• Script
• Alphabetic
• Uppercase
• Lowercase
• White_Space
• Noncharacter_Code_Point
• Default_Ignorable_Code_Point
• ANY, ASCII, ASSIGNED

请注意，像 \w 、 \d 、 \s 、 \b 这样的单字母字符类缩写，以及它们的大写补码，以及像 \p{alpha} 这样的 POSIX 发音名称，它们本身是根据以下定义的 UTS#18’s Annex C on Compatibility Properties 中的 Unicode 字符属性。

据我所知，目前满足 Level 1 requirements of UTS#18 for Basic Unicode Support 的 只有 正则表达式引擎是 Perl 、 ICU’s regex library for C and C++ 、 Java 7’s Pattern class 和 Matthew Barnett’s excellent regexp library for Python 2 and Python 3 。 Android 中使用的正则表达式实际上是 ICU 的，而不是人们想象的 Java 的，因此使用 Unicode 效果更好。

对于 Java 7，您必须使用 UNICODE_CHARACTER_CLASS 模式编译标志或嵌入的 (?U) 来获得 RL1.2a ( \w &c) 的东西。对于 PCRE ，您似乎需要嵌入 (*PCRE_UCP) ，或将其用作编译标志。这可能取决于您的 php 版本是如何构建的，这可能是一个问题。

Russ Cox 的 RE2 library ，具有可用于 C 和 C++ 的绑定(bind)，加上 Perl regex engine plugin 和现在的 the standard regex library used by Go programming language ，支持两个最重要的属性，通用类别和脚本。

PCRE 和 PHP

我相信 PCRE 距离满足 RL 1.2 对属性的要求还有很长的路要走。它同时处理 General Category 和 Script 属性，这是两个最重要和最常用的属性，但似乎没有让您获得其他九个必需属性。其与 POSIX 兼容的属性 lkike alpha 、 upper 、 lower 和 space 被专门记录为仅 7 位 ASCII，这违反了 RL 1.2a 。但是，PCRE 还提供以下特价商品:

Xan 字母数字:属性 L 和 N 的并集

Xps POSIX 空格:属性 Z 或制表符、NL、VT、FF、CR

Xsp Perl 空格:属性 Z 或制表符、NL、FF、CR

Xwd Perl 字:属性 Xan 或下划线

请注意，PCRE 的 \p{Xan} 仍然不同于 Unicode 所说的 \p{alnum} 必须表示的含义，因为它缺少例如组合标记和某些字母符号。 Perl \p{alnum} 遵循 Unicode 定义。总的来说，PCRE 的 \p{Xwd} 与 Unicode(和 Perl)的不同之处在于它缺少额外的字母和其余的 \p{GC=Connector_Punctuation} 字符。 UTS#18 的下一个修订版还将 \p{Join_Control} 添加到 \p{word} 字符集。

更多房产

在满足 RL 1.2 和 RL 1.2a 的这四个中，除了 Java 7 之外，其他所有版本都满足(或非常接近满足，有时使用替代语法，如 \N{…} 代替 \p{name=…} 语法) the new RL 2.7 on Full Properties 来自提议本月早些时候发布的 UTS#18 更新，部分内容如下:

RL2.7 Full Properties

To meet this requirement, an implementation shall support all of the properties listed below that are in the supported version of Unicode, with values that match the Unicode definitions for that version.

To meet requirement RL2.7, the implementation must satisfy the Unicode definition of the properties for the supported version of Unicode, rather than other possible definitions. However, the names used by the implementation for these properties may differ from the formal Unicode names for the properties. For example, if a regex engine already has a property called "Alphabetic", for backwards compatibility it may need to use a distinct name, such as "Unicode_Alphabetic", for the corresponding property listed in RL1.2.

[table omitted for brevity —tchrist]

The Name and Name_Alias properties are used in \p{name=…} and \N{…}. The data in NamedSequences.txt is also used in \N{…}. For more information see Section 2.5, Name Properties. The Script and Script_Extensions properties are used in \p{scx=…}. For more information, see Section 1.2.2, Script_Property. The list excludes contributory, obsolete, and deprecated properties, most provisional properties, and the Unicode_1_Name and Unicode_Radical_Stroke properties. The properties in gray are covered by RL1.2 Properties. For more information on properties, see UAX #44, Unicode Character Database [UAX44].

Unicode 属性探索工具

您可能希望方便地探索 Unicode 字符属性的三个独立工具是 uniprops ，
unichars 和 *uninames 。它们也可作为较大的 Unicode::Tussle suite from CPAN 的一部分。

快速演示:

$ uniprops -a 3b1
U+03B1 ‹α› \N{GREEK SMALL LETTER ALPHA}
    \w \pL \p{LC} \p{L_} \p{L&} \p{Ll}
    All Any Alnum Alpha Alphabetic Assigned Greek Is_Greek InGreek Cased Cased_Letter LC
       Changes_When_Casemapped CWCM Changes_When_Titlecased CWT Changes_When_Uppercased CWU Ll L Gr_Base
       Grapheme_Base Graph GrBase Grek Greek_And_Coptic ID_Continue IDC ID_Start IDS Letter L_
       Lowercase_Letter Lower Lowercase Print Word XID_Continue XIDC XID_Start XIDS X_POSIX_Alnum
       X_POSIX_Alpha X_POSIX_Graph X_POSIX_Lower X_POSIX_Print X_POSIX_Word
    Age=1.1 Bidi_Class=L Bidi_Class=Left_To_Right BC=L Block=Greek Block=Greek_And_Coptic BLK=Greek
       Canonical_Combining_Class=0 Canonical_Combining_Class=Not_Reordered CCC=NR
       Canonical_Combining_Class=NR Decomposition_Type=None DT=None East_Asian_Width=A
       East_Asian_Width=Ambiguous EA=A Grapheme_Cluster_Break=Other GCB=XX Grapheme_Cluster_Break=XX
       Script=Greek Hangul_Syllable_Type=NA Hangul_Syllable_Type=Not_Applicable HST=NA
       Joining_Group=No_Joining_Group JG=NoJoiningGroup Joining_Type=Non_Joining JT=U Joining_Type=U
       Line_Break=AL Line_Break=Alphabetic LB=AL Numeric_Type=None NT=None Numeric_Value=NaN NV=NaN
       Present_In=1.1 IN=1.1 Present_In=2.0 IN=2.0 Present_In=2.1 IN=2.1 Present_In=3.0 IN=3.0 Present_In=3.1
       IN=3.1 Present_In=3.2 IN=3.2 Present_In=4.0 IN=4.0 Present_In=4.1 IN=4.1 Present_In=5.0 IN=5.0
       Present_In=5.1 IN=5.1 Present_In=5.2 IN=5.2 Present_In=6.0 IN=6.0 SC=Grek Script=Grek
       Sentence_Break=LO Sentence_Break=Lower SB=LO Word_Break=ALetter WB=LE Word_Break=LE

$ unichars '\pN' '\D' '\p{Latin}'
 Ⅰ      8544  02160  ROMAN NUMERAL ONE
 Ⅱ      8545  02161  ROMAN NUMERAL TWO
 Ⅲ      8546  02162  ROMAN NUMERAL THREE
 Ⅳ      8547  02163  ROMAN NUMERAL FOUR
 Ⅴ      8548  02164  ROMAN NUMERAL FIVE
 Ⅵ      8549  02165  ROMAN NUMERAL SIX
 Ⅶ      8550  02166  ROMAN NUMERAL SEVEN
 Ⅷ      8551  02167  ROMAN NUMERAL EIGHT
 (etc)

$ uninames Old English
 æ  00E6        LATIN SMALL LETTER AE
        = latin small ligature ae (1.0)
        = ash (from Old English æsc)
        * Danish, Norwegian, Icelandic, Faroese, Old English, French, IPA
        x (latin small ligature oe - 0153)
        x (cyrillic small ligature a ie - 04D5)
 ð  00F0        LATIN SMALL LETTER ETH
        * Icelandic, Faroese, Old English, IPA
        x (latin capital letter eth - 00D0)
        x (greek small letter delta - 03B4)
        x (partial differential - 2202)
 þ  00FE        LATIN SMALL LETTER THORN
        * Icelandic, Old English, phonetics
        * Runic letter borrowed into Latin script
        x (runic letter thurisaz thurs thorn - 16A6)
 œ  0153        LATIN SMALL LIGATURE OE
        = ethel (from Old English eðel)
        * French, IPA, Old Icelandic, Old English, ...
        x (latin small letter ae - 00E6)
        x (latin letter small capital oe - 0276)
 ƿ  01BF        LATIN LETTER WYNN
        = wen
        * Runic letter borrowed into Latin script
        * replaced by "w" in modern transcriptions of Old English
        * uppercase is 01F7
        x (runic letter wunjo wynn w - 16B9)
 ǣ  01E3        LATIN SMALL LETTER AE WITH MACRON
        * Old Norse, Old English
        : 00E6 0304
 ⁊  204A        TIRONIAN SIGN ET
        * Irish Gaelic, Old English, ...
        x (ampersand - 0026)