awk - 您如何分辨哪些字符属于哪些字符类？

Question

我们经常看到人们问为什么他们的字符串与正则表达式不匹配的问题，有时答案归结为他们期望一个字符成为字符类的一部分，而不是他们或他们试图使用一个字符的简写形式(例如\d的[[:digit:]])存在于其他一些工具中，但根本不是awk语言的一部分。因此，考虑到这一点，我正在为awk中哪些字符类中存在哪些字符的问题创建一个规范答案。

Answer 1

下面的脚本将为您列出的数字范围为0-127的字符的语言环境生成每个字符类中的字符集(如果您的awk支持它们，则还要加上\s，\S，\w和\W扩展名)。在http://www.asciitable.com/和https://en.wikipedia.org/wiki/ASCII的第一个表中。对于print "\t"输出的水平制表符，第一个引用使用TAB，另一个HT作为缩写-我更喜欢TAB，所以我在下面使用它。他们都使用Space来表示print " "输出的char，因此即使我更常将其称为“空白char”，我也在下面进行了说明:

$ cat prtCharClasses.awk
# From the gawk manual, https://www.gnu.org/software/gawk/manual/gawk.html#Bracket-Expressions:
#   [:alnum:]   Alphanumeric characters
#   [:alpha:]   Alphabetic characters
#   [:blank:]   Space and TAB characters
#   [:cntrl:]   Control characters
#   [:digit:]   Numeric characters
#   [:graph:]   Characters that are both printable and visible (a space is printable but not visible, whereas an ‘a’ is both)
#   [:lower:]   Lowercase alphabetic characters
#   [:print:]   Printable characters (characters that are not control characters)
#   [:punct:]   Punctuation characters (characters that are not letters, digits, control characters, or space characters)
#   [:space:]   Space characters (these are: space, TAB, newline, carriage return, formfeed and vertical tab)
#   [:upper:]   Uppercase alphabetic characters
#   [:xdigit:]  Characters that are hexadecimal digits
#   \s          Matches any whitespace character. Think of it as shorthand for ‘[[:space:]]’.
#   \S          Matches any character that is not whitespace. Think of it as shorthand for ‘[^[:space:]]’.
#   \w          Matches any word-constituent character—that is, it matches any letter, digit, or underscore. Think of it as shorthand for ‘[[:alnum:]_]’.
#   \W          Matches any character that is not word-constituent. Think of it as shorthand for ‘[^[:alnum:]_]’.

BEGIN {
    asciiMax = (asciiMax == "" ? 127 : asciiMax)

    numClasses = split("\
        [[:alpha:]]     \
        [[:digit:]]     \
        [[:alnum:]]     \
        [[:lower:]]     \
        [[:upper:]]     \
        [[:xdigit:]]    \
        [[:punct:]]     \
        [[:cntrl:]]     \
        [[:graph:]]     \
        [[:print:]]     \
        [[:blank:]]     \
        [[:space:]]     \
        \\s             \
        \\S             \
        \\w             \
        \\W             \
    ", classes)

    # Map the control chars and white space in the 0-127 range to
    # their abbreviations to make them visible in the output:
    split("NUL SOH STX ETX EOT ENQ ACK BEL BS TAB LF VT FF CR SO SI DLE DC1 DC2 DC3 DC4 NAK SYN ETB CAN EM SUB ESC FS GS RS US Space", map)
    map[128] = "DEL"

    for (asciiNr=0; asciiNr<=asciiMax; asciiNr++) {
        char = sprintf("%c", asciiNr)
        chars[++numChars] = char
    }

    for (classNr in classes) {
        class = classes[classNr]
        for (charNr in chars) {
            char = chars[charNr]
            if ( char ~ class ) {
                classChars[classNr,charNr]
            }
        }
    }

    for (classNr=1; classNr<=numClasses; classNr++) {
        class = classes[classNr]
        printf "%-12s =", class
        for (charNr=1; charNr<=numChars; charNr++) {
            if ( (classNr,charNr) in classChars ) {
                char = chars[charNr]
                printf " %s", (charNr in map ? map[charNr] : char)
            }
        }
        print ""
    }
}

这是C语言环境中chars 0-127的输出，如果您具有不同的语言环境，则输出将有所不同，因此运行上面的脚本以查看它在您的语言环境中是什么:

$ awk -f prtCharClasses.awk file
[[:alpha:]]  = A B C D E F G H I J K L M N O P Q R S T U V W X Y Z a b c d e f g h i j k l m n o p q r s t u v w x y z
[[:digit:]]  = 0 1 2 3 4 5 6 7 8 9
[[:alnum:]]  = 0 1 2 3 4 5 6 7 8 9 A B C D E F G H I J K L M N O P Q R S T U V W X Y Z a b c d e f g h i j k l m n o p q r s t u v w x y z
[[:lower:]]  = a b c d e f g h i j k l m n o p q r s t u v w x y z
[[:upper:]]  = A B C D E F G H I J K L M N O P Q R S T U V W X Y Z
[[:xdigit:]] = 0 1 2 3 4 5 6 7 8 9 A B C D E F a b c d e f
[[:punct:]]  = ! " # $ % & ' ( ) * + , - . / : ; < = > ? @ [ \ ] ^ _ ` { | } ~
[[:cntrl:]]  = NUL SOH STX ETX EOT ENQ ACK BEL BS TAB LF VT FF CR SO SI DLE DC1 DC2 DC3 DC4 NAK SYN ETB CAN EM SUB ESC FS GS RS US DEL
[[:graph:]]  = ! " # $ % & ' ( ) * + , - . / 0 1 2 3 4 5 6 7 8 9 : ; < = > ? @ A B C D E F G H I J K L M N O P Q R S T U V W X Y Z [ \ ] ^ _ ` a b c d e f g h i j k l m n o p q r s t u v w x y z { | } ~
[[:print:]]  = Space ! " # $ % & ' ( ) * + , - . / 0 1 2 3 4 5 6 7 8 9 : ; < = > ? @ A B C D E F G H I J K L M N O P Q R S T U V W X Y Z [ \ ] ^ _ ` a b c d e f g h i j k l m n o p q r s t u v w x y z { | } ~
[[:blank:]]  = TAB Space
[[:space:]]  = TAB LF VT FF CR Space
\s           = TAB LF VT FF CR Space
\S           = NUL SOH STX ETX EOT ENQ ACK BEL BS SO SI DLE DC1 DC2 DC3 DC4 NAK SYN ETB CAN EM SUB ESC FS GS RS US ! " # $ % & ' ( ) * + , - . / 0 1 2 3 4 5 6 7 8 9 : ; < = > ? @ A B C D E F G H I J K L M N O P Q R S T U V W X Y Z [ \ ] ^ _ ` a b c d e f g h i j k l m n o p q r s t u v w x y z { | } ~ DEL
\w           = 0 1 2 3 4 5 6 7 8 9 A B C D E F G H I J K L M N O P Q R S T U V W X Y Z _ a b c d e f g h i j k l m n o p q r s t u v w x y z
\W           = NUL SOH STX ETX EOT ENQ ACK BEL BS TAB LF VT FF CR SO SI DLE DC1 DC2 DC3 DC4 NAK SYN ETB CAN EM SUB ESC FS GS RS US Space ! " # $ % & ' ( ) * + , - . / : ; < = > ? @ [ \ ] ^ ` { | } ~ DEL

请注意 \s， \S， \w和 \W是仅在某些工具中可用的扩展名，例如GNU awk。上面没有 \d和 \D，它们是某些工具中可用的扩展，这些工具支持PCRE作为 [:digit:]的简写，但不包括awk的任何变体。如果您想使用 [:digit:]的简写形式，那么 [0-9]似乎可以跨语言环境移植，但我会予以纠正。

如果您需要查看超过127的字符，则可以在命令行上设置 asciiMax，例如:

$ awk -v asciiMax=255 -f prtCharClasses.awk
[[:alpha:]]  = A B C D E F G H I J K L M N O P Q R S T U V W X Y Z a b c d e f g h i j k l m n o p q r s t u v w x y z ª µ º À Á Â Ã Ä Å Æ Ç È É Ê Ë Ì Í Î Ï Ð Ñ Ò Ó Ô Õ Ö Ø Ù Ú Û Ü Ý Þ ß à á â ã ä å æ ç è é ê ë ì í î ï ð ñ ò ó ô õ ö ø ù ú û ü ý þ ÿ
[[:digit:]]  = 0 1 2 3 4 5 6 7 8 9
[[:alnum:]]  = 0 1 2 3 4 5 6 7 8 9 A B C D E F G H I J K L M N O P Q R S T U V W X Y Z a b c d e f g h i j k l m n o p q r s t u v w x y z ª µ º À Á Â Ã Ä Å Æ Ç È É Ê Ë Ì Í Î Ï Ð Ñ Ò Ó Ô Õ Ö Ø Ù Ú Û Ü Ý Þ ß à á â ã ä å æ ç è é ê ë ì í î ï ð ñ ò ó ô õ ö ø ù ú û ü ý þ ÿ
[[:lower:]]  = a b c d e f g h i j k l m n o p q r s t u v w x y z µ ß à á â ã ä å æ ç è é ê ë ì í î ï ð ñ ò ó ô õ ö ø ù ú û ü ý þ ÿ
[[:upper:]]  = A B C D E F G H I J K L M N O P Q R S T U V W X Y Z À Á Â Ã Ä Å Æ Ç È É Ê Ë Ì Í Î Ï Ð Ñ Ò Ó Ô Õ Ö Ø Ù Ú Û Ü Ý Þ
[[:xdigit:]] = 0 1 2 3 4 5 6 7 8 9 A B C D E F a b c d e f
[[:punct:]]  = ! " # $ % & ' ( ) * + , - . / : ; < = > ? @ [ \ ] ^ _ ` { | } ~ ¡ ¢ £ ¤ ¥ ¦ § ¨ © « ¬ ® ¯ ° ± ² ³ ´ ¶ · ¸ ¹ » ¼ ½ ¾ ¿ × ÷
[[:cntrl:]]  = NUL SOH STX ETX EOT ENQ ACK BEL BS TAB LF VT FF CR SO SI DLE DC1 DC2 DC3 DC4 NAK SYN ETB CAN EM SUB ESC FS GS RS US DEL                                
[[:graph:]]  = ! " # $ % & ' ( ) * + , - . / 0 1 2 3 4 5 6 7 8 9 : ; < = > ? @ A B C D E F G H I J K L M N O P Q R S T U V W X Y Z [ \ ] ^ _ ` a b c d e f g h i j k l m n o p q r s t u v w x y z { | } ~ ¡ ¢ £ ¤ ¥ ¦ § ¨ © ª « ¬ ® ¯ ° ± ² ³ ´ µ ¶ · ¸ ¹ º » ¼ ½ ¾ ¿ À Á Â Ã Ä Å Æ Ç È É Ê Ë Ì Í Î Ï Ð Ñ Ò Ó Ô Õ Ö × Ø Ù Ú Û Ü Ý Þ ß à á â ã ä å æ ç è é ê ë ì í î ï ð ñ ò ó ô õ ö ÷ ø ù ú û ü ý þ ÿ
[[:print:]]  = Space ! " # $ % & ' ( ) * + , - . / 0 1 2 3 4 5 6 7 8 9 : ; < = > ? @ A B C D E F G H I J K L M N O P Q R S T U V W X Y Z [ \ ] ^ _ ` a b c d e f g h i j k l m n o p q r s t u v w x y z { | } ~   ¡ ¢ £ ¤ ¥ ¦ § ¨ © ª « ¬ ® ¯ ° ± ² ³ ´ µ ¶ · ¸ ¹ º » ¼ ½ ¾ ¿ À Á Â Ã Ä Å Æ Ç È É Ê Ë Ì Í Î Ï Ð Ñ Ò Ó Ô Õ Ö × Ø Ù Ú Û Ü Ý Þ ß à á â ã ä å æ ç è é ê ë ì í î ï ð ñ ò ó ô õ ö ÷ ø ù ú û ü ý þ ÿ
[[:blank:]]  = TAB Space  
[[:space:]]  = TAB LF VT FF CR Space  
\s           = TAB LF VT FF CR Space  
\S           = NUL SOH STX ETX EOT ENQ ACK BEL BS SO SI DLE DC1 DC2 DC3 DC4 NAK SYN ETB CAN EM SUB ESC FS GS RS US ! " # $ % & ' ( ) * + , - . / 0 1 2 3 4 5 6 7 8 9 : ; < = > ? @ A B C D E F G H I J K L M N O P Q R S T U V W X Y Z [ \ ] ^ _ ` a b c d e f g h i j k l m n o p q r s t u v w x y z { | } ~ DEL                                 ¡ ¢ £ ¤ ¥ ¦ § ¨ © ª « ¬ ­ ® ¯ ° ± ² ³ ´ µ ¶ · ¸ ¹ º » ¼ ½ ¾ ¿ À Á Â Ã Ä Å Æ Ç È É Ê Ë Ì Í Î Ï Ð Ñ Ò Ó Ô Õ Ö × Ø Ù Ú Û Ü Ý Þ ß à á â ã ä å æ ç è é ê ë ì í î ï ð ñ ò ó ô õ ö ÷ ø ù ú û ü ý þ ÿ
\w           = 0 1 2 3 4 5 6 7 8 9 A B C D E F G H I J K L M N O P Q R S T U V W X Y Z _ a b c d e f g h i j k l m n o p q r s t u v w x y z ª µ º À Á Â Ã Ä Å Æ Ç È É Ê Ë Ì Í Î Ï Ð Ñ Ò Ó Ô Õ Ö Ø Ù Ú Û Ü Ý Þ ß à á â ã ä å æ ç è é ê ë ì í î ï ð ñ ò ó ô õ ö ø ù ú û ü ý þ ÿ
\W           = NUL SOH STX ETX EOT ENQ ACK BEL BS TAB LF VT FF CR SO SI DLE DC1 DC2 DC3 DC4 NAK SYN ETB CAN EM SUB ESC FS GS RS US Space ! " # $ % & ' ( ) * + , - . / : ; < = > ? @ [ \ ] ^ ` { | } ~ DEL                                   ¡ ¢ £ ¤ ¥ ¦ § ¨ © « ¬ ­ ® ¯ ° ± ² ³ ´ ¶ · ¸ ¹ » ¼ ½ ¾ ¿ × ÷