regex - 使用正则表达式 (.NET) 标记复杂字符串(Snort 规则)

Question

我需要 Regex 向导的帮助。我正在尝试编写一个简单的解析器，它可以标记 Snort 规则(Snort，IDS/IPS 软件)的选项列表。问题是，我似乎无法找到一个可行的公式来根据终止分号将各个规则选项分开。我编写的公式将括号之间的所有选项抓取到一个捕获组中。

我正在使用优秀的 RegExr GSkinner 站点上的工具以及来自 Emerging Threats 的一些示例规则选项(我解析了规则 header ——这很容易标记化):

(msg:"ET DELETED Majestic-12 Spider Bot User-Agent (MJ12bot)"; flow:to_server,established; content:"|0d 0a|User-Agent\: MJ12bot|0d 0a|"; classtype:trojan-activity; reference:url,www.majestic12.co.uk/; reference:url,doc.emergingthreats.net/2003409; reference:url,www.emergingthreats.net/cgi-bin/cvsweb.cgi/sigs/POLICY/POLICY_Majestic-12; sid:2003409; rev:4;)
(msg:"ET DELETED Majestic-12 Spider Bot User-Agent Inbound (MJ12bot)"; flow:to_server,established; content:"|0d 0a|User-Agent\: MJ12bot"; classtype:trojan-activity; reference:url,www.majestic12.co.uk/; reference:url,doc.emergingthreats.net/2007762; reference:url,www.emergingthreats.net/cgi-bin/cvsweb.cgi/sigs/POLICY/POLICY_Majestic-12; sid:2007762; rev:4;)
(msg:"ET POLICY McAfee Update User Agent (McAfee AutoUpdate)"; flow:to_server,established; content:"User-Agent|3a| "; http_header; nocase; content:"McAfee AutoUpdate"; http_header; pcre:"/User-Agent\x3a[^\n]+McAfee AutoUpdate/i"; classtype:not-suspicious; reference:url,doc.emergingthreats.net/2003381; reference:url,www.emergingthreats.net/cgi-bin/cvsweb.cgi/sigs/POLICY/POLICY_McAffee; sid:2003381; rev:6;)
(msg:"ET DELETED Metacafe.com family filter off"; flow:established,to_server; content:"POST"; http_method; content:"Host|3a| www.metacafe.com"; http_header; fast_pattern:6,16; content:"submit=Continue+-+I%27m+over+18"; classtype:policy-violation; reference:url,doc.emergingthreats.net/2006367; reference:url,www.emergingthreats.net/cgi-bin/cvsweb.cgi/sigs/POLICY/POLICY_Metacafe; sid:2006367; rev:7;)

这是公式:

([a-zA-Z0-9_:]+(?:[\w\s.,\-/=<>+!\[\]\(\)\{\}\"|\\;'?`~@#$%^&*])+;)

问题是，它不处理冒号。因此上面的两条规则将不会正确解析它们的“内容”选项。但在 RegExr 上，每个选项都将以蓝色突出显示，包括终止分号，但不包括分号后面的空格。如果我将其输入 .NET，我应该能够执行 Regex.Split 并正确拆分所有标记。

如果我将冒号添加到字符列表，那么在 RegExr 上，整套规则将被标记化为单个文本 block ，这不是我想要的。进一步尝试调整公式会导致 Adob​​e Flash 崩溃，这表明我遇到了 Flash 或 RegExr 中的错误。

我不排除编写自己的字符串分词器，但我希望正则表达式可以让我免于处理诸如计算我的开引号、转义字符、空格等的事情。

Snort 规则选项通常采用以下格式:

option:value;
option:"string value";
option:!"negated string value";
option:>num;
option:param1,param2,param3;

但是有几个选项的值往往具有更多“异国情调”的格式，例如 byte_test。每个人最喜欢的“pcre”，基本上是执行与 perl 兼容的正则表达式的选项。因此，任何此类分词器在遇到包含正则表达式的“pcre”关键字时都必须避免混淆。

想法？


编辑:下面这个真的很接近:

([\w]+:?(?:[\x20]|)?(?:[\x00-\xff])*?;)

但是，根据 RegExr，它会被 pcre 语法弄乱:

(msg:"ET WEB_SPECIFIC_APPS Horde 3.0.9-3.1.0 Help Viewer Remote PHP Exploit"; flow:established,to_server; content:"/services/help/"; nocase; http_uri; pcre:"/module=[^\;]*\;.*\"/UGi"; classtype:web-application-attack; reference:url,www.milw0rm.com/exploits/1660; reference:cve,2006-1491; reference:bugtraq,17292; reference:url,doc.emergingthreats.net/2002867; reference:url,www.emergingthreats.net/cgi-bin/cvsweb.cgi/sigs/WEB_SPECIFIC_APPS/WEB_Horde; sid:2002867; rev:9; http_method;)

在上面，除了 ]*\;.*\"/ 之外，每个选项都被突出显示为一个不同的分组。我认为 \x00-\xff 会得到所有，但看起来我正在使用惰性匹配。贪婪匹配得到一切，包括选项之间的所有空格，这是我不想要的。所以我需要以某种方式修改用于处理标记化 pcre 文本的正则表达式。


Edit2:这样做的技巧:

([\w]+:?(?:[\x20]|)?(?<!\\)\"?.*?(?<!\\)\"?;)

我不得不尝试一些使用引号字符串的正则表达式示例。终于意识到我正在盯着避免引用被转义的负面回顾。这似乎也解决了任何其他转义字符，因为转义字符只出现在未转义的引号内。

Answer 1

无需环顾四周。只需仔细编写正则表达式以精确匹配您的需要。通过像这样在冗长的自由间距模式下编写，这变得更加清晰(并且更容易维护):(尽管 VB.NET 语法使这样做很尴尬)

Dim RegexObj As New Regex(
    "# Match set of Snort rules enclosed within parentheses." & chr(10) & _
    "\(                              # Literal opening parentheses." & chr(10) & _
    "(?:                             # Group for one or more rules." & chr(10) & _
    "  \w+                           # Required rule name." & chr(10) & _
    "  (?:                           # Group for optional rule value." & chr(10) & _
    "    :                           # Rule name/values separated by :" & chr(10) & _
    "    (?:                         # Group for rule value alternatives." & chr(10) & _
    "      ""                        # Either a double quoted string," & chr(10) & _
    "      [^""\\]*                  # {normal} Use ""Unrolling the Loop""." & chr(10) & _
    "      (?:                       # Begin {(special normal*)*} construct." & chr(10) & _
    "        \\.                     # {special} == escaped anything." & chr(10) & _
    "        [^""\\]*                # More {normal*} non-quote, non-escapes." & chr(10) & _
    "      )*                        # Finish {(special normal*)*} construct." & chr(10) & _
    "      ""                        # Closing quote." & chr(10) & _
    "    | '[^'\\]*(?:\\.[^'\\]*)*'  # or a single quoted string," & chr(10) & _
    "    | [^;]+                     # or one or more non semi-colons." & chr(10) & _
    "    )                           # End group for rule value options." & chr(10) & _
    "  )?                            # Rule value is optional." & chr(10) & _
    "  ; \s*                         # Rule ends with ;, optional ws." & chr(10) & _
    ")+                              # One or more rules." & chr(10) & _
    "\)                              # LiteraL closing parentheses.", 
    RegexOptions.IgnorePatternWhitespace)
Dim MatchResults As Match = RegexObj.Match(SubjectString)
While MatchResults.Success
    ' matched text: MatchResults.Value
    ' match start: MatchResults.Index
    ' match length: MatchResults.Length
    MatchResults = MatchResults.NextMatch()
End While

此正则表达式演示了如何使用 Jeffrey Friedl 的“展开循环” 高效技术来正确匹配可能包含转义字符的带引号的字符串。 (参见:MRE3)

哦，是的，还有一件事......伊卡洛斯找到你了!