java - 从正则表达式的日志文件中解析 FIX 协议(protocol)消息？

Question

我需要从一个大型日志文件中提取所有 FIX 消息，其中可能有超过 10000 到 20000 条 FIX 协议(protocol)消息。我期望获取的 FIX 协议(protocol)消息将以 8=FIX 开头并以 |10= 结尾，一些 CheckSum 值可以是任何值，然后后跟分隔符 '|'。

例如8=FIXT.1.1|9=449|35=AE|34=1734|49=REPOFIXUAT|52=20140402-11:38:34|56=TR_UAT_VENDOR|1128=8|15=英镑|31=1.7666|32=50000000.00|55=英镑/美元|60 =20140402-11:07:33|63=B|64=20140415|65=OR|75=20140402|150=F|167=FOR|194=1.7654|195=0.0012|460=4|571=7852455|1003=2美元|1056=8833 0000.00|1057=N|552=1|54=2|37=20140402-12:36:48|11=NOREF|453=4|448=零|447=D|452=3|448=MBY2|447=D|452=1|448=LMEB|447=D|452=16|448=D或|447=D|452=11|826=0|78=1|79=默认|80=50000000.00|5967=88330000.00|10=111|

目前，我正在使用此模式进行正则表达式

模式pattern = Pattern.compile("8=FIX(.*?)10=(.*?)|");

但在上面的模式中，我只能提取消息直到10=，但不能提取校验和值，并且可能存在某些 FIX 消息自定义标记类似于 8410=TEST| 的情况。如下:

8=FIXT.1.1|9=73|35=0|34=560|49=RTNSFIXUAT|8410=TEST|52=20140403-01:50:21|56=TR_UAT_VELOCITY|1128=8|10=206|

在上面，我会得到值

"8=FIXT.1.1|9=73|35=0|34=560|49=RTNSFIXUAT|84" (X - I want complete messages till tag 10 checksum value 206)

请找到如下日志文件片段:

>02-04-14 11:38:33.559|QFJ Message Processor|input/REPOFIXInput1|INFO|quickfix.outgoing: 8=FIXT.1.1|9=71|35=0|34=1731|49=TR_UAT_VENDOR|52=20140402-11:38:33.557|56=REPOFIXUAT|10=147|
02-04-14 11:38:34.713|SocketConnectorIoProcessor-1.0|input/REPOFIXInput1|INFO|quickfix.incoming: 8=FIXT.1.1|9=449|35=AE|34=1734|1128=8|49=REPOFIXUAT|56=TR_UAT_VENDOR|52=20140402-11:38:34|552=1|54=2|37=20140402-12:36:48|11=NOREF|826=0|78=1|79=default|80=50000000.00|5967=88330000.00|453=4|448=ZERO|452=3|447=D|448=MBY2|452=1|447=D|448=LMEB|452=16|447=D|448=DOR|452=11|447=D|571=7852455|1003=2 USD|150=F|32=50000000.00|15=GBP|1056=88330000.00|31=1.6666|194=1.6654|195=0.0012|64=20140415|63=B|60=20140402-11:07:33|75=20140402|1057=N|460=4|167=FOR|65=OR|55=GBP/USD|10=111|
02-04-14 11:38:35.004|QFJ Message Processor|input/REPOFIXInput1|INFO|Received FIX application message: 8=FIXT.1.1|9=449|35=AE|34=1734|49=REPOFIXUAT|52=20140402-11:38:34|56=TR_UAT_VENDOR|1128=8|15=GBP|31=1.7666|32=50000000.00|55=GBP/USD|60=20140402-11:07:33|63=B|64=20140415|65=OR|75=20140402|150=F|167=FOR|194=1.7654|195=0.0012|460=4|571=7852455|1003=2 USD|1056=88330000.00|1057=N|552=1|54=2|37=20140402-12:36:48|11=NOREF|453=4|448=ZERO|447=D|452=3|448=MBY2|447=D|452=1|448=LMEB|447=D|452=16|448=DOR|447=D|452=11|826=0|78=1|79=default|80=50000000.00|5967=88330000.00|10=111|

Answer 1

如果我理解正确的话，你想保留:

• 8=FIX 之后但 |10=... 之前的所有内容
• 还有|10=value|中的值

这是一个例子:

String input = ">02-04-14 11:38:33.559|QFJ Message Processor|input/REPOFIXInput1|INFO|quickfix.outgoing: 8=FIXT.1.1|9=71|35=0|34=1731|49=TR_UAT_VENDOR|52=20140402-11:38:33.557|56=REPOFIXUAT|10=147|\r\n"
                       + "02-04-14 11:38:34.713|SocketConnectorIoProcessor-1.0|input/REPOFIXInput1|INFO|quickfix.incoming: 8=FIXT.1.1|9=449|35=AE|34=1734|1128=8|49=REPOFIXUAT|56=TR_UAT_VENDOR|52=20140402-11:38:34|552=1|54=2|37=20140402-12:36:48|11=NOREF|826=0|78=1|79=default|80=50000000.00|5967=88330000.00|453=4|448=ZERO|452=3|447=D|448=MBY2|452=1|447=D|448=LMEB|452=16|447=D|448=DOR|452=11|447=D|571=7852455|1003=2 USD|150=F|32=50000000.00|15=GBP|1056=88330000.00|31=1.6666|194=1.6654|195=0.0012|64=20140415|63=B|60=20140402-11:07:33|75=20140402|1057=N|460=4|167=FOR|65=OR|55=GBP/USD|10=111|\r\n"
                       + "02-04-14 11:38:35.004|QFJ Message Processor|input/REPOFIXInput1|INFO|Received FIX application message: 8=FIXT.1.1|9=449|35=AE|34=1734|49=REPOFIXUAT|52=20140402-11:38:34|56=TR_UAT_VENDOR|1128=8|15=GBP|31=1.7666|32=50000000.00|55=GBP/USD|60=20140402-11:07:33|63=B|64=20140415|65=OR|75=20140402|150=F|167=FOR|194=1.7654|195=0.0012|460=4|571=7852455|1003=2 USD|1056=88330000.00|1057=N|552=1|54=2|37=20140402-12:36:48|11=NOREF|453=4|448=ZERO|447=D|452=3|448=MBY2|447=D|452=1|448=LMEB|447=D|452=16|448=DOR|447=D|452=11|826=0|78=1|79=default|80=50000000.00|5967=88330000.00|10=111|";
    Pattern p = Pattern.compile("8=FIX(.+)(?<=\\|)10=(.+?)(?=\\|)", Pattern.MULTILINE);
    Matcher m = p.matcher(input);
    while (m.find()) {
        System.out.println(m.group());
        System.out.println("\t" + m.group(1));
        System.out.println("\t" + m.group(2));
    }

输出

8=FIXT.1.1|9=71|35=0|34=1731|49=TR_UAT_VENDOR|52=20140402-11:38:33.557|56=REPOFIXUAT|10=147
    T.1.1|9=71|35=0|34=1731|49=TR_UAT_VENDOR|52=20140402-11:38:33.557|56=REPOFIXUAT|
    147
8=FIXT.1.1|9=449|35=AE|34=1734|1128=8|49=REPOFIXUAT|56=TR_UAT_VENDOR|52=20140402-11:38:34|552=1|54=2|37=20140402-12:36:48|11=NOREF|826=0|78=1|79=default|80=50000000.00|5967=88330000.00|453=4|448=ZERO|452=3|447=D|448=MBY2|452=1|447=D|448=LMEB|452=16|447=D|448=DOR|452=11|447=D|571=7852455|1003=2 USD|150=F|32=50000000.00|15=GBP|1056=88330000.00|31=1.6666|194=1.6654|195=0.0012|64=20140415|63=B|60=20140402-11:07:33|75=20140402|1057=N|460=4|167=FOR|65=OR|55=GBP/USD|10=111
    T.1.1|9=449|35=AE|34=1734|1128=8|49=REPOFIXUAT|56=TR_UAT_VENDOR|52=20140402-11:38:34|552=1|54=2|37=20140402-12:36:48|11=NOREF|826=0|78=1|79=default|80=50000000.00|5967=88330000.00|453=4|448=ZERO|452=3|447=D|448=MBY2|452=1|447=D|448=LMEB|452=16|447=D|448=DOR|452=11|447=D|571=7852455|1003=2 USD|150=F|32=50000000.00|15=GBP|1056=88330000.00|31=1.6666|194=1.6654|195=0.0012|64=20140415|63=B|60=20140402-11:07:33|75=20140402|1057=N|460=4|167=FOR|65=OR|55=GBP/USD|
    111
8=FIXT.1.1|9=449|35=AE|34=1734|49=REPOFIXUAT|52=20140402-11:38:34|56=TR_UAT_VENDOR|1128=8|15=GBP|31=1.7666|32=50000000.00|55=GBP/USD|60=20140402-11:07:33|63=B|64=20140415|65=OR|75=20140402|150=F|167=FOR|194=1.7654|195=0.0012|460=4|571=7852455|1003=2 USD|1056=88330000.00|1057=N|552=1|54=2|37=20140402-12:36:48|11=NOREF|453=4|448=ZERO|447=D|452=3|448=MBY2|447=D|452=1|448=LMEB|447=D|452=16|448=DOR|447=D|452=11|826=0|78=1|79=default|80=50000000.00|5967=88330000.00|10=111
    T.1.1|9=449|35=AE|34=1734|49=REPOFIXUAT|52=20140402-11:38:34|56=TR_UAT_VENDOR|1128=8|15=GBP|31=1.7666|32=50000000.00|55=GBP/USD|60=20140402-11:07:33|63=B|64=20140415|65=OR|75=20140402|150=F|167=FOR|194=1.7654|195=0.0012|460=4|571=7852455|1003=2 USD|1056=88330000.00|1057=N|552=1|54=2|37=20140402-12:36:48|11=NOREF|453=4|448=ZERO|447=D|452=3|448=MBY2|447=D|452=1|448=LMEB|447=D|452=16|448=DOR|447=D|452=11|826=0|78=1|79=default|80=50000000.00|5967=88330000.00|
    111

注释

• 此示例未进行性能优化。
• 您可能想要删除此 Pattern 的“多行”标志，并使用常量 Pattern 单独处理每一行
• 如果这样做，您将必须为每一行重新初始化一个 Matcher，但您可以在 if 语句而不是 while 语句中调用 find(假设每行有一个日志)
• 存储匹配值由您决定