java - 使用 sed/awk/whatever 从 *nix 上的日志中获取异常

Question

我想从 tomcat 创建的日志文件中获取异常。
是的，我做了一些研究，但是因为我没有任何使用 sed 或 awk 的经验 - 调整我发现的内容以适应我的需要有点困难。
下面显示了一个示例文件以供处理:

    at org.springframework.scheduling.quartz.QuartzJobBean.execute(QuartzJobBean.java:86)
Caused by: org.apache.cxf.transport.http.HTTPException: HTTP response '503: Service Temporarily Unavailable' when communicating with http://66.66.66.66:1234/aaa/bbb/ccc
    at org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.handleResponseInternal(HTTPConduit.java:1546)
    at org.apache.cxf.interceptor.MessageSenderInterceptor$MessageSenderEndingInterceptor.handleMessage(MessageSenderInterceptor.java:62)
    ... 39 more
2014-10-24 11:40:01,558 ERROR [aaa.bbb.ccc.ddd.SomeClass] - some exception on parsing '2007/11/45' bla bla
javax.xml.ws.WebServiceException: Could not send Message.
    at org.apache.cxf.jaxws.JaxWsClientProxy.invoke(JaxWsClientProxy.java:145)
    at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:525)
Caused by: org.apache.cxf.transport.http.HTTPException: HTTP response '503: Service Temporarily Unavailable' when communicating with http://66.66.66.66:1234/aaa/bbb/ccc
    at org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.handleResponseInternal(HTTPConduit.java:1546)
    at org.apache.cxf.frontend.ClientProxy.invokeSync(ClientProxy.java:88)
    at org.apache.cxf.jaxws.JaxWsClientProxy.invoke(JaxWsClientProxy.java:134)
    ... 32 more
2014-10-24 11:40:01,561 ERROR [aaa.bbb.ccc.ddd] - some error with id = 1214
java.lang.NullPointerException
    at sun.reflect.GeneratedMethodAccessor181.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:525)
2014-10-24 11:44:48,253 INFO [org.springframework.orm.hibernate3.annotation.AnnotationSessionFactoryBean] - Closing Hibernate SessionFactory
2014-10-24 11:44:48,253 INFO [org.hibernate.impl.SessionFactoryImpl] - closing
2014-10-24 11:44:48 org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap
2014-10-24 11:44:50 org.apache.coyote.http11.Http11Protocol destroy
INFO: Stopping Coyote HTTP/1.1 on http-8096

如我们所见，有:2 个完全异常(我们想要)，1 个部分异常(我们不想要)。为了简化此示例，我删除了一些重要的内容，例如我们不需要的 log4j:ERRORs。

到目前为止我试过:
AWK(这是我使用 AWK 的第一天，请不要笑 :D)。它非常简单。
它在每一行的开头找到“/t”(制表符)+ at +“”(空格)。如果在它匹配给定条件(异常和日期)之前的 2 行，它也会打印它们。它工作得很好，但它也打印出部分异常，这是我们不想要的。

BEGIN {
    preprevious = "";
    previous = "";
}
/^\tat / {
    if( previous != "" ) {
        if(preprevious ~ /20[0-9][0-9]-[0-9][0-9]-[0-9][0-9]/){
            print preprevious;
            preprevious = "";
        }
        if(previous ~ /.*Exception/) {
            print previous;
            previous = "";
        }
    }
    print;
    next;
}
 { preprevious = previous;
    previous = $0; }

我是这样运行的:

awk -f awkScript testFileExceptions.txt

和 Bash 脚本中的 SED(我更喜欢)

#!/bin/sh
if [ "$#" -eq  "2" ]
    then
        tail -n $2 $1 | sed -n "/ ERROR \[/,/20[0-9][0-9]-[0-9][0-9]-[0-9][0-9]/p"
    else
        echo "usage: scriptName filePath amountOfLastLinesInFile"
fi

它匹配'""+ERROR+""'(ERROR两边有空格，所以不会匹配log4j:ERROR)到现在为止。它有点工作......
缺点:
1)如果异常之间有一些额外的行，它将起作用 - 像这样:

        at org.apache.cxf.jaxws.JaxWsClientProxy.invoke(JaxWsClientProxy.java:134)
        ... 32 more
2014-10-24 11:40:01,561 INFO [aaa.bbb.ccc.ddd] - AAAAAAAAAAAAAAAAAAAAAAAA
2014-10-24 11:40:01,561 ERROR [aaa.bbb.ccc.ddd] - some error with id = 1214
java.lang.NullPointerException
        at sun.reflect.GeneratedMethodAccessor181.invoke(Unknown Source)

如果不是，则不会显示第二个异常
2)它还会打印出最后匹配的行(日期匹配的那一行)

总而言之，我想要的结果是:

2014-10-24 11:40:01,558 ERROR [aaa.bbb.ccc.ddd.SomeClass] - some exception on parsing '2007/11/45' bla bla javax.xml.ws.WebServiceException: Could not send Message.
        at org.apache.cxf.jaxws.JaxWsClientProxy.invoke(JaxWsClientProxy.java:145)
        at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:525) 
Caused by: org.apache.cxf.transport.http.HTTPException: HTTP response '503: Service Temporarily Unavailable' when communicating with http://66.66.66.66:1234/aaa/bbb/ccc
        at org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.handleResponseInternal(HTTPConduit.java:1546)
        at org.apache.cxf.frontend.ClientProxy.invokeSync(ClientProxy.java:88)
        at org.apache.cxf.jaxws.JaxWsClientProxy.invoke(JaxWsClientProxy.java:134)
        ... 32 more

2014-10-24 11:40:01,561 ERROR [aaa.bbb.ccc.ddd] - some error with id = 1214 java.lang.NullPointerException
        at sun.reflect.GeneratedMethodAccessor181.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:525)

我还想将不同的异常保存到不同的文件中(假设是 exceptionOutputNNN.txt，例如 exceptionOutput001.txt)，但我稍后会弄清楚，在我弄清楚如何做到这一点之后应该不会那么难对于 XML...:P

嗯..就是这样。我希望有人能帮助我:)
干杯

编辑:请注意，异常可以以“... NN more”和简单的“\tat org.*”结尾

Answer 1

还是不确定自己到底想要什么
这应该适用于您想要的输出

 awk '/^[0-9]+/{x=0}/ERROR/{x=1}x' file

输出

2014-10-24 11:40:01,558 ERROR [aaa.bbb.ccc.ddd.SomeClass] - some exception on parsing '2007/11/45' bla bla
javax.xml.ws.WebServiceException: Could not send Message.
    at org.apache.cxf.jaxws.JaxWsClientProxy.invoke(JaxWsClientProxy.java:145)
    at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:525)
Caused by: org.apache.cxf.transport.http.HTTPException: HTTP response '503: Service Temporarily     Unavailable' when communicating with http://66.66.66.66:1234/aaa/bbb/ccc
at org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.handleResponseInternal(HTTPConduit.java:1546)
    at org.apache.cxf.frontend.ClientProxy.invokeSync(ClientProxy.java:88)
    at org.apache.cxf.jaxws.JaxWsClientProxy.invoke(JaxWsClientProxy.java:134)
    ... 32 more
2014-10-24 11:40:01,561 ERROR [aaa.bbb.ccc.ddd] - some error with id = 1214
java.lang.NullPointerException
    at sun.reflect.GeneratedMethodAccessor181.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:525)

编辑:为您的原始文件

 awk 'a=/^[0-9]+/{x=0}a&&/ERROR/{x=1}x' file

或

 awk '(/^[0-9]/&&x=/ERROR/)||x' file