iis - 将 IIS 日志读取到 pandas 数据帧

Question

我有一个 IIS 日志文件，其中的行采用以下格式:

61.245.163.59 - [16/May/2013:23:55:09 +0530] "GET /ehrm/Recruitment/Images/divider.gif HTTP/1.1" 404 1245 "http://www.example.com/ehrm/Recruitment/MyApplication.aspx?PRF_ID=000005&digest=6LL4BTSuW9YnE5R4T8k27Q" "Mozilla/5.0 (Windows NT 6.1; rv:20.0) Gecko/20100101 Firefox/20.0" GET /ehrm/Recruitment/Images/divider.gif - HTTP/1.1 www.example.com

我想从中获取一些列并构建一个数据框。在下面的方法中，它只是构建一个包含一列的数据框。我想让每个分割列成为数据帧的一列？另外就是日志文件行的长度并不唯一，那么如何通过这样的分割来提高取值的准确性呢？

log_list = []
for line in f:
    ip = (line.split(' ')[0])
    time = (line.split(' ')[2])
    method = (line.split(' ')[4])
    status = (line.split(' ')[7])
    bytes = (line.split(' ')[8])
    referrer = (line.split(' ')[9])
    agent = (line.split(' ')[10])
    data = ip + ' ' + time + ' ' + method + ' ' + status + ' ' + bytes + ' ' + referrer + ' ' + agent
    log_list.append(data)
df = pandas.DataFrame(log_list)

Answer 1

以下代码应该可以完成您想要做的事情:

from pandas import read_csv
log_file = 'filename.log'
df = read_csv(log_file, sep=r'\s+', usecols=[0, 2, 4, 7, 8, 9, 10])

read_csv documentation .