python - 使用 python 解析多行 whatsapp 文本不起作用

Question

我正在尝试准备一个 whatsapp 文件以供分析。我需要将它分为三列:时间、姓名和消息。文本包含对话中的一些消息，这些消息有换行符。当我将它加载到数据框中时，这些消息显示为它们自己的行，而不是一条消息的一部分。

4/16/19, 15:22 - ‪+254 123 123‬: Hi my T. L

4/16/19, 15:22 - ‪+254 123 124‬: Was up

4/17/19, 06:28 - member: Hi team details are Thursday 18 April, 

Venue: Hilton Hotel

Time: 07:30am

Come on time team!

4/17/19, 12:17 - member: Hi guys

4/17/19, 12:18 - member: How many are coming tomorrow?

我试过两种方法:

1. 按照这些解决方案中的指示直接使用正则表达式直接解析 here和 here在 stackoverflow 和 this 上还有博客

2. 间接创建一个文件，将这些多行消息编译成一行，如发现的那样here

两种方法都失败了:(我最喜欢的是第二种方法，只是因为你能够创建一个可以被其他平台使用的文件，例如excel, tableau...

对于方法 2:

import re
from datetime import datetime

start = True

with open('test.txt', "r", encoding='utf-8') as infile, open("output.txt", "w", encoding='utf-8') as outfile:
    for line in infile:
        time = re.search(r'\d\d\/\d\d\/\d\d,.(24:00|2[0-3]:[0-5][0-9]|[0-1][0-9]:[0-5][0-9])', line)
        sender = re.search(r'-(.*?):', line)
        if sender and time:
            date = datetime.strptime(
                time.group(),
                '%m/%d/%Y, %H:%M')
            sender = sender.group()
            text = line.rsplit(r'].+: ', 1)[-1]
            new_line = str(date) + ',' + sender + ',' + text
            if not start: new_line =  new_line
            outfile.write(new_line)
        else:
            outfile.write(' ' + line)
        start = False

我希望我最终不会得到:

4/17/19, 06:28 - member: Hi team details are Thursday 18 April, 

Venue: Hilton Hotel

Time: 07:30am

Come on time team!

并得到:

4/17/19, 06:28 - member: Hi team details are Thursday 18 April, Venue: Hilton Hotel Time: 07:30am Come on time team!

此外，将其输出为数据框，日期时间、成员和消息都正确完成。

Answer 1

正则表达式

您将需要以下正则表达式:

^(\d{1,2})\/(\d{1,2})\/(\d\d), (24:00|2[0-3]:[0-5][0-9]|[0-1][0-9]:[0-5][0-9]) - (\S[^:]*?): (.*)$

在线测试正则表达式 in sandbox .

代码

接收到的数据被形成为 DateFrame 的对象。最后，例如，将 DateFrame 对象保存在 CSV 文件中。

import re
from datetime import datetime
import pandas as pd

with open('test.txt', "r", encoding='utf-8') as infile:
    outputData = { 'date': [], 'sender': [], 'text': [] }
    for line in infile:
        matches = re.match(r'^(\d{1,2})\/(\d{1,2})\/(\d\d), (24:00|2[0-3]:[0-5][0-9]|[0-1][0-9]:[0-5][0-9]) - ((\S[^:]*?): )?(.*)$', line)
        if matches:
          outputData['date'].append(
            datetime(
              int(matches.group(3))+2000,
              int(matches.group(1)),
              int(matches.group(2)),
              hour=int(matches.group(4)[0:2]),
              minute=int(matches.group(4)[3:])
            ))
          outputData['sender'].append(matches.group(6) or "{undefined}")
          outputData['text'].append(matches.group(7))

        elif len(outputData['text']) > 0:
          outputData['text'][-1] += "\n" + line[0:-1]

    outputData = pd.DataFrame(outputData)
    outputData.to_csv('output.csv', index=False, line_terminator='\n', encoding='utf-8')

在线测试完整in sandbox .