python - 使用 pandas/python 格式化 txt 文件

Question

我有一个来自实验室设备的 txt 文件，它以以下格式保存数据:

Run1 
Selected data
        Time (s)    Charge Q (nC)   Charge density q (nC/g) Mass (g)    
Initial -   21.53   -2.81E-01   -1.41E-03   200.0   
Flow    -   0.00    0.00E+00    0.00E+00    0.0 
Charge (in Coulomb) temporal evolution
3.61    2.44e-11
4.11    2.44e-11
4.61    2.44e-11
5.11    3.66e-11
5.63    3.66e-11
6.14    2.44e-11
6.66    3.66e-11
7.14    3.66e-11
7.67    2.44e-11
8.19    3.66e-11
8.70    2.44e-11
9.20    2.44e-11
9.72    2.44e-11
10.23   2.44e-11
10.73   2.44e-11

Run2 
Selected data
        Time (s)    Charge Q (nC)   Charge density q (nC/g) Mass (g)    
Initial -   21.53   -2.81E-01   -1.41E-03   200.0   
Flow    -   0.00    0.00E+00    0.00E+00    0.0 
Charge (in Coulomb) temporal evolution
3.61    2.44e-11
4.11    2.44e-11
4.61    2.44e-11
5.11    3.66e-11
5.63    3.66e-11
6.14    2.44e-11
6.66    3.66e-11
7.14    3.66e-11
7.67    2.44e-11
8.19    3.66e-11

Run3 
Selected data
        Time (s)    Charge Q (nC)   Charge density q (nC/g) Mass (g)    
Initial -   21.53   -2.81E-01   -1.41E-03   200.0   
Flow    -   0.00    0.00E+00    0.00E+00    0.0 
Charge (in Coulomb) temporal evolution
3.61    2.44e-11
4.11    2.44e-11
4.61    2.44e-11
5.11    3.66e-11
5.63    3.66e-11
6.14    2.44e-11
6.66    3.66e-11
7.14    3.66e-11
7.67    2.44e-11
8.19    3.66e-11
8.70    2.44e-11
9.20    2.44e-11

我的测试文件夹中有多个这样的文件。我希望简化和自动化对这些数据集所做的分析，因为对于另一种设备，我通过更简单的代码获得了类似的成功。

我想要做的是从每个具有 FileName 的文件中提取 3 次运行中每一次的 2 列测试数据，并导出到一个逗号分隔的文本文件，其中 filename = FileName-Run#.txt

到目前为止，我所做的就是尝试将文本文件内容转换为列表列表，然后尝试将数字数据单独处理为新的 csv，但这效果不佳，因为我无法检测到我感兴趣的列数据的长度。

这里的其他几个问题在这方面有所帮助，包括如何在文件夹内的文件上运行代码(如果有效的话)。

我使用了 jupyter 笔记本 - 如果有用的话我可以分享我在这里编写的代码，尽管我羞于展示它。

Answer 1

试试这个:

import re
from pathlib import Path

input_path = Path("path/to/input_folder")
output_path = Path("path/to/output_folder")
run_name_pattern = re.compile("Run\d+")
data_line_pattern = re.compile("(.+?) +(.+?)")


def write_output(input_file: Path, run_name: str, data: str):
    output_file = output_path / f"{input_file.stem}-{run_name}.csv"
    with output_file.open("w") as fp_out:
        fp_out.write(data)


for input_file in input_path.glob("*.txt"):
    with input_file.open() as fp:
        run_name, data, start_reading = "", "", False

        for line in fp:
            # If a line matches "Run...", start a new run name
            if run_name_pattern.match(line):
                run_name = line.strip()
            # If the line matches "Charge (in Coulomb)...",
            # read in the data, starting with the next line
            elif line.startswith("Charge (in Coulomb) temporal evolution"):
                start_reading = True
            # For the data lines, replace spaces in the middle with a comma
            elif start_reading and line != "\n":
                data += data_line_pattern.sub(r"\1,\2", line)
            # If we encounter a blank line, that means the end of data.
            # Flush the data to disk.
            elif line == "\n":
                write_output(input_file, run_name, data)
                run_name, data, start_reading = "", "", False
        else:
            # If we have reached the end of the file but there still
            # data we haven't written to disk, flush it
            if data:
                write_output(input_file, run_name, data)