python - 从文本文件中提取数据并将其写入 csv 或平面文件

Question

我正在做一个项目，涉及以某种格式创建美国联邦代码的关系数据库管理系统。我已经从官方源代码中获得了完整的代码，但结构不是很好。我已经设法使用 GITHUB 上的一些代码将以下格式的美国代码抓取到文本文件中。

可以使用 Python 脚本将其写入以下格式的某些 csv 或平面文件中来完成吗？

我是 Python 新手，但有人告诉我，使用 Python 可以轻松完成此操作。

最终输出将是具有以下架构的平面文件或 csv 文件:

示例:

**Title | Text | Chapter | text | Section | Text | Section text**


1     |  GENERAL PROVISIONS  |  1 | RULES OF CONSTRUCTION | 2 | "County" as including "parish", and so forth | The word "county" includes a parish, or any other equivalent subdivision of a State or Territory of the United States.

输入将是一个文本文件，其数据如下所示。

Sample data:

-CITE-
    1 USC Sec. 2                                                01/15/2013

-EXPCITE-
    TITLE 1 - GENERAL PROVISIONS
    CHAPTER 1 - RULES OF CONSTRUCTION

-HEAD-
    Sec. 2. "County" as including "parish", and so forth

-STATUTE-
      The word "county" includes a parish, or any other equivalent
    subdivision of a State or Territory of the United States.

-SOURCE-
    (July 30, 1947, ch. 388, 61 Stat. 633.)

-End-



-CITE-
    1 USC Sec. 3                                                01/15/2013

-EXPCITE-
    TITLE 1 - GENERAL PROVISIONS
    CHAPTER 1 - RULES OF CONSTRUCTION

-HEAD-
    Sec. 3. "Vessel" as including all means of water transportation

-STATUTE-
      The word "vessel" includes every description of watercraft or
    other artificial contrivance used, or capable of being used, as a
    means of transportation on water.

-SOURCE-
    (July 30, 1947, ch. 388, 61 Stat. 633.)

-End-

Answer 1

如果您想使用像 pyparsing 这样强大的解析器除了正则表达式，以下内容应该适合您:

import csv, re
from pyparsing import Empty, FollowedBy, Group, LineEnd, Literal, \
                      OneOrMore, Optional, Regex, SkipTo, Word
from pyparsing import alphanums, alphas, nums

def section(header, other):
    return Literal('-'+header+'-').suppress() + other

def tc(header, next_item):
    # <header> <number> - <name>
    begin = Literal(header).suppress()
    number = Word(nums)\
             .setResultsName('number')\
             .setParseAction(compress_whitespace)
    dash = Literal('-').suppress()
    name = SkipTo(Literal(next_item))\
           .setResultsName('name')\
           .setParseAction(compress_whitespace)
    return begin + number + dash + name

def compress_whitespace(s, loc, toks):
    return [re.sub(r'\s+', ' ', tok).strip() for tok in toks]

def parse(data):
    # should match anything that looks like a header
    header = Regex(re.compile(r'-[A-Z0-9]+-'))

    # -CITE- (ignore)
    citation = SkipTo('-EXPCITE-').suppress()
    cite_section = section('CITE', citation)

        # -EXPCITE- (parse)
    # grab title number, title name, chapter number, chapter name
    title = Group(tc('TITLE', 'CHAPTER'))\
            .setResultsName('title')
    chapter = Group(tc('CHAPTER', '-HEAD-'))\
              .setResultsName('chapter')
    expcite_section = section('EXPCITE', title + chapter)

    # -HEAD- (parse)
    # two possible forms of section number:
    # > Sec. 1. <head_text>
    # > CHAPTER 1 - <head_text>
    sec_number1 = Literal("Sec.").suppress() \
                  + Regex(r'\d+\w?.')\
                    .setResultsName('section')\
                    .setParseAction(lambda s, loc, toks: toks[0][:-1])
    sec_number2 = Literal("CHAPTER").suppress() \
                  + Word(nums)\
                    .setResultsName('section') \
                  + Literal("-")
    sec_number = sec_number1 | sec_number2
    head_text = SkipTo(header)\
                .setResultsName('head')\
                .setParseAction(compress_whitespace)
    head = sec_number + head_text
    head_section = section('HEAD', head)

    # -STATUTE- (parse)
    statute = SkipTo(header)\
              .setResultsName('statute')\
              .setParseAction(compress_whitespace)
    statute_section = section('STATUTE', statute)

    # -End- (ignore)
    end_section = SkipTo('-End-', include=True)

    # do parsing
    parser = OneOrMore(Group(cite_section \
                             + expcite_section \
                             + head_section \
                             + Optional(statute_section) \
                             + end_section))
    result = parser.parseString(data)

    return result

def write_to_csv(parsed_data, filename):
    with open(filename, 'w') as f:
        writer = csv.writer(f, lineterminator='\n')
        for item in parsed_data:
            if 'statute' not in item:
                continue
            row = [item['title']['number'],
                   item['title']['name'],
                   item['chapter']['number'],
                   item['chapter']['name'],
                   item['section'],
                   item['head'],
                   item['statute']]
            writer.writerow(row)



# your data is assumed to be in <source.txt>
with open('source.txt', 'r') as f:
    data = f.read()
result = parse(data)
write_to_csv(result, 'output.txt')

输出:参见http://pastie.org/8654063 。

这当然比使用正则表达式更冗长，但在我看来它也更易于维护和扩展。 (当然，这需要学习如何在 pyparsing 中进行基本操作的开销，这不一定是微不足道的。)

<小时/>

为了响应您的请求 - 我已更新解析器以容纳您链接到我的文件中出现的所有文本。它现在应该对异常换行符/标点符号更加强大。

根据您的要求，具有部分枚举(并且缺少 -STATUTE- 部分)的引文不再包含在输出中。