python - 处理文件的简单脚本的偏执狂、过多的日志记录和异常处理。这是正常的吗？

Question

我发现自己将 python 用于许多文件管理脚本，如下所示。在网上寻找示例时，我很惊讶示例中的日志记录和异常处理功能如此之少。每次我写一个新脚本时，我的意图都不是像下面那样结束，但如果它处理文件，那么无论我的偏执狂接管什么，最终结果都不像我在网上看到的例子。由于我是新手，我想知道这是否正常。如果不是，那么您如何处理未知数和删除有值(value)信息的恐惧？

def flatten_dir(dirname):
    '''Flattens a given root directory by moving all files from its sub-directories and nested 
    sub-directories into the root directory and then deletes all sub-directories and nested 
    sub-directories. Creates a backup directory preserving the original structure of the root
    directory and restores this in case of errors.
    '''
    RESTORE_BACKUP = False
    log.info('processing directory "%s"' % dirname)
    backup_dirname = str(uuid.uuid4())
    try:
        shutil.copytree(dirname, backup_dirname)
        log.debug('directory "%s" backed up as directory "%s"' % (dirname,backup_dirname))
    except shutil.Error:
        log.error('shutil.Error: Error while trying to back up the directory')
        sys.stderr.write('the program is terminating with an error\n')
        sys.stderr.write('press consult the log file\n')
        sys.stderr.flush()
        time.sleep(0.25)
        print 'Press any key to quit this program.'
        msvcrt.getch()
        sys.exit()

    for root, dirs, files in os.walk(dirname, topdown=False):
        log.debug('os.walk passing: (%s, %s, %s)' % (root, dirs, files))
        if root != dirname:
            for file in files:
                full_filename = os.path.join(root, file)
                try:
                    shutil.move(full_filename, dirname)
                    log.debug('"%s" copied to directory "%s"' % (file,dirname))
                except shutil.Error:
                    RESTORE_BACKUP = True
                    log.error('file "%s" could not be copied to directory "%s"' % (file,dirname))
                    log.error('flagging directory "%s" for reset' % dirname)
            if not RESTORE_BACKUP:
                try:
                    shutil.rmtree(root)
                    log.debug('directory "%s" deleted' % root)
                except shutil.Error:
                    RESTORE_BACKUP = True
                    log.error('directory "%s" could not be deleted' % root)
                    log.error('flagging directory "%s" for reset' % dirname)
        if RESTORE_BACKUP:
            break
    if RESTORE_BACKUP:
        RESTORE_FAIL = False
        try:
            shutil.rmtree(dirname)
        except shutil.Error:
            log.error('modified directory "%s" could not be deleted' % dirname)
            log.error('manual restoration from backup directory "%s" necessary' % backup_dirname)
            RESTORE_FAIL = True 
        if not RESTORE_FAIL:
            try:
                os.renames(backup_dirname, dirname)
                log.debug('back up of directory "%s" restored' % dirname)
                print '>'
                print '>******WARNING******'
                print '>There was an error while trying to flatten directory "%s"' % dirname
                print '>back up of directory "%s" restored' % dirname
                print '>******WARNING******'
                print '>'
            except WindowsError:
                log.error('backup directory "%s" could not be renamed to original directory name' % backup_dirname)
                log.error('manual renaming of backup directory "%s" to original directory name "%s" necessary' % (backup_dirname,dirname))
                print '>'
                print '>******WARNING******'
                print '>There was an error while trying to flatten directory "%s"' % dirname
                print '>back up of directory "%s" was NOT restored successfully' % dirname
                print '>no information is lost'
                print '>check the log file for information on manually restoring the directory'
                print '>******WARNING******'
                print '>'
    else:
        try:
            shutil.rmtree(backup_dirname)
            log.debug('back up of directory "%s" deleted' % dirname)
            log.info('directory "%s" successfully processed' % dirname)
            print '>directory "%s" successfully processed' % dirname
        except shutil.Error:
            log.error('backup directory "%s" could not be deleted' % backup_dirname)
            log.error('manual deletion of backup directory "%s" necessary' % backup_dirname)
            print '>'
            print '>******WARNING******'
            print '>directory "%s" successfully processed' % dirname
            print '>cleanup of backup directory "%s" failed' % backup_dirname
            print '>manual cleanup necessary'
            print '>******WARNING******'
            print '>'

Answer 1

学会放手(或者我是如何学会忍受炸弹的)...

问问自己:您到底害怕什么，如果发生了您将如何处理？在您提供的示例中，您希望避免数据丢失。您处理它的方式是查找您认为错误的每种条件组合，并在其上放置大量日志记录。事情仍然会出错，并且不清楚大量的日志记录是否是处理它的好方法。勾勒出您要实现的目标:

for each file in a tree
  if file is below the root
    move it into the root
if nothing went wrong
  delete empty subtrees

那么在这个过程中会出现什么样的问题呢？好吧，由于底层文件系统的原因，移动文件操作可能会以多种方式失败。我们能否将它们全部列出并提供处理它们的好方法？不……但总的来说，你会以同样的方式处理它们。有时错误只是一个错误，不管它是什么。

所以在这种情况下，如果发生任何错误，那么您想要中止并撤消任何更改。您决定这样做的方法是创建备份副本并在出现问题时恢复它。但是你最有可能的错误是文件系统已满，在这种情况下这些步骤可能会失败......好吧，这是一个很常见的问题 - 如果你随时担心未知错误，你如何停止你的恢复路径不出错？

一般的答案是确保您先完成任何中间工作，然后再执行一个麻烦的(希望是原子的)步骤。在你的情况下，你需要翻转你的恢复。与其构建副本作为备份，不如构建结果的副本。如果一切顺利，您可以将新结果换成旧的原始树。或者，如果你真的很偏执，你可以把这一步留给人类。这样做的好处是，如果出现问题，您可以中止并丢弃您构建的部分状态。

然后你的结构变成:

make empty result directory
for every file in the tree
  copy file into new result
on failure abort otherwise
  move result over old source directory

顺便说一句，您当前的脚本中有一个错误，此伪代码使该错误更加明显:如果您在不同分支中具有相同名称的文件，它们将在新的扁平化版本中相互覆盖。

关于这个伪代码的第二点是所有的错误处理都在同一个地方(即将创建新目录和递归副本包装在一个 try block 中并捕获它之后的所有错误)，这解决了你原来的问题关于日志记录/错误检查与实际工作代码的比例过大的问题。

backup_dirname = str(uuid.uuid4())
try:
    shutil.mkdir(backup_dirname)
    for root, dirs, files in os.walk(dirname, topdown=False):
        for file in files:
            full_filename = os.path.join(root, file)
            target_filename = os.path.join(backup_dirname,file)
            shutil.copy(full_filename, target_filename)
catch Exception, e:
    print >>sys.stderr, "Something went wrong %s" % e
    exit(-1)
shutil.move(back_dirname,root)      # I would do this bit by hand really