gpt4 book ai didi

python - watchdog(python) - 仅监视一种文件格式并忽略 'PatternMatchingEventHandler' 中的其他所有内容

转载 作者:行者123 更新时间:2023-12-01 06:50:02 26 4
gpt4 key购买 nike

我正在运行来自 this article 的代码并进行了一些更改以监视仅一种格式的文件创建/添加,即指定目录中的 .csv

现在的问题是:

只要添加的新文件不是 .csv 格式,我的程序就会中断(停止监视,但继续运行);为了弥补这一点,这是我使用 ignore_patterns 参数所做的事情(但在添加其他格式的新文件后程序仍然停止监视):
PatternMatchingEventHandler(patterns="*.csv",ignore_patterns=["*~"],ignore_directories=True,case_sensitive=True)

完整代码为:

import time
import csv
from datetime import datetime
from watchdog.observers import Observer
from watchdog.events import PatternMatchingEventHandler
from os import path
from pandas import read_csv
# class that takes care of everything
class file_validator(PatternMatchingEventHandler):
def __init__(self, source_path):
# setting parameters for 'PatternMatchingEventHandler'
super(file_validator, self).__init__(patterns="*.csv", ignore_patterns=["*~"], ignore_directories=True, case_sensitive=True)
self.source_path = source_path
self.print_info = None

def on_created(self, event):
# this is the new file that was created
new_file = event.src_path
# details of each new .csv file
# demographic details
file_name = path.basename(new_file)
file_size = f"{path.getsize(new_file) / 1000} KiB"
file_creation = f"{datetime.fromtimestamp(path.getmtime(new_file)).strftime('%Y-%m-%d %H:%M:%S')}"
new_data = read_csv(new_file)
# more details
number_columns = new_data.shape[1]
data_types_data = [
('float' if i == 'float64' else ('int' if i == 'int64' else ('character' if i == 'object' else i))) for i in
[x.name for x in list(new_data.dtypes)]]
null_count_data = list(dict(new_data.isna().sum()).values())
print(f"{file_name}, {file_size}, {file_creation}, {number_columns}")
# trying to access this info, but of no help
self.print_info = f"{file_name}, {file_size}, {file_creation}, {number_columns}"

def return_logs(self):
return self.print_info

# main function
if __name__ == "__main__":
some_path = "C:\\Users\\neevaN_Reddy\\Documents\\learning dash\\"
my_validator = file_validator(source_path=some_path)
my_observer = Observer()
my_observer.schedule(my_validator, some_path, recursive=True)
my_observer.start()
try:
while True:
time.sleep(1)
except KeyboardInterrupt:
my_observer.stop()
my_observer.join()
# # this doesn't print anything
print(my_validator.return_logs)

编辑 1(在 Quentin Pradet 发表评论之后):根据您在评论中的建议,我将我的论点更改为:

super(file_validator, self).__init__(patterns="*.csv",
# ignore_patterns=["*~"],
ignore_directories=True,
case_sensitive=True)

当我复制其他格式的文件时(我尝试使用 .ipynb 文件),我看到这个错误(程序甚至在之后停止监视 .csv 文件)这个):

Exception in thread Thread-1:
Traceback (most recent call last):
File "C:\Users\neevaN_Reddy\AppData\Local\Programs\Python\Python37\lib\threading.py", line 926, in _bootstrap_inner
self.run()
File "C:\Users\neevaN_Reddy\AppData\Local\Programs\Python\Python37\lib\site-packages\watchdog\observers\api.py", line 199, in run
self.dispatch_events(self.event_queue, self.timeout)
File "C:\Users\neevaN_Reddy\AppData\Local\Programs\Python\Python37\lib\site-packages\watchdog\observers\api.py", line 368, in dispatch_events
handler.dispatch(event)
File "C:\Users\neevaN_Reddy\AppData\Local\Programs\Python\Python37\lib\site-packages\watchdog\events.py", line 454, in dispatch
_method_map[event_type](event)
File "C:/Users/neevaN_Reddy/Documents/Work/Project-Aretaeus/diabetes_risk project/file validation using a class.py", line 26, in on_created
new_data = read_csv(new_file)
File "C:\Users\neevaN_Reddy\AppData\Local\Programs\Python\Python37\lib\site-packages\pandas\io\parsers.py", line 685, in parser_f
return _read(filepath_or_buffer, kwds)
File "C:\Users\neevaN_Reddy\AppData\Local\Programs\Python\Python37\lib\site-packages\pandas\io\parsers.py", line 463, in _read
data = parser.read(nrows)
File "C:\Users\neevaN_Reddy\AppData\Local\Programs\Python\Python37\lib\site-packages\pandas\io\parsers.py", line 1154, in read
ret = self._engine.read(nrows)
File "C:\Users\neevaN_Reddy\AppData\Local\Programs\Python\Python37\lib\site-packages\pandas\io\parsers.py", line 2059, in read
data = self._reader.read(nrows)
File "pandas/_libs/parsers.pyx", line 881, in pandas._libs.parsers.TextReader.read
File "pandas/_libs/parsers.pyx", line 896, in pandas._libs.parsers.TextReader._read_low_memory
File "pandas/_libs/parsers.pyx", line 950, in pandas._libs.parsers.TextReader._read_rows
File "pandas/_libs/parsers.pyx", line 937, in pandas._libs.parsers.TextReader._tokenize_rows
File "pandas/_libs/parsers.pyx", line 2132, in pandas._libs.parsers.raise_parser_error
pandas.errors.ParserError: Error tokenizing data. C error: Expected 1 fields in line 4, saw 2

显然,pandas 存在一些错误,这意味着我的 on_created 函数也会针对非 .csv 的文件格式触发,我认为这意味着必须在 ignore_patterns 参数中添加某些内容,以便在添加其他格式的文件时不会触发 on_created 函数。

最佳答案

您可以尝试将pattern作为列表而不是字符串发送,例如。 patterns=["*.csv"]

关于python - watchdog(python) - 仅监视一种文件格式并忽略 'PatternMatchingEventHandler' 中的其他所有内容,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/59063532/

26 4 0
Copyright 2021 - 2024 cfsdn All Rights Reserved 蜀ICP备2022000587号
广告合作:1813099741@qq.com 6ren.com