parquet - parquet 可以支持并发写操作吗？

Question

是否可以对 parquet 格式执行分布式并发写入？

是否可以在写入拼花文件时读取它们？

如果有并发读/写的方法，我有兴趣了解。

提前感谢您的帮助。

Answer 1

我最终得到了 Parquet 开发人员的回答:这两个问题的答案都是否定的:

Parquet writers are not thread-safe and files cannot be read or written by different readers or writers concurrently. Parquet doesn't expose flush/sync operations to the user (for good reason) so there isn't a way to reliably do this anyway.