sql - Synapse Serverless SQL 中的大型表的 CETAS 超时

Question

我尝试使用 CETAS ( CREATE EXTERNAL TABLE AS SELECT * FROM <table> ) 语句从 Azure Synapse Serverless SQL 池中现有的外部表创建新的外部表。我选择的表是一个非常大的外部表，基于存储在 ADLS Gen 2 存储中的大约 30 GB parquet 格式的数据构建，但查询总是在大约 30 分钟后超时。我尝试过使用高级存储，也尝试了大多数(如果不是全部)提出的建议 here也是如此，但它没有帮助，查询仍然超时。我在 Synapse Studio 中遇到的错误是:-

Statement ID: {550AF4B4-0F2F-474C-A502-6D29BAC1C558} | Query hash: 0x2FA8C2EFADC713D | Distributed request ID: {CC78C7FD-ED10-4CEF-ABB6-56A3D4212A5E}. Total size of data scanned is 0 megabytes, total size of data moved is 0 megabytes, total size of data written is 0 megabytes. Query timeout expired.

核心用例是，假设我只有外部表名称，我想创建一个数据副本，在 Azure 存储本身中创建该外部表。

有没有办法解决这个超时问题或者更好的方法来解决这个问题？

Answer 1

这是无服务器的限制。

Query timeout expired

The error Query timeout expired is returned if the query executed morethan 30 minutes on serverless SQL pool. This is a limit of serverlessSQL pool that cannot be changed. Try to optimize your query byapplying best practices, or try to materialize parts of your queriesusing CETAS. Check is there a concurrent workload running on theserverless pool because the other queries might take the resources. Inthat case you might split the workload on multiple workspaces.

Self-help for serverless SQL pool - Query Timeout Expired

The core use case is that assuming I only have the external table name, I want to create a copy of the data over which that external table is created in Azure storage itself.

在数据工厂复制作业、Spark 作业或 AzCopy 中执行此操作很简单。