python - 使用 python 进行 Azure ML - (SSLError(SSLError ('The write operation timed out' ,),),) 执行表存储实体查询时

Question

您好，我正在尝试通过执行从表存储帐户查询数据的 python 脚本来开始 Azure ML 算法。我用这个来做:

entities_Azure=table_session.query_entities(table_name=table_name, 
                                                filter="PartitionKey eq '" + partitionKey + "'",
                                                select='PartitionKey,RowKey,Timestamp,value',
                                                next_partition_key = next_pk,
                                                next_row_key = next_rk, top=1000)  

我传入调用这段代码所在的函数时所需的变量，并通过在 Azure ML 中包含一个 zip 文件来包含该函数。

我认为错误是由于查询时间太长或类似的原因造成的，但它必须花费很长时间，因为我可能必须查询大量数据....我看了这篇SO post Windows Azure Storage Table connection timed out 其中我认为关于达到这些查询的指定阈值是一个类似的问题，但我不知道如何避免它。程序运行时间大约只有1.5分钟就超时了..

关于为什么会发生这种情况以及我如何解决它有什么想法吗？

编辑:

根据 Peter Pan - MSFT 的建议，我运行了一个更具体的查询:

entities_Azure=table_service.query_entities(table_name='#######',select='PartitionKey,RowKey,Timestamp,value', next_partition_key = None, next_row_key = None, top=2)

这返回了以下错误日志:

Error 0085: The following error occurred during script evaluation, please view the output log for more information:  
---------- Start of error message from Python interpreter ----------  
data:text/plain,Caught exception while executing function: Traceback (most recent call last):    

File "C:\server\invokepy.py", line 169, in 
batch odfs = mod.azureml_main(*idfs)    

File "C:\temp\azuremod.py", line 61, in 
azureml_main entities_Azure=table_service.query_entities(table_name='######',select='PartitionKey,RowKey,Timestamp,value', next_partition_key = None, next_row_key = None, top=2)    

File "./Script Bundle\azure\storage\table\tableservice.py", line 421, in query_entities
 response = self._perform_request(request)    

File "./Script Bundle\azure\storage\storageclient.py", line 171, in _perform_request
 resp = self._filter(request)    

File "./Script Bundle\azure\storage\table\tableservice.py", line 664, in _perform_request_worker
 return self._httpclient.perform_request(request)    

File "./Script Bundle\azure\storage\_http\httpclient.py", line 181, in perform_request
 self.send_request_body(connection, request.body)    

File "./Script Bundle\azure\storage\_http\httpclient.py", line 145, in send_request_body
 connection.send(None)    

File "./Script Bundle\azure\storage\_http\requestsclient.py", line 81, in send
 self.response = self.session.request(self.method, self.uri, data=request_body, headers=self.headers, timeout=self.timeout)    

File "C:\pyhome\lib\site-packages\requests\sessions.py", line 456, in request
 resp = self.send(prep, **send_kwargs)    

File "C:\pyhome\lib\site-packages\requests\sessions.py", line 559, in send
 r = adapter.send(request, **kwargs)    

File "C:\pyhome\lib\site-packages\requests\adapters.py", line 382, in send
 raise SSLError(e, request=request) 

SSLError: The write operation timed out    
---------- End of error message from Python  interpreter 
---------- Start time: UTC 11/18/2015 11:39:32 End time: UTC 11/18/2015 11:40:53

希望这能让我们更深入地了解情况!

Answer 1

我尝试用自己生成的数据填充表存储，并希望通过执行像您这样的查询来重现您的问题，但失败了。

我发现 REST API(用于 python 包装的 REST API 的 Azure 存储 sdk)的表存储查询超时问题。表服务 REST API 的页面 (https://msdn.microsoft.com/en-us/library/azure/dd894042.aspx)“查询超时和分页”说道:

A query against the Table service may return a maximum of 1,000 items at one time and may execute for a maximum of five seconds. If the result set contains more than 1,000 items, if the query did not complete within five seconds, or if the query crosses the partition boundary, the response includes headers which provide the developer with continuation tokens to use in order to resume the query at the next item in the result set. Continuation token headers may be returned for a Query Tables operation or a Query Entities operation.

Note that the total time allotted to the request for scheduling and processing the query is 30 seconds, including the five seconds for query execution.

It is possible for a query to return no results but to still return a continuation header.

我认为该问题是由于达到这些指定的阈值而引起的。

此外，我在数据输入和输出中使用了Reader模块，并使用Azure Table设置数据源，成功读取了1000个实体，并且快速进行 Azure ML Studio 实验。

对于这种场景，我建议您可以使用指定的查询过滤器来查询您的表存储，如下所示:

entities_Azure=table_session.query_entities(table_name=table_name,
      filter="PartitionKey eq '" + partitionKey + "' and Rowkey eq '" + rowkey + "'",
      select='PartitionKey,RowKey,Timestamp,value',
      next_partition_key = next_pk,
      next_row_key = next_rk, top=1000) 

我们可以通过这段代码来判断问题是连接问题还是阈值问题。

如有任何疑问，请随时告诉我。