azure - 从 Azure 表存储中高效检索大量实体

Question

有哪些方法可以优化从 Azure 表存储到 .NET 应用程序的单个分区检索大量实体 (~250K) 的过程？

Answer 1

据我所知，有两种方法可以优化从单个分区从 Azure 表存储到 .NET 应用程序的大量实体的检索。

1.如果你不需要获取实体的所有属性，我建议你可以使用服务器端投影。

A single entity can have up to 255 properties and be up to 1 MB in size. When you query the table and retrieve entities, you may not need all the properties and can avoid transferring data unnecessarily (to help reduce latency and cost). You can use server-side projection to transfer just the properties you need.

来自:Azure Storage Table Design Guide: Designing Scalable and Performant Tables（Server-side projection）

更多详细信息，您可以引用以下代码:

string filter = TableQuery.GenerateFilterCondition(
        "PartitionKey", QueryComparisons.Equal, "Sales");
List<string> columns = new List<string>() { "Email" };
TableQuery<EmployeeEntity> employeeQuery =
        new TableQuery<EmployeeEntity>().Where(filter).Select(columns);

var entities = employeeTable.ExecuteQuery(employeeQuery);
foreach (var e in entities)
{
        Console.WriteLine("RowKey: {0}, EmployeeEmail: {1}", e.RowKey, e.Email);
}

2.如果您只想显示表的消息，则无需同时获取所有实体。您可以获得部分结果。如果您想获得其他结果，可以使用延续标记。这将提高表查询性能。

A query against the table service may return a maximum of 1,000 entities at one time and may execute for a maximum of five seconds. If the result set contains more than 1,000 entities, if the query did not complete within five seconds, or if the query crosses the partition boundary, the Table service returns a continuation token to enable the client application to request the next set of entities. For more information about how continuation tokens work, see Query Timeout and Pagination.

来自:Azure Storage Table Design Guide: Designing Scalable and Performant Tables(Retrieving large numbers of entities from a query)

通过显式使用延续 token ，您可以控制应用程序何时检索下一个数据段。

更多详细信息，您可以引用以下代码:

string filter = TableQuery.GenerateFilterCondition(
        "PartitionKey", QueryComparisons.Equal, "Sales");
TableQuery<EmployeeEntity> employeeQuery =
        new TableQuery<EmployeeEntity>().Where(filter);

TableContinuationToken continuationToken = null;

do
{
        var employees = employeeTable.ExecuteQuerySegmented(
        employeeQuery, continuationToken);
    foreach (var emp in employees)
    {
    ...
    }
    continuationToken = employees.ContinuationToken;
} while (continuationToken != null);

此外，我建议您可以关注表分区的可扩展性目标。

单个表分区的目标吞吐量(1 KB 实体)每秒最多 2000 个实体

如果您达到此分区的可扩展性目标，存储服务将受到限制。