Azure 数据工厂 ForEachActivity : @item() evaluating to an empty string in . NET SDK

Question

我正在使用 .NET SDK 以编程方式创建 Azure 数据工厂管道。我正在尝试使用 ForEachActivity 并在其中引用 DatasetReference 中的 @item() 。但是，@item() 似乎正在计算空字符串。下面的代码片段演示了 ForEachActivity 和后续管道的创建:

ForEachActivity forEachActivity = new ForEachActivity
{
    Name = "ForEachDbContainer",
    BatchCount = 20,  // You can adjust this based on your needs
    //IsSequential = false,
    Items = new Expression { Value = "@array(['Source_db1_container1'])" },
    Activities = new List<Activity>
    {
        new CopyActivity
        {
            Name = "CosmosDBCopyActivity",
            Inputs = new List<DatasetReference> { new DatasetReference { ReferenceName = "@item()" } },
            Outputs = new List<DatasetReference> { new DatasetReference { ReferenceName = "Destination_db1_container1" } },
            Source = new DocumentDbCollectionSource { },
            Sink = new DocumentDbCollectionSink { },
        }
    }
};

PipelineResource pipeline = new PipelineResource
{
    Activities = new List<Activity> { forEachActivity }
};
client.Pipelines.CreateOrUpdate(resourceGroupName, dataFactoryName, "YourPipelineName", pipeline);

执行上述代码时，我遇到以下错误:

Unhandled exception. Microsoft.Rest.Azure.CloudException: The document creation or update failed because of invalid reference ''.

谁能帮我弄清楚为什么 @item() 计算结果为空字符串，以及如何解决这个问题？

Answer 1

您遇到的问题可能与您在 DatasetReference 对象中使用 @item() 表达式的方式有关。

@item() 表达式用于引用 ForEachActivity 的 Items 数组中的当前项目。但是，在您的代码中，您使用 @item() 表达式作为 DatasetReference 对象的 ReferenceName，这是不正确的。

您应该使用以下代码为每个事件创建，例如:

ForEachActivity forEachActivity = new ForEachActivity
{
    Name = "ForEachDbContainer",
    BatchCount = 20, 
    //IsSequential = false,
    Items = new Expression { Value = "@CreateArray(["your-collection-1","Collection-2"])" },
    Activities = new List<Activity>
    {
        new CopyActivity
        {
            Name = "CosmosDBCopyActivity",
                        Inputs = new List<DatasetReference>
                        {
                            new DatasetReference()
                            {
                                ReferenceName = datasetName //<your-source-datasetname>
                            }
                        },
                        Outputs = new List<DatasetReference>
                        {
                            new DatasetReference
                            {
                                ReferenceName = dataset2 //<Your-destination-datasetname>
                            }
                        },
            Source = new DocumentDbCollectionSource { },
            Sink = new DocumentDbCollectionSink { },
        }
    }
};

PipelineResource pipeline = new PipelineResource
{
    Activities = new List<Activity> { forEachActivity }
};
client.Pipelines.CreateOrUpdate(resourceGroupName, dataFactoryName, "YourPipelineName", pipeline);

您可以使用以下代码为 cosmosdb 创建数据集。

为 cosmos DB 创建数据集时，在集合中将值指定为 "@item()"

代码:

var containerDataset = new DatasetResource
{
    Properties = new CosmosDbSqlApiCollectionDataset
    {
        LinkedServiceName = new LinkedServiceReference { ReferenceName = "xxxxx" },
        CollectionName = "@item()"
    }
};
adfClient.Datasets.CreateOrUpdate(resourceGroupName, dataFactoryName, datasetName, containerDataset);

引用:

• CosmosDBSqlApiCollectionDataset Class (Azure.ResourceManager.DataFactory.Models) - Azure for .NET Developers | Microsoft Learn
• ForEachActivity.Items Property (Microsoft.Azure.Management.DataFactory.Models) - Azure for .NET Developers | Microsoft Learn