gpt4 book ai didi

python - 无法使用python为azure数据工厂中的每个事件创建

转载 作者:行者123 更新时间:2023-12-02 08:31:45 24 4
gpt4 key购买 nike

我正在尝试使用 python 创建 azure 数据工厂管道和资源。我成功完成了某些 ADF 事件,例如查找、复制 .. 但我在这里面临的问题是我尝试使用 FOR Each 事件将几个表从 SQL 复制到 blob,并且抛出以下错误

您将如何在每个事件中创建事件?非常感谢任何意见。谢谢!

引用号:https://learn.microsoft.com/en-us/python/api/azure-mgmt-datafactory/azure.mgmt.datafactory.models.foreachactivity?view=azure-python

错误消息

TypeError: 'CopyActivity' object is not iterable

代码块

## Lookup Activity
ls_sql_name = 'ls_'+project_name+'_'+src_svr_type+'_dev'
linked_service_name =LinkedServiceReference(reference_name=ls_sql_name)
lkp_act_name ='Get Table Names'
sql_reader_query = "SELECT top 3 name from sys.tables where name like '%dim'"
source = SqlSource(sql_reader_query= sql_reader_query)
dataset= {"referenceName": "ds_sql_Dim_input","type": "DatasetReference"}
LookupActivity_ = LookupActivity(name =lkp_act_name, linked_service_name= linked_service_name, source = source, dataset = dataset
,first_row_only =False)


#create copy activity
ds_name = 'ds_sql_dim_input' #these datasets already created
dsOut_name ='ds_blob_dim_output' #these datasets already created

copy_act_name = 'Copy SQL to Blob(parquet)'
sql_reader_query = {"value": "@item().name","type": "Expression"}
sql_source = SqlSource(sql_reader_query=sql_reader_query)
blob_sink = ParquetSink()
dsin_ref = DatasetReference(reference_name=ds_name)
dsOut_ref = DatasetReference(reference_name=dsOut_name)
copy_activity = CopyActivity(name=copy_act_name,inputs=[dsin_ref], outputs=[dsOut_ref], source=sql_source, sink=blob_sink)

## For Each Activity
pl_name ='pl_Test'
items= {"value": "@activity('Get Table Names').output.value","type": "Expression"}
dependsOn = [{"activity": "Get Table Names","dependencyConditions": ["Succeeded"]}]
ForEachActivity_= ForEachActivity(name = 'Copy tables in loop',items=items,depends_on=dependsOn ,activities =copy_activity)

params_for_pipeline = {}
p_obj = PipelineResource(activities=[LookupActivity_,ForEachActivity_], parameters=params_for_pipeline)
p = adf_client.pipelines.create_or_update(rg_name, df_name, pl_name, p_obj)

最佳答案

Activities 需要是 Activity 的列表,并且您传递的是单个 Activity。尝试创建一个列表并向其中添加复制事件,然后在事件参数中传递该列表。

关于python - 无法使用python为azure数据工厂中的每个事件创建,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/60342953/

24 4 0
Copyright 2021 - 2024 cfsdn All Rights Reserved 蜀ICP备2022000587号
广告合作:1813099741@qq.com 6ren.com