azure - 如何将数据存储 URI 传递给 azureml.fsspec.AzureMachineLearningFileSystem Python SDK？

Question

我已经注册了一个 ADLS 数据存储。

datastore = mlclient.datastores.get(ds_name)

from azureml.fsspec import AzureMachineLearningFileSystem

#azureml://subscriptions/<subid>/resourcegroups/<rgname>/workspaces/<workspace_name>/datastore/datastorename
ds_url = f"azureml://subscriptions/{subscriptionID}/resourcegroups/{RG}/workspaces/{ws_name}/datastore/adls/paths/iris-processed/*"
fs = AzureMachineLearningFileSystem(ds_url)
fs.ls()

即使我使用 datastore.id，我也会收到以下错误:

ValueError: azureml://subscriptions/xx/resourcegroups/xx/workspaces/xx/datastore/adls/paths/iris-processed/* is not a valid datastore uri: azureml://subscriptions/([^\/]+)/resourcegroups/([^\/]+)/(?:Microsoft.MachineLearningServices/)?workspaces/([^\/]+)/datastores/([^\/]+)/paths/(.*)

Answer 1

ValueError:azureml://subscriptions/xx/resourcegroups/xx/workspaces/xx/datastore/adls/paths/iris-processed/* is not a valid datastoreuri:azureml://subscriptions/([^/]+)/resourcegroups/([^/]+)/(?:Microsoft.MachineLearningServices/)workspaces/([^/]+)/datastores/([^/]+)/paths/(.*)

当您在 URI 中传递错误的参数(例如 Susbcriptionid、资源组、工作空间名称、数据存储名称和路径)时，会出现上述错误。

我尝试使用相同的代码在 Uri 中使用正确的参数，并得到了预期的结果。

代码:

from azureml.fsspec import AzureMachineLearningFileSystem

subscription_id = 'Subscription-id'
resource_group = 'Your-resource-group'
workspace_name = 'Workspacename'
input_datastore_name = 'datastore1'
path_on_datastore = 'folder1/'

#azureml://subscriptions/<subid>/resourcegroups/<rgname>/workspaces/<workspace_name>/datastore/datastorename
ds_url =  f'azureml://subscriptions/{subscription_id}/resourcegroups/{resource_group}/workspaces/{workspace_name}/datastores/{input_datastore_name}/paths/{path_on_datastore}'
fs = AzureMachineLearningFileSystem(ds_url)
f_list = fs.ls()
print(f_list)

输出:

['datastore1/folder1/09-05-2023 (1).html', 'datastore1/folder1/09-05-2023.html', 'datastore1/folder1/10-05-2023.html', 'datastore1/folder1/10-05=2023.html', 'datastore1/folder1/11-05-2023.html', 'datastore1/folder1/12-05-2023 (1).html', 'datastore1/folder1/12-05-2023.html', 'datastore1/folder1/timezone.csv']

引用: Is there a way to get list of folders from a datastore in Azure ML studio with Python SDK v2 - Stack Overflow作者:khemanth958。