python - 将 Azure Blob CSV 读取到 Python Pandas DF 中的问题

Question

我正在尝试访问存储在 Azure blob 中的 csv 并将其读入 python 脚本中的 pandas 数据帧中。但我遇到了导入和实际读取 csv 的问题。我至少能够使用我的 python 脚本看到它的存在，如下所示:

import os, uuid, sys
from io import StringIO
import pandas as pd
from azure.storage.filedatalake import DataLakeServiceClient
from azure.core._match_conditions import MatchConditions
from azure.storage.filedatalake._models import ContentSettings
from azure.storage.blob import BlobServiceClient, BlobClient, ContainerClient, BlobService

try:  
    global service_client
    storage_account_name = 'ACCOUNT_NAME'
    storage_account_key = 'ACCOUNT_KEY'
    storage_connection_string = 'ACCOUNT_STRING'
    storage_container_name = 'CONTAINER_NAME'
    csv_path = '<PATH_TO>/FILE.csv'

    service_client = DataLakeServiceClient(account_url="{}://{}.dfs.core.windows.net".format(
        "https", storage_account_name), credential=storage_account_key)

    file_system_client = service_client.get_file_system_client(file_system=storage_container_name)

    print('GET PATH(S)')
    paths = file_system_client.get_paths(path=csv_path)
    for path in paths:
        print(path.name + '\n')

    blob_service = BlobService(account_name=storage_account_name, account_key=storage_account_key)
    blobstring = blob_service.get_blob_to_text(storage_container_name,csv_path)
    df = pd.read_csv(StringIO(blobstring))

except Exception as e:
    print(e)

finally:
    print('DONE')

问题是我无法正确地将 csv 读入我的 df 中。另外，我遇​​到了实际使用 BlobService 的问题，因为每次我尝试运行脚本时，都会收到错误:

导入错误:无法从“azure.storage.blob”导入名称“BlobService”

我的 azure 的 pip freeze 看起来像这样:

azure-common==1.1.25
azure-core==1.5.0
azure-storage-blob==12.3.1
azure-storage-common==2.1.0
azure-storage-file-datalake==12.0.1

我在这里做错了什么？

Answer 1

根据您提供的代码，您使用类BlobService从Azure Blob存储下载文件。该类位于 SDK azure.storage 0.20.0 中。但是您安装了 sdk azure.storage.blob。所以你会得到错误。由于您已经安装了 sdk azure.storage.blob，我们可以使用类 BlobClient下载 blob。

例如

from azure.storage.blob import BlobServiceClient, BlobClient, ContainerClient
#download csv file from Azure blob
sas_url = "your blob sas url"
blob_client = BlobClient.from_blob_url(sas_url)
downloaded_blo = blob_client.download_blob()

#read csv file
import pandas as pd
df = pd.read_csv(StringIO(downloaded_blob.content_as_text()) )