gpt4 book ai didi

azure - python 中的错误日志记录不适用于 azure databricks

转载 作者:行者123 更新时间:2023-12-02 23:43:32 26 4
gpt4 key购买 nike

没有人回答与此问题相关的问题

我尝试在 azure 数据 block 中使用 python 实现错误日志记录。如果我在 python(pycharm) 中尝试以下代码,它按预期工作。但当我在 azure databricks(python) 中尝试相同的代码时,它没有创建文件,并且不向文件中写入任何内容。我尝试创建一个文件在第二代 Azure 数据湖中。我已经给出了带有数据挂载点的路径Lake Store gen2。

你能帮忙解释一下为什么Python代码没有按预期工作吗? azure 数据砖(Python)

# importing module
import logging

dbutils.fs.mkdirs('/dbfs/mnt/sales/region/country/sample/newfile.txt')

# Create and configure logger
logging.basicConfig(filename="/dbfs/mnt/sales/region/
country/sample/newfile.txt",
format='%(asctime)s %(message)s',
filemode='a')

# Creating an object
logger = logging.getLogger()

# Setting the threshold of logger to DEBUG
logger.setLevel(logging.DEBUG)

# Test messages
logger.debug("Harmless debug Message")
logger.info("Just an information")
logger.warning("Its a Warning")
logger.error("Did you try to divide by zero")
logger.critical("Internet is down")


If i open the file i expect the output to be like below which is
happening with python but the same is not working with azure data
bricks(python)

2019-06-06 00:19:23,881 Harmless debug Message
2019-06-06 00:19:23,881 Just an information
2019-06-06 00:19:23,881 Its a Warning
2019-06-06 00:19:23,881 Did you try to divide by zero
2019-06-06 00:19:23,881 Internet is down
2019-06-06 00:19:33,447 Harmless debug Message
2019-06-06 00:19:33,447 Just an information
2019-06-06 00:19:33,447 Its a Warning
2019-06-06 00:19:33,447 Did you try to divide by zero
2019-06-06 00:19:33,447 Internet is down

最佳答案

在 Databricks 中,您已在路径 /dbfs/mnt/sales 处安装了 Blob 存储容器(或 ADLS Gen2 文件系统)。您无法对 blob 存储支持的文件进行随机写入,并且 Python 日志库只是默默地失败。

https://docs.databricks.com/data/databricks-file-system.html#local-file-api-limitations

要测试这个:

# this works
with open('/dbfs/mnt/container-name/my-app.log', 'w') as fid:
fid.write('this is a message')

# this fails, since append to existing file is a random write operation
with open('/dbfs/mnt/container-name/my-app.log', 'a') as fid:
fid.write('this message will not work')

关于azure - python 中的错误日志记录不适用于 azure databricks,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/56466855/

26 4 0
Copyright 2021 - 2024 cfsdn All Rights Reserved 蜀ICP备2022000587号
广告合作:1813099741@qq.com 6ren.com