azure - Spacy无法加载模型: en_core_web_md in Azure databricks

Question

所以 spacy 似乎已成功安装在 Azure databricks 集群中的笔记本中。

%shpython -m spacy 下载 en_core_web_md

然后我在单元格中使用以下命令对其进行验证

%sh

python -m spacy 验证

这将返回以下结果:

⠙ 正在加载兼容性表...✔ 已加载兼容性表

====================== 已安装的模型 (spaCy v2.3.2) ================= =====ℹ spaCy安装:/databricks/conda/envs/databricks-ml/lib/python3.7/site-packages/spacy

类型名称 型号版本
软件包 en-core-web-md en_core_web_md 2.3.1 ✔

因此中英文模型显示为已加载。但是，当运行此命令时

nlp = spacy.load("en_core_web_md")

出现以下错误消息:

操作系统错误:[E050] 找不到模型“en_core_web_md”。它似乎不是快捷方式链接、Python 包或数据目录的有效路径。

Answer 1

您可以按照以下步骤在 Azure Databricks 上安装和加载 spaCy 包。

第 1 步:使用 pip 安装 spaCy 并下载 spaCy 模型。

%sh
/databricks/python3/bin/pip install spacy 
/databricks/python3/bin/python3 -m spacy download en_core_web_sm

笔记本输出:

第 2 步:使用 spaCy 运行示例。

import spacy

# Load English tokenizer, tagger, parser, NER and word vectors
nlp = spacy.load("en_core_web_sm")

# Process whole documents
text = ("When Sebastian Thrun started working on self-driving cars at "
        "Google in 2007, few people outside of the company took him "
        "seriously. “I can tell you very senior CEOs of major American "
        "car companies would shake my hand and turn away because I wasn’t "
        "worth talking to,” said Thrun, in an interview with Recode earlier "
        "this week.")
doc = nlp(text)

# Analyze syntax
print("Noun phrases:", [chunk.text for chunk in doc.noun_chunks])
print("Verbs:", [token.lemma_ for token in doc if token.pos_ == "VERB"])

# Find named entities, phrases and concepts
for entity in doc.ents:
    print(entity.text, entity.label_)

笔记本输出: