apache-spark - spark nlp 中的多语言 bert

Question

我想知道在 sparknlp 中是否可以使用经过预训练的多语言 Bert？
如您所知，Bert 接受过 109 种语言的预训练。我想知道所有这些语言是否也都在 spark bert 中？
谢谢

Answer 1

从 spark nlp documentation 可以看出:

Models
Spark NLP offers more than 250 pre-trained models in 46 languages.

List of languages: Afrikaans, Arabic, Armenian, Basque, Bengali, Breton, Bulgarian, Catalan, Czech, Dutch, English, Esperanto, Finnish, French, Galician, German, Greek, Hausa, Hebrew, Hindi, Hungarian, Indonesian, Irish, Italian, Japanese, Latin, Latvian, Marathi, Norwegian, Persian, Polish, Portuguese, Romanian, Russian, Slovak, Slovenian, Somali, Southern Sotho, Spanish, Swahili, Swedish, Tswana, Turkish, Ukrainian, Zulu

还有这个 repository它保留了 Spark NLP 库的管道和模型的预训练版本。