python - 在 Synapse 的 SQL 数据库中保存 PySpark 数据帧时出现错误 "IllegalArgumentException: KrbException: Cannot locate default realm"

Question

我尝试将 PySpark 数据帧保存在 Synapse 的 SQL 数据库中:

test = spark.createDataFrame([Row("Sarah", 28), Row("Anne", 5)], ["Name", "Age"])
test.write\
    .format("jdbc")\
    .option("url", "jdbc:sqlserver://XXXX.sql.azuresynapse.net:1433;database=azlsynddap001;encrypt=true;trustServerCertificate=false;hostNameInCertificate=*.sql.azuresynapse.net;loginTimeout=30;Authentication=ActiveDirectoryIntegrated")\
    .option("forwardSparkAzureStorageCredentials", "true")\
    .option("dbTable", "test_CP")\
    .save()

我收到以下错误:

IllegalArgumentException: KrbException: Cannot locate default realm

这是错误的详细信息:

---------------------------------------------------------------------------
IllegalArgumentException                  Traceback (most recent call last)
/tmp/ipykernel_7675/2619697817.py in <module>
      1 test = spark.createDataFrame([Row("Sarah", 28), Row("Anne", 5)], ["Name", "Age"])
----> 2 test.write\
      3     .format("jdbc")\
      4     .option("url", "jdbc:sqlserver://XXXX.sql.azuresynapse.net:1433;database=XXXX;encrypt=true;trustServerCertificate=false;hostNameInCertificate=*.sql.azuresynapse.net;loginTimeout=30;Authentication=ActiveDirectoryIntegrated")\
      5     .option("forwardSparkAzureStorageCredentials", "true")\

/opt/spark/python/lib/pyspark.zip/pyspark/sql/readwriter.py in save(self, path, format, mode, partitionBy, **options)
   1105             self.format(format)
   1106         if path is None:
-> 1107             self._jwrite.save()
   1108         else:
   1109             self._jwrite.save(path)

~/cluster-env/env/lib/python3.8/site-packages/py4j/java_gateway.py in __call__(self, *args)
   1302 
   1303         answer = self.gateway_client.send_command(command)
-> 1304         return_value = get_return_value(
   1305             answer, self.gateway_client, self.target_id, self.name)
   1306 

/opt/spark/python/lib/pyspark.zip/pyspark/sql/utils.py in deco(*a, **kw)
    115                 # Hide where the exception came from that shows a non-Pythonic
    116                 # JVM exception message.
--> 117                 raise converted from None
    118             else:
    119                 raise

IllegalArgumentException: KrbException: Cannot locate default realm

这个错误从何而来？我用谷歌到处搜索但找不到我做错了什么。我也看到这个post和我有同样的问题。

Answer 1

您可以尝试使用专用的突触连接器而不是 JDBC，更多详细信息 here

此外，如果您正在使用 databricks，那么您可以尝试使用 databricks SQL DW 连接器详细信息 here