java - 类型错误 : 'JavaPackage' object is not callable AWS Glue Pyspark

Question

我正在尝试按照 AWS documentation 在我的 ubuntu Virtual box 上设置 AWS Glue 环境。

我已经完成了必要的工作，例如下载 aws 胶水库、spark 包和按照建议设置 spark home。之后，我无法初始化胶水上下文并面临以下错误。

from awsglue.context import GlueContext
from pyspark.context import SparkContext

glueContext = GlueContext(SparkContext.getOrCreate())
or 
glueContext = GlueContext(sc)

错误:

TypeError          Traceback (most recent call last)
<ipython-input-15-0798793d4033> in <module>
----> 1 glueContext = GlueContext(SparkContext.getOrCreate())

~/aws-glue-libs-glue-1.0/PyGlue.zip/awsglue/context.py in __init__(self, sparkContext, **options)
     43         super(GlueContext, self).__init__(sparkContext)
     44         register(sparkContext)
---> 45         self._glue_scala_context = self._get_glue_scala_context(**options)
     46         self.create_dynamic_frame = DynamicFrameReader(self)
     47         self.write_dynamic_frame = DynamicFrameWriter(self)

~/aws-glue-libs-glue-1.0/PyGlue.zip/awsglue/context.py in _get_glue_scala_context(self, **options)
     64 
     65         if min_partitions is None:
---> 66             return self._jvm.GlueContext(self._jsc.sc())
     67         else:
     68             return self._jvm.GlueContext(self._jsc.sc(), min_partitions, target_partitions)

TypeError: 'JavaPackage' object is not callable

Answer 1

将 aws-glue-libs jar 文件复制到 Spark Jar 文件夹。这意味着将 jar 文件从\aws-glue-libs\jarsv1\文件夹复制到\spark-2.4.3-bin-spark-2.4.3-bin-hadoop2.8\jars 文件夹