python - 使用来自 StringIndexer 的标签进行 IndexToString 转换

Question

如何通过从 labelIndexer 获取标签，使用 IndexToString 进行转换？

labelIndexer = StringIndexer(inputCol="shutdown_reason", outputCol="label")

idx_to_string = IndexToString(inputCol="prediction", outputCol="predictedValue")

Answer 1

How can I convert using IndexToString by taking the labels from labelIndexer?

你不能。 labelIndexer 是一个 StringIndexer，要获取标签，您需要 StringIndexerModel。 拟合模型:

from pyspark.ml.feature import *

df = spark.createDataFrame([
    ("foo", ), ("bar", )
]).toDF("shutdown_reason")

labelIndexerModel = labelIndexer.fit(df)

使用标签:

idx_to_string.setLabels(labelIndexerModel.labels)
idx_to_string.getLabels()
# ['foo', 'bar']

和转换:

df_with_prediction = labelIndexerModel.transform(df).withColumnRenamed(
    "label", "prediction"
)

idx_to_string.transform(df_with_prediction).show()
# +---------------+----------+--------------+
# |shutdown_reason|prediction|predictedValue|
# +---------------+----------+--------------+
# |            foo|       0.0|           foo|
# |            bar|       1.0|           bar|
# +---------------+----------+--------------+