scala - 如何在 Spark 中将 RowMatrix 更改为数组或将其导出为 CSV？

Question

我在 Scala 中得到了这段代码:

val mat: CoordinateMatrix = new CoordinateMatrix(data)
val rowMatrix: RowMatrix = mat.toRowMatrix()

val svd: SingularValueDecomposition[RowMatrix, Matrix] = rowMatrix.computeSVD(100, computeU = true)

val U: RowMatrix = svd.U // The U factor is a RowMatrix.
val S: Vector = svd.s // The singular values are stored in a local dense vector.
val V: Matrix = svd.V // The V factor is a local dense matrix.

val uArray: Array[Double] = U.toArray // doesn't work, because there is not toArray function in RowMatrix type
val sArray: Array[Double] = S.toArray // works good
val vArray: Array[Double] = V.toArray // works good

如何将 U 更改为 uArray 或类似类型，以便打印到 CSV 文件中？

Answer 1

这是一个基本操作，考虑到 U 是 RowMatrix，您必须执行以下操作:

val U = svd.U

rows() is a RowMatrix method that allows you to get an RDD from your RowMatrix by row.

您只需要在 RowMatrix 上应用行并映射 RDD[Vector] 即可创建一个数组，您可以将其连接到创建 RDD[String] 的字符串中。

val rdd = U.rows.map( x => x.toArray.mkString(","))

现在您需要做的就是保存 RDD:

rdd.saveAsTextFile(path)