scala - 使用scala添加其他列的长度作为值的列

Question

我的任务是计算每一列的长度并将消息添加到“errorMsg”列。我能够根据长度过滤记录，但无法在新列中附加消息。

例如。我只想找出新列“ErrorMsg”上带有消息的无效记录

记录长度=4

InputDataFrame-             
+------+
| value|
+------+
|Pra   |
|Akshay|
|  Raju|
|Shakti|
|xyz   |
+------+

输出数据帧

+------+------------------------+
| value|ErrorMsg                |
+------+------------------------+
|Pra   |Less Than total Length
|Akshay|Greater than total length 
|Shakti|Greater than total length
|xyx   |Less than total length
+------+-------------------------

raju 是我的真实记录，它转到没有消息的有效记录。

Answer 1

下面会得到想要的结果。

val df = Seq("Pra", "Akshay", "Raju", "Shakti", "xyz").toDF("value")
df
 .filter(not(length($"value") === 4))
 .withColumn("ErrorMsg", when(length($"value") > lit(4), "Greater than total length").otherwise("Less Than total Length"))
 .show(10000, false)

+------+-------------------------+
|value |ErrorMsg                 |
+------+-------------------------+
|Pra   |Less Than total Length   |
|Akshay|Greater than total length|
|Shakti|Greater than total length|
|xyz   |Less Than total Length   |
+------+-------------------------+