python - 使用 pySpark 计算月末差异

Question

我有一个要求，需要满足以下请求:

IF EndOfMonth(to_date(df.col1, 'DD.MM.YYYY')) >= EOM(CURRENT_DATE):
THEN "OldCustomer"
ELSE "NewCustomer"

输入数据框:

col1 (stringtype)
20190523
20200428
20130819

Current_date:从 python import datetime 获取

from datetime import date

now = date.today()

输出应该是这样的:

col1 (Stringtype)  col1_formated  customer(stringtype)
 20190523          23.05.2019     oldcustomer
 20200428          28.04.2020     newcustomer
 20130819          19.03.2013     oldcustomer

请帮忙进行eom格式编码？这真的很有帮助。

Answer 1

last_day函数就是您正在寻找的:

df.withColumn("col1_formated", date_format(to_date("col1", "yyyyMMdd"), "dd.MM.yyyy")) \
  .withColumn("customer",
              when(last_day(to_date("col1", "yyyyMMdd")) >= last_day(current_date()),
                   "OldCustomer").otherwise("NewCustomer")
             ).show()