python - 从旧数据框创建子列

Question

我正在尝试创建一个带有子列的 DataFrame，其中子列是 Dataframes。

数据:

import pandas as pd
from pandas import Timestamp

df1={'Open': {Timestamp('2020-12-15 01:05:00'): 152.28, Timestamp('2020-12-15 01:10:00'): 151.59, Timestamp('2020-12-15 01:15:00'): 152.19}, 'High': {Timestamp('2020-12-15 01:05:00'): 152.28, Timestamp('2020-12-15 01:10:00'): 152.39, Timestamp('2020-12-15 01:15:00'): 152.38}, 'Low': {Timestamp('2020-12-15 01:05:00'): 150.0, Timestamp('2020-12-15 01:10:00'): 151.34, Timestamp('2020-12-15 01:15:00'): 150.67}, 'Close': {Timestamp('2020-12-15 01:05:00'): 151.58, Timestamp('2020-12-15 01:10:00'): 152.21, Timestamp('2020-12-15 01:15:00'): 151.12}, 'price': {Timestamp('2020-12-15 01:05:00'): 149.305, Timestamp('2020-12-15 01:10:00'): nan, Timestamp('2020-12-15 01:15:00'): nan}, 'executedQty': {Timestamp('2020-12-15 01:05:00'): 6.991142857142856, Timestamp('2020-12-15 01:10:00'): nan, Timestamp('2020-12-15 01:15:00'): nan}, 'side': {Timestamp('2020-12-15 01:05:00'): 1.0, Timestamp('2020-12-15 01:10:00'): nan, Timestamp('2020-12-15 01:15:00'): nan}}
df2={'Open': {Timestamp('2020-12-15 01:05:00'): 5.385, Timestamp('2020-12-15 01:10:00'): 5.403, Timestamp('2020-12-15 01:15:00'): 5.419}, 'High': {Timestamp('2020-12-15 01:05:00'): 5.417999999999999, Timestamp('2020-12-15 01:10:00'): 5.428999999999999, Timestamp('2020-12-15 01:15:00'): 5.42}, 'Low': {Timestamp('2020-12-15 01:05:00'): 5.3839999999999995, Timestamp('2020-12-15 01:10:00'): 5.395, Timestamp('2020-12-15 01:15:00'): 5.351}, 'Close': {Timestamp('2020-12-15 01:05:00'): 5.406000000000001, Timestamp('2020-12-15 01:10:00'): 5.414, Timestamp('2020-12-15 01:15:00'): 5.37}, 'price': {Timestamp('2020-12-15 01:05:00'): nan, Timestamp('2020-12-15 01:10:00'): nan, Timestamp('2020-12-15 01:15:00'): nan}, 'executedQty': {Timestamp('2020-12-15 01:05:00'): nan, Timestamp('2020-12-15 01:10:00'): nan, Timestamp('2020-12-15 01:15:00'): nan}, 'side': {Timestamp('2020-12-15 01:05:00'): nan, Timestamp('2020-12-15 01:10:00'): nan, Timestamp('2020-12-15 01:15:00'): nan}}
df3={'Open': {Timestamp('2020-12-15 01:05:00'): 12.455, Timestamp('2020-12-15 01:10:00'): 12.429, Timestamp('2020-12-15 01:15:00'): 12.442}, 'High': {Timestamp('2020-12-15 01:05:00'): 12.458, Timestamp('2020-12-15 01:10:00'): 12.456, Timestamp('2020-12-15 01:15:00'): 12.443}, 'Low': {Timestamp('2020-12-15 01:05:00'): 12.425999999999998, Timestamp('2020-12-15 01:10:00'): 12.425, Timestamp('2020-12-15 01:15:00'): 12.383}, 'Close': {Timestamp('2020-12-15 01:05:00'): 12.435, Timestamp('2020-12-15 01:10:00'): 12.442, Timestamp('2020-12-15 01:15:00'): 12.401}, 'price': {Timestamp('2020-12-15 01:05:00'): nan, Timestamp('2020-12-15 01:10:00'): nan, Timestamp('2020-12-15 01:15:00'): nan}, 'executedQty': {Timestamp('2020-12-15 01:05:00'): nan, Timestamp('2020-12-15 01:10:00'): nan, Timestamp('2020-12-15 01:15:00'): nan}, 'side': {Timestamp('2020-12-15 01:05:00'): nan, Timestamp('2020-12-15 01:10:00'): nan, Timestamp('2020-12-15 01:15:00'): nan}}

df1=pd.DataFrame(df1)
df2=pd.DataFrame(df2)
df3=pd.DataFrame(df3)

类似这样的输出是预期的，但我希望它们共享时间戳索引。因此，一行将包含来自一个索引下所有数据帧的所有数据。

df3=pd.DataFrame()
dfList=[df1,df2,df3]

for df in dfList:
    cols = pd.MultiIndex.from_frame(df, ['Open','High','Low','Close','price','executedQty' ,'side'])
    df=pd.DataFrame(df, columns=cols)
    df3=df3.join(df)

print(df3)

                     df1
                       Open    High     Low   Close    price  executedQty  \
2020-12-15 01:05:00  152.28  152.28  150.00  151.58  149.305     6.991143   
2020-12-15 01:10:00  151.59  152.39  151.34  152.21      NaN          NaN   
2020-12-15 01:15:00  152.19  152.38  150.67  151.12      NaN          NaN   

                     side  
2020-12-15 01:05:00   1.0  
2020-12-15 01:10:00   NaN  
2020-12-15 01:15:00   NaN 
 
                     df2
                      Open   High    Low  Close  price  executedQty  side
2020-12-15 01:05:00  5.385  5.418  5.384  5.406    NaN          NaN   NaN
2020-12-15 01:10:00  5.403  5.429  5.395  5.414    NaN          NaN   NaN
2020-12-15 01:15:00  5.419  5.420  5.351  5.370    NaN          NaN   NaN

                     df3
                       Open    High     Low   Close  price  executedQty  side
2020-12-15 01:05:00  12.455  12.458  12.426  12.435    NaN          NaN   NaN
2020-12-15 01:10:00  12.429  12.456  12.425  12.442    NaN          NaN   NaN
2020-12-15 01:15:00  12.442  12.443  12.383  12.401    NaN          NaN   NaN

我还想在创建其他 DataFrame 的循环中使用此函数。这是因为有超过 3 个 DataFrame，它们是通过请求中的数据创建的，否则我将不得不在 concat 之前命名每个 DataFrame，这对我的用例来说太多了。

像这样

dfList=[df1,df2,df3]

dataFrame=pd.DataFrame

for d in dfList:
    df=requestFuncThatCreatesDf(d)
    dataFrame=dataFrame.concat([df],key=(d))

Answer 1

您可以将 pd.concat 沿 axis=1 与可选的 keys 参数一起使用来连接数据帧，以便生成的帧共享相同的 Timestamp 索引并具有 MultiIndex 列:

pd.concat([df1, df2, df3], axis=1, keys=('df1', 'df2', 'df3'))

编辑(如果你想动态生成对应于 dfList 中数据帧顺序的键):

dfs = [requestFuncThatCreatesDf(d) for d in dfList]
pd.concat(dfs, axis=1, keys=[f'df{i + 1}' for i in range(len(dfs))])

结果:

                        df1                                                      df2                                                 df3                                               
                       Open    High     Low   Close    price executedQty side   Open   High    Low  Close price executedQty side    Open    High     Low   Close price executedQty side
2020-12-15 01:05:00  152.28  152.28  150.00  151.58  149.305    6.991143  1.0  5.385  5.418  5.384  5.406   NaN         NaN  NaN  12.455  12.458  12.426  12.435   NaN         NaN  NaN
2020-12-15 01:10:00  151.59  152.39  151.34  152.21      NaN         NaN  NaN  5.403  5.429  5.395  5.414   NaN         NaN  NaN  12.429  12.456  12.425  12.442   NaN         NaN  NaN
2020-12-15 01:15:00  152.19  152.38  150.67  151.12      NaN         NaN  NaN  5.419  5.420  5.351  5.370   NaN         NaN  NaN  12.442  12.443  12.383  12.401   NaN         NaN  NaN