python - 通过两个字符串名称过滤具有非常量形状的数据框 - Pandas

Question

我试图通过两个字符串名称过滤数据框，但问题是字符串可以位于数据框的任何一个系列中，并且系列的数量是可变的。如何过滤数据框的每个系列，然后将它们合并到一个数据框中？

import pandas as pd
import os

# Directories of Statements:
cdir = "Current Directory"
odir = "Output Directory"

# Find all CSVs in cdir:
excels = [filename for filename in os.listdir(cdir) if filename.endswith(".csv")]

# Define concat_csv Function:
def concat_csv(csv_file):
    df_csv = pd.read_csv(os.path.join(cdir, csv_file), header=None, index_col=None) # Load CSV into dataframe
    df_final = pd.DataFrame() # Create empty dataframe
    for col in df_csv: # For all columns in the dataframe filter rows by string 1 or 2 then create new dataframe
        df_i = df_csv[(df_csv[col].str.contains("string1")==True) or (df_csv[col].str.contains("string2")==True)] # Use row if string equals string 1 or 2
        df_final = df_final.concat(df_i, axis=1) # Concat all rows that contain string 1 or 2 to a new dataframe

    # Send final dataframe to CSV in output directory:
    df_final.to_csv(os.path.join(odir, os.path.splitext(os.path.basename(csv_file))[0] + ".csv"), encoding='utf-8')

# Apply concat_csv to all CSVs in cdir:
for f in excels:
    concat_csv(os.path.join(cdir, f))

这是我在 Scott Boston 的推荐后使用的最终代码:

...
# Define concat_csv Function:
def concat_csv(csv_file):
    df_csv = pd.read_csv(os.path.join(cdir, csv_file), header=None, index_col=None) # Load CSV into data frame

    df = df_csv[df_csv.isin(["string 1", "string2"]).any(axis=1)] # Filter data frame by UGL data
    df2 = df.dropna(axis=1, how="all") # Drop columns with all empty cells
    try:
        df_final = df2.set_index([0]) # Set index to column 1
    except:
        df_final = df2

# Send final dataframe to CSV in output directory:
df_final.to_csv(os.path.join(odir, os.path.splitext(os.path.basename(csv_file))[0] + ".csv"), encoding='utf-8')

# Apply concat_csv to all CSVs in cdir:
for f in excels:
    concat_csv(os.path.join(cdir, f))

Answer 1

IIUC:

您有一个包含 N 个系列的数据框，您想要检查两个字符串是否出现在任何系列中，并仅使用这些行构建一个新的数据框。

构建通用数据

df = pd.DataFrame({'A':np.random.choice(list('ABCDEFG'),size=26),'B':np.random.choice(list('FGHIJKLMN'),size=26)})

查找任意列中出现“G”或“F”的所有记录

df_final = df[df.isin(['G','F']).any(axis=1)]

print(df_final)

输出:

    A  B
0   G  I
2   G  G
4   A  G
7   F  N
8   F  M
10  C  F
11  A  G
14  F  G
16  G  H
18  F  L
19  D  G