java - Spring batch FlatfileitemReader 读取格式错误的行

Question

在我的项目中，我使用 Spring 批处理并使用 FlatFileItemReader/FieldSetMapper 读取文件。一些输入文件有问题。几条记录的行被剪切/格式错误。
假设输入文件有 4 列。几列未正确形成。谁能帮我解决这个问题？(如果需要我可以解释更多)
文件.csv

"id","name","age","salary"
"1","user1","28","1000"
"2","user2","27","2000"
"3","user3","26
    ","3000"
"4","user4","25","
    4000"
"5","
        user5","24","5000"
"6","user6","23","6000"
"7","user7","22","7000"
"8","user8","21","8000"

Answer 1

我在使用 FlatFileItemReader 读取格式错误的行时遇到了类似的问题。在这种情况下，您可以使用 DefaultRecordSeparatorPolicy 作为 FlatFileItemReader 中的 RecordSeparatorPolicy。它所做的是在读取一行后检查 endOfRecord。如果读取的行有任何未注释的引号，它会读取另一行以规范化输入。您还可以覆盖该行为。

flatFileItemReader.setRecordSeparatorPolicy(new DefaultRecordSeparatorPolicy());

引用DefaultRecordSeparatorPolicy API 文档以获取更多信息

@Bean
public FlatFileItemReader<YourClassName> itemReader(@Value("${input}") Resource resource) {
    FlatFileItemReader<YourClassName> flatFileItemReader = new FlatFileItemReader<>();
    flatFileItemReader.setResource(resource);
    flatFileItemReader.setName("CSV-Reader");
    flatFileItemReader.setLinesToSkip(1);
    // override default comment '#' from file parsing
    flatFileItemReader.setComments(new String[] {});
    // checks for multi-line csv inputs for very lage row
    flatFileItemReader.setRecordSeparatorPolicy(new DefaultRecordSeparatorPolicy());
    flatFileItemReader.setLineMapper(lineMapper());
    return flatFileItemReader;
}

@Bean
public LineMapper<YourClassName> lineMapper() {
    DelimitedLineTokenizer lineTokenizer = new DelimitedLineTokenizer();
    lineTokenizer.setDelimiter(DelimitedLineTokenizer.DELIMITER_COMMA);
    lineTokenizer.setQuoteCharacter(DelimitedLineTokenizer.DEFAULT_QUOTE_CHARACTER);
    lineTokenizer.setStrict(false);
    lineTokenizer.setNames(COLUMN_NAMES);

    BeanWrapperFieldSetMapper<YourClassName> fieldSetMapper = new BeanWrapperFieldSetMapper<>();
    fieldSetMapper.setTargetType(YourClassName.class);

    DefaultLineMapper<YourClassName> defaultLineMapper = new DefaultLineMapper<>();
    defaultLineMapper.setLineTokenizer(lineTokenizer);
    defaultLineMapper.setFieldSetMapper(fieldSetMapper);
    return defaultLineMapper;
}