java - FlatFileItemReader 制表符分隔符不起作用

Question

我从 Spring 开始检查这个项目: https://github.com/spring-guides/gs-batch-processing

来源:https://spring.io/guides/gs/batch-processing/

我将“sample-data.csv”中的“,”替换为“tab”:

Jill    Doe
Joe Doe
Justin  Doe
Jane    Doe
John    Doe

然后我将新的定界符添加到阅读器:

@Bean
public FlatFileItemReader<Person> reader() {
    return new FlatFileItemReaderBuilder<Person>()
        .name("personItemReader")
        .resource(new ClassPathResource("sample-data.csv"))
        .delimited()
        .delimiter(DelimitedLineTokenizer.DELIMITER_TAB) // NEW DELIMITER
        .names(new String[]{"firstName", "lastName"})
        .fieldSetMapper(new BeanWrapperFieldSetMapper<Person>() {{
            setTargetType(Person.class);
        }})
        .build();
}

启动时出现此错误:

Caused by: org.springframework.batch.item.file.transform.IncorrectTokenCountException: Incorrect number of tokens found in record: expected 2 actual 1
    at org.springframework.batch.item.file.transform.AbstractLineTokenizer.tokenize(AbstractLineTokenizer.java:142) ~[spring-batch-infrastructure-4.0.1.RELEASE.jar:4.0.1.RELEASE]
    at org.springframework.batch.item.file.mapping.DefaultLineMapper.mapLine(DefaultLineMapper.java:43) ~[spring-batch-infrastructure-4.0.1.RELEASE.jar:4.0.1.RELEASE]
    at org.springframework.batch.item.file.FlatFileItemReader.doRead(FlatFileItemReader.java:180) ~[spring-batch-infrastructure-4.0.1.RELEASE.jar:4.0.1.RELEASE]
    ... 50 common frames omitted

我试过使用“@”分隔符 -> 它有效。出于某种原因，我无法使用制表符分隔符...

当然在我的真实项目中，我有一个带有“制表符”分隔符的输入文件...

这里有什么解决方案吗？

Answer 1

您不能那样设置制表符分隔符。由于制表符 ('\t') 不包含任何实际文本，它会被 FlatFileItemReaderBuilder.java 中静态 DelimitedBuilder 类中的 DelimitedLineTokenizer 忽略。可以使用您在问题中提供的上述代码设置任何非空白分隔符。

FlatFileItemReaderBuilder sourceCode

这就是 LineTokenizer 实例在 FlatFileItemReaderBuilder.java 中的构建方式。

public DelimitedLineTokenizer build() {
        Assert.notNull(this.fieldSetFactory, "A FieldSetFactory is required.");
        Assert.notEmpty(this.names, "A list of field names is required");

        DelimitedLineTokenizer tokenizer = new DelimitedLineTokenizer();

        tokenizer.setNames(this.names.toArray(new String[this.names.size()]));

        // the hasText ignores the tab delimiter.

        if(StringUtils.hasText(this.delimiter)) {
            tokenizer.setDelimiter(this.delimiter);
        }
// more code

因此，要解决此问题，您需要提供明确配置有制表符分隔符的 DelimitedLineTokenizer 类型的 bean。

在您的 spring 配置文件中使用以下代码来设置制表符分隔符:

@Bean
public FlatFileItemReader<Person> reader() {
    return new FlatFileItemReaderBuilder<Person>().name("personItemReader")
            .resource(new ClassPathResource("sample-data.csv"))
            .lineMapper(lineMapper()).build();
}

@Bean
public DefaultLineMapper<Person> lineMapper(){
      DefaultLineMapper<Person> lineMapper = new DefaultLineMapper<>();
      lineMapper.setLineTokenizer(lineTokenizer());
      lineMapper.setFieldSetMapper(new BeanWrapperFieldSetMapper<Person>() {
                {
                    setTargetType(Person.class);
                }
            });
      return lineMapper;
}

@Bean
public DelimitedLineTokenizer lineTokenizer() {
    DelimitedLineTokenizer tokenizer = new DelimitedLineTokenizer(DelimitedLineTokenizer.DELIMITER_TAB);
    tokenizer.setNames(new String[] { "firstName", "lastName" });
    return tokenizer;
}