java - Spring批处理在xml读取错误后不继续处理记录

Question

我配置了一个 Spring Batch，以便在读取 xml 文件时出现错误时跳过错误记录。为了跳过坏记录，skipPolicy 实现始终返回 true。该作业需要继续处理其余记录，但在我的例子中，它在坏记录完成后停止。

@Configuration
@Import(DataSourceConfig.class)
@EnableWebMvc
@ComponentScan(basePackages = "org.nova.batch")
@EnableBatchProcessing
public class BatchIssueConfiguration {
private static final Logger LOG =LoggerFactory.getLogger(BatchIssueConfiguration.class);
    @Autowired
    private JobBuilderFactory jobBuilderFactory;
    @Autowired
    private StepBuilderFactory stepBuilderFactory;

    @Bean(name = "jobRepository")
    public JobRepository jobRepository(DataSource dataSource, PlatformTransactionManager transactionManager) throws Exception {
        JobRepositoryFactoryBean factory = new JobRepositoryFactoryBean();
        factory.setDatabaseType("derby");
        factory.setDataSource(dataSource);
        factory.setTransactionManager(transactionManager);
        return factory.getObject();
    }
    @Bean
    public Step stepSGR() throws IOException{
        return stepBuilderFactory.get("ETL_STEP").<SigmodRecord.Issue,SigmodRecord.Issue>chunk(1)
                //.processor(itemProcessor())
                .writer(itemWriter())
                .reader(multiReader())
                .faultTolerant()
                .skipLimit(Integer.MAX_VALUE)
                .skipPolicy(new FileVerificationSkipper())
                .skip(Throwable.class)
                .build();
    }

    @Bean
    public SkipPolicy   fileVerificationSkipper(){
        return new FileVerificationSkipper();
    }


    @Bean
    @JobScope
    public MultiResourceItemReader<SigmodRecord.Issue> multiReader() throws IOException{
        MultiResourceItemReader<SigmodRecord.Issue> mrir = new MultiResourceItemReader<SigmodRecord.Issue>();
        //FileSystemResource [] files = new FileSystemResource [{}];
        ResourcePatternResolver rpr = new PathMatchingResourcePatternResolver();
        Resource[] resources = rpr.getResources("file:c:/temp/Sigm*.xml");
        mrir.setResources( resources);
        mrir.setDelegate(xmlItemReader());
        return mrir;
    }
}

public class FileVerificationSkipper implements SkipPolicy {

    private static final Logger LOG = LoggerFactory.getLogger(FileVerificationSkipper.class);

    @Override
    public boolean shouldSkip(Throwable t, int skipCount) throws SkipLimitExceededException {
        LOG.error("There is an error {}",t);
        return true;
    }

}

该文件的输入包含“&”，这会导致读取错误，即

<title>Notes of DDTS & n Apparatus for Experimental Research</title>

抛出以下错误:

org.springframework.dao.DataAccessResourceFailureException: Error reading XML stream; nested exception is javax.xml.stream.XMLStreamException: ParseError at [row,col]:[127,25]
Message: The entity name must immediately follow the '&' in the entity reference.

我的配置中是否存在任何错误，不允许其余记录继续处理。

Answer 1

要跳过某些类型的异常，我们可以提及跳过策略，我们可以在其中编写用于跳过异常的自定义逻辑。就像下面的代码一样。

        @Bean
            public Step stepSGR() throws IOException{
                return stepBuilderFactory.get("ETL_STEP").<SigmodRecord.Issue,SigmodRecord.Issue>chunk(1)
                        //.processor(itemProcessor())
                        .writer(itemWriter())
                        .reader(multiReader())
                        .faultTolerant()
                        .skipPolicy(new FileVerificationSkipper())
                        .build();
            }

        public class FileVerificationSkipper implements SkipPolicy {

        private static final Logger LOG = LoggerFactory.getLogger(FileVerificationSkipper.class);

        @Override
        public boolean shouldSkip(Throwable t, int skipCount) throws SkipLimitExceededException {
            LOG.error("There is an error {}",t);
            if (t instanceof DataAccessResourceFailureException)          
              return true;
        }

    }

或者您可以像下面这样简单地设置。

     @Bean
     public Step stepSGR() throws IOException{
       return stepBuilderFactory.get("ETL_STEP").<SigmodRecord.Issue,SigmodRecord.Issue>chunk(1)
                        //.processor(itemProcessor())
                        .writer(itemWriter())
                        .reader(multiReader())
                        .faultTolerant()
                        .skipLimit(Integer.MAX_VALUE)
                        .skip(DataAccessResourceFailureException.class)
                        .build();
            }