After a chat that got deeper into this issue, the real "issue" is that the ItemReader is being created once instead of once per REST call (aka once per execution of the job). To fix this, make the ItemReader step scoped. This will allow you to get a new instance of the reader with each run which also enables the ability to inject the file name base on job parameters.

From a code perspective, change this:

@Bean
public ItemReader<Person> reader() {

to this:

@Bean
@StepScope
public FlatFileItemReader<Person> reader() {

The reason for the return type change is that Spring Batch will automatically register ItemStreams for you. However when using @StepScope we only see the return you define. In this case, ItemReader doesn't extend/implement ItemStream so we don't know you're returning something we should be auto-registering. By returning FlatFileItemReader we can see all the interfaces/etc and can apply the "magic" for you.

You have to build job with JobbuilderFactory:

    @Configuration
@EnableBatchProcessing
public class BatchConfiguration {

    @Autowired
    public JobBuilderFactory jobBuilderFactory;

    @Autowired
    public StepBuilderFactory stepBuilderFactory;

    @Bean
    public SomeReader<Some> reader() {

        // some reader configuration
        return reader;
    }

    @Bean
    public SomeProcessor processor() {
        return new SomeProcessor();
    }

    @Bean
    public SomeWriter<Person> writer() {
        // some config
        return writer;
    }

    @Bean
    public Job someJob() {
        return jobBuilderFactory.get("someJob")
                .flow(step1())
                .end()
                .build();
    }

    @Bean
    public Step step1() {
        return stepBuilderFactory.get("step1")
                .<Some, Some> chunk(10)
                .reader(reader())
                .processor(processor())
                .writer(writer())
                .build();
    }
    }

Start job in rest controller:

@RestController
@AllArgsConstructor
@Slf4j
public class BatchStartController {

    JobLauncher jobLauncher;

    Job job;

    @GetMapping("/job")
    public void startJob() {
    //some parameters
        Map<String, JobParameter> parameters = new HashMap<>();
        JobExecution jobExecution = jobLauncher.run(job, new JobParameters(parameters));
        }    }

And one important detail - add in application.properties:

spring.batch.job.enabled=false

to prevent job self start.

I did read about AsyncItemProcessor and AsyncItemWriter, but I am not sure if that is something I should use in this scenario.

Yes, AsyncItemProcessor and AsyncItemWriter are suitable for your use case. The AsyncItemProcessor will execute the logic (your rest call) of the delegate ItemProcessor for an item on a new thread. Once the item completes, the Future of the result is passed to the AsynchItemWriter to be written. The AsynchItemWriter will then unwrap the Future and write the item. The advantage of these components is that you don't have to deal with Futures wrapping, unwrapping, etc yourself.

You can find:

• More details here: https://docs.spring.io/spring-batch/4.0.x/reference/html/spring-batch-integration.html#asynchronous-processors
• An example here: https://github.com/mminella/scaling-demos/blob/master/single-jvm-demos/src/main/java/io/spring/batch/scalingdemos/asyncprocessor/AsyncProcessorJobApplication.java

Hope this helps.

You can do that using a Controller with a JobLauncher and Job. The barebones of the controller would be like this

@RestController
public class MyController{
    // Usually given by Spring Batch
    private JobLauncher jobLauncher;
    // Your Job
    private Job job;
    // Ctor
    public MyController(JobLauncher jobLauncher, Job job, ...){}

    @PostMapping("/")
    public String launchJob(...){
        ...
        // Create JobParameters and launch
        JobParameters jobparameters = new Job Parameters();
        jobLauncher.run(job, jobParameters);
        ...
    }
}

SimpleJobLauncher, the implementation of JobLauncher, uses a synchronous executor by default, you'll probably want to change it to an Async one depending of your requirements

It is possible to create your own custom ItemWriter.

In your case, please add the spring-boot-starter-web dependency to either your pom.xml or build.gradle

Example:

package com.example.batch;

import lombok.extern.log4j.Log4j2;
import org.springframework.batch.item.ItemWriter;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.http.ResponseEntity;
import org.springframework.web.client.RestTemplate;

import java.util.List;

@Log4j2
public class RestItemWriter implements ItemWriter<String> {
    @Autowired
    RestTemplate restTemplate;

    public RestItemWriter(RestTemplate restTemplate) {
        this.restTemplate = restTemplate;
    }

    @Override
    public void write(List<? extends String> items) throws Exception {
        ResponseEntity<Users> users = restTemplate.getForEntity("https://jsonplaceholder.typicode.com/users/1", Users.class);

        log.info("Status code is: " + users.getStatusCode());
    }
}


package com.example.batch;

import com.fasterxml.jackson.annotation.JsonInclude;
import lombok.Getter;
import lombok.Setter;

@Getter
@Setter
@JsonInclude(JsonInclude.Include.NON_NULL)
public class Users {

    public String id;
    public String name;
    public String username;
    public String email;
    public String phone;
}

More information about custom item writers here