you can implement StepExecutionListener at itemreader. Then you can get readcount which is corresponds your line number.

 public class ExampleItemReader implements ItemReader<String>, StepExecutionListener {

    public synchronized String read() throws Exception {
        return "";
    }

    @Override
    public ExitStatus afterStep(StepExecution executionContext) {
        if (executionContext.getReadCount() > 8000) {
            return ExitStatus.COMPLETED;
        }
        return ExitStatus.EXECUTING;
    }

    @Override
    public void beforeStep(StepExecution arg0) {

    }

}

advise reading spring batch patterns

Without seeing you ItemReader config I can't really be sure but if you are using something like FlatFileItemReader to parse a csv, if in strict mode it will validate the number of columns.

Assuming you reader looks like this, that is:

<bean id="iItemReader" class="org.springframework.batch.item.file.FlatFileItemReader" scope="step">
    <property name="linesToSkip" value="1"/>
    <property name="comments" value="#" />
    <property name="encoding" value="UTF-8"/>
    <property name="lineMapper" >
        <bean class="org.springframework.batch.item.file.mapping.DefaultLineMapper">
            <property name="lineTokenizer">
                <bean class="org.springframework.batch.item.file.transform.DelimitedLineTokenizer">
                    <property name="delimiter" value=","/>
                    <property name="names">
                        <list >
                            <value>First_Field</value>
                        <value>Second_Field</value>
                        </list>
                    </property>
                    <property name="strict" value="true"/>
                </bean>
            </property>
            <property name="fieldSetMapper">
                <bean class="uk.co.package.FieldSetMapper">
                    <property name="dateFormat" value="yyyy-MM-dd HH:mm:ss"/>
                </bean>
            </property>
        </bean>
    </property>

</bean>

It will throw a FlatFileParseException for any lines that can't be processed. This includes the line number and can be handled in a listener.

if you need to know how many lines where read, it's available in spring batch itself, take a look at the StepExecution

The method getReadCount() should give you the number you are looking for.

You need to add a step execution listener to your step in your xml configuration. To do that (copy/pasted from spring documentation):

<step id="step1">
    <tasklet>
        <chunk reader="reader" writer="writer" commit-interval="10"/>
        <listeners>
            <listener ref="chunkListener"/>
        </listeners>
   </tasklet>
</step>

where "chunkListner" is a bean of yours annotated with a method annotated with @AfterStep to tell spring batch to call it after your step.

you should take a look at the spring reference for step configuration

Hope that helps,

I realized that I can get the lineNumber value into MyObject by overriding the DefaultLineMapper with my own LineMapper in this way:

import org.springframework.batch.item.file.FlatFileParseException;
import org.springframework.batch.item.file.LineMapper;
import org.springframework.batch.item.file.transform.LineTokenizer;
import org.springframework.beans.factory.InitializingBean;
import org.springframework.util.Assert;

import my.model.MyObject;

public class MyLineMapper<T> implements LineMapper<MyObject>, InitializingBean {

    private LineTokenizer tokenizer;

    private ResourceFieldSetMapper fieldSetMapper;

    public MyObject mapLine(String line, int lineNumber) throws Exception {
        try{
            MyObject r = fieldSetMapper.mapFieldSet(tokenizer.tokenize(line));
            // this is the modification
            r.setLineNumber(lineNumber);
            return r;
        }
        catch(Exception ex){
            throw new FlatFileParseException("Parsing error at line: " + lineNumber + 
                    ", input=[" + line + "]", ex, line, lineNumber); 
        }
    }

    public void setLineTokenizer(LineTokenizer tokenizer) {
        this.tokenizer = tokenizer;
    }

    public void setFieldSetMapper(ResourceFieldSetMapper fieldSetMapper) {
        this.fieldSetMapper = fieldSetMapper;
    }

    public void afterPropertiesSet() {
        Assert.notNull(tokenizer, "The LineTokenizer must be set");
        Assert.notNull(fieldSetMapper, "The FieldSetMapper must be set");
    }

}

Thanks for your help! I hope this works for someone!

Blessings!

You could write your own SystemCommandTasklet. This tasklet is used to execute a system command. You could use the split command to split the files. Subsequently you can use partitioner to execute the processing of separate files.

More information about this tasklet Here

I want to count the number of lines in a file using batch

Specific solution

From a command line:

F:\test>for /f "usebackq" %b in (`type abc.csv ^| find "" /v /c`) do @echo line count is %b
line count is 1

From a batch file (countlines.cmd):

@echo off
Setlocal EnableDelayedExpansion
  for /f "usebackq" %%b in (`type abc.csv ^| find "" /v /c`) do (
    echo line count is %%b
    )
  )

Example:

F:\test>countlines
line count is 1
F:\test>

Flexible solution

Use the following batch file (countlines.cmd):

@echo off
Setlocal EnableDelayedExpansion
for /f "usebackq" %%a in (`dir /b /s %1`)  do (
  echo processing file %%a
  for /f "usebackq" %%b in (`type %%a ^| find "" /v /c`) do (
    echo line count is %%b
    set /a lines += %%b
    )
  )
echo total lines is %lines%

Notes:

• Total number of lines is stored in %lines%.
• Batch file supports wildcards.
• Tweak echo ... commands as appropriate.

Usage:

countlines filename_expression

Example:

F:\test>countlines *.csv
processing file F:\test\abc.csv
line count is 1
processing file F:\test\def.csv
line count is 1
total lines is 2

Further Reading

• An A-Z Index of the Windows CMD command line - An excellent reference for all things Windows cmd line related.
• dir - Display a list of files and subfolders.
• find - Search for a text string in a file & display all the lines where it is found.
• for /f - Loop command against the results of another command.

Using the JobExplorer, you can look at the previous StepExecutions to get the number of items read, etc in each step. You can read more about the JobExplorer here: http://docs.spring.io/spring-batch/trunk/apidocs/org/springframework/batch/core/explore/JobExplorer.html

Update

You actually don't even need to use the JobExplorer. Since you have the JobExecution, you already have references to all the StepExecutions. Each StepExecution contains the number of items read, processed, written, skipped, etc.