Utilizing Parallelism and Concurrency in Spring Batch for maximum performance

[jgoldverg]

Hello all,

I have a couple questions about parallel steps that I have been unable to find googling. My current set up is I have written my own HTTP reader that uses the java 11 HttpClient to download files and then my FileSystem writer that uses fileChannels to insert chunks into the file in proper order. Both have been fully tested and work but what I am currently trying to do is maximize performance which lead me to the following questions.

1. When using ThreadPoolTaskExecutor with a Step why does each thread require a Hikari pool connection? To me this implies that each thread will be doing some form of transaction on the JobRepository. An explanation or a link to the code that would explain why this is necessary for the framework and if there is anyway for users of Spring-Batch to define such behavior.

2. Should steps share a ThreadPoolTaskExecutor or should each step have its own ThreadPoolTaskExecutor?
   My current approach has been to run a large amount of steps concurrently(16+) where each step downloads 1 file and then each step has a thread pool to download a commit-interval amount of chunks of the file per-thread. I really love this mapping as its extremely simple to implement and my readers & writer are 100% lock free.

3. With Project Loom coming out in the next LTS will Spring Batch be able to fully utilize the new kind of threads being released and what kind of benefits can we hope to expect?

Any advice on how to maximize the gains from parallelism and concurrency would be great. I can share my performance results and source code if needed.

[fmbenhassine]

You asked several questions, I will add a comment per question so we can discuss each point in a thread if needed.

My current set up is I have written my own HTTP reader that uses the java 11 HttpClient to download files and then my FileSystem writer that uses fileChannels to insert chunks into the file in proper order

This setup is not clear to me. Please share your code.

Any advice on how to maximize the gains from parallelism and concurrency would be great.

As mentioned above, I can't understand your setup, but if the input is a set of files that can be processed independently, I see at least two options to maximize the throughput:

• Use different steps in parallel, where each step processes a file concurrently (ie each step is a multi-threaded step)
• Use a partitioned step, where each worker processes a file concurrently (ie each worker is a multi-threaded)

If that is not enough, you can go remote with parallel job instances, one job instance per file that you scale on different machines.

[jgoldverg]

For specific Reader and Writer impl here are two example links that demonstrate what I am trying to do :
HttpReader
FileSystemWriter

For job definition

The flow is that each step reads and writes a single file from Location A to Location B. In the current example my file source is an Http Server(Nginx for ex) and the destination is the local file system.

My current implementation does two things to increase throughput:

1. Multi-threaded steps which means that every step has a ThreadPoolTaskExecutor assigned to it enabling the use of many threads to read in a single file in independent chunks.
2. Parallel Steps, in this case we are using a ThreadPoolTaskExecutor to run many steps(file transfers) independently and concurrently.

[fmbenhassine]

1. Multi-threaded steps which means that every step has a ThreadPoolTaskExecutor assigned to it enabling the use of many threads to read in a single file in independent chunks.

Getting bytes out of disc is typically fast (especially with modern hardware), so I would not bother reading data from a single file using multiple threads. But if that works for you, then go for it.

2. Parallel Steps, in this case we are using a ThreadPoolTaskExecutor to run many steps(file transfers) independently and concurrently.

That's a good idea 👍 Using parallel steps is the way to go to run independent tasks concurrently.

[fmbenhassine]

1. When using ThreadPoolTaskExecutor with a Step why does each thread require a Hikari pool connection?

How did you come to this conclusion?

[jgoldverg]

I noticed an interesting behaviour when my Hikari pool's size was less than the number of actively running steps. Steps would be blocking for a new connection from the DB pool which I found to be limiting performance. In response to this I decided to size the pool appropriately to always guarantee each step thread has a Hikari pool connection to use but this is not always feasible as the number of parallel steps can change.

The way i noticed this behaviour was by using JProfiler.

[fmbenhassine]

Yes, you need to make sure you have enough db connections from the pool, this is mentioned in the docs here for multi-threaded steps, but the same idea applies when having parallel steps as well:

Note also that there may be limits placed on concurrency by any pooled resources used in your step, such as a DataSource. Be sure to make the pool in those resources at least as large as the desired number of concurrent threads in the step.

[fmbenhassine]

2. Should steps share a ThreadPoolTaskExecutor or should each step have its own ThreadPoolTaskExecutor?

Threads and executors are expensive to create, so they should be limited. A rightly-sized thread pool can be shared between multiple steps

[fmbenhassine]

3. With Project Loom coming out in the next LTS will Spring Batch be able to fully utilize the new kind of threads being released and what kind of benefits can we hope to expect?

Yes! We are planning to redesign the concurrency model to leverage virtual threads. We are going to open a discussion on this topic. Stay tuned!

[jgoldverg]

Wonderful I have been thoroughly excited to see project loom integrated into Spring and Spring Batch :).

[fmbenhassine]

I believe (and hope!) I answered all your questions. Please let me know if you need further details on the matter. Thank you.