When you run larger workloads, Dataflow likely tries to split the work into
more splits and may be also autoscaling to add more workers. So the number
of parallel connections to the Database will go up if the workload is
higher. So probably try adjusting the Database settings to allow more
parallel
Hello Cham,
The Runner in use is Dataflow Runner.
The last 28 lines aren't available in Cloud logging as well.
The Code shared above works just fine with 2-3 records but starts to fail
when we try with a bigger source data payload.
Does it look like multiple threads trying to acquire a write lock
Which runner are you using ?
Also, do you have the bottom of the StackTrace here ? It's possibly due to
Docker containers running the Java SDK not having access to your database,
but I'm not sure based on the information provided.
Thanks,
Cham
On Tue, Feb 21, 2023 at 11:32 AM Somnath Chouwdhury
