Hi All,
We are using flink batch mode to process s3 files. However, recently we are
seeing the errors like:
Caused by: java.util.concurrent.CompletionException:
org.apache.flink.runtime.jobmanager.scheduler.NoResourceAvailableException:
Slot request bulk is not fulfillable! Could not allocate the required slot
within slot request timeout
at
java.base/java.util.concurrent.CompletableFuture.encodeThrowable(Unknown
Source)
at
java.base/java.util.concurrent.CompletableFuture.completeThrowable(Unknown
Source)
at
java.base/java.util.concurrent.CompletableFuture$UniApply.tryFire(Unknown
Source)
... 36 more
Caused by:
org.apache.flink.runtime.jobmanager.scheduler.NoResourceAvailableException:
Slot request bulk is not fulfillable! Could not allocate the required slot
within slot request timeout
at
org.apache.flink.runtime.jobmaster.slotpool.PhysicalSlotRequestBulkCheckerImpl.lambda$schedulePendingRequestBulkWithTimestampCheck$0(PhysicalSlotRequestBulkCheckerImpl.java:86)
... 29 more
Caused by: java.util.concurrent.TimeoutException: Timeout has occurred:
300000 ms
... 30 more
I can see the job manager & task manager still got memory to run jobs.
Anyone can help me on this?
Job manager: JVM: 5.02 GB / 6.00 GB
jobmanager.heap.size 6144m
jobmanager.memory.heap.size 6442450944b
jobmanager.memory.jvm-metaspace.size 4294967296b
jobmanager.memory.jvm-overhead.max 1073741824b
jobmanager.memory.jvm-overhead.min 1073741824b
jobmanager.memory.off-heap.size 134217728b
