I am trying to switch to Flink 1.11 with the new EMR release 6.1. I have
created 3 nodes EMR cluster with Flink 1.11. When I am running my job its
working fine only issue is I am not able to see any logs in the job manager
and task manager. I am seeing below exception in stdout of job manager
09:21:36.761 [flink-akka.actor.default-dispatcher-17] ERROR
org.apache.flink.runtime.rest.handler.taskmanager.TaskManagerLogFileHandler
- Failed to transfer file from TaskExecutor
container_1599202660950_0009_01_000003.
java.util.concurrent.CompletionException: org.apache.flink.util.
FlinkException: The file LOG does not exist on the TaskExecutor.
at org.apache.flink.runtime.taskexecutor.TaskExecutor
.lambda$requestFileUploadByFilePath$25(TaskExecutor.java:1742)
~[flink-dist_2.12-1.11.0.jar:1.11.0]
at java.util.concurrent.CompletableFuture$AsyncSupply.run(
CompletableFuture.java:1604) ~[?:1.8.0_252]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor
.java:1149) ~[?:1.8.0_252]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor
.java:624) ~[?:1.8.0_252]
at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_252]
Caused by: org.apache.flink.util.FlinkException: The file LOG does not
exist on the TaskExecutor.
... 5 more
09:21:36.773 [flink-akka.actor.default-dispatcher-17] ERROR
org.apache.flink.runtime.rest.handler.taskmanager.TaskManagerLogFileHandler
- Unhandled exception.
org.apache.flink.util.FlinkException: The file LOG does not exist on the
TaskExecutor.
at org.apache.flink.runtime.taskexecutor.TaskExecutor
.lambda$requestFileUploadByFilePath$25(TaskExecutor.java:1742)
~[flink-dist_2.12-1.11.0.jar:1.11.0]
at java.util.concurrent.CompletableFuture$AsyncSupply.run(
CompletableFuture.java:1604) ~[?:1.8.0_252]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor
.java:1149) ~[?:1.8.0_252]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor
.java:624) ~[?:1.8.0_252]
at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_252]
09:21:57.399 [flink-akka.actor.default-dispatcher-15] ERROR
org.apache.flink.runtime.rest.handler.taskmanager.TaskManagerLogFileHandler
- Failed to transfer file from TaskExecutor
container_1599202660950_0009_01_000003.
TaskManagerLogFileHandler - Unhandled exception.
org.apache.flink.util.FlinkException
Is Any new setting I have to do in flink 1.11 as the same job is working
fine on the previous version of EMR with Flink 1.10
--
Thanks & Regards,
Anuj Jain
<http://www.cse.iitm.ac.in/%7Eanujjain/>
