Hi Piotrek,
Sorry for late response.
I have another problem with setting logs. I think the logging issue comes
from using Flink on my host machine and running a job on a jobmanager in
K8s. I'm managing the issue. But, this is what I got in /log folder of my
host machine:
2020-12-14 15:04:51,329 INFO org.apache.flink.client.cli.CliFrontend
[] -
--------------------------------------------------------------------------------
2020-12-14 15:04:51,331 INFO org.apache.flink.client.cli.CliFrontend
[] - Starting Command Line Client (Version: 1.12.0, Scala: 2.11,
Rev:fc00492, Date:2020-12-02T08:49:16+01:00)
2020-12-14 15:04:51,331 INFO org.apache.flink.client.cli.CliFrontend
[] - OS current user: folani
2020-12-14 15:04:51,332 INFO org.apache.flink.client.cli.CliFrontend
[] - Current Hadoop/Kerberos user: <no hadoop dependency found>
2020-12-14 15:04:51,332 INFO org.apache.flink.client.cli.CliFrontend
[] - JVM: OpenJDK 64-Bit Server VM - Oracle Corporation - 1.8/25.212-b04
2020-12-14 15:04:51,332 INFO org.apache.flink.client.cli.CliFrontend
[] - Maximum heap size: 3538 MiBytes
2020-12-14 15:04:51,332 INFO org.apache.flink.client.cli.CliFrontend
[] - JAVA_HOME: (not set)
2020-12-14 15:04:51,332 INFO org.apache.flink.client.cli.CliFrontend
[] - No Hadoop Dependency available
2020-12-14 15:04:51,332 INFO org.apache.flink.client.cli.CliFrontend
[] - JVM Options:
2020-12-14 15:04:51,332 INFO org.apache.flink.client.cli.CliFrontend
[] -
-Dlog.file=/home/folani/Softwares/flink-1.12.0-bin-scala_2.11/flink-1.12.0/log/flink-folani-client-hralaptop.log
2020-12-14 15:04:51,332 INFO org.apache.flink.client.cli.CliFrontend
[] -
-Dlog4j.configuration=file:/home/folani/Softwares/flink-1.12.0-bin-scala_2.11/flink-1.12.0/conf/log4j-cli.properties
2020-12-14 15:04:51,332 INFO org.apache.flink.client.cli.CliFrontend
[] -
-Dlog4j.configurationFile=file:/home/folani/Softwares/flink-1.12.0-bin-scala_2.11/flink-1.12.0/conf/log4j-cli.properties
2020-12-14 15:04:51,332 INFO org.apache.flink.client.cli.CliFrontend
[] -
-Dlogback.configurationFile=file:/home/folani/Softwares/flink-1.12.0-bin-scala_2.11/flink-1.12.0/conf/logback.xml
2020-12-14 15:04:51,332 INFO org.apache.flink.client.cli.CliFrontend
[] - Program Arguments:
2020-12-14 15:04:51,333 INFO org.apache.flink.client.cli.CliFrontend
[] - stop
2020-12-14 15:04:51,334 INFO org.apache.flink.client.cli.CliFrontend
[] - -p
2020-12-14 15:04:51,334 INFO org.apache.flink.client.cli.CliFrontend
[] - cc4a9a04e164fff8628b3ac59e5fbe80
2020-12-14 15:04:51,334 INFO org.apache.flink.client.cli.CliFrontend
[] - Classpath:
/home/folani/Softwares/flink-1.12.0-bin-scala_2.11/flink-1.12.0/lib/flink-csv-1.12.0.jar:/home/folani/Softwares/flink-1.12.0-bin-scala_2.11/flink-1.12.0/lib/flink-json-1.12.0.jar:/home/folani/Softwares/flink-1.12.0-bin-scala_2.11/flink-1.12.0/lib/flink-shaded-zookeeper-3.4.14.jar:/home/folani/Softwares/flink-1.12.0-bin-scala_2.11/flink-1.12.0/lib/flink-table_2.11-1.12.0.jar:/home/folani/Softwares/flink-1.12.0-bin-scala_2.11/flink-1.12.0/lib/flink-table-blink_2.11-1.12.0.jar:/home/folani/Softwares/flink-1.12.0-bin-scala_2.11/flink-1.12.0/lib/log4j-1.2-api-2.12.1.jar:/home/folani/Softwares/flink-1.12.0-bin-scala_2.11/flink-1.12.0/lib/log4j-api-2.12.1.jar:/home/folani/Softwares/flink-1.12.0-bin-scala_2.11/flink-1.12.0/lib/log4j-core-2.12.1.jar:/home/folani/Softwares/flink-1.12.0-bin-scala_2.11/flink-1.12.0/lib/log4j-slf4j-impl-2.12.1.jar:/home/folani/Softwares/flink-1.12.0-bin-scala_2.11/flink-1.12.0/lib/flink-dist_2.11-1.12.0.jar:::
2020-12-14 15:04:51,334 INFO org.apache.flink.client.cli.CliFrontend
[] -
--------------------------------------------------------------------------------
2020-12-14 15:04:51,336 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: jobmanager.rpc.address, localhost
2020-12-14 15:04:51,336 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: jobmanager.rpc.port, 6123
2020-12-14 15:04:51,337 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: jobmanager.memory.process.size, 1024m
2020-12-14 15:04:51,337 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: taskmanager.memory.process.size, 1024m
2020-12-14 15:04:51,337 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: taskmanager.numberOfTaskSlots, 1
2020-12-14 15:04:51,337 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: parallelism.default, 1
2020-12-14 15:04:51,337 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: state.backend, filesystem
2020-12-14 15:04:51,337 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: state.checkpoints.dir, s3://state/checkpoints
2020-12-14 15:04:51,337 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: state.savepoints.dir, s3://state/savepoints
2020-12-14 15:04:51,337 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: s3.path-style-access, true
2020-12-14 15:04:51,337 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: s3.endpoint, http://172.17.0.3:30090
2020-12-14 15:04:51,338 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: s3.access-key, minio
2020-12-14 15:04:51,338 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: s3.secret-key, ******
2020-12-14 15:04:51,338 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: jobmanager.execution.failover-strategy, region
2020-12-14 15:04:51,338 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: metrics.reporter.prom.class,
org.apache.flink.metrics.prometheus.PrometheusReporter
2020-12-14 15:04:51,338 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: metrics.reporter.prom.host, localhost
2020-12-14 15:04:51,338 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: metrics.reporter.prom.port, 9250-9260
2020-12-14 15:04:51,339 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: taskmanager.network.detailed-metrics, true
2020-12-14 15:04:51,339 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: metrics.latency.granularity, "subtask"
2020-12-14 15:04:51,339 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: metrics.latency.interval, 1000
2020-12-14 15:04:51,339 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: web.backpressure.refresh-interval, 1000
2020-12-14 15:04:51,339 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: metrics.system-resource, true
2020-12-14 15:04:51,339 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: metrics.system-resource-probing-interval, 1000
2020-12-14 15:04:51,358 INFO org.apache.flink.client.cli.CliFrontend
[] - Loading FallbackYarnSessionCli
2020-12-14 15:04:51,392 INFO org.apache.flink.core.fs.FileSystem
[] - Hadoop is not in the classpath/dependencies. The extended set of
supported File Systems via Hadoop is not available.
2020-12-14 15:04:51,449 INFO
org.apache.flink.runtime.security.modules.HadoopModuleFactory [] - Cannot
create Hadoop Security Module because Hadoop cannot be found in the
Classpath.
2020-12-14 15:04:51,454 INFO
org.apache.flink.runtime.security.modules.JaasModule [] - Jaas file
will be created as /tmp/jaas-1159138828547882777.conf.
2020-12-14 15:04:51,458 INFO
org.apache.flink.runtime.security.contexts.HadoopSecurityContextFactory [] -
Cannot install HadoopSecurityContext because Hadoop cannot be found in the
Classpath.
2020-12-14 15:04:51,459 INFO org.apache.flink.client.cli.CliFrontend
[] - Running 'stop-with-savepoint' command.
2020-12-14 15:04:51,466 INFO org.apache.flink.client.cli.CliFrontend
[] - Suspending job "cc4a9a04e164fff8628b3ac59e5fbe80" with a savepoint.
2020-12-14 15:04:51,470 INFO
org.apache.flink.client.deployment.DefaultClusterClientServiceLoader [] -
Could not load factory due to missing dependencies.
2020-12-14 15:04:51,485 INFO org.apache.flink.configuration.Configuration
[] - Config uses fallback configuration key 'jobmanager.rpc.address' instead
of key 'rest.address'
2020-12-14 15:05:51,973 WARN
org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannel [] -
Force-closing a channel whose registration task was not accepted by an event
loop: [id: 0xe1d20630]
java.util.concurrent.RejectedExecutionException: event executor terminated
at
org.apache.flink.shaded.netty4.io.netty.util.concurrent.SingleThreadEventExecutor.reject(SingleThreadEventExecutor.java:926)
~[flink-dist_2.11-1.12.0.jar:1.12.0]
at
org.apache.flink.shaded.netty4.io.netty.util.concurrent.SingleThreadEventExecutor.offerTask(SingleThreadEventExecutor.java:353)
~[flink-dist_2.11-1.12.0.jar:1.12.0]
at
org.apache.flink.shaded.netty4.io.netty.util.concurrent.SingleThreadEventExecutor.addTask(SingleThreadEventExecutor.java:346)
~[flink-dist_2.11-1.12.0.jar:1.12.0]
at
org.apache.flink.shaded.netty4.io.netty.util.concurrent.SingleThreadEventExecutor.execute(SingleThreadEventExecutor.java:828)
~[flink-dist_2.11-1.12.0.jar:1.12.0]
at
org.apache.flink.shaded.netty4.io.netty.util.concurrent.SingleThreadEventExecutor.execute(SingleThreadEventExecutor.java:818)
~[flink-dist_2.11-1.12.0.jar:1.12.0]
at
org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannel$AbstractUnsafe.register(AbstractChannel.java:471)
~[flink-dist_2.11-1.12.0.jar:1.12.0]
at
org.apache.flink.shaded.netty4.io.netty.channel.SingleThreadEventLoop.register(SingleThreadEventLoop.java:87)
~[flink-dist_2.11-1.12.0.jar:1.12.0]
at
org.apache.flink.shaded.netty4.io.netty.channel.SingleThreadEventLoop.register(SingleThreadEventLoop.java:81)
~[flink-dist_2.11-1.12.0.jar:1.12.0]
at
org.apache.flink.shaded.netty4.io.netty.channel.MultithreadEventLoopGroup.register(MultithreadEventLoopGroup.java:86)
~[flink-dist_2.11-1.12.0.jar:1.12.0]
at
org.apache.flink.shaded.netty4.io.netty.bootstrap.AbstractBootstrap.initAndRegister(AbstractBootstrap.java:323)
~[flink-dist_2.11-1.12.0.jar:1.12.0]
at
org.apache.flink.shaded.netty4.io.netty.bootstrap.Bootstrap.doResolveAndConnect(Bootstrap.java:155)
~[flink-dist_2.11-1.12.0.jar:1.12.0]
at
org.apache.flink.shaded.netty4.io.netty.bootstrap.Bootstrap.connect(Bootstrap.java:139)
~[flink-dist_2.11-1.12.0.jar:1.12.0]
at
org.apache.flink.shaded.netty4.io.netty.bootstrap.Bootstrap.connect(Bootstrap.java:123)
~[flink-dist_2.11-1.12.0.jar:1.12.0]
at
org.apache.flink.runtime.rest.RestClient.submitRequest(RestClient.java:333)
~[flink-dist_2.11-1.12.0.jar:1.12.0]
at
org.apache.flink.runtime.rest.RestClient.sendRequest(RestClient.java:272)
~[flink-dist_2.11-1.12.0.jar:1.12.0]
at
org.apache.flink.runtime.rest.RestClient.sendRequest(RestClient.java:214)
~[flink-dist_2.11-1.12.0.jar:1.12.0]
at
org.apache.flink.client.program.rest.RestClusterClient.lambda$null$23(RestClusterClient.java:666)
~[flink-dist_2.11-1.12.0.jar:1.12.0]
at
java.util.concurrent.CompletableFuture.uniCompose(CompletableFuture.java:952)
[?:1.8.0_212]
at
java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:926)
[?:1.8.0_212]
at
java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
[?:1.8.0_212]
at
java.util.concurrent.CompletableFuture.postFire(CompletableFuture.java:561)
[?:1.8.0_212]
at
java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:580)
[?:1.8.0_212]
at
java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:442)
[?:1.8.0_212]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[?:1.8.0_212]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[?:1.8.0_212]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_212]
2020-12-14 15:05:51,981 ERROR
org.apache.flink.shaded.netty4.io.netty.util.concurrent.DefaultPromise.rejectedExecution
[] - Failed to submit a listener notification task. Event loop shut down?
java.util.concurrent.RejectedExecutionException: event executor terminated
at
org.apache.flink.shaded.netty4.io.netty.util.concurrent.SingleThreadEventExecutor.reject(SingleThreadEventExecutor.java:926)
~[flink-dist_2.11-1.12.0.jar:1.12.0]
at
org.apache.flink.shaded.netty4.io.netty.util.concurrent.SingleThreadEventExecutor.offerTask(SingleThreadEventExecutor.java:353)
~[flink-dist_2.11-1.12.0.jar:1.12.0]
at
org.apache.flink.shaded.netty4.io.netty.util.concurrent.SingleThreadEventExecutor.addTask(SingleThreadEventExecutor.java:346)
~[flink-dist_2.11-1.12.0.jar:1.12.0]
at
org.apache.flink.shaded.netty4.io.netty.util.concurrent.SingleThreadEventExecutor.execute(SingleThreadEventExecutor.java:828)
~[flink-dist_2.11-1.12.0.jar:1.12.0]
at
org.apache.flink.shaded.netty4.io.netty.util.concurrent.SingleThreadEventExecutor.execute(SingleThreadEventExecutor.java:818)
~[flink-dist_2.11-1.12.0.jar:1.12.0]
at
org.apache.flink.shaded.netty4.io.netty.util.concurrent.DefaultPromise.safeExecute(DefaultPromise.java:841)
~[flink-dist_2.11-1.12.0.jar:1.12.0]
at
org.apache.flink.shaded.netty4.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:498)
~[flink-dist_2.11-1.12.0.jar:1.12.0]
at
org.apache.flink.shaded.netty4.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:183)
~[flink-dist_2.11-1.12.0.jar:1.12.0]
at
org.apache.flink.shaded.netty4.io.netty.channel.DefaultChannelPromise.addListener(DefaultChannelPromise.java:95)
~[flink-dist_2.11-1.12.0.jar:1.12.0]
at
org.apache.flink.shaded.netty4.io.netty.channel.DefaultChannelPromise.addListener(DefaultChannelPromise.java:30)
~[flink-dist_2.11-1.12.0.jar:1.12.0]
at
org.apache.flink.runtime.rest.RestClient.submitRequest(RestClient.java:337)
~[flink-dist_2.11-1.12.0.jar:1.12.0]
at
org.apache.flink.runtime.rest.RestClient.sendRequest(RestClient.java:272)
~[flink-dist_2.11-1.12.0.jar:1.12.0]
at
org.apache.flink.runtime.rest.RestClient.sendRequest(RestClient.java:214)
~[flink-dist_2.11-1.12.0.jar:1.12.0]
at
org.apache.flink.client.program.rest.RestClusterClient.lambda$null$23(RestClusterClient.java:666)
~[flink-dist_2.11-1.12.0.jar:1.12.0]
at
java.util.concurrent.CompletableFuture.uniCompose(CompletableFuture.java:952)
[?:1.8.0_212]
at
java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:926)
[?:1.8.0_212]
at
java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
[?:1.8.0_212]
at
java.util.concurrent.CompletableFuture.postFire(CompletableFuture.java:561)
[?:1.8.0_212]
at
java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:580)
[?:1.8.0_212]
at
java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:442)
[?:1.8.0_212]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[?:1.8.0_212]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[?:1.8.0_212]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_212]
2020-12-14 15:05:51,981 ERROR org.apache.flink.client.cli.CliFrontend
[] - Error while running the command.
org.apache.flink.util.FlinkException: Could not stop with a savepoint job
"cc4a9a04e164fff8628b3ac59e5fbe80".
at
org.apache.flink.client.cli.CliFrontend.lambda$stop$5(CliFrontend.java:539)
~[flink-dist_2.11-1.12.0.jar:1.12.0]
at
org.apache.flink.client.cli.CliFrontend.runClusterAction(CliFrontend.java:919)
~[flink-dist_2.11-1.12.0.jar:1.12.0]
at org.apache.flink.client.cli.CliFrontend.stop(CliFrontend.java:531)
~[flink-dist_2.11-1.12.0.jar:1.12.0]
at
org.apache.flink.client.cli.CliFrontend.parseAndRun(CliFrontend.java:986)
~[flink-dist_2.11-1.12.0.jar:1.12.0]
at
org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:1047)
~[flink-dist_2.11-1.12.0.jar:1.12.0]
at
org.apache.flink.runtime.security.contexts.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:30)
[flink-dist_2.11-1.12.0.jar:1.12.0]
at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1047)
[flink-dist_2.11-1.12.0.jar:1.12.0]
Caused by: java.util.concurrent.TimeoutException
at
java.util.concurrent.CompletableFuture.timedGet(CompletableFuture.java:1771)
~[?:1.8.0_212]
at
java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1915)
~[?:1.8.0_212]
at
org.apache.flink.client.cli.CliFrontend.lambda$stop$5(CliFrontend.java:537)
~[flink-dist_2.11-1.12.0.jar:1.12.0]
... 6 more
Thank you in advance.
Folani
--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
