Yes, it does sound quite a lot like FLINK-10225. I assume it is only
happening for some task executors and not all of them?
Unfortunately I don't think this issue will be fixed anytime soon.
On 1/28/2021 12:59 PM, Daniel Peled wrote:
Hi,
We have followed the instructions in the following link ""Enabling
Queryable State" with kubernetes:
https://ci.apache.org/projects/flink/flink-docs-stable/deployment/resource-providers/standalone/kubernetes.html#enabling-queryable-state
<https://ci.apache.org/projects/flink/flink-docs-stable/deployment/resource-providers/standalone/kubernetes.html#enabling-queryable-state>
*When the replicas of the task-manager pods is 1 we get NO error
But when the replicas is greater than 1 for example 7 we get the
following error when trying to access flink state:
We think it might be related to jira issue *FLINK-10225
<https://issues.apache.org/jira/browse/FLINK-10225> *that has been
abandoned*
java.util.concurrent.ExecutionException: java.lang.RuntimeException:
Failed request 1.
Caused by:
org.apache.flink.queryablestate.exceptions.UnknownLocationException:
Could not retrieve location of state=stopJobValueState of
job=d5d14923157f5c3d3c4b2e1b7c02a942. Potential reasons are: i) the
state is not ready, or ii) the job does not exist.
Caused by:
org.apache.flink.queryablestate.exceptions.UnknownLocationException:
Could not retrieve location of state=stopJobValueState of
job=d5d14923157f5c3d3c4b2e1b7c02a942. Potential reasons are: i) the
state is not ready, or ii) the job does not exist.
at
org.apache.flink.queryablestate.client.proxy.KvStateClientProxyHandler.getKvStateLookupInfo(KvStateClientProxyHandler.java:247)
at
org.apache.flink.queryablestate.client.proxy.KvStateClientProxyHandler.getState(KvStateClientProxyHandler.java:164)
at
org.apache.flink.queryablestate.client.proxy.KvStateClientProxyHandler.executeActionAsync(KvStateClientProxyHandler.java:131)
at
org.apache.flink.queryablestate.client.proxy.KvStateClientProxyHandler.handleRequest(KvStateClientProxyHandler.java:121)
at
org.apache.flink.queryablestate.client.proxy.KvStateClientProxyHandler.handleRequest(KvStateClientProxyHandler.java:63)
at
org.apache.flink.queryablestate.network.AbstractServerHandler$AsyncRequestTask.run(AbstractServerHandler.java:258)
at
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
BR,
Danny
