; Yun
>
>
>
> ----------Original Mail --
> *Sender:*Chirag Dewan
> *Send Date:*Tue Jun 8 18:06:07 2021
> *Recipients:*User , Yun Gao
> *Subject:*Re: Multiple Exceptions during Load Test in State Access APIs
> with RocksDB
> Hi,
>
> Although
Hi Chirag,
Logically Integer type should not have this issue. Sorry that from the current
description I
have not found other issues, could you also share the code in the main method
that
adds the KeyProcessFunction into the job ? Very thanks!
Best,
Yun
---
Thanks for the reply Yun.
The key is an Integer type. Do you think there can be hash collisions for
Integers?
It somehow works on single TM now. No errors for 1m records.But as soon as we
move to 2 TMs, we get all sort of errors - 'Position Out of Bound', key not in
Keygroup etc.
This also caus
Hi Chirag,
As far as I know, If you are running a single job, I think all th pods share
the same
state.checkpoints.dir configuration should be as expected, and it is not
necessary
to configuraiton the rocksdb local dir since Flink will chosen a default dir.
Regarding the latest exception, I t
Hi,
Although this looks like a problem to me, I still cant conclude it.
I tried reducing my TM replicas from 2 to 1 with 4 slots and 4 cores each. I
was hoping that with single TM there will be file write conflicts. But that
doesn't seem to be the case as still get the:
Caused by: org.apache.fl
Hi,
I think I got my issue. Would help if someone can confirm it :)
I am using a NFS filesystem for storing my checkpoints and my Flink cluster is
running on a K8 with 2 TMs and 2 JMs.
All my pods share the NFS PVC with state.checkpoint.dir and we also missed
setting the RocksDB local dir.
Does
Thanks for the reply Yun. I strangely don't see any nulls. And infact this
exception comes on the first few records and then job starts processing
normally.
Also, I don't see any reason for Concurrent access to the state in my code.
Could more CPU cores than task slots to the Task Manager be th
Hi Chirag,
If be able to produce the exception, could you first add some logs to print
the value of valueState, valueState.value(), inEvent and
inEvent.getPriceDelta() ?
I think either object being null would cause NullPointerException here.
For the second exception, I found a similar issue[1],
