Hi Gordon,
Thanks for the reply.
So is it true to say that the KPL RateLimit would not get enforced when the
sink parallelism is >1? If multiple subtasks are writing to the same shard
and each has their own RateLimit, it is possible that the RateLimit is
crossed.
If that's the case, can you sugg
Hi all,
I think Steve's occurrence of the warning was from the consumer side.
For the Flink Kinesis Consumer, this could most likely occur due to
excessive ListShard API calls on the target Kinesis stream. The consumer
uses this API to discover shards, at a fixed interval.
The problem with the cu
Hi Steve,
We've encountered this also. We have way more than enough shards, but were
still getting exceptions.
We think we know what is the reason, we would love for someone to
approve/reject.
What we suspect is happening is as follows:
The KPL's RateLimit parameter is tracking the amount of byt
If it's running in parallel aren't you just adding readers which maxes out
your provisioned throughput? probably doesn't belong in here but rather a
Kinesis thing, but i suggest increasing your number of shards?
--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
I am getting this error from the Flink Kinesis Connector. I have a native
KCL app running in parallel with no problems.
Any help would be appreciated
Thanks so much!!
Steve
flink-sbistl919-taskexecutor-0-CACSVML-15736.log:2018-11-09 07:46:11,579
WARN org.apache.flink.streaming.connectors.k
