Thanks Danny, Let me comeback with results.
> > On Feb 23, 2022, at 3:41 AM, Danny Cranmer <dannycran...@apache.org> wrote: > > > Hello Vijay, > > > Once i do that my flink consumer need to be restarted with changed > > parallelism. > Why is this? The Flink consumer continuously scans for new shards, and will > auto scale up/down the number of shard consumer threads to accommodate > Kinesis resharding. Flink job/operator parallelism does not need to match the > number of Kinesis shards. > > > How do i solve this problem of restarting from existing checkpoint which > > was created with respect to N shards but now while restating we have more > > or less shards? > The consumer should handle this. When the source starts it discovers active > shards on the stream and attaches available state on a per shard basis. What > is the problem here, is there an Exception you can share? > > Thanks, > > >> On Wed, Feb 23, 2022 at 11:25 AM Vijayendra Yadav <contact....@gmail.com> >> wrote: >> Hi Team, >> >> I am running flink 1.11 kinesis consumer with say N kinesis shards, but i >> want to increase/decrease shards to N+M or N-M. >> Once i do that my flink consumer need to be restarted with changed >> parallelism. >> But i am unable to restart from existing checkpoint because of change in >> number of shards. >> >> How do i solve this problem of restarting from existing checkpoint which was >> created with respect to N shards but now while restating we have more or >> less shards? >> >> Thanks, >> Vijay
