Re: Validating my understanding of SHARD_DISCOVERY_INTERVAL_MILLIS

2020-07-21 Thread Tzu-Li (Gordon) Tai
Hi Vijay, Your assumption is correct that the discovery interval does not affect the interval of fetching records. As a side note, you can actually disable shard discovery, by setting the value to -1. The FlinkKinesisProducer would then only call ListShards once at job startup. Cheers, Gordon O

Validating my understanding of SHARD_DISCOVERY_INTERVAL_MILLIS

2020-07-09 Thread Vijay Balakrishnan
Hi, I see these 2 constants- SHARD_GETRECORDS_INTERVAL_MILLIS & SHARD_DISCOVERY_INTERVAL_MILLIS. My understanding was SHARD_GETRECORDS_INTERVAL_MILLIS defines how often records are fetched from Kinesis Data Stream(KDS). Code seems to be doing this in ShardConsumer.run()-->getRecords() SHARD_DISCO