[
https://issues.apache.org/jira/browse/CAMEL-20958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17872038#comment-17872038
]
Mark Wimpory commented on CAMEL-20958:
--------------------------------------
Mmmm yes I can see what you mean, but when a shard is closed no new messages
can be written to it. So when you reach the end of the shard there is no need
to continue asking for a new subscription. So I would say on first start its
fine, because we dont know we are at the end of the shard, and in fact the
subscriber may want to read from the shard. But once that finishes we can stop
until the next restart.
> camel-aws - Kinesis consumer loops on closed shards
> ---------------------------------------------------
>
> Key: CAMEL-20958
> URL: https://issues.apache.org/jira/browse/CAMEL-20958
> Project: Camel
> Issue Type: Bug
> Components: camel-aws2-kinesis
> Affects Versions: 4.6.0
> Environment: Camel 4.6.0
> Localstack 3.0.2
> Reproducer attached.
> Reporter: Mark Wimpory
> Priority: Major
> Fix For: 4.x
>
> Attachments: camel-reproducer.zip
>
>
> When shards are merged the consumer reads any remaining messages from those
> shards and then notes that the shard has closed. However it continuously
> loops as in the log exert below. This continues on a restart.
> Note this occurs no matter what type of failure option (silent, fail etc)
> I have not tested so far what happens when the shard reaches its expired
> state (due to a localstack bug)
>
> 2024-07-09 12:01:43 [ForkJoinPool.commonPool-worker-4] WARN
> o.a.c.c.a.k.Kinesis2Consumer - The shard with id=shardId-000000000000 on
> stream ingestion-death-stream reached CLOSE status
> 2024-07-09 12:01:43 [ForkJoinPool.commonPool-worker-3] WARN
> o.a.c.c.a.k.Kinesis2Consumer - The shard with id=shardId-000000000001 on
> stream ingestion-death-stream reached CLOSE status
> 2024-07-09 12:01:43 [ForkJoinPool.commonPool-worker-4] INFO
> o.a.c.c.a.k.c.KinesisDefaultResumeAdapter - Resuming from offset
> 49653759514412492265585802359002135113225509352150925314 for key
> ingestion-death-stream
> 2024-07-09 12:01:43 [ForkJoinPool.commonPool-worker-3] INFO
> o.a.c.c.a.k.c.KinesisDefaultResumeAdapter - Resuming from offset
> 49653759514434793010784332982141252979858928386588016658 for key
> ingestion-death-stream
> 2024-07-09 12:01:43 [ForkJoinPool.commonPool-worker-4] DEBUG
> o.a.c.c.a.k.Kinesis2Consumer - Obtained new ShardIterator
> AAAAAAAAAAFhGdwQ2Ac96sw4ScfsOEaTn+V+4vvB9RE9cnHLqD+s4rLMSSioedJLXVbHf47GfSV8ktk53sJmQXMFb+EZOOA/PaByEh66RTC7iBK+8BCZrrL5wb4wNHV7M1qtua1Cu7NrsqEZli3+HgagoG4V7bopK5xPcezzfDN9EWclJRCnlq3Z9nJRSmB4JCmIweWsX8VrM7SHnaVabQ3Kd8l9zoj7
> for shard shardId-000000000000 on stream ingestion-death-stream
> 2024-07-09 12:01:43 [ForkJoinPool.commonPool-worker-3] DEBUG
> o.a.c.c.a.k.Kinesis2Consumer - Obtained new ShardIterator
> AAAAAAAAAAEGg3GnGpu+nfbhufxJ7/V1iSjlD7G24d03xeQATtF9pV1dK21qLxxKpdS8g/CLPdV3MZaJaZoXkthM0J4xNvLOAkCwPpEsl9ympInzr/3QgNqFHqhGopNBlGKMyCdQUYi0UErBuUJ8dPKCq+IYRNVQvhSGg4fF4zXAlDi7jsXC7HZ2X9Xo3Q4XF/E7yEhi1YnFKAmMOmzstDkUtJd+U3MO
> for shard shardId-000000000001 on stream ingestion-death-stream
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
