[
https://issues.apache.org/jira/browse/CAMEL-20958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17872047#comment-17872047
]
Claus Ibsen commented on CAMEL-20958:
-------------------------------------
Okay so maybe we can detect the CLOSED and do a WARN or INFO log and saying
that its closed and that no new messages will be consumed, and then make the
consumer a noop mode.
You are welcome to attempt to implement this and send a PR against the main
branch
> camel-aws - Kinesis consumer loops on closed shards
> ---------------------------------------------------
>
> Key: CAMEL-20958
> URL: https://issues.apache.org/jira/browse/CAMEL-20958
> Project: Camel
> Issue Type: Bug
> Components: camel-aws2-kinesis
> Affects Versions: 4.6.0
> Environment: Camel 4.6.0
> Localstack 3.0.2
> Reproducer attached.
> Reporter: Mark Wimpory
> Priority: Major
> Fix For: 4.x
>
> Attachments: camel-reproducer.zip
>
>
> When shards are merged the consumer reads any remaining messages from those
> shards and then notes that the shard has closed. However it continuously
> loops as in the log exert below. This continues on a restart.
> Note this occurs no matter what type of failure option (silent, fail etc)
> I have not tested so far what happens when the shard reaches its expired
> state (due to a localstack bug)
>
> 2024-07-09 12:01:43 [ForkJoinPool.commonPool-worker-4] WARN
> o.a.c.c.a.k.Kinesis2Consumer - The shard with id=shardId-000000000000 on
> stream ingestion-death-stream reached CLOSE status
> 2024-07-09 12:01:43 [ForkJoinPool.commonPool-worker-3] WARN
> o.a.c.c.a.k.Kinesis2Consumer - The shard with id=shardId-000000000001 on
> stream ingestion-death-stream reached CLOSE status
> 2024-07-09 12:01:43 [ForkJoinPool.commonPool-worker-4] INFO
> o.a.c.c.a.k.c.KinesisDefaultResumeAdapter - Resuming from offset
> 49653759514412492265585802359002135113225509352150925314 for key
> ingestion-death-stream
> 2024-07-09 12:01:43 [ForkJoinPool.commonPool-worker-3] INFO
> o.a.c.c.a.k.c.KinesisDefaultResumeAdapter - Resuming from offset
> 49653759514434793010784332982141252979858928386588016658 for key
> ingestion-death-stream
> 2024-07-09 12:01:43 [ForkJoinPool.commonPool-worker-4] DEBUG
> o.a.c.c.a.k.Kinesis2Consumer - Obtained new ShardIterator
> AAAAAAAAAAFhGdwQ2Ac96sw4ScfsOEaTn+V+4vvB9RE9cnHLqD+s4rLMSSioedJLXVbHf47GfSV8ktk53sJmQXMFb+EZOOA/PaByEh66RTC7iBK+8BCZrrL5wb4wNHV7M1qtua1Cu7NrsqEZli3+HgagoG4V7bopK5xPcezzfDN9EWclJRCnlq3Z9nJRSmB4JCmIweWsX8VrM7SHnaVabQ3Kd8l9zoj7
> for shard shardId-000000000000 on stream ingestion-death-stream
> 2024-07-09 12:01:43 [ForkJoinPool.commonPool-worker-3] DEBUG
> o.a.c.c.a.k.Kinesis2Consumer - Obtained new ShardIterator
> AAAAAAAAAAEGg3GnGpu+nfbhufxJ7/V1iSjlD7G24d03xeQATtF9pV1dK21qLxxKpdS8g/CLPdV3MZaJaZoXkthM0J4xNvLOAkCwPpEsl9ympInzr/3QgNqFHqhGopNBlGKMyCdQUYi0UErBuUJ8dPKCq+IYRNVQvhSGg4fF4zXAlDi7jsXC7HZ2X9Xo3Q4XF/E7yEhi1YnFKAmMOmzstDkUtJd+U3MO
> for shard shardId-000000000001 on stream ingestion-death-stream
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
