I have a bit more information from the debug logs. For the partition that
isn't working (Partition 4), I see messages like this
from org.apache.samza.checkpoint.kafka.KafkaCheckpointManager:
Adding checkpoint Checkpoint [offsets={SystemStreamPartition [kafka,
com.stormpath.iam.indexing.document, 4]=448}] for taskName Partition 4
It's always 448. To pick another partition that seems to be working fine,
Partition 0, similar log messages come:
Adding checkpoint Checkpoint [offsets={SystemStreamPartition [kafka,
com.stormpath.iam.indexing.document, 0]=28445}] for taskName Partition 0
But the number is steadily increasing (not a fixed amount, but consistently
going up).
I hope this gives a clue.
On Mon, Apr 25, 2016 at 2:33 PM Jason Erickson <ja...@stormpath.com> wrote:
> I have a puzzling issue with one of my samza tasks. It works correctly
> except for messages on one partition. I have 9 partitions on the topic. If
> I send 1000 messages, I only receive about 890 of them. I have checked with
> kafka-console-consumer with partition keys that I know don’t get processed
> by my samza job and the console consumer DOES see the message, so I know
> it’s getting written to the topic and that at least a vanilla consumer can
> see it just fine.
>
>
> I’m happy to share whatever interesting configuration information would
> help narrow this down, but right now, I’m a bit mystified about what I
> would even share.
>
>
> I’m running
>
> * samza-kafka_2.10 version 0.9.1
>
> * kafka_2.10 version 0.8.2.1 on the client
>
> * kafka broker 0.9.0.0
>
>
