Re: Samza task not receiving on one partition

2016-05-01 Thread Jason Erickson
Will do. On Fri, Apr 29, 2016 at 5:31 PM Yi Pan wrote: > Hi, Jason, > > Sorry to hear that it caused you a long time to figure this out. Could you > open a JIRA on this issue s.t. we can do some further investigation to see > whether there is explicit indication from Kafka consumer on this proble

Re: Samza task not receiving on one partition

2016-04-29 Thread Yi Pan
Hi, Jason, Sorry to hear that it caused you a long time to figure this out. Could you open a JIRA on this issue s.t. we can do some further investigation to see whether there is explicit indication from Kafka consumer on this problem and we can report correctly? -Yi On Fri, Apr 29, 2016 at 2:45

Re: Samza task not receiving on one partition

2016-04-29 Thread Jason Erickson
The issue was that there was a message that exceeded the default max.message.bytes for a kafka consumer. But rather than give any sort of error message, the thread responsible for consuming that partition would simply hang on that message. The other partition threads would continue along happily.

Re: Samza task not receiving on one partition

2016-04-25 Thread Jason Erickson
My earlier answer to this was replying to the wrong message. I am launching this with ThreadJobFactory. On Mon, Apr 25, 2016 at 4:26 PM Jagadish Venkatraman wrote: > Are you running in Yarn or as a local process? > > If you're running in Yarn, you can use the AppMaster UI to see what the > jobm

Re: Samza task not receiving on one partition

2016-04-25 Thread Jason Erickson
I am running it with ThreadJobFactory. On Mon, Apr 25, 2016 at 4:34 PM Jason Erickson wrote: > I have a bit more information from the debug logs. For the partition that > isn't working (Partition 4), I see messages like this > from org.apache.samza.checkpoint.kafka.KafkaCheckpointManager: > > A

Re: Samza task not receiving on one partition

2016-04-25 Thread Jason Erickson
I have a bit more information from the debug logs. For the partition that isn't working (Partition 4), I see messages like this from org.apache.samza.checkpoint.kafka.KafkaCheckpointManager: Adding checkpoint Checkpoint [offsets={SystemStreamPartition [kafka, com.stormpath.iam.indexing.document,

Re: Samza task not receiving on one partition

2016-04-25 Thread Jagadish Venkatraman
Are you running in Yarn or as a local process? If you're running in Yarn, you can use the AppMaster UI to see what the jobmodel is. On Mon, Apr 25, 2016 at 2:33 PM, Jason Erickson wrote: > I have a puzzling issue with one of my samza tasks. It works correctly > except for messages on one partit