Re: Samza task not receiving on one partition

2016-04-25 Thread Jason Erickson
My earlier answer to this was replying to the wrong message. I am launching this with ThreadJobFactory. On Mon, Apr 25, 2016 at 4:26 PM Jagadish Venkatraman wrote: > Are you running in Yarn or as a local process? > > If you're running in Yarn, you can use the AppMaster UI to see what the > jobm

Re: Review Request 44920: SAMZA-680 Refactor the Samza AppMaster to support other cluster managers

2016-04-25 Thread Yi Pan (Data Infrastructure)
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/44920/#review130448 --- Fix it, then Ship it! Overall, ltgm. Only a few comments to be

Re: Samza task not receiving on one partition

2016-04-25 Thread Jason Erickson
I am running it with ThreadJobFactory. On Mon, Apr 25, 2016 at 4:34 PM Jason Erickson wrote: > I have a bit more information from the debug logs. For the partition that > isn't working (Partition 4), I see messages like this > from org.apache.samza.checkpoint.kafka.KafkaCheckpointManager: > > A

Re: Samza task not receiving on one partition

2016-04-25 Thread Jason Erickson
I have a bit more information from the debug logs. For the partition that isn't working (Partition 4), I see messages like this from org.apache.samza.checkpoint.kafka.KafkaCheckpointManager: Adding checkpoint Checkpoint [offsets={SystemStreamPartition [kafka, com.stormpath.iam.indexing.document,

Re: Samza task not receiving on one partition

2016-04-25 Thread Jagadish Venkatraman
Are you running in Yarn or as a local process? If you're running in Yarn, you can use the AppMaster UI to see what the jobmodel is. On Mon, Apr 25, 2016 at 2:33 PM, Jason Erickson wrote: > I have a puzzling issue with one of my samza tasks. It works correctly > except for messages on one partit

Samza task not receiving on one partition

2016-04-25 Thread Jason Erickson
I have a puzzling issue with one of my samza tasks. It works correctly except for messages on one partition. I have 9 partitions on the topic. If I send 1000 messages, I only receive about 890 of them. I have checked with kafka-console-consumer with partition keys that I know don’t get processed by

Re: Review Request 46644: SAMZA-889 - Change log not working properly with In memory Store

2016-04-25 Thread Navina Ramesh
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/46644/ --- (Updated April 25, 2016, 5:25 p.m.) Review request for samza, Boris Shkolnik, C

Review Request 46644: SAMZA-889 - Change log not working properly with In memory Store

2016-04-25 Thread Navina Ramesh
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/46644/ --- Review request for samza, Boris Shkolnik, Chris Pettitt, and Yi Pan (Data Infras

Samza Metrics topics with Elastic Search 2.0

2016-04-25 Thread George @paytm.com
Hi all, ES 2.0 doesn't allow . in field name anymore. As a result, writing metrics to ES will give the following errors :response=>{"create"=>{"_index"=>"samza_metrics-2016.04.21", "_type"=>"samza_metrics", "_id"=>"AVQ6FgE-3mvK_4wEEW2o", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception"

Re: Review Request 44920: SAMZA-680 Refactor the Samza AppMaster to support other cluster managers

2016-04-25 Thread Yi Pan (Data Infrastructure)
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/44920/#review130350 --- Partial review... Will continue Mon. samza-core/src/main/java/or