Re: Review Request 41884: SAMZA-802: KafkaSystemAdmin needs to handle empty topic offsets

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/41884/#review113037 --- samza-kafka/src/main/scala/org/apache/samza/system/kafka/KafkaSyst

hello-samza doesn't run on CDH5.4

Hi, Any ideas why Samza can't start a container while running hello-samza on CDH5.4? It runs fine on local grid. But when running on cluster I can see it talks fine to YARN's ResourceManager and Kafka, get's resources, but fails to start container. At first, I thought it's because CDH5.4 uses YA

Question on zero downtime deployment of Samza jobs

Hi Samza devs and users I know this will be tricky in Samza because Samza Kafka consumer is not coordinated externally, but do you have any idea how to deploy samza jobs with zero downtime? Thank you Best, Jae

Re: Question on zero downtime deployment of Samza jobs

FYI: As part of an Uber internship project, we were working on exactly this problem. Our approach was to do a rolling restart of all the containers wherein we start a "replica" container for each primary container and let it "catch up" before we do the switch. Of course this doesn't guarantee zero

Re: Question on zero downtime deployment of Samza jobs

Hi, Chinmay, That's awesome! Could you share some design doc of this feature? We would love to have this feature in LinkedIn as well! -Yi On Wed, Jan 6, 2016 at 10:02 AM, Chinmay Soman wrote: > FYI: As part of an Uber internship project, we were working on exactly this > problem. Our approach

Re: hello-samza doesn't run on CDH5.4

Hi, Stanislav, Could it be related to SAMZA-727? Your exception stack trace is different though. Besides that, Samza 0.10 requires minimum YARN 2.6.1 now, due to a token expiration issue in YARN 2.6.0 client lib. -Yi On Wed, Jan 6, 2016 at 7:52 AM, Stanislav Los wrote: > Hi, > > Any ideas why

Re: hello-samza doesn't run on CDH5.4

Hi Yi, Our cluster doesn't have Kerberos security in place, so I don't know how it could be related. CDH5.4 runs on YARN 2.6.0. I compiled Samza with YARN 2.6.0 and YARN 2.6.1. In both cases I get the same error. Should token expiration be a problem for the cluster without Kerberos? Regards. On

Re: hello-samza doesn't run on CDH5.4

Token expiration is still a problem w/o Kerberos in YARN 2.6.0 client. But the exception stack trace from that bug is also different from yours. I am suspecting there is some incompatible dependency libraries between CDH5.4 distro and Samza that caused your serialization problems here. On Wed, Jan

Re: hello-samza doesn't run on CDH5.4

So, for now, I made it work by replacing any generic hadoop-* jar in hello-samza distribution with related jar from CDH: [stanis...@hcm001.us-east-1.mgnt.cc samza]$ ls -l deploy/samza-cdh/lib/ | grep hadoop -rwxr-xr-x 1 stanislav stanislav21533 Jan 6 15:19 hadoop-annotations-2.6.0-cdh5.4.0.j

Re: hello-samza doesn't run on CDH5.4

Glad to hear that it has worked. Yeah, I guess right now, if there is any dependency conflicts, your solution would be the way to resolve it. Could you open a JIRA to enhance the hello-samza tutorial docs to address this issue? Thanks! -Yi On Wed, Jan 6, 2016 at 12:47 PM, Stanislav Los wrote: