Re: Review Request 51346: SAMZA-974 - Support finite datasources in Samza that have a notion of End-Of-Stream

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51346/ --- (Updated Aug. 25, 2016, 11:52 p.m.) Review request for samza, Boris Shkolnik, C

Re: Review Request 51346: SAMZA-974 - Support finite datasources in Samza that have a notion of End-Of-Stream

> On Aug. 25, 2016, 6:13 p.m., Prateek Maheshwari wrote: > > samza-core/src/main/java/org/apache/samza/task/AsyncRunLoop.java, line 306 > > > > > > I think we only need the size(). Is the underlying data structure >

Re: Review Request 51346: SAMZA-974 - Support finite datasources in Samza that have a notion of End-Of-Stream

> On Aug. 25, 2016, 6:13 p.m., Prateek Maheshwari wrote: > > samza-core/src/main/scala/org/apache/samza/system/SystemConsumers.scala, > > line 140 > > > > > > Minor: Add documentation about what this is. All change

Re: Review Request 51346: SAMZA-974 - Support finite datasources in Samza that have a notion of End-Of-Stream

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51346/ --- (Updated Aug. 25, 2016, 11:43 p.m.) Review request for samza, Boris Shkolnik, C

Re: Review Request 51346: SAMZA-974 - Support finite datasources in Samza that have a notion of End-Of-Stream

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51346/#review146813 --- samza-api/src/main/java/org/apache/samza/system/IncomingMessageEn

Re: Job coordinator stream and job redeployment

Thanks for confirming, Jake :) On Thu, Aug 25, 2016 at 11:18 AM Jacob Maes wrote: > Hey David, > > Yes, the config is rewritten by the JobRunner each time the job is > restarted as dictated by the "resetJobConfig" property (which defaults to > true) here: > > https://github.com/apache/samza/blob

Re: Job coordinator stream and job redeployment

Hey David, Yes, the config is rewritten by the JobRunner each time the job is restarted as dictated by the "resetJobConfig" property (which defaults to true) here: https://github.com/apache/samza/blob/944dd02e1d00bcce59f1fcc33ecbb2a8acd95870/samza-core/src/main/scala/org/apache/samza/job/JobRunner

Re: Job coordinator stream and job redeployment

After digging around a bit using kafka-console-consumer.sh, I'm able to peek into the coordinator stream and see the config entries. Looks like a redeployment would just append the new configs to this topic. And I assume the bootstrapping process will have to consume the entire stream from the beg