Thanks for confirming, Jake :) On Thu, Aug 25, 2016 at 11:18 AM Jacob Maes <jacob.m...@gmail.com> wrote:
> Hey David, > > Yes, the config is rewritten by the JobRunner each time the job is > restarted as dictated by the "resetJobConfig" property (which defaults to > true) here: > > https://github.com/apache/samza/blob/944dd02e1d00bcce59f1fcc33ecbb2a8acd95870/samza-core/src/main/scala/org/apache/samza/job/JobRunner.scala#L85 > > The version parameter is not the version of the config, but the version of > the coordinator stream protocol. It will only increment if there's a change > to the way Samza reads/writes coordinator stream messages. > > -Jake > > On Thu, Aug 25, 2016 at 8:44 AM, David Yu <david...@optimizely.com> wrote: > > > After digging around a bit using kafka-console-consumer.sh, I'm able to > > peek into the coordinator stream and see the config entries. > > > > Looks like a redeployment would just append the new configs to this > topic. > > And I assume the bootstrapping process will have to consume the entire > > stream from the beginning to get a consolidated/up-to-date job model. Am > I > > understanding this correctly? > > > > Also, according to the doc, the keys of the coordinator stream is > > ["<version-number>", "<message-type>", "<key>"]. From what I saw, the > > <version-number> doesn't seem to change. It stays at 1 even after I > > redeployed the job with new configs. I was assuming that it would get > > incremented with every deployment/update. > > > > Thanks, > > David > > > > On Wed, Aug 24, 2016 at 5:39 PM David Yu <david...@optimizely.com> > wrote: > > > > > Hi, > > > > > > I'm trying to understand role of the coordinator stream during a job > > > redeployment. > > > > > > From the Samza documentation, I'm seeing the following about the > > > coordinator stream: > > > > > > The Job Coordinator bootstraps configuration from the coordinator > stream > > > each time upon job start-up. It periodically catches up with any new > data > > > written to the coordinator stream and updates the Job Model. > > > > > > However, it is unclear to me how this will work with "--config-path" > when > > > we need to redeploy a job. Does the provided config first gets > persisted > > to > > > the coordinator stream, updating the previous model, or will it simply > be > > > ignored? > > > > > > Thanks, > > > David > > > > > >
