After digging around a bit using kafka-console-consumer.sh, I'm able to peek into the coordinator stream and see the config entries.
Looks like a redeployment would just append the new configs to this topic. And I assume the bootstrapping process will have to consume the entire stream from the beginning to get a consolidated/up-to-date job model. Am I understanding this correctly? Also, according to the doc, the keys of the coordinator stream is ["<version-number>", "<message-type>", "<key>"]. From what I saw, the <version-number> doesn't seem to change. It stays at 1 even after I redeployed the job with new configs. I was assuming that it would get incremented with every deployment/update. Thanks, David On Wed, Aug 24, 2016 at 5:39 PM David Yu <david...@optimizely.com> wrote: > Hi, > > I'm trying to understand role of the coordinator stream during a job > redeployment. > > From the Samza documentation, I'm seeing the following about the > coordinator stream: > > The Job Coordinator bootstraps configuration from the coordinator stream > each time upon job start-up. It periodically catches up with any new data > written to the coordinator stream and updates the Job Model. > > However, it is unclear to me how this will work with "--config-path" when > we need to redeploy a job. Does the provided config first gets persisted to > the coordinator stream, updating the previous model, or will it simply be > ignored? > > Thanks, > David >
