GitHub user shanthoosh opened a pull request:
https://github.com/apache/samza/pull/879
SAMZA-2059: Persist configuration in coordinator stream for standalone.
Prior to Samza 1.0, users plugged in the properties of an I/O system
through a configuration file. Samza employed rewriters in the user-defined
order to compute the configuration of a job.
Post Samza 1.0, we introduced new abstractions viz` StreamDescriptor` and
`SystemDescriptor` in samza, with the purpose of performing configuration
expansion for predefined systems at run-time.
Configuration computed at run-time is not persisted at a centralized
storage in samza-standalone. This breaks the functionality of the tools viz
checkpoint-tool, coordinator-stream-writer, etc in samza standalone. This patch
addresses this problem by storing the configuration in coordinator stream for
standalone.
In the follow up PR's:
1. We'll switch from zookeeper to coordinator-stream as JobModel storage
layer in standalone
2. Samza tools(checkpoint-tool) will be migrated to read the configuration
from coordinator stream rather than the disk.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/shanthoosh/samza
standalone-coordinator-stream-for-config
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/samza/pull/879.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #879
----
commit 5095eb5dcf447646494eed3cf2b3e9ea8fcf7580
Author: Shanthoosh Venkataraman <spvenkat@...>
Date: 2019-01-11T21:51:52Z
SAMZA-2059: Storing configuration in coordinator stream for standalone.
----
---
