> On April 28, 2015, 5:43 p.m., Yi Pan (Data Infrastructure) wrote: > > samza-core/src/main/scala/org/apache/samza/container/SamzaContainer.scala, > > line 457 > > <https://reviews.apache.org/r/33453/diff/2/?file=939981#file939981line457> > > > > Question: shouldn't the base dir to be determined by container ID, not > > jobId? > > Navina Ramesh wrote: > I think containerId is not really relevant because the store partition > directories are uniquely named across the containers. The ownership of > containers and partition directory is persisted in the config stream (going > forward). I will verify this and confirm here.
Verified. Store partitions have unique names and are agnostic of the container to which they belong. - Navina ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33453/#review81838 ----------------------------------------------------------- On April 22, 2015, 9:54 p.m., Navina Ramesh wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/33453/ > ----------------------------------------------------------- > > (Updated April 22, 2015, 9:54 p.m.) > > > Review request for samza, Yan Fang, Chris Riccomini, Naveen Somasundaram, and > Yi Pan (Data Infrastructure). > > > Repository: samza > > > Description > ------- > > Changed default to yarn cwd instead of io.tmpDir and refactored code > > > Diffs > ----- > > samza-core/src/main/scala/org/apache/samza/config/ShellCommandConfig.scala > 1a2dd4413f56e53dbeeb47b5637d7b0c50522f02 > samza-core/src/main/scala/org/apache/samza/container/SamzaContainer.scala > 720fbdceafc4fe69b048d81a677e874d13e6d22f > samza-core/src/main/scala/org/apache/samza/storage/TaskStorageManager.scala > f68a7fee24614fce101e91c4f933d9b4e65dda0a > > samza-kv-rocksdb/src/main/scala/org/apache/samza/storage/kv/RocksDbKeyValueStore.scala > 66c2a0dc2e38e21f951727a30f0987776ac52fe2 > > Diff: https://reviews.apache.org/r/33453/diff/ > > > Testing > ------- > > Tested locally using hello-samza. > Note: you have to set an environment variable LOGGED_STORE_BASE_DIR pointing > to the new location to persist the changelog attached stores. Otherwise, it > will default to YARN's cwd and will not re-use local state. > > > Thanks, > > Navina Ramesh > >
