Re: [DISCUSS] (SAMZA-865) REST API for starting and stopping Samza jobs

2016-01-27 Thread Jacob Maes
There were no questions in my initial email, so I thought I'd add a couple to seed the discussion: 1. Does a separate Samza REST service sound useful for your Samza implementation(s)? 2. Is the API useful and extensible for your purposes? 3. Is anyone working on something similar? Thanks, Jake On

Re: ChangeLog Question for TTL rocksDB stores

2016-01-27 Thread Jacob Maes
Here's my understanding. The others can correct me if I'm mistaken. Samza provides the changelog functionality by intercepting RocksDB "put" and "delete" operations. However, TTL is managed by RocksDB internally and there aren't any hooks exposed in the RocksDB JNI. So there are 2 problems that ar

ChangeLog Question for TTL rocksDB stores

2016-01-27 Thread David Garcia
So, I saw this very scary message: ERROR - e.kv.RocksDbKeyValueStore$ - sessionJoinStore is a TTL based store, changelog is not supported for TTL based stores, use at your own discretion A few of questions: 1.) Does this mean that this store is NOT backed by the changelog? 2.) Provided tha

[DISCUSS] (SAMZA-865) REST API for starting and stopping Samza jobs

2016-01-27 Thread Jacob Maes
Hey folks, I wanted to solicit feedback on the design for a REST API for starting and stopping jobs. At LinkedIn, this API is going to help us integrate other services/features with Samza and provide a cluster-management-agnostic means of managing Samza jobs. As mentioned in the design doc, despi