Review Request 31520: SAMZA-579

2015-02-26 Thread Chris Riccomini
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31520/ --- Review request for samza. Bugs: SAMZA-579 https://issues.apache.org/jira/br

Re: Samza checkpoints in ZK

2015-02-26 Thread Chris Riccomini
Hey Dotan, Samza has the checkpoint-tool.sh script, which can be used to read checkpoints for a given task. The MetricsSnapshotReporter can also be used to read metrics from a Samza job to check its offset progress. I don't believe that there's anything on the OS side that's plug and play, but yo

Handling defaults and windowed aggregates in stream queries

2015-02-26 Thread Milinda Pathirage
Hi devs, I ask about $subject in calcite-dev. You can find the archived discussion at [1]. I think your thoughts are also valuable in this discussion in calcite list. I discovered the requirement for a default window operator when I tried to integrate streamscan (I was using tablescan prevously)

Re: Samza checkpoints in ZK

2015-02-26 Thread Dotan Patrich
Hi Chris, Thanks for the info! very helpful! Seems very reasonable, by the way, it all started when I was looking for some open source monitoring tool for Samza/Kafka to see which tasks are the bottleneck in terms of performance. Do you have any experience with such a tool (other than the internal

Re: Samza checkpoints in ZK

2015-02-26 Thread Chris Riccomini
Hey Dotan, The high-level (ZK-based) Kafka consumer (not Samza's) currently uses ZK to store offsets. They (Kafka) are moving away from this when they re-write their new NIO-based consumer. They will adopt the strategy of storing offsets in a Kafka topic, just like Samza has for years. The main m

Re: Review Request 31392: SAMZA-555

2015-02-26 Thread Yan Fang
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31392/#review74231 --- samza-core/src/main/scala/org/apache/samza/container/RunLoop.scala