Re: [DISCUSS] Samza 0.11.0 release

2016-08-26 Thread Yi Pan
Hi, Nicholas, Thank your so much for pointing out the perf degradation. I synced up w/ Jiangjie Qin from our Kafka team and confirmed that we are running 0.10 Kafka brokers w/o the timestamp message format in 0.10. I also took a look at SAMZA-855 patch and it is far from a working state yet. Given

Re: [DISCUSS] Samza 0.11.0 release

2016-08-24 Thread Nicolas Maquet
Hi, We are looking at upgrading to Kafka 0.10.0 in part for the new message format which includes a timestamp field. Kafka 0.10.0 is backwards compatible with 0.8.x clients but we are concerned about the performance impact, see http://kafka.apache.org/documentation.html#upgrade_10_performance_impa

Re: [DISCUSS] Samza 0.11.0 release

2016-08-24 Thread Yi Pan
Hi, Nicolas, Could you explain to me why Samza is blocking you from upgrading your Kafka brokers to 0.10? At LinkedIn, we are running Samza 0.10 w/ Kafka 0.10 brokers. This is a valid combination since Kafka 0.10 brokers should be backward compatible w/ 0.8.2 clients (which is the version Samza us

Re: [DISCUSS] Samza 0.11.0 release

2016-08-22 Thread Nicolas Maquet
Hi all, Would it be possible to consider including the following in the 0.11.0 target release? SAMZA-855: Upgrade Samza's Kafka client version to 0.10.0.0 We are looking at upgrading our Kafka cluster to 0.10 and Samza is currently preventing us from doing so. Regards, Nicolas Maquet 2016-0

[DISCUSS] Samza 0.11.0 release

2016-08-22 Thread xinyu liu
Hi, All, There have been quite a lot of new features added to master since 0.10 release to warrant a new major release. At LinkedIn, we've done functional and performance testing against master in the past weeks, and deployed jobs with the latest build in production. We will continue to test for s