Hi, We are looking at upgrading to Kafka 0.10.0 in part for the new message format which includes a timestamp field. Kafka 0.10.0 is backwards compatible with 0.8.x clients but we are concerned about the performance impact, see http://kafka.apache.org/documentation.html#upgrade_10_performance_impact.
Cheers, Nicolas On 25 Aug 2016 6:20 a.m., "Yi Pan" <nickpa...@gmail.com> wrote: > Hi, Nicolas, > > Could you explain to me why Samza is blocking you from upgrading your Kafka > brokers to 0.10? At LinkedIn, we are running Samza 0.10 w/ Kafka 0.10 > brokers. This is a valid combination since Kafka 0.10 brokers should be > backward compatible w/ 0.8.2 clients (which is the version Samza uses). I > would recommend that you try to upgrade the Kafka brokers w/ Samza 0.10. If > there is any issues you see, we will help to investigate and fix. > > Meanwhile, we are planning to schedule Samza releases more regularly and > frequent in terms of time. Hence, although SAMZA-855 is not planned in > 0.11, it will be included in the immediate version after that. > > Let me know if you have any further concerns. > > On Mon, Aug 22, 2016 at 2:48 PM, Nicolas Maquet <nmaq...@gmail.com> wrote: > > > Hi all, > > > > Would it be possible to consider including the following in the 0.11.0 > > target release? > > > > SAMZA-855: Upgrade Samza's Kafka client version to 0.10.0.0 > > > > We are looking at upgrading our Kafka cluster to 0.10 and Samza is > > currently preventing us from doing so. > > > > Regards, > > > > Nicolas Maquet > > > > > > > > 2016-08-23 9:06 GMT+12:00 xinyu liu <xinyuliu...@gmail.com>: > > > > > Hi, All, > > > > > > There have been quite a lot of new features added to master since 0.10 > > > release to warrant a new major release. At LinkedIn, we've done > > functional > > > and performance testing against master in the past weeks, and deployed > > jobs > > > with the latest build in production. We will continue to test for > > stability > > > in the next few weeks. > > > > > > Here are the JIRAs of main features that will be included in this > release > > > (sorted in chronological order): > > > > > > SAMZA-876: Add AvroDataFileHdfsWriter > > > SAMZA-849: Split-deployment of samza framework > > > SAMZA-905: Host Affinity - Continuous offset > > > SAMZA-906: Host Affinity - Minimize task reassignment when container > > count > > > changes > > > SAMZA-924: Disk Quotas: Add disk space monitoring > > > SAMZA-956: Disk Quotas: Add throttler and disk quota enforcement > > > SAMZA-680: Inverting JobCoordinator and SamzaAppMaster logic > > > SAMZA-863: Multithreading support in Samza > > > > > > And the JIRAs that I think we should try and get done before 0.11.0 is > > > released: > > > > > > SAMZA-980: Automatically restart job containers on change in upstream > > > partitioning > > > SAMZA-988: Update failure testing job configs with job coordinator > > > properties > > > SAMZA-997: Add documentation about disk space monitoring > > > SAMZA-977: User doc for samza multithreading > > > SAMZA-998: Documentation updates for refactored JC > > > SAMZA-1000: Hello-samza latest is broken after Samza 10.1 release > > > > > > > > > Here's what I purpose: > > > > > > 1. Cut an 0.11.0 release branch. > > > 2. Work on getting as many of the pending JIRAs done as possible. > > > 2. Target a release vote on the week of Sep 5. > > > > > > Thoughts? > > > > > > Xinyu > > > > > >
