Thanks Penghui for driving this. From my side, it would be good to have those schema related changes to be merged and included in 2.4.0 release. Since we have made fair amount of efforts in making pulsar easily integrated with other data systems. I also have a pull request for adding schema documentation after all the schema related changes merged.
Some other comments inline. - Sijie On Fri, May 17, 2019 at 4:43 PM PengHui Li <codelipeng...@gmail.com> wrote: > Hi all > > It is about 3 months since last major release (2.3.0). I am planning to > start the 2.4.0 release soon (probably next week). > > > I would like to check the current status for the in-progress features, to > see what need to be merged and included in 2.4.0. > > > Here is the list of items still in progress. Can the owner confirm if we > should include them in 2.4.0? Also if they are not started or need more > time to complete, can we push some of them to next release? > > > *in-process features :* > > 1. Delayed message delivery > 2. Key value schema compatibility checker > 3. Rate limiter for replicator > 4. Snappy compression for java client > 5. Tenant based bookie isolation > 6. Secondary bookie isolation group > 7. Reader for pulsar-spout in pulsar-storm > 8. Support key value schema versioning > 9. Pulsar IO Source Connector for Couchbase > <https://github.com/apache/pulsar/pull/4210> > > *issues plan to fix in 2.4.0:* > > 1. Update debezium connector documentation > I have seen an increasing adoption in using debezium, especially for postgres cdc. I would suggest us including this change for 2.4.0. Otherwise users will be difficult to use debezium for postgres cdc. > 2. Pulsar SQL is not able to read the latest message > I think that is not an easy fix. It requires a bookkeeper client change. I think @zyong has worked on a change to fall back to use v3 protocol to read explicit LAC for addressing this problem. Since it requires a bookkeeper release, I don't think we should block 2.4.0 for this. This fix can go with 2.4.1 or 2.5.0. > > Please let me know if I miss anything. > > Thanks. > Penghui > > > Ezequiel Lovelle <ezequiellove...@gmail.com> 于2019年4月24日周三 上午4:28写道： > > > Sorry for my confusion, I thought the intention was listing both, wip and > > done features. > > > > > Let's bring the discussion back to the list of features to be included > in > > 2.4.0 > > > > Do we want to include #4062 <https://github.com/apache/pulsar/pull/4062> > > in > > 2.4.0? I'm very interested in this one. > > > > Congrats PengHui :) > > -- > > *Ezequiel Lovelle* > > > > > > On Tue, 23 Apr 2019 at 15:00, Sijie Guo <guosi...@gmail.com> wrote: > > > > > On Wed, Apr 24, 2019 at 1:51 AM Matteo Merli <matteo.me...@gmail.com> > > > wrote: > > > > > > > > I don't think the original email was trying to list all the > features. > > > It > > > > > was trying to bring up all the *in-progress* features > > > > > to discuss > > > > > > > > I don't really want to drag the conversation but I think it difficult > > > > to characterize the unilaterally declaration of a "feature freeze > > > > immediate now" for a call for discussion. > > > > > > > > > > Since I am mentoring Penghui for the release, I take the responsibility > > for > > > this and > > > apologize for any misleading words he was using in this email. > > > > > > Let's bring the discussion back to the list of features to be included > in > > > 2.4.0, and help > > > him successfully cut a release. > > > > > > - Sijie > > > > > > > > > > > > > > > > > > > > > > On Tue, Apr 23, 2019 at 10:30 AM Sijie Guo <guosi...@gmail.com> > wrote: > > > > > > > > > > Hi Lovelle, > > > > > > > > > > I don't think the original email was trying to list all the > features. > > > It > > > > > was trying to bring up all the *in-progress* features > > > > > to discuss, so that the community can wrap up the tasks for the > > release > > > > > manager to release 2.4.0. If an in-progress > > > > > feature that is not needed for 2.4.0, people can comment to remove > it > > > > from > > > > > the list; if a new feature is not showed > > > > > up in the in-progress, people can comment to bring it up. (I don't > > mean > > > > to > > > > > jump in to claim a process here. but I am > > > > > here to help a new release manager to get familiar with a release > > > process > > > > > based on my experiences with pulsar and > > > > > other ASF projects) > > > > > > > > > > So I guess that's why negative acks is omitted from that list, > since > > it > > > > is > > > > > already merged in 2.4.0. At the point when > > > > > the email was sent, interceptors are not merged. Hope this can > clean > > up > > > > the > > > > > confusion. > > > > > > > > > > I hope this email thread should be brought back to focus on what > > > > > in-progress features should be included in 2.4.0 and > > > > > what should be pushed to 2.5.0, and if there are features that > > people > > > is > > > > > already working on but not shown in the issues list. > > > > > So that the release manager can know when to start cutting the > > release. > > > > > > > > > > Thanks, > > > > > Sijie > > > > > > > > > > > > > > > On Tue, Apr 23, 2019 at 9:29 PM Ezequiel Lovelle < > > > > ezequiellove...@gmail.com> > > > > > wrote: > > > > > > > > > > > Hi! > > > > > > > > > > > > My 2 cents, as Matteo said, the feature which is worth mentioning > > is > > > > > > negative ack instead of its interceptor. > > > > > > > > > > > > Idk If it's as a relevant feature, but in pull #4031 > > > > > > <https://github.com/apache/pulsar/pull/4031> we've just finish > > > > > > adding support for seek() in Reader impl. > > > > > > > > > > > > -- > > > > > > *Ezequiel Lovelle* > > > > > > > > > > > > > > > > > > On Tue, 23 Apr 2019 at 02:24, Sijie Guo <guosi...@gmail.com> > > wrote: > > > > > > > > > > > > > It is the time for the community to discuss. Since 2.4 has > > already > > > > > > > accumulated 1+ month changes, we should be ready to cut a > release > > > > soon. > > > > > > So > > > > > > > ideally we should just wrap the in progress changes and release > > > > 2.4.0. If > > > > > > > the features are not started yet, we can push those features to > > > > 2.5.0. > > > > > > That > > > > > > > what we have been doing in the past releases - We are trying to > > do > > > > > > releases > > > > > > > in 1~2 months or so. > > > > > > > > > > > > > > Unless there are strong requirements that a unstarted feature > is > > > > needed > > > > > > to > > > > > > > be included. We can bring it for discussion. > > > > > > > > > > > > > > Sijie > > > > > > > > > > > > > > On Tue, Apr 23, 2019 at 1:06 PM Jerry Peng < > > > > jerry.boyang.p...@gmail.com> > > > > > > > wrote: > > > > > > > > > > > > > > > Are we collecting a list of still in-progress features for > 2.4 > > or > > > > are > > > > > > we > > > > > > > > collecting a list of all the features going into 2.4? > > > > > > > > > > > > > > > > On Mon, Apr 22, 2019 at 9:36 PM Sijie Guo < > guosi...@gmail.com> > > > > wrote: > > > > > > > > > > > > > > > > > I think Penghui was just trying to collect a list of > features > > > to > > > > be > > > > > > > > > included in 2.4.0 and have a discussion thread to include > any > > > > missed > > > > > > > > items. > > > > > > > > > So that he can have a better estimation on when to cut the > > > 2.4.0 > > > > > > > release. > > > > > > > > > “Feature Freeze” is probably not the right word here, but > it > > > is a > > > > > > > common > > > > > > > > > concept in a software release cycle. We can discuss what > > would > > > > be the > > > > > > > > right > > > > > > > > > term in a separated thread. Let’s focus on giving him the > > list > > > of > > > > > > > > features > > > > > > > > > to be included for 2.4.0. So he can know what to wait for. > > > > > > > > > > > > > > > > > > - Sijie > > > > > > > > > > > > > > > > > > On Tue, Apr 23, 2019 at 11:34 AM Jerry Peng < > > > > > > > jerry.boyang.p...@gmail.com > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > Yup the PMC usually need to vote on something like this > and > > > be > > > > in > > > > > > > > > agreement > > > > > > > > > > before such a freeze can take into effect. > > > > > > > > > > > > > > > > > > > > -Jerry > > > > > > > > > > > > > > > > > > > > On Mon, Apr 22, 2019 at 8:24 PM Matteo Merli < > > > > > > matteo.me...@gmail.com > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > Negative acks also would be a new feature of 2.4, not > > just > > > > its > > > > > > > > exposure > > > > > > > > > > in > > > > > > > > > > > interceptors. > > > > > > > > > > > > > > > > > > > > > > On Mon, Apr 22, 2019 at 8:07 PM Matteo Merli < > > > > > > > matteo.me...@gmail.com > > > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > NodeJS client was meant for 2.4 as well. > > > > > > > > > > > > > > > > > > > > > > > > On Mon, Apr 22, 2019 at 8:06 PM Matteo Merli < > > > > > > > > matteo.me...@gmail.com > > > > > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > >> Replicated subscriptions are meant for 2.4 > > > > > > > > > > > >> > > > > > > > > > > > >> On Mon, Apr 22, 2019 at 7:58 PM PengHui Li < > > > > > > > > codelipeng...@gmail.com > > > > > > > > > > > > > > > > > > > > > >> wrote: > > > > > > > > > > > >> > > > > > > > > > > > >>> Hi all, > > > > > > > > > > > >>> > > > > > > > > > > > >>> > > > > > > > > > > > >>> There are already a lot of great features in 2.4.0. > > It > > > is > > > > > > time > > > > > > > to > > > > > > > > > > > release > > > > > > > > > > > >>> 2.4.0 now. I am calling a feature freeze for 2.4.0. > > We > > > > are > > > > > > not > > > > > > > > > going > > > > > > > > > > to > > > > > > > > > > > >>> accept any new features from now. People still have > > > time > > > > to > > > > > > > > > complete > > > > > > > > > > > the > > > > > > > > > > > >>> ongoing features and any bug fixes. The code change > > > will > > > > be > > > > > > > > freeze > > > > > > > > > > in 2 > > > > > > > > > > > >>> weeks. > > > > > > > > > > > >>> > > > > > > > > > > > >>> > > > > > > > > > > > >>> I am listing the in-progress features as below. > > Please > > > > let me > > > > > > > > know > > > > > > > > > > if I > > > > > > > > > > > >>> miss anything. > > > > > > > > > > > >>> > > > > > > > > > > > >>> > > > > > > > > > > > >>> - Go Function > > > > > > > > > > > >>> > > > > > > > > > > > >>> - Go Schema > > > > > > > > > > > >>> > > > > > > > > > > > >>> - Key_Shared Subscription > > > > > > > > > > > >>> > > > > > > > > > > > >>> - Delayed Message > > > > > > > > > > > >>> > > > > > > > > > > > >>> - Kerberos Support > > > > > > > > > > > >>> > > > > > > > > > > > >>> - Multi-Hosts service url support in pulsar admin > > > > > > > > > > > >>> > > > > > > > > > > > >>> - Multi-version Schema support > > > > > > > > > > > >>> > > > > > > > > > > > >>> - Interceptor for negative ack send > > > > > > > > > > > >>> > > > > > > > > > > > >>> - Pulsar IO connector for InfluxDB sink > > > > > > > > > > > >>> > > > > > > > > > > > >>> > > > > > > > > > > > >>> - Penghui > > > > > > > > > > > >>> > > > > > > > > > > > >> -- > > > > > > > > > > > >> -- > > > > > > > > > > > >> Matteo Merli > > > > > > > > > > > >> <matteo.me...@gmail.com> > > > > > > > > > > > >> > > > > > > > > > > > > -- > > > > > > > > > > > > -- > > > > > > > > > > > > Matteo Merli > > > > > > > > > > > > <matteo.me...@gmail.com> > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > -- > > > > > > > > > > > Matteo Merli > > > > > > > > > > > <matteo.me...@gmail.com> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
