Thanks all, I added a note to the KIP about deprecating the old
Producer#sendOffsetsToTransaction method.
I think this is sufficient for now, and keeps this KIP well scoped. If we
have reason to do a more aggressive
cleanup that doesn't violate any compatibility policy, someone can pick
that up in another KIP.
Cheers,
Sophie
On Wed, Apr 14, 2021 at 3:05 PM Ismael Juma <ism...@juma.me.uk> wrote:
> Kafka protocol support has a different compatibility policy. For example,
> we still support 0.8.x protocol versions to this day.
>
> Ismael
>
> On Wed, Apr 14, 2021 at 2:43 PM Gary Russell <gruss...@vmware.com> wrote:
>
> > There is also all that downgrade stuff in the TxnOffsetCommitRequest
> that
> > will downgrade to "alpha" if the broker is < 2.5 (if
> > ProducerConfig.AUTO_DOWNGRADE_TXN_COMMIT is set).
> >
> > If the intention is to eventually pull support for alpha/v1 altogether
> > then why not remove all that too?
> > ________________________________
> > From: Sophie Blee-Goldman <sop...@confluent.io.INVALID>
> > Sent: Wednesday, April 14, 2021 5:22 PM
> > To: dev <dev@kafka.apache.org>
> > Subject: Re: [DISCUSS] KIP-732: Deprecate eos-alpha and replace eos-beta
> > with eos-v2
> >
> > Well, the method being deprecated has a consumerGroupId parameter... if
> > it's a "producer only"
> > application then do they just set this as null, or empty string?
> >
> > Btw: this method actually just invokes the other overload, which accepts
> a
> > ConsumerGroupMetadata
> > instead of a String consumerGroupId. So even a "producer only"
> application
> > could use this, and just
> > do the same:
> >
> > public void sendOffsetsToTransaction(Map<TopicPartition,
> > OffsetAndMetadata> offsets,
> > String consumerGroupId) throws
> > ProducerFencedException {
> > sendOffsetsToTransaction(offsets, new
> > ConsumerGroupMetadata(consumerGroupId));
> > }
> >
> >
> > On Wed, Apr 14, 2021 at 2:19 PM Matthias J. Sax <mj...@apache.org>
> wrote:
> >
> > > I don't think we should deprecate the producer method, because there
> are
> > > also "producer only" application that may use this method.
> > >
> > > For example, source connectors.
> > >
> > >
> > > -Matthias
> > >
> > > On 4/14/21 2:12 PM, Sophie Blee-Goldman wrote:
> > > > Hey Gary, thanks for the suggestion. My impression is that the tech
> > debt
> > > > burden of maintaining
> > > > the eos-alpha style code in the Producer is significantly less than
> > what
> > > we
> > > > have to do in Streams,
> > > > since in the Producers eyes eos-alpha reduces to a subset of
> > eos-beta/v2
> > > in
> > > > which only a single
> > > > Producer/transactionalId is active across the entire consumer group.
> > > >
> > > > That said, we may as well encourage non-Streams users to migrate
> > towards
> > > > eos-v2 anyways and at
> > > > least reduce the surface area of the API if nothing else.
> > > >
> > > > I don't have a strong opinion either way, so unless anyone objects
> I'll
> > > add
> > > > this deprecation to the KIP
> > > > as well:
> > > >
> > > > @Deprecated
> > > > public void sendOffsetsToTransaction(Map<TopicPartition,
> > > OffsetAndMetadata>
> > > > offsets, String consumerGroupId) ;
> > > >
> > > >
> > > > On Wed, Apr 14, 2021 at 1:54 PM Gary Russell <gruss...@vmware.com>
> > > wrote:
> > > >
> > > >> Sophie,
> > > >>
> > > >> Why not also deprecate Producer.sendOffsetsToTransaction method
> > variant
> > > >> with the consumerGroupId parameter?
> > > >>
> > > >> This would encourage non-Streams EOS users to move to beta/v2
> > > >> implementations, and the fallback to alpha/v1 mode code can also be
> > > removed
> > > >> in 4.0.
> > > >> ________________________________
> > > >> From: Sophie Blee-Goldman <sop...@confluent.io.INVALID>
> > > >> Sent: Wednesday, April 14, 2021 4:38 PM
> > > >> To: dev <dev@kafka.apache.org>
> > > >> Subject: Re: [DISCUSS] KIP-732: Deprecate eos-alpha and replace
> > eos-beta
> > > >> with eos-v2
> > > >>
> > > >> Thanks all. I've updated the KIP to clarify these will be removed in
> > > 4.0,
> > > >> and added a note about
> > > >> logging a warning as well. I agree that some users may set the
> config
> > > value
> > > >> directly and could
> > > >> miss the deprecation notice.
> > > >>
> > > >> I'll call for a vote today since there seems to be consensus, but
> > please
> > > >> let me know if you have
> > > >> any more late-breaking ideas
> > > >>
> > > >> -Sophie
> > > >>
> > > >> On Wed, Apr 14, 2021 at 1:36 PM Ismael Juma <ism...@juma.me.uk>
> > wrote:
> > > >>
> > > >>> Hey John,
> > > >>>
> > > >>> Yes, we should definitely do that. It's meant to be the standard
> > > >> procedure
> > > >>> for configs that are deprecated.
> > > >>>
> > > >>> Ismael
> > > >>>
> > > >>> On Wed, Apr 14, 2021 at 12:02 PM John Roesler <vvcep...@apache.org
> >
> > > >> wrote:
> > > >>>
> > > >>>> Hello again, Sophie,
> > > >>>>
> > > >>>> I've just had a late-breaking thought about your KIP. It
> > > >>>> might be worthwhile also printing a WARN log when the
> > > >>>> deprecated configs are used. People may not always be
> > > >>>> referencing the StreamsConfig fields in their source code
> > > >>>> when they set these configs, so it would be good to make an
> > > >>>> extra effort to raise the visibility of this deprecation.
> > > >>>>
> > > >>>> Thanks,
> > > >>>> -John
> > > >>>>
> > > >>>> On Wed, 2021-04-14 at 10:47 -0700, Guozhang Wang wrote:
> > > >>>>> Thanks Sophie for writing the KIP! I'm +1 on the proposal.
> > > >>>>>
> > > >>>>> On Wed, Apr 14, 2021 at 8:56 AM Ismael Juma <ism...@juma.me.uk>
> > > >> wrote:
> > > >>>>>
> > > >>>>>> Thanks Sophie. This makes sense to me. One question: do we want
> to
> > > >> be
> > > >>>> a bit
> > > >>>>>> clearer about the removal plans? That is, can we say that the
> > > >>>> deprecated
> > > >>>>>> configs will be removed in 4.0 (instead of likely to be
> removed)?
> > > >> The
> > > >>>>>> implication would be that exactly-once would only work with 2.5+
> > > >>> while
> > > >>>> at
> > > >>>>>> least once would work with all versions. 4.0 is probably 1.5-2
> > > >> years
> > > >>>> away,
> > > >>>>>> so this seems reasonable to me.
> > > >>>>>>
> > > >>>>>> Ismael
> > > >>>>>>
> > > >>>>>> On Tue, Apr 13, 2021 at 7:53 PM Sophie Blee-Goldman
> > > >>>>>> <sop...@confluent.io.invalid> wrote:
> > > >>>>>>
> > > >>>>>>> Hey all,
> > > >>>>>>>
> > > >>>>>>> I'd like to kick off discussion on a small KIP to move towards
> a
> > > >>>> unified
> > > >>>>>>> EOS and clean up the current options. Please give it a pass and
> > > >> let
> > > >>>> me
> > > >>>>>> know
> > > >>>>>>> what you think.
> > > >>>>>>>
> > > >>>>>>> KIP-732: Deprecate eos-alpha and replace eos-beta with eos-v2
> > > >>>>>>> <
> > > >>>>>>>
> > > >>>>>>
> > > >>>>
> > > >>>
> > > >>
> > >
> >
> https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FKAFKA%2FKIP-732%253A%2BDeprecate%2Beos-alpha%2Band%2Breplace%2Beos-beta%2Bwith%2Beos-v2&data=04%7C01%7Cgrussell%40vmware.com%7C1c6861b832134829580708d8ff8b81c7%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637540321944021820%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=ZAUDx2MgPi6gaSj8C87iG68qAo3NhnOtbW2JGafKeq4%3D&reserved=0
> > > >>>>>>>>
> > > >>>>>>> KAFKA-12574: Deprecate eos-alpha
> > > >>>>>>> <
> > > >>
> > >
> >
> https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FKAFKA-12574&data=04%7C01%7Cgrussell%40vmware.com%7C1c6861b832134829580708d8ff8b81c7%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637540321944021820%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=QxLbmSN4kdMMnpsf6W%2BT33vcdeG11yCPkSDcfsaSaj4%3D&reserved=0
> > > >>>
> > > >>>>>>>
> > > >>>>>>> Thanks,
> > > >>>>>>> Sophie
> > > >>>>>>>
> > > >>>>>>
> > > >>>>>
> > > >>>>>
> > > >>>>
> > > >>>>
> > > >>>>
> > > >>>
> > > >>
> > > >
> > >
> >
>
