> >>>> could
>> > > >>>> > define which converter it wants to use; for backward
>> compatibility
>> > > >>>> purposes
>> > > >>>> > we use a header converter by default that serializ
gt;>> > headers in the same way -- with string-like representations by
> > > >>>> default. I
> > > >>>> > think this is a safe assumption for the short term, and if we
> need
> > > >>>> more
> > > >>>>
he desired type, and to support all of the sensible
> > >>>> conversions
> > >>>> > between Connect's primitives and logical types. So, a connector
or
> > >>>> > transform could always call `header.valueAsObje
t; > >>>> > transforms that are implemented to expect a certain type of header
> > >>>> value,
> > >>>> > such as an integer or boolean or timestamp? We could solve this
> > >>>> problem
> > >>>> > (for the m
>>>> > between Connect's primitives and logical types. So, a connector or
> >>>> > transform could always call `header.valueAsObject()` to get the raw
> >>>> > representation from the converter, but a connector or transfo
> `header.valueAsTimestamp()` to return a java.util.Date value that is
>>>> > described by Connect's Timestamp logical type). We can convert
>>>> between most
>>>> > primitive and logical types (e.g., anything to a STRING, INT32 to
>>>>
ARRAY to
>>> FLOAT32,
>>> > INT32 to STRUCT, BYTE_ARRAY to anything, etc.), so these can throw a
>>> > `DataException`.
>>> >
>>> > I've refined this approach over the last few months, and have a PR for
>>> a
>>> > complete prot
; the
>> > KIP) some relatively obvious SMTs for copying header values into record
>> > key/value fields, and extracting record key/value fields into header
>> values.
>> >
>> > @Michael, would you mind if I edited KIP-145 to reflect this proposal? I
>> >
gt; Comments? Thoughts?
> >
> > Best regards,
> >
> > Randall
> >
> >
> > On Thu, Oct 19, 2017 at 2:10 PM, Michael André Pearce <
> > michael.andre.pea...@me.com> wrote:
> >
> >> @rhauch
> >>
> >> Here is the previous
nts? Thoughts?
>>
>> Best regards,
>>
>> Randall
>>
>>
>> On Thu, Oct 19, 2017 at 2:10 PM, Michael André Pearce <
>> michael.andre.pea...@me.com> wrote:
>>
>>> @rhauch
>>>
>>> Here is the previous discussion thread, just reigniting so we can d
thread, just reigniting so we can discuss
>> against the original kip thread
>>
>>
>> Cheers
>>
>> Mike
>>
>> Sent from my iPhone
>>
>>> On 5 May 2017, at 02:21, Michael Pearce wrote:
>>>
>>> Hi Ewen,
>>>
dea?
> >
> > Did it help?
> >
> > Cheers
> > Mike
> >
> > Sent using OWA for iPhone
> > ________________
> > From: Michael Pearce
> > Sent: Wednesday, May 3, 2017 10:11:55 AM
> > To: dev@kafka.apache.org
> >
howing the idea?
>
> Did it help?
>
> Cheers
> Mike
>
> Sent using OWA for iPhone
>
> From: Michael Pearce
> Sent: Wednesday, May 3, 2017 10:11:55 AM
> To: dev@kafka.apache.org
> Subject: Re: [DISCUSS] KIP 145 - Expose Record He
rror
> >> makers of replicating the messaging from one cluster to another. Eg
> like
> >> mirror make should just be taking and transferring the headers as is.
> >>
> >> We don't actually use this inside our company, so not exposing
icating the messaging from one cluster to another. Eg
> like
> >> mirror make should just be taking and transferring the headers as is.
> >>
> >> We don't actually use this inside our company, so not exposing this
> isn't
> >> a
o mirror
> >> makers of replicating the messaging from one cluster to another. Eg
> like
> >> mirror make should just be taking and transferring the headers as is.
> >>
> >> We don't actually use this inside our company, so not exposing this
> isn't
> >&
>> tools like replicator that do.
>>
>> And as good citizens think we should complete the work and expose the
>> headers same as in the record to at least allow them to replicate the
>> messages as is. Note Steph seems to want it.
>>
>> Cheers
>> Mike
>>
>>
; messages as is. Note Steph seems to want it.
>
> Cheers
> Mike
>
> Sent using OWA for iPhone
> ____
> From: Gwen Shapira
> Sent: Monday, May 1, 2017 2:36:34 PM
> To: dev@kafka.apache.org
> Subject: Re: [DISCUSS] KIP 145 - Expose Record Headers in Kafka Con
to want it.
Cheers
Mike
Sent using OWA for iPhone
From: Gwen Shapira
Sent: Monday, May 1, 2017 2:36:34 PM
To: dev@kafka.apache.org
Subject: Re: [DISCUSS] KIP 145 - Expose Record Headers in Kafka Connect
Hi,
I'm excited to see the community expanding
Hi,
I'm excited to see the community expanding Connect in this direction!
Headers + Transforms == Fun message routing.
I like how clean the proposal is, but I'm concerned that it kinda deviates
from how Connect handles data elsewhere.
Unlike Kafka, Connect doesn't look at all data as byte-arrays,
Hi All,
Now KIP-82 is committed I would like to discuss extending the work to expose it
in Kafka Connect, its primary focus being so connectors that may do similar
tasks as MirrorMakers, either Kafka->Kafka or JMS-Kafka would be able to
replicate the headers.
It would be ideal but not mandatory
21 matches
Mail list logo
