Hi.
I'm the author of the KIP-616. I got acquainted with the KIP-513 and I
think overall it's a good idea. But I think workaround only on the scala
side is not enough. We could consider moving a bit further and change serde
hierarchy on the java side to something like this:
KeySerializer-----↘
KeySerde---↘
KeyDeserializer---↗ ↓
Serde
ValueSerializer---↘ ↑
ValueSerde-↗
ValueDeserializer-↗
I understand that this is a bit too revolutionary, but I think this is the
right direction.
Regarding KIP-616 and KIP-513 connection. It's hard to say at the current
moment how should we implement these kips: together or separately. It looks
like there is no consensus on the implementation details for these kips.
If we decided to create a new `KeySerde` and `ValueSerde` on the scala side
(current plan without my proposal above), we could use their companions for
default instances. In this case, it looks like we don't need to do KIP-616.
But what about backward compatibility? What should we do with
`org.apache.kafka.streams.scala.Serdes`? Deprecate it?
- Yuriy
On Sun, May 31, 2020 at 1:24 AM John Roesler <vvcep...@apache.org> wrote:
> Hi Mykhailo,
>
> Wow, I really dropped the ball here. I have just looked over your KIP
> again, and now I see how you don’t need to change every dsl method, only
> Consumed, Materialized, etc.
>
> I think this would be a good addition. Yuriy has just proposed KIP-616 to
> fix some other problems with the implicit serdes. I’m wondering if these
> two kips have any joint opportunities we should consider, or if it’s better
> to continue to consider them separately.
>
> Thanks,
> John
>
> On Wed, Jan 22, 2020, at 16:18, Михаил Ерёменко wrote:
> > Hi, John!
> >
> > Sorry for the late reply. I am not really familiar with this mail list
> > discussions, so I have not seen your mails.
> >
> > Regarding your question:
> > > I guess what
> > I'm struggling with is why you actually want to have different key and
> > serdes for the same type
> >
> > I think good example will be (and it is actually what we do in ours
> > project) using confluent schema registry in conjunction with kafka
> > streams. Some models can be used as keys as well as values. When we
> > define schema registry compatible serde, we have to specify is it for
> > key or not. We can of course create two serdes for the same model, but
> > in this case implicit semantic will not work because scala doesn’t know
> > which implicit to pick. And things become even more complicated in case
> > if you will try to derive your serdes (we derive serdes in ours
> > project).
> >
> > One more thing:
> > > every method in the streams-scala DSL.
> >
> > So far we've just changed
> > org.apache.kafka.streams.scala.ImplicitConversions and
> > org.apache.kafka.streams.scala.kstream.Materialized and it works for
> > us. Also we did introduce default serdes for primitive types.
> >
> > Regards,
> > Mykhailo
>
