Hi Pushkar, Just wanted to say, as someone with battle scars from ActiveMQ and Camel, there's very many good reasons to avoid Java serialization on a messaging system. What if you need to tail a topic from the console? What if your testers want to access in their pytests? Etc. And that's not even getting into the minefield of Java serialization compatibility between one VM and another.
That said, if using another serialization format like Avro or JSON isn't feasible, you can implement your own serializer/deserializer. At it's heart, a Kafka producer sends a byte array and a consumer receives a byte array. So a custom serialiser to turn a list or map into bytes is easily doable using an ObjectInputStream. And a consumer can use deserialiser built on an ObjectOutputStream at the other end. Kind regards, Liam Clarke-Hutchinson On Tue, 12 May 2020, 5:13 pm Pushkar Deole, <pdeole2...@gmail.com> wrote: > And by the way, confluent has provided KafkaAvroSerializer/Deserialier. > Can't they be used to do conversion for java types? > > On Tue, May 12, 2020 at 10:09 AM Pushkar Deole <pdeole2...@gmail.com> > wrote: > > > Ok... so jackson json serialization is the way to go for hashmaps as > well? > > > > On Mon, May 11, 2020 at 7:57 PM John Roesler <vvcep...@apache.org> > wrote: > > > >> Oh, my mistake. I thought this was a different thread :) > >> > >> You might want to check, but I don’t think there is a kip for a map > >> serde. Of course, you’re welcome to start one. > >> > >> Thanks, > >> John > >> > >> On Mon, May 11, 2020, at 09:14, John Roesler wrote: > >> > Hi Pushkar, > >> > > >> > I don’t think there is. You’re welcome to start one if you think it > >> > would be a useful addition. > >> > > >> > Before worrying about it further, though, you might want to check the > >> > InMemoryKeyValueStore implementation, since my answer was from memory. > >> > > >> > Thanks, > >> > John > >> > > >> > On Mon, May 11, 2020, at 03:47, Pushkar Deole wrote: > >> > > John, > >> > > is there KIP in progress for supporting Java HashMap also? > >> > > > >> > > On Sun, May 10, 2020, 00:47 John Roesler <vvcep...@apache.org> > wrote: > >> > > > >> > > > Yes, that’s correct. It’s only for serializing the java type > >> ‘byte[]’. > >> > > > > >> > > > On Thu, May 7, 2020, at 10:37, Pushkar Deole wrote: > >> > > > > Thanks John... I got to finish the work in few days so need to > >> get it > >> > > > > quick, so looking for something ready. I will take a look at > >> jackson > >> > > > json. > >> > > > > > >> > > > > By the way, what is the byteArrayserializer? As the name > >> suggests, it is > >> > > > > for byte arrays so won't work for java ArrayList, right? > >> > > > > > >> > > > > On Thu, May 7, 2020 at 8:44 PM John Roesler < > vvcep...@apache.org> > >> wrote: > >> > > > > > >> > > > > > Hi Pushkar, > >> > > > > > > >> > > > > > If you’re not too concerned about compactness, I think Jackson > >> json > >> > > > > > serialization is the easiest way to serialize complex types. > >> > > > > > > >> > > > > > There’s also a kip in progress to add a list serde. You might > >> take a > >> > > > look > >> > > > > > at that proposal for ideas to write your own. > >> > > > > > > >> > > > > > Thanks, > >> > > > > > John > >> > > > > > > >> > > > > > On Thu, May 7, 2020, at 08:17, Nicolas Carlot wrote: > >> > > > > > > Won't say it's a good idea to use java serialized classes > for > >> > > > messages, > >> > > > > > but > >> > > > > > > you should use a byteArraySerializer if you want to do such > >> things > >> > > > > > > > >> > > > > > > Le jeu. 7 mai 2020 à 14:32, Pushkar Deole < > >> pdeole2...@gmail.com> a > >> > > > > > écrit : > >> > > > > > > > >> > > > > > > > Hi All, > >> > > > > > > > > >> > > > > > > > I have a requirement to store a record with key as java > >> String and > >> > > > > > value as > >> > > > > > > > java's ArrayList in the kafka topic. Kafka has by default > >> provided > >> > > > a > >> > > > > > > > StringSerializer and StringDeserializer, however for java > >> > > > ArrayList, > >> > > > > > how > >> > > > > > > > can get serializer. Do I need to write my own? Can someone > >> share if > >> > > > > > someone > >> > > > > > > > already has written one? > >> > > > > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > -- > >> > > > > > > *Nicolas Carlot* > >> > > > > > > > >> > > > > > > Lead dev > >> > > > > > > | | nicolas.car...@chronopost.fr > >> > > > > > > > >> > > > > > > > >> > > > > > > *Veuillez noter qu'à partir du 20 mai, le siège Chronopost > >> déménage. > >> > > > La > >> > > > > > > nouvelle adresse est : 3 boulevard Romain Rolland 75014 > Paris* > >> > > > > > > > >> > > > > > > [image: Logo Chronopost] > >> > > > > > > | chronopost.fr <http://www.chronopost.fr/> > >> > > > > > > Suivez nous sur Facebook < > >> https://fr-fr.facebook.com/chronopost> et > >> > > > > > Twitter > >> > > > > > > <https://twitter.com/chronopost>. > >> > > > > > > > >> > > > > > > [image: DPD Group] > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > > >
