So basically prepend the information in the header of each message? Wouldn't this require changes in all readers/writers?
On Aug 20, 2013, at 9:19 AM, Guozhang Wang <wangg...@gmail.com> wrote: > Hello Mark, > > The Kafka message format starts with a magic byte indicating what kind of > serialization is used for this message. And if this byte indicates Avro, > you can layout your message as starting with the schemaId and then followed > by message payload. Upon consumption, you can first get the schemaId, query > Avro for the schema given the id, and then use schema to deserialize the > message. > > Guozhang > > > On Tue, Aug 20, 2013 at 7:08 AM, Mark <static.void....@gmail.com> wrote: > >> Can someone break down how message serialization would work with Avro? >> I've read instead of adding a schema to every single event it would be wise >> to add some sort of fingerprint with each message to identify which schema >> it should used. What I'm having trouble understanding is, how do we read >> the fingerprint without a schema? Don't we need the schema to deserialize? >> Same question goes for working with Hadoop.. how does the input format >> know which schema to use? >> >> Thanks > > > > > -- > -- Guozhang
