Hello Mark, The Kafka message format starts with a magic byte indicating what kind of serialization is used for this message. And if this byte indicates Avro, you can layout your message as starting with the schemaId and then followed by message payload. Upon consumption, you can first get the schemaId, query Avro for the schema given the id, and then use schema to deserialize the message.
Guozhang On Tue, Aug 20, 2013 at 7:08 AM, Mark <static.void....@gmail.com> wrote: > Can someone break down how message serialization would work with Avro? > I've read instead of adding a schema to every single event it would be wise > to add some sort of fingerprint with each message to identify which schema > it should used. What I'm having trouble understanding is, how do we read > the fingerprint without a schema? Don't we need the schema to deserialize? > Same question goes for working with Hadoop.. how does the input format > know which schema to use? > > Thanks -- -- Guozhang
