I did a Samza proof of concept project recently and I ended up writing this code:
https://gist.github.com/ldcasillas-progreso/871af3c1a1790be975fd In the end, however, I switched the project from Avro to JSON. The issue is that Avro is designed to work with its self-describing container file format, which embeds the schema used to write the records in the file. Avro’s schema evolution features rely on this embedded schema; when the embedded schema and the reader’s schema are not equal, Avro uses its special rules to translate the old data to the new schema. But when you’re working with Kafka/Samza, there is no container file. Therefore, none of the schema evolution tools work. Therefore, if you change your Avro schema, you likely won’t be able to read any of the old messages again. There’s a Kafka Avro schema registry project that aims to fix this: https://github.com/confluentinc/schema-registry I tried it but the released version just was not mature enough—which is why I ended up using JSON. But I did write a Serde that encodes/decodes the Avro objects in JSON: https://gist.github.com/ldcasillas-progreso/3611d40d2833aa62c1b3 Hope this helps. On 11/17/15, 12:32 AM, "Selina Tech" <swucaree...@gmail.com> wrote: >Dear All: > Do you know where I can find the tutorial or sample code for writing >Avro type message to Kafka and reading Avro type message from Kafka in >Samza? > I am wondering how should I serialized GenericRecord to byte and >deserialized it? > Your comments/suggestion are highly appreciated. > >Sincerely, >Selina ----------- This message and any files or text attached to it are intended only for the recipients named above, and contain information that is confidential or privileged. If you are not an intended recipient, you must not read, copy, use or disclose this communication. Please also notify the sender by replying to this message, and then delete all copies of it from your system. Este mensaje y cualquier archivo o texto adjunto es dirigido solamente a los destinatarios especificados en el encabezado y contiene información confidencial y/o privilegiada. Si usted no es el destinatario no deberá leer, copiar, usar o divulgar el contenido. Por favor notifique al remitente, respondiendo a esté mensaje y elimine todas las copias del mismo de su sistema.
