Thank you all for the lightning fast replies. @Yin: I am actually passing SerDe properties and they are working fine. Actually, this is how I pass the the full namespace of the class I want to use. I receive the Properties object fine, it's the Configuration one that is null. :(
@Hari: Perfect Hari, I think this is exactly what I need. @Edward: Thanks for the suggestion. I have seen that and I think twitter also has one called Elephant Bird but neither supports all the functionalities I need. I've implement support both Deserializer and Serializer for Protocol Buffers (the former works flawlessly, the latter is the one with the problem). In my implementation I also support other data format types (apart from Protocol Buffers), and I also implemented new Input and Output file formats. Hopefully I'll be able to opensource this soon. On Mon, Oct 14, 2013 at 6:06 PM, Edward Capriolo <edlinuxg...@gmail.com>wrote: > Have you seen? > > https://github.com/edwardcapriolo/hive-protobuf/ > > > On Mon, Oct 14, 2013 at 8:16 AM, Rui Martins <ru...@ruibm.com> wrote: > >> Hi hive users, >> >> I am writing a custom SerDe that loads any protocol buffer generated >> class. >> For flexibility this class can live in a jar external to the SerDe's jar >> and then I just use the Hive Configuration class passed in the initiliaze >> to dynamically load it and set the schema for the Hive table. >> >> >> http://grepcode.com/file/repository.cloudera.com/content/repositories/releases/org.apache.hadoop.hive/hive-serde/0.7.0-cdh3u0/org/apache/hadoop/hive/serde2/Serializer.java#Serializer >> >> When I use my custom SerDe as a Deserializer it all works well, I get a >> Configuration and I correctly load the ProtoBuffer class from the external >> Jar. >> >> However, when I use the SerDe as a Serializer, the Configuration is >> always set to null so I have no way of loading the external class from the >> Jar. >> >> My questions are: >> >> * 1) Is the initialize(..) method in Serializer supposed to always pass >> a null Configuration?* >> * >> * >> * 2) Is there a way of creating or retrieving the current Hadoop/Hive >> Configuration when this parameter is passed as null?* >> * >> * >> >> Thank you, >> rui >> > >
