Hi, I've written a SerDe and I'd like it to be able handle compressed data (gzip). Hadoop detects and decompresses on the fly so if you have a compressed data set and you don't need to perform any custom interpretation of it as you go, hadoop and hive will handle it. Is there a way to get Hive to notice the data is compressed, decompress it then push it through the custom SerDe? Or will I have to either a. add some decompression logic to my SerDe (possibly impossible) b. decompress the data before pushing it into a table with my SerDe
Thanks! Pat
