Re: Greetings and question

2017-07-17 Thread Fabian Hueske
Hi Robert, I don't think anybody is working on a ORC file sink. Are you interested in a sink for data streams or a batch sink? Implementing a batch sink shouldn't be very hard. You can either implement an OutputFormat the internally uses the ORC Java API or you try to use Flink's HadoopOutputForm

Re: Greetings and question

2017-07-16 Thread Tzu-Li (Gordon) Tai
Hi Robert, Thanks for your interest in contributing that. AFAIK, I don’t think there’s any ongoing efforts yet in an ORC table sink. I’ll loop in Fabian (CC'ed) who might know more about this. The only complicated consideration in designing sinks is to consider the delivery guarantees it will pr

Greetings and question

2017-07-16 Thread Robert Rapplean
Hey, everyone. I have a need for Flink to write to ORCFile tables in the near future. Could someone educate me on the current challenges that might make that hard to do? I've worked quite a bit with the HCat libraries, and may be overconfident about how complicated this is. Is anyone currently wor