+1. Sounds good to me. -Jakob
On Mon, Apr 22, 2019 at 9:00 AM Kailash Dayanand <kdayan...@lyft.com.invalid> wrote: > > Friendly remainder. Any thoughts on this approach ? > > On Tue, Apr 9, 2019 at 11:36 AM Kailash Dayanand <kdayan...@lyft.com> wrote: > > > cc'ing few folks who are interested in this discussion. > > > > On Tue, Apr 9, 2019 at 11:35 AM Kailash Dayanand <kdayan...@lyft.com> > > wrote: > > > >> Hello, > >> > >> I am looking to contribute a ProtoParquetWriter support which can be used > >> in Bulk format for the StreamingFileSink api. There has been earlier > >> discussions on this in the user mailing list: https://goo.gl/ya2StL and > >> thought it would be a good addition to have. > >> > >> For implementation, looking at the current APIs present at > >> ProtoParquetWriter with the parguet project (http://tinyurl.com/y378be42), > >> it looks like there is some different in the interface between Avro and > >> Proto writes (ProtoParquetWriter does not have a builder class as well as > >> not interface with Outputfile). Due to this, I was looking at directly > >> extending the ParquetWriter within Flink to define the Builder static class > >> and have newer interfaces. This is needed as the bulk writer takes a > >> builder to crate the ParquetWriter in the bulkWriter.Factory. ( > >> http://tinyurl.com/yyg9cn9b) > >> > >> Any thoughts if this is a reasonable approach? > >> > >> Thanks > >> Kailash > >> > >
