Re: Add Bucket File System Table Sink

2019-09-21 Thread Jun Zhang
Hi, Kurt      Thank you very much for your suggestion.      I have studied the design documentation of FLIP-63. I have no comments here.      With your suggestion from the previous few days, I am writing a design documentation about the Bucket FileSystem Table Sink and considering how to support

Re: Add Bucket File System Table Sink

2019-09-20 Thread Kurt Young
- > *From:* Fabian Hueske > *Date:* Fri,Sep 20,2019 9:38 PM > *To:* Jun Zhang <825875...@qq.com> > *Cc:* dev , user > *Subject:* Re: Add Bucket File System Table Sink > > Hi Jun, > > Thank you very much for your contribution. > > I think a Bucketing F

Re: Add Bucket File System Table Sink

2019-09-20 Thread Jun Zhang
Hi??Fabian ?? Thank you very much for your suggestion. This is when I use flink sql to write data to hdfs at work. I feel that it is inconvenient. I wrote this function, and then I want to contribute it to the community. This is my first PR , some processes may not be clear, I am very sorry.

Re: Add Bucket File System Table Sink

2019-09-20 Thread Fabian Hueske
Hi Jun, Thank you very much for your contribution. I think a Bucketing File System Table Sink would be a great addition. Our code contribution guidelines [1] recommend to discuss the design with the community before opening a PR. First of all, this ensures that the design is aligned with Flink's

Re: Add Bucket File System Table Sink

2019-09-16 Thread Kurt Young
3 and see if there is a better solution to > combine these two functions. I am very willing to join this development. > > > > -- 原始邮件 -- > *发件人:* "Kurt Young"; > *发送时间:* 2019年9月17日(星期二) 中午11:19 > *收件人:* "Jun Zhang"<825875..

Re: Add Bucket File System Table Sink

2019-09-16 Thread Kurt Young
Thanks. Let me clarify a bit more about my thinkings. Generally, I would prefer we can concentrate the functionalities about connector, especially some standard & most popular connectors, like kafka, different file system with different formats, etc. We should make these core connectors as powerful

Re: Add Bucket File System Table Sink

2019-09-16 Thread Kurt Young
Hi Jun, Thanks for bringing this up, in general I'm +1 on this feature. As you might know, there is another ongoing efforts about such kind of table sink, which covered in newly proposed partition support reworking[1]. In this proposal, we also want to introduce a new file system connector, which