Re: Add Bucket File System Table Sink

2019-09-20 Thread Jun Zhang
Hi??Fabian ?? Thank you very much for your suggestion. This is when I use flink sql to write data to hdfs at work. I feel that it is inconvenient. I wrote this function, and then I want to contribute it to the community. This is my first PR , some processes may not be clear, I am very sorry.

Re: Add Bucket File System Table Sink

2019-09-20 Thread Fabian Hueske
Hi Jun, Thank you very much for your contribution. I think a Bucketing File System Table Sink would be a great addition. Our code contribution guidelines [1] recommend to discuss the design with the community before opening a PR. First of all, this ensures that the design is aligned with Flink's

Re: Add Bucket File System Table Sink

2019-09-16 Thread Kurt Young
3 and see if there is a better solution to > combine these two functions. I am very willing to join this development. > > > > -- 原始邮件 -- > *发件人:* "Kurt Young"; > *发送时间:* 2019年9月17日(星期二) 中午11:19 > *收件人:* "Jun Zhang"<825875..

?????? Add Bucket File System Table Sink

2019-09-16 Thread Jun Zhang
Hi Kurt: Thanks. When I encountered this problem, I found a File System Connector, but its function is not powerful enough and rich. I also found that it is built into Flink, there are many unit tests that refer to it, so I dare not easily modify it to enrich its functions

Re: Add Bucket File System Table Sink

2019-09-16 Thread Kurt Young
Thanks. Let me clarify a bit more about my thinkings. Generally, I would prefer we can concentrate the functionalities about connector, especially some standard & most popular connectors, like kafka, different file system with different formats, etc. We should make these core connectors as powerful

Re: Add Bucket File System Table Sink

2019-09-16 Thread Kurt Young
Hi Jun, Thanks for bringing this up, in general I'm +1 on this feature. As you might know, there is another ongoing efforts about such kind of table sink, which covered in newly proposed partition support reworking[1]. In this proposal, we also want to introduce a new file system connector, which

Add Bucket File System Table Sink

2019-09-16 Thread Jun Zhang
Hello everyone: I am a user and fan of flink. I also want to join the flink community. I contributed my first PR a few days ago. Can anyone help me to review my code? If there is something wrong, hope I would be grateful if you can give some advice. This PR is mainly in the proce