Re: SplitEnumerator for Bigquery Source.

2022-10-18 Thread Lavkesh Lahngir
egards, > > Yuxia > > > > - 原始邮件 - > > 发件人: "Lavkesh Lahngir" > > 收件人: "dev" > > 发送时间: 星期一, 2022年 10 月 17日 下午 10:42:29 > > 主题: SplitEnumerator for Bigquery Source. > > > > Hii Everybody, > > we are trying to impl

Re: SplitEnumerator for Bigquery Source.

2022-10-18 Thread Martijn Visser
teing splits . > option2 seems isnot flexible and is too coarse-grained. > option4 need extrac efforts to wrting the data again. > > Best regards, > Yuxia > > - 原始邮件 - > 发件人: "Lavkesh Lahngir" > 收件人: "dev" > 发送时间: 星期一, 2022年 10 月 17日 下午 10:42:

Re: SplitEnumerator for Bigquery Source.

2022-10-17 Thread yuxia
need extrac efforts to wrting the data again. Best regards, Yuxia - 原始邮件 - 发件人: "Lavkesh Lahngir" 收件人: "dev" 发送时间: 星期一, 2022年 10 月 17日 下午 10:42:29 主题: SplitEnumerator for Bigquery Source. Hii Everybody, we are trying to implement a google bigquery source on flink. W

SplitEnumerator for Bigquery Source.

2022-10-17 Thread Lavkesh Lahngir
Hii Everybody, we are trying to implement a google bigquery source on flink. We were thinking of taking time partition and column information as config. I was thinking of how to parallelize the source and how to generate splits. I read the code of Hive source, where we could generate hadoop file sp