Re: [DISCUSS] FLIP-226: Introduce Schema Evolution on Table Store

2022-05-12 Thread Jingsong Li
Thank Jark~ Hi all, I just created a vote thread [1]. Feel free to -1 if you think there is something wrong with the design. [1] https://lists.apache.org/thread/lg5txz95mgko4mp6fqcwt1dd1hbjctjy Best, Jingsong On Thu, May 12, 2022 at 4:14 PM Jark Wu wrote: > Thank Jingsong for the explanation

Re: [DISCUSS] FLIP-226: Introduce Schema Evolution on Table Store

2022-05-12 Thread Jark Wu
Thank Jingsong for the explanation. I don't have other concerns. Best, Jark On Thu, 12 May 2022 at 09:53, Jingsong Li wrote: > Hi all, > > If there are no more comments, I'm going to start a vote. > > Best, > Jingsong > > On Tue, May 10, 2022 at 10:37 AM Jingsong Li > wrote: > > > Hi Jark, > >

Re: [DISCUSS] FLIP-226: Introduce Schema Evolution on Table Store

2022-05-11 Thread Jingsong Li
Hi all, If there are no more comments, I'm going to start a vote. Best, Jingsong On Tue, May 10, 2022 at 10:37 AM Jingsong Li wrote: > Hi Jark, > > Thanks for your feedback. > > > 1) Does table-store support evolve schemas multiple times during a > checkpoint? > > In this case this checkpoint

Re: [DISCUSS] FLIP-226: Introduce Schema Evolution on Table Store

2022-05-09 Thread Jingsong Li
Hi Jark, Thanks for your feedback. > 1) Does table-store support evolve schemas multiple times during a checkpoint? In this case this checkpoint is split into multiple commits, e.g.: - commit1: write 1 million rows - commit1: write 1 million rows - commit2: evolve mode 1 - commit3: write 1 milli

Re: [DISCUSS] FLIP-226: Introduce Schema Evolution on Table Store

2022-05-09 Thread Jark Wu
Thanks for proposing this exciting feature, Jingsong! I only have a few questions: 1) Does table-store support evolve schemas multiple times during a checkpoint? For example, cp1 -> write 1M rows (may flush file store) -> evolve schema1 -> write 1M rows (may flush file store again) -> evolve sche

Re: [DISCUSS] FLIP-226: Introduce Schema Evolution on Table Store

2022-05-09 Thread Caizhi Weng
Hi all! +1 for this FLIP. By adding schema information into data files we can not only support schema evolution, which is a very useful feature for data storages, but also make it easier for table store to integrate with other systems. For example timestamp type in Hive does not support precision