Hi Timo,
Thanks for the FLIP, I also like the idea and option 3 sounds good to me.
I would like to discuss a case which is not mentioned in the current FLIP.
How are the "System column"s expressed in intermediate result, e.g. Join?
E.g. `SELECT * FROM t1 JOIN t2`, I guess it should not include "system
columns" from t1 and t2 as you proposed, and for `SELECT t1.$rowtime, *
FROM t1 JOIN t2`, it should also be valid.
Then the question is how to you plan to implement the "system columns", do
we need to add it to `RelNode` level? Or we just need to do it in the
parsing/validating phase?
I'm not sure that Calcite's "system column" feature is fully ready for this
since the code about this part is imported from the earlier project before
it gets into Apache, and has not been considered much in the past
development.
Jing Ge <j...@ververica.com.invalid> 于2023年7月26日周三 00:01写道：
> Hi Timo,
>
> Thanks for your proposal. It is a very pragmatic feature. Among all options
> in the FLIP, option 3 is one I prefer too and I'd like to ask some
> questions to understand your thoughts.
>
> 1. I did some research on pseudo columns, just out of curiosity, do you
> know why most SQL systems do not need any prefix with their pseudo column?
> 2. Some platform providers will use ${variable_name} to define their own
> configurations and allow them to be embedded into SQL scripts. Will there
> be any conflict with option 3?
>
> Best regards,
> Jing
>
> On Tue, Jul 25, 2023 at 7:00 PM Konstantin Knauf <kna...@apache.org>
> wrote:
>
> > Hi Timo,
> >
> > this makes sense to me. Option 3 seems reasonable, too.
> >
> > Cheers,
> >
> > Konstantin
> >
> > Am Di., 25. Juli 2023 um 12:53 Uhr schrieb Timo Walther <
> > twal...@apache.org
> > >:
> >
> > > Hi everyone,
> > >
> > > I would like to start a discussion about introducing the concept of
> > > "System Columns" in SQL and Table API.
> > >
> > > The subject sounds bigger than it actually is. Luckily, Flink SQL
> > > already exposes the concept of metadata columns. And this proposal is
> > > just a slight adjustment for how metadata columns can be used as system
> > > columns.
> > >
> > > The biggest problem of metadata columns currently is that a catalog
> > > implementation can't provide them by default because they would affect
> > > `SELECT *` when adding another one.
> > >
> > > Looking forward to your feedback on FLIP-348:
> > >
> > >
> > >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-348%3A+Support+System+Columns+in+SQL+and+Table+API
> > >
> > > Thanks,
> > > Timo
> > >
> >
> >
> > --
> > https://twitter.com/snntrable
> > https://github.com/knaufk
> >
>
--
Best,
Benchao Li
