>
> `SELECT * FROM (SELECT $rowtime, * FROM t);`
> Am I right that it will show `$rowtime` in output ?
Yes, all explicitly selected columns become a part of the result (and
intermediate) schema, and hence propagate.
On Thu, Jul 27, 2023 at 2:40 PM Alexey Leonov-Vendrovskiy <
vendrov...@gmail.com> wrote:
> Thank you, Timo, for starting this FLIP!
>
> I propose the following change:
>
> Remove the requirement that DESCRIBE need to show system columns.
>
>
> Some concrete vendor specific catalog implementations might prefer this
> approach.
> Usually the same system columns are available on all (or family) of
> tables, and it can be easily captured in the documentation.
>
> For example, BigQuery does exactly this: there, pseudo-columns do not show
> up in the table schema in any place, but can be accessed via reference.
>
> So I propose we:
> a) Either we say that DESCRIBE doesn't show system columns,
> b) Or leave this vendor-specific / or configurable via flag (if needed).
>
> Regards,
> Alexey
>
> On Thu, Jul 27, 2023 at 3:27 AM Sergey Nuyanzin <snuyan...@gmail.com>
> wrote:
>
>> Hi Timo,
>>
>> Thanks for the FLIP.
>> I also tend to think that Option 3 is better.
>>
>> I would be also interested in a question mentioned by Benchao Li.
>> And a similar question about nested queries like
>> `SELECT * FROM (SELECT $rowtime, * FROM t);`
>> Am I right that it will show `$rowtime` in output ?
>>
>>
>> On Thu, Jul 27, 2023 at 6:58 AM Benchao Li <libenc...@apache.org> wrote:
>>
>> > Hi Timo,
>> >
>> > Thanks for the FLIP, I also like the idea and option 3 sounds good to
>> me.
>> >
>> > I would like to discuss a case which is not mentioned in the current
>> FLIP.
>> > How are the "System column"s expressed in intermediate result, e.g.
>> Join?
>> > E.g. `SELECT * FROM t1 JOIN t2`, I guess it should not include "system
>> > columns" from t1 and t2 as you proposed, and for `SELECT t1.$rowtime, *
>> > FROM t1 JOIN t2`, it should also be valid.
>> > Then the question is how to you plan to implement the "system columns",
>> do
>> > we need to add it to `RelNode` level? Or we just need to do it in the
>> > parsing/validating phase?
>> > I'm not sure that Calcite's "system column" feature is fully ready for
>> this
>> > since the code about this part is imported from the earlier project
>> before
>> > it gets into Apache, and has not been considered much in the past
>> > development.
>> >
>> >
>> > Jing Ge <j...@ververica.com.invalid> 于2023年7月26日周三 00:01写道：
>> >
>> > > Hi Timo,
>> > >
>> > > Thanks for your proposal. It is a very pragmatic feature. Among all
>> > options
>> > > in the FLIP, option 3 is one I prefer too and I'd like to ask some
>> > > questions to understand your thoughts.
>> > >
>> > > 1. I did some research on pseudo columns, just out of curiosity, do
>> you
>> > > know why most SQL systems do not need any prefix with their pseudo
>> > column?
>> > > 2. Some platform providers will use ${variable_name} to define their
>> own
>> > > configurations and allow them to be embedded into SQL scripts. Will
>> there
>> > > be any conflict with option 3?
>> > >
>> > > Best regards,
>> > > Jing
>> > >
>> > > On Tue, Jul 25, 2023 at 7:00 PM Konstantin Knauf <kna...@apache.org>
>> > > wrote:
>> > >
>> > > > Hi Timo,
>> > > >
>> > > > this makes sense to me. Option 3 seems reasonable, too.
>> > > >
>> > > > Cheers,
>> > > >
>> > > > Konstantin
>> > > >
>> > > > Am Di., 25. Juli 2023 um 12:53 Uhr schrieb Timo Walther <
>> > > > twal...@apache.org
>> > > > >:
>> > > >
>> > > > > Hi everyone,
>> > > > >
>> > > > > I would like to start a discussion about introducing the concept
>> of
>> > > > > "System Columns" in SQL and Table API.
>> > > > >
>> > > > > The subject sounds bigger than it actually is. Luckily, Flink SQL
>> > > > > already exposes the concept of metadata columns. And this
>> proposal is
>> > > > > just a slight adjustment for how metadata columns can be used as
>> > system
>> > > > > columns.
>> > > > >
>> > > > > The biggest problem of metadata columns currently is that a
>> catalog
>> > > > > implementation can't provide them by default because they would
>> > affect
>> > > > > `SELECT *` when adding another one.
>> > > > >
>> > > > > Looking forward to your feedback on FLIP-348:
>> > > > >
>> > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/FLINK/FLIP-348%3A+Support+System+Columns+in+SQL+and+Table+API
>> > > > >
>> > > > > Thanks,
>> > > > > Timo
>> > > > >
>> > > >
>> > > >
>> > > > --
>> > > > https://twitter.com/snntrable
>> > > > https://github.com/knaufk
>> > > >
>> > >
>> >
>> >
>> > --
>> >
>> > Best,
>> > Benchao Li
>> >
>>
>>
>> --
>> Best regards,
>> Sergey
>>
>
