Re: Accessing columns from input stream table during Window operations

Thanks Dian, Guowei. I think it makes sense to roll with this approach. On Tue, Apr 20, 2021 at 8:29 AM Guowei Ma wrote: > Hi, Sumeet > Thanks you for the sharing. As Dian suggested, I think you could use b as > your `group_by`'s key and so the b could be output directly. > I think it is more si

Re: Accessing columns from input stream table during Window operations

Hi, Sumeet Thanks you for the sharing. As Dian suggested, I think you could use b as your `group_by`'s key and so the b could be output directly. I think it is more simple. Best, Guowei On Mon, Apr 19, 2021 at 7:31 PM Dian Fu wrote: > Hi Sumeet, > > Thanks for the sharing. > > Then I guess you

Re: Accessing columns from input stream table during Window operations

Hi Sumeet, Thanks for the sharing. Then I guess you could use `.group_by(col('w'), input.a, input.b)`. Since the value for input.a is always the same, it’s equal to group_by(col(‘w'), input.b) logically. The benefit is that you could access input.a directly in the select clause. Regards, Dian

Re: Accessing columns from input stream table during Window operations

Hi Guowei, Let me elaborate the use case with an example. Sample input table looks like this: timea b c - t0 a0 b0 1 t1 a0 b1 2 t2 a0 b2 3 t3 a0 b0 6 t4 a0 b1 7 t5 a0 b2 8 Basically, every time interval there are new readings fro

Re: Accessing columns from input stream table during Window operations

Hi Sumeet, 1) Regarding to the above exception, it’s a known issue and has been fixed in FLINK-21922 [1]. It will be available in the coming 1.12.3. You could also cherry-pick that fix to 1.12.2 and build from source following the instruction

Re: Accessing columns from input stream table during Window operations

Thanks Guowei. I'm trying out Over Windows, as follows: input \ .over_window( Over.partition_by(col(input.a)) \ .order_by(input.Timestamp) \ .preceding(lit(10).seconds) \ .alias('w')) \ .select( input.b, input.c.avg.over(col('w'))) \ .exe

Re: Accessing columns from input stream table during Window operations

Hi, Sumeet Maybe you could try the Over Windows[1], which could keep the "non-group-key" column. [1] https://ci.apache.org/projects/flink/flink-docs-release-1.12/dev/table/tableApi.html#over-windows Best, Guowei On Mon, Apr 19, 2021 at 3:25 PM Sumeet Malhotra wrote: > Thanks Guowei! Regardin

Re: Accessing columns from input stream table during Window operations

Thanks Guowei! Regarding "input.c.avg" you're right, that doesn't cause any issues. It's only when I want to use "input.b". My use case is to basically emit "input.b" in the final sink as is, and not really perform any aggregation on that column - more like pass through from input to sink. What's

Re: Accessing columns from input stream table during Window operations

Hi, Sumeet For "input.b" I think you should aggregate the non-group-key column[1]. But I am not sure why the "input.c.avg.alias('avg_value')" has resolved errors. Would you mind giving more detailed error information? [1] https://ci.apache.org/projects/flink/flink-docs-release-1.12/dev/tabl