Re: Flink SQL - Join Lookup Table

2020-07-21 Thread Jark Wu
the workaround you suggested in the short term. > > > > Thanks! > > Kelly > > > > *From: *Leonard Xu > *Date: *Monday, July 20, 2020 at 7:49 PM > *To: *Danny Chan > *Cc: *Kelly Smith , Flink ML < > user@flink.apache.org> > *Subject: *Re: Flink SQL - Join L

Re: Flink SQL - Join Lookup Table

2020-07-21 Thread Jingsong Li
n, and look into > the workaround you suggested in the short term. > > > > Thanks! > > Kelly > > > > *From: *Leonard Xu > *Date: *Monday, July 20, 2020 at 7:49 PM > *To: *Danny Chan > *Cc: *Kelly Smith , Flink ML < > user@flink.apache.org> > *Su

Re: Flink SQL - Join Lookup Table

2020-07-21 Thread Kelly Smith
suggested in the short term. Thanks! Kelly From: Leonard Xu Date: Monday, July 20, 2020 at 7:49 PM To: Danny Chan Cc: Kelly Smith , Flink ML Subject: Re: Flink SQL - Join Lookup Table Hi, kelly Looks like you want to use fact table(from Kafka) to join a dimension table(From filesystem

Re: Flink SQL - Join Lookup Table

2020-07-20 Thread Leonard Xu
Hi, kelly Looks like you want to use fact table(from Kafka) to join a dimension table(From filesystem), dimension table is one kind of Temporal Table, temporal table join syntax you could refer Danny's post[1]. But `FileSystemTableSource` did not implement `LookupTableSource` interface yet wh

Re: Flink SQL - Join Lookup Table

2020-07-20 Thread Danny Chan
Seems you want a temporal table join instead of a two stream join, if that is your request, you should use syntax Join LookupTable FOR SYSTEM_TIME AS OF … See [1] for details. [1]  https://ci.apache.org/projects/flink/flink-docs-master/dev/table/streaming/joins.html#join-with-a-temporal-table

Re: Flink SQL - Join Lookup Table

2020-07-20 Thread godfrey he
JIRA: https://issues.apache.org/jira/browse/FLINK-18651 godfrey he 于2020年7月21日周二 上午9:46写道: > hi Kelly, > As the exception message mentioned: currently, we must cast the time > attribute to regular TIMESTAMP type, > then we can do regular join. Because time attribute will be out-of-order > after

Re: Flink SQL - Join Lookup Table

2020-07-20 Thread godfrey he
hi Kelly, As the exception message mentioned: currently, we must cast the time attribute to regular TIMESTAMP type, then we can do regular join. Because time attribute will be out-of-order after regular join, and then we can't do window aggregate based on the time attribute. We can improve it tha

Flink SQL - Join Lookup Table

2020-07-20 Thread Kelly Smith
Hi folks, I have a question Flink SQL. What I want to do is this: * Join a simple lookup table (a few rows) to a stream of data to enrich the stream by adding a column from the lookup table. For example, a simple lookup table: CREATE TABLE LookupTable ( `computeClass` STRING, `m