Re: Apache Flink - How to use/invoke LookupTableSource/Function

2021-07-08 Thread M Singh
Thanks Jink for such a great explanation and references.   I will follow-up your references to understand the concepts you have explain so well. Mans On Wednesday, July 7, 2021, 11:21:39 PM EDT, JING ZHANG wrote: Hi Mans,Before coming to the next part, we may need some backgrounds abo

Re: Apache Flink - How to use/invoke LookupTableSource/Function

2021-07-07 Thread JING ZHANG
Hi Mans, *Before coming to the next part, we may need some backgrounds about lookup join and temporal join.* 1. Lookup join is typically used to enrich a table with data that is queried from an external system. It requires right table to be backed by a lookup source connector. Its syntax is same wi

Re: Apache Flink - How to use/invoke LookupTableSource/Function

2021-07-07 Thread M Singh
Hi Jing: Thanks for your explanation and references.  I looked at your reference (https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/dev/table/sql/queries/joins/#lookup-join) and have a few question regarding the example code: CREATE TEMPORARY TABLE Customers ( id INT, name

Re: Apache Flink - How to use/invoke LookupTableSource/Function

2021-07-06 Thread JING ZHANG
Hi Mans, `LookupTableSource` used to look up rows from external storage system by given keys, it's very suitable for Key/Value storage system (e.g Redis, HBase), or storage system with key concept (e.g, Mysql, Hive). ` ScanTableSource` is used to scan all rows from an external storage system. Some

Apache Flink - How to use/invoke LookupTableSource/Function

2021-07-06 Thread M Singh
Hey Folks: I am trying to understand how LookupTableSource works and have a few questions:  1. Are there other examples/documentation on how create a query that uses it vs ScanTableSource ?2. Are there any best practices for using this interface ?3. How does the planner decide to use LookupTableS