Re: Enrich stream with SQL api

2021-01-15 Thread Marek Maj
Good point, thanks for the clarification best regards, Marek pt., 15 sty 2021 o 09:32 Dawid Wysakowicz napisaƂ(a): > The LookupTableSource is used when you join based on processing time, as > described in here[1]. Moreover it supports only equi lookups, therefore it > won't work with range quer

Re: Enrich stream with SQL api

2021-01-15 Thread Dawid Wysakowicz
The LookupTableSource is used when you join based on processing time, as described in here[1]. Moreover it supports only equi lookups, therefore it won't work with range queries as in your case. Best, Dawid [1] https://ci.apache.org/projects/flink/flink-docs-release-1.12/dev/table/streaming/join

Re: Enrich stream with SQL api

2021-01-15 Thread Marek Maj
Hi Dawid, thanks for your answers! I guess CDC and Debezium will be the right choice for that case. Iit's integration with oracle however, is in incubating phase as documentation states [1], we will need to investigate further. I was hoping there will be some way to incorporate LookupCache [2] in

Re: Enrich stream with SQL api

2021-01-14 Thread Dawid Wysakowicz
Hi Marek, I am afraid I don't have a good answer for your question. The problem indeed is that the JDBC source can work only as a bounded source. As you correctly pointed out, as of now mixing bounded with unbounded sources does not work with checkpointing, which we want to address in the FLIP-147

Enrich stream with SQL api

2021-01-12 Thread Marek Maj
Hello, I am trying to use Flink SQL api to join two tables. My stream data source is kafka (defined through catalog and schema registry) and my enrichment data is located in relational database (JDBC connector). I think this setup reflects quite common use case Enrichment table definition looks li