Hi kant, In my understanding, "treat batch as a special case of streaming" just means runtime implementation. So currently, we cannot consume a database and it's changes as a stream. But You can put the change log into massage queue system like Kafka using a CDC system like Maxwell or Canal. Then you can consume it as a stream.
About the joining, maybe you can try Dimension Join[1] in blink. Personally, I think reading from a database and it's continuous changes as a stream is a good idea, and I see some other users requesting this feature as well. kant kodali <kanth...@gmail.com> 于2020年1月12日周日 下午8:07写道： > Hi, > > Thanks for the response! The Documentation says Blink will treat batch as > a special case of streaming. if so, can I stream from a database? or put it > in another way. If I have two streams A & B where Stream A is writing to > Database and Stream B reading from the database. will stream B keeps > running forever picking up all the changes written to the database by > Stream A? If not, is there any to achieve that sort of functionality? > > The reason I may want to do something like this is I may want to do join > on two tables that continuously get populated however I cannot specify the > time constraint on the state which is why I would want to write to external > database such that I can treat it as an infinite store and then do join as > the data come in continuously. > > Thanks! > > > > > On Sat, Jan 11, 2020 at 6:42 PM Benchao Li <libenc...@gmail.com> wrote: > >> Hi kant, >> >> Blink has merged most of it's functionality into 1.9-release, and we have >> two planners [1] in 1.9: >> >> - Blink Planner >> - Old Planner (Legacy Planner) >> >> You can try out blink planner by [2]. >> Hope this helps. >> >> [1] >> https://ci.apache.org/projects/flink/flink-docs-release-1.9/dev/table/common.html#main-differences-between-the-two-planners >> [2] >> https://ci.apache.org/projects/flink/flink-docs-release-1.9/dev/table/common.html#create-a-tableenvironment >> >> >> kant kodali <kanth...@gmail.com> 于2020年1月12日周日 上午7:48写道： >> >>> Hi All, >>> >>> Are blink changes merged into flink 1.9? It looks like there are a lot >>> of features and optimizations in Blink and if they aren't merged into flink >>> 1.9 I am not sure on which one to use? is there any plan towards merging it? >>> >>> Thanks! >>> >> >> >> -- >> >> Benchao Li >> School of Electronics Engineering and Computer Science, Peking University >> Tel:+86-15650713730 >> Email: libenc...@gmail.com; libenc...@pku.edu.cn >> >> -- Benchao Li School of Electronics Engineering and Computer Science, Peking University Tel:+86-15650713730 Email: libenc...@gmail.com; libenc...@pku.edu.cn
