Zhanghao is correct. You can use what is called "keyed state". It's like a cache.
https://nightlies.apache.org/flink/flink-docs-release-1.19/docs/dev/datastream/fault-tolerance/state/ > On Mar 28, 2024, at 7:48 PM, Zhanghao Chen <zhanghao.c...@outlook.com> wrote: > > Hi, > > You can maintain a cache manually in your operator implementations. You can > load the initial cached data on the operator open() method before the > processing starts. However, this would set up a cache per task instance. If > you'd like to have a cache shared by all processing tasks without > duplication, you might set up a Redis service externally for that purpose. > > Best, > Zhanghao Chen > From: Ganesh Walse <ganesh.wa...@gmail.com> > Sent: Friday, March 29, 2024 4:45 > To: user@flink.apache.org <user@flink.apache.org> > Subject: Flink cache support > > Hi Team, > > In my project my requirement is to cache data from the oracle database where > the number of tables are more and the same data will be required for all the > transactions to process. > > Can you please suggest the approach where cache should be 1st loaded in > memory then stream processing should start. > > Thanks & regards, > Ganesh Walse.
