A more common approach would be that Ignite has all your data and Spark has a subset. Ignite SQL is generally faster than Spark SQL, since it can use indexes, etc. But it’s not magic; it can’t query data it doesn’t know about.
> On 19 Jul 2023, at 04:40, Arunima Barik <arunimabari...@gmail.com> wrote: > > How does write through work? I mean if I add a row in Ignite dataframe, how > does it reflect to spark? > > I have 50 rows in Ignite and all 100 rows in Spark. > If I perform a union all, wont the performance degrade? > I mean if will get slower than just querying spark > > On Tue, 18 Jul, 2023, 10:43 pm Stephen Darlington, > <stephen.darling...@gridgain.com <mailto:stephen.darling...@gridgain.com>> > wrote: >> Write through works regardless of how you insert data into Ignite. >> >> I’m not clear what you mean by federated query. Are the records in Spark a >> subset of those in the cache? >> >> Assuming not, create a data frame with a SQL query against Ignite. Create a >> data frame with a SQL query against your Spark data frame. Union together. >> >> > On 13 Jul 2023, at 08:27, Arunima Barik <arunimabari...@gmail.com >> > <mailto:arunimabari...@gmail.com>> wrote: >> > >> > Hello Team >> > >> > I want to build a read write through Ignite cache over Spark >> > >> > If I have 50 rows in the cache and entire 100 row data in spark then how >> > can I use federated queries? >> > >> > Also, how to perform write through using Sql queries? >> > >> > Regards >> > Arunima >>
