Just an example. I mean to say a part of my data resides in Ignite, not the complete data.
Rest data is present in Spark. On Wed, 19 Jul, 2023, 2:07 pm Stephen Darlington, < stephen.darling...@gridgain.com> wrote: > Why you would have Ignite, a horizontally scalable, in-memory database, to > store 100 records? > > On 19 Jul 2023, at 04:37, Arunima Barik <arunimabari...@gmail.com> wrote: > > I have a huge dataset and I am keeping few (say 100) rows in Ignite and > the entire dataset remains in Spark > > When I query Ignite I want to write an Sql query to perform the same. > > Does option 1 still hold good? > > On Tue, 18 Jul, 2023, 10:40 pm Stephen Darlington, < > stephen.darling...@gridgain.com> wrote: > >> “Correct” is hard to quantify without knowing your use case, but option 1 >> is probably what you want. Spark pushes down SQL execution to Ignite, so >> you get all the distribution, use of indexes, etc. >> >> > On 14 Jul 2023, at 16:12, Arunima Barik <arunimabari...@gmail.com> >> wrote: >> > >> > Hello team >> > >> > What is the correct way out of these? >> > >> > 1. Write a spark dataframe to ignite >> > Read the same back and perform spark.sql() on that >> > >> > 2. Write the spark dataframe to ignite >> > Connect to server via a thin client >> > Perform client.sql() >> > >> > Regards >> > Arunima >> >> >
