It's possible to do that but not sure the community has any ready-to-be-used code sample to share.
As for already available solutions, GridGain provides so-called Hadoop Connector that solves data loading task: https://docs.gridgain.com/docs/bdb-getting-started#section-gridgain-hadoop-connector - Denis On Tue, Jun 4, 2019 at 1:49 PM Edward Chen <[email protected]> wrote: > Hi, I am trying to load Hdfs data into cache from Spark. it is working > in local mode, but failed in spark-submit yarn mode. it tried to find > Ignite Home path in the cluster. yes, it is true that Ignite is not > installed in cluster. but why needed ? Ignite instance is created inside > my java code and all of ignite jars are packaged. why need to find > ignite info outside of my jar. > > My ignite version is 2.3. > > Anyone has a working example of Spark submit + Yarn mode ? any web link > is welcome. > > Thanks. Edward > >
