Re: Integrating Spark with Ignite File System

2015-04-11 Thread Devl Devel
Hi Dmitriy, Thanks for the input, I think as per my previous email it would be good to have a bridge project that for example, creates a IgniteFS RDD, similar to the JDBC or HDFS one in which we can extract blocks and populate RDD partitions, I'll post this proposal on your list. Thanks Devl O

Re: Integrating Spark with Ignite File System

2015-04-11 Thread Reynold Xin
Welcome, Dmitriy, to the Spark dev list! On Sat, Apr 11, 2015 at 1:14 AM, Dmitriy Setrakyan wrote: > Hello Everyone, > > I am one of the committers to Apache Ignite and have noticed some talks on > this dev list about integrating Ignite In-Memory File System (IgniteFS) > with Spark. We definite

Integrating Spark with Ignite File System

2015-04-11 Thread Dmitriy Setrakyan
Hello Everyone, I am one of the committers to Apache Ignite and have noticed some talks on this dev list about integrating Ignite In-Memory File System (IgniteFS) with Spark. We definitely like the idea. If you have any questions about Apache Ignite at all, feel free to forward them to the Ignite