Re: Transform MongoDB Aggregation into Spark Job

2015-08-04 Thread Jörn Franke
Hi, I think the combination of Mongodb and Spark is a little bit unlucky. Why don't you simply use mongodb? If you want to process a lot of data you should use hdfs or cassandra as storage. Mongodb is not suitable for heterogeneous processing of large scale data. Best regards Best regards, L

Transform MongoDB Aggregation into Spark Job

2015-08-04 Thread Deepesh Maheshwari
Hi, I am new to Apache Spark and exploring spark+kafka intergration to process data using spark which i did earlier in MongoDB Aggregation. I am not able to figure out to handle my use case. Mongo Document : { "_id" : ObjectId("55bfb3285e90ecbfe37b25c3"), "url" : " http://www.z.com/ne