We had identified a problem of running MapReduce on top of Swift mainly on the objects that cannot be renamed without a data copy. Also, there are some significant impact on job completion time for large input data and latency sensitive jobs.
You can refer the following research paper which talks about performance study on swift & Hadoop. http://sc14.supercomputing.org/sites/all/themes/sc14/files/archive/tech_poster/poster_files/post192s2-file2.pdf > On Mar 23, 2015, at 2:17 PM, Dariusz Chrząścik <dari...@chrzascik.com> wrote: > > Hello, > > In my Sahara deployment, I am considering using a swift as an input/output > data store. However, I am wondering if swift is eligible for big data > processing. Does anyone have some experiences with such configuration? Is it > efficient? Can you possibly point me to any articles, reports that compare > hdfs performance with swift when running Hadoop Jobs over it? I have done > some research in that matter but without success. I'd be grateful for any > piece of advice. > > Regards, > Dariusz Chrząścik > > _______________________________________________ > Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack > Post to : openstack@lists.openstack.org > Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
_______________________________________________ Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack Post to : openstack@lists.openstack.org Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
