Hi Jerry, We using RGW and RBD in our OpenStack clusters and as stand alone clusters. We have six large clusters and adding more. Most of any issues we have faced have been self inflicted such as not currently supporting bucket names like host names. Some S3 tools only work that way which causes some of our developer customers grief. We are addressing that. We have built extensive testing frameworks around S3 RGW testing using a OpenStack or AWS EC2 or Google Cloud Platform to dynamically spin up worker nodes to distribute load for stressing and performance monitoring.
I'm actually building a project called IQStack that will be released on github soon that does that plus other OpenStack testing for scalability. Anyway, there may be some incompatibilities depending on the feature set but most can be abstracted away and addressed. Just as a footnote, I finished running long load testing against AWS S3 and RGW. After tuning our load balancers, firewall rules and a few other tweaks I was able to get parity with AWS S3 up to 10GbE (max size of my load balancer I was testing with). We use several CDNs on video clips so my tests were with 2MB byte-range requests. Thanks, Chris On Fri, Mar 20, 2015 at 6:06 PM, Craig Lewis <cle...@centraldesktop.com> wrote: > I have found a few incompatibilities, but so far they're all on the Ceph > side. One example I remember was having to change the way we delete > objects. The function we originally used fetches a list of object > versions, and deletes all versions. Ceph is implementing objects versions > now (I believe that'll ship with Hammer), so we had to call a different > function to delete the object without iterating over the versions. > > AFAIK, that code should work fine if we point it at Amazon. I haven't > tried it though. > > > I've been using RGW (with replication) in production for 2 years now, > although I'm not large. So far, all of my RGW issues have been Ceph > issues. Most of my issues are caused by my under-powered hardware, or > shooting myself in the foot with aggressive optimizations. Things are > better with my journals on SSD, but the best thing I did was slow down with > my changes. For example, I have 7 OSD nodes and 72 OSDs. When I add new > OSDs, I add a couple at a time instead of adding all the disks in a node at > once. Guess how I learned that lesson. :-) > > > > On Wed, Mar 18, 2015 at 10:03 AM, Jerry Lam <jerry....@oicr.on.ca> wrote: > >> Hi Chris, >> >> Thank you for your reply. >> We are also thinking about using the S3 API but we are concerned about >> how compatible it is with the real S3. For instance, we would like to >> design the system using pre-signed URL for storing some objects. I read the >> ceph documentation, it does not mention if it supports it or not. >> >> My question is do you guys find that the code using the RADOS S3 API >> can easily run in Amazon S3 without any change? If no, how much effort it >> is needed to make it compatible? >> >> Best Regards, >> >> Jerry >> From: Chris Jones <cjo...@cloudm2.com> >> Date: Tuesday, March 17, 2015 at 4:39 PM >> To: Jerry Lam <jerry....@oicr.on.ca> >> Cc: "ceph-users@lists.ceph.com" <ceph-users@lists.ceph.com> >> Subject: Re: [ceph-users] RADOS Gateway Maturity >> >> Hi Jerry, >> >> I currently work at Bloomberg and we currently have a very large Ceph >> installation in production and we use the S3 compatible API for rados >> gateway. We are also re-architecting our new RGW and evaluating a different >> Apache configuration for a little better performance. We only use replicas >> right now, no erasure coding yet. Actually, you can take a look at our >> current configuration at https://github.com/bloomberg/chef-bcpc. >> >> -Chris >> >> On Tue, Mar 17, 2015 at 10:40 AM, Jerry Lam <jerry....@oicr.on.ca> wrote: >> >>> Hi Ceph user, >>> >>> I’m new to Ceph but I need to use Ceph as the storage for the Cloud we >>> are building in house. >>> Did anyone use RADOS Gateway in production? How mature it is in terms of >>> compatibility with S3 / Swift? >>> Anyone can share their experience on it? >>> >>> Best Regards, >>> >>> Jerry >>> >>> _______________________________________________ >>> ceph-users mailing list >>> ceph-users@lists.ceph.com >>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >>> >>> >> >> >> -- >> Best Regards, >> Chris Jones >> >> <http://www.cloudm2.com> >> >> cjo...@cloudm2.com >> (p) 770.655.0770 >> >> This message is intended exclusively for the individual or entity to >> which it is addressed. This communication may contain information that is >> proprietary, privileged or confidential or otherwise legally exempt from >> disclosure. If you are not the named addressee, you are not authorized to >> read, print, retain, copy or disseminate this message or any part of it. >> If you have received this message in error, please notify the sender >> immediately by e-mail and delete all copies of the message. >> >> _______________________________________________ >> ceph-users mailing list >> ceph-users@lists.ceph.com >> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >> >> > -- Best Regards, Chris Jones <http://www.cloudm2.com> cjo...@cloudm2.com (p) 770.655.0770 This message is intended exclusively for the individual or entity to which it is addressed. This communication may contain information that is proprietary, privileged or confidential or otherwise legally exempt from disclosure. If you are not the named addressee, you are not authorized to read, print, retain, copy or disseminate this message or any part of it. If you have received this message in error, please notify the sender immediately by e-mail and delete all copies of the message.
_______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
