Re: [ceph-users] Mapping rbd with read permission

2014-10-08 Thread Ramakrishnan Periyasamy
Hi, Thanks Ilya for reply and I require some more clarifications, correct me if somewhere am wrong. Am able to map rbd with --read-only option using user specific keyring for pool3 since it is having "rwx" but unable to map for pool1 where capabilities are "rx"/"r" (i.e. tried both). User spe

Re: [ceph-users] Openstack keystone with Radosgw

2014-10-08 Thread lakshmi k s
Good workaround. But it did not work. Not sure what this error is all about now. gateway@gateway:~$ openssl x509 -in /home/gateway/ca.pem -pubkey | certutil -d /var/lib/ceph/nss -A -n ca -t "TCu,Cu,Tuw" certutil: function failed: SEC_ERROR_LEGACY_DATABASE: The certificate/key database is in an

Re: [ceph-users] Openstack keystone with Radosgw

2014-10-08 Thread Mark Kirkwood
As a workaround check if your rgw host has openssl and certutil installed, if so you can copy the relevant unconverted certs over to it and convert 'em there. On 09/10/14 15:07, lakshmi k s wrote: Tried aptitude as well, but no luck. Ceph users, have you tried to install libnss3-tools or cert

Re: [ceph-users] Openstack keystone with Radosgw

2014-10-08 Thread lakshmi k s
Tried aptitude as well, but no luck. Ceph users, have you tried to install libnss3-tools or certutil tool on debian/ubuntu? If so, how did you go about this problem. On Wednesday, October 8, 2014 7:01 PM, Mark Kirkwood wrote: Ok, so that is the thing to get sorted. I'd suggest posting

Re: [ceph-users] Openstack keystone with Radosgw

2014-10-08 Thread Mark Kirkwood
Ok, so that is the thing to get sorted. I'd suggest posting the error(s) you are getting perhaps here (someone else might know), but definitely to one of the Debian specific lists. In the meantime perhaps try installing the packages with aptitude rather than apt-get - if there is some fancy fo

Re: [ceph-users] How to restore a Ceph cluster from its cluster map?

2014-10-08 Thread Craig Lewis
I asked a similiar question before, about backing up maps: http://lists.ceph.com/pipermail/ceph-users-ceph.com/2013-August/022798.html The short answer is you can't. There are maps that you can't dump, so you don't have the ability to make a complete snapshot of the cluster. The long answer is t

Re: [ceph-users] Openstack keystone with Radosgw

2014-10-08 Thread lakshmi k s
Thanks Mark. I have been trying to install this on controller node. But for some reason, I am unable to install certutil or libnss3-tools on debian. I am not sure how to proceed. On Wednesday, October 8, 2014 6:26 PM, Mark Kirkwood wrote: If you are using ceph + radosgw packages they sho

Re: [ceph-users] Basic Ceph questions

2014-10-08 Thread Craig Lewis
Comments inline. On Tue, Oct 7, 2014 at 5:51 PM, Marcus White wrote: > Hello, > Some basic Ceph questions, would appreciate your help:) Sorry about > the number and detail in advance! > > a. Ceph RADOS is strongly consistent and different from usual object, > does that mean all metadata also, co

Re: [ceph-users] Openstack keystone with Radosgw

2014-10-08 Thread Mark Kirkwood
If you are using ceph + radosgw packages they should be built with the nss option (--with-nss), so nothing to do there. For the server running keystone you need to do: (root) $ mkdir /var/ceph/nss (root) $ openssl x509 -in /etc/keystone/ssl/certs/ca.pem -pubkey | \ certutil -d /var/ceph

Re: [ceph-users] Rados Gateway and Swift create containers/buckets that cannot be opened

2014-10-08 Thread Mark Kirkwood
Yes. I ran into that as well - I used WSGIChunkedRequest On in the virtualhost config for the *keystone* server [1] as indicated in issue 7796. Cheers Mark [1] i.e, not the rgw. On 08/10/14 22:58, Ashish Chandra wrote: Hi Mark, Good you got the solution. But since you have already done au

Re: [ceph-users] accept: got bad authorizer

2014-10-08 Thread Gregory Farnum
Check your clock sync on that node. That's the usual cause of this issue. -Greg On Wednesday, October 8, 2014, Nathan Stratton wrote: > I have one out of 16 of my OSDs doing something odd. The logs show some > sort of authentication issue. If I restart the OSD things are fine, but in > a few hou

Re: [ceph-users] Basic Ceph questions

2014-10-08 Thread Marcus White
Just a bump:) Is this the right list or should I be posting in devel? MW On Tue, Oct 7, 2014 at 5:51 PM, Marcus White wrote: > Hello, > Some basic Ceph questions, would appreciate your help:) Sorry about > the number and detail in advance! > > a. Ceph RADOS is strongly consistent and different

[ceph-users] accept: got bad authorizer

2014-10-08 Thread Nathan Stratton
I have one out of 16 of my OSDs doing something odd. The logs show some sort of authentication issue. If I restart the OSD things are fine, but in a few hours it happens again and I have to restart it to get things back up. 2014-10-08 06:46:46.858260 7f43f62a0700 0 auth: could not find secret_id=

Re: [ceph-users] rbd and libceph kernel api

2014-10-08 Thread Shawn Edwards
On Wed, Oct 8, 2014 at 2:35 AM, Ilya Dryomov wrote: > On Wed, Oct 8, 2014 at 2:19 AM, Shawn Edwards > wrote: > > Are there any docs on what is possible by writing/reading from the rbd > > driver's sysfs paths? Is it documented anywhere? > > > > I've seen at least one blog post: > > http://www.s

Re: [ceph-users] max_bucket limit -- safe to disable?

2014-10-08 Thread Yehuda Sadeh
On Wed, Oct 8, 2014 at 10:00 AM, Daniel Schneller wrote: > Ok. How can I tell if stuff is stuck in a queue? > What to look for? Correlate a slow request/response that you see in the rgw log to the same in the osd log. Check how much time it the osd thought it took to process. If there's a huge d

Re: [ceph-users] max_bucket limit -- safe to disable?

2014-10-08 Thread Yehuda Sadeh
Right. I misread the log. Note the following request: 2.547155 7f036ffc7700 1 -- 10.102.4.11:0/1009401 --> 10.102.4.14:6809/7428 -- osd_op(client.78418684.0:27514711 .bucket.meta.:default.78418684.122043 [call version.read,getxattrs,stat] 5.3b7d1197 ack+read e16034) v4 -- ?+0 0x7f026802e2c0 con 0

Re: [ceph-users] RadosGW over HTTPS

2014-10-08 Thread Yehuda Sadeh
On Wed, Oct 8, 2014 at 9:21 AM, Marco Garcês wrote: > I believe so: > 2014-10-08 18:19:38.438133 7f9119b90700 2 > RGWDataChangesLog::ChangesRenewThread: start > 2014-10-08 18:19:44.151527 7f90ea7fc700 20 enqueued request req=0x1b9e400 > 2014-10-08 18:19:44.151558 7f90ea7fc700 20 RGWWQ: > 2014-10-

Re: [ceph-users] RadosGW over HTTPS

2014-10-08 Thread Marco Garcês
I believe so: 2014-10-08 18:19:38.438133 7f9119b90700 2 RGWDataChangesLog::ChangesRenewThread: start 2014-10-08 18:19:44.151527 7f90ea7fc700 20 enqueued request req=0x1b9e400 2014-10-08 18:19:44.151558 7f90ea7fc700 20 RGWWQ: 2014-10-08 18:19:44.151561 7f90ea7fc700 20 req: 0x1b9e400 2014-10-08 18:1

Re: [ceph-users] Openstack keystone with Radosgw

2014-10-08 Thread lakshmi k s
Hello Mark, Thanks for your reply. Where should I be installing NSS package? On Gateway or Openstack Controller node? On both, I could not execute the following command as it resulted in bunch of errors. openssl x509 -in /etc/keystone/ssl/certs/ca.pem -pubkey | certutil -d /var/ceph/nss -A -n

Re: [ceph-users] RBD on openstack glance+cinder CoW?

2014-10-08 Thread Jonathan Proulx
On Wed, Oct 8, 2014 at 10:10 AM, Sebastien Han wrote: > Hum I just tried on a devstack and on firefly stable, it works for me. > > Looking at your config it seems that the glance_api_version=2 is put in the > wrong section. > Please move it to [DEFAULT] and let me know if it works. That was exac

Re: [ceph-users] RadosGW over HTTPS

2014-10-08 Thread Yehuda Sadeh
Does the header reach the gateway? Bump up the log (debug rgw = 20) to verify. Yehuda On Wed, Oct 8, 2014 at 8:05 AM, Marco Garcês wrote: > Same thing: > curl -D - -H "Server-Port-Secure: 443" -H "X-Auth-Key: > QoakiyY0tg8jULacsJLsmAbyZHJbY5g/Rc/dOHK3" -H "X-Auth-User: > frontend:swf0002" https:

Re: [ceph-users] RadosGW over HTTPS

2014-10-08 Thread Marco Garcês
Same thing: curl -D - -H "Server-Port-Secure: 443" -H "X-Auth-Key: QoakiyY0tg8jULacsJLsmAbyZHJbY5g/Rc/dOHK3" -H "X-Auth-User: frontend:swf0002" https://gateway.local/auth HTTP/1.1 204 Server: Tengine/2.0.3 Date: Wed, 08 Oct 2014 15:04:27 GMT Content-Type: application/json Connection: keep-alive X-S

Re: [ceph-users] RadosGW over HTTPS

2014-10-08 Thread Yehuda Sadeh
Try passing in 'Server-Port-Secure: 443' header to the auth request. Yehuda On Wed, Oct 8, 2014 at 7:41 AM, Marco Garcês wrote: > Hi David, > > I am indeed using Tengine 2.0.3, but I feel very strange that the > default config is returning X-Storage-Url in the headers, in http, not > https as th

Re: [ceph-users] RadosGW over HTTPS

2014-10-08 Thread Marco Garcês
Hi David, I am indeed using Tengine 2.0.3, but I feel very strange that the default config is returning X-Storage-Url in the headers, in http, not https as the original request. I will try your options, and perhaps downgrading to 1.5.*, and report back. Thank you! Marco Garcês #sysadmin Maputo -

Re: [ceph-users] RadosGW over HTTPS

2014-10-08 Thread David Moreau Simard
Hi Marco, While I do not have a RadosGW implementation right now, I do have a successful setup with tengine and Swift - it should be pretty similar. What version of tengine are you trying to use ? It dates back to a while.. but I remember having issues with the 2.0.x branch of tengine. We packa

Re: [ceph-users] Network hardware recommendations

2014-10-08 Thread Nathan Stratton
On Wed, Oct 8, 2014 at 9:25 AM, Massimiliano Cuttini wrote: > > > > Il 08/10/2014 14:39, Nathan Stratton ha scritto: > > On Wed, Oct 8, 2014 at 8:15 AM, Massimiliano Cuttini > wrote: > >> If you want to build up with Viatta. >> And this give you the possibility to have a fully feature OS. >> W

Re: [ceph-users] RBD on openstack glance+cinder CoW?

2014-10-08 Thread Sebastien Han
Hum I just tried on a devstack and on firefly stable, it works for me. Looking at your config it seems that the glance_api_version=2 is put in the wrong section. Please move it to [DEFAULT] and let me know if it works. On 08 Oct 2014, at 14:28, Nathan Stratton wrote: > On Tue, Oct 7, 2014 at 5

Re: [ceph-users] RadosGW over HTTPS

2014-10-08 Thread Marco Garcês
I made some tests: curl -D - -H "X-Auth-Key: QoakiyY0tg8jULacsJLsmAbyZHJbY5g/Rc/dOHK3" -H "X-Auth-User: frontend:swf0002" https://gateway.local/auth HTTP/1.1 204 Server: Tengine/2.0.3 Date: Wed, 08 Oct 2014 14:04:18 GMT Content-Type: application/json Connection: keep-alive X-Storage-Url: http://gat

Re: [ceph-users] Network hardware recommendations

2014-10-08 Thread Massimiliano Cuttini
Il 08/10/2014 14:39, Nathan Stratton ha scritto: On Wed, Oct 8, 2014 at 8:15 AM, Massimiliano Cuttini mailto:m...@phoenixweb.it>> wrote: If you want to build up with Viatta. And this give you the possibility to have a fully feature OS. What kind of hardware would you use to build

Re: [ceph-users] How to restore a Ceph cluster from its cluster map?

2014-10-08 Thread Aegeaner
Thanks Wido, When I describe a ceph cluster is "down", I mean something is wrong with the ceph software, someone mistakenly changed the configuration file, making the conf in many nodes inconsistent, e.g. wrong fs_id, inconsistent OSD / host mapping, etc. I'm not talking about OSD failures,

Re: [ceph-users] Network hardware recommendations

2014-10-08 Thread Nathan Stratton
On Wed, Oct 8, 2014 at 8:15 AM, Massimiliano Cuttini wrote: > If you want to build up with Viatta. > And this give you the possibility to have a fully feature OS. > What kind of hardware would you use to build up a switch? > Hard to beat the Quanta T3048-LY2, 48 10 gig, 4 40 gig. Same chip as C

Re: [ceph-users] RBD on openstack glance+cinder CoW?

2014-10-08 Thread Nathan Stratton
On Tue, Oct 7, 2014 at 5:35 PM, Jonathan Proulx wrote: > Hi All, > > We're running Firefly on the ceph side and Icehouse on the OpenStack > side & I've pulled the recommended nova branch from > https://github.com/angdraug/nova/tree/rbd-ephemeral-clone-stable-icehouse > > according to > http://cep

Re: [ceph-users] Network hardware recommendations

2014-10-08 Thread Massimiliano Cuttini
If you want to build up with Viatta. And this give you the possibility to have a fully feature OS. What kind of hardware would you use to build up a switch? Il 08/10/2014 09:10, Christian Balzer ha scritto: On Wed, 08 Oct 2014 00:45:06 + Scott Laird wrote: IIRC, one thing to look out for

[ceph-users] RadosGW over HTTPS

2014-10-08 Thread Marco Garcês
Hi there, I am using RadosGW over NGINX, with Swift API, and everything is working great, over HTTP, but with HTTPS, I keep getting errors, and I'm guessing is something on the gateway itself. Does anyone have a working HTTPS gateway with nginx? Can you provide it, so I can compare to mine? If I

Re: [ceph-users] Rados Gateway and Swift create containers/buckets that cannot be opened

2014-10-08 Thread Ashish Chandra
Hi Mark, Good you got the solution. But since you have already done authenticating RadosGW with Keystone, I am having one issue that you can help with. For me I get an error "411 Length Required" with Keystone token authentication. To fix this I use "WSGIChunkedRequest On" in rgw.conf as mentioned

Re: [ceph-users] How to restore a Ceph cluster from its cluster map?

2014-10-08 Thread Wido den Hollander
On 10/08/2014 11:00 AM, Aegeaner wrote: > > Hi all! > > For production use, I want to use two ceph clusters at the same time. > One is the master cluster, and the other is the replication cluster, > which syncs RBD snapshots with master cluster at fixed time (every day, > e.g.), by the way this a

Re: [ceph-users] How to restore a Ceph cluster from its cluster map?

2014-10-08 Thread Marco Garcês
Im in on this thread. *Marco Garcês* *#sysadmin* Maputo - Mozambique *[Phone]* +258 84 4105579 *[Skype]* marcogarces On Wed, Oct 8, 2014 at 11:00 AM, Aegeaner wrote: > > Hi all! > > For production use, I want to use two ceph clusters at the same time. > One is the master cluster, and the other

[ceph-users] How to restore a Ceph cluster from its cluster map?

2014-10-08 Thread Aegeaner
Hi all! For production use, I want to use two ceph clusters at the same time. One is the master cluster, and the other is the replication cluster, which syncs RBD snapshots with master cluster at fixed time (every day, e.g.), by the way this article describes: http://ceph.com/dev-notes/increment

Re: [ceph-users] Federated gateways (our planning use case)

2014-10-08 Thread David Barker
I've had some luck putting a load balancer infront of multiple zones to get around the multiple URL issue. You can get the LB to send POST/DELETE et al to the primary zone, but GET requests can be distributed to multiple zones. The only issue is the replication delay; your data may not be available

Re: [ceph-users] rbd and libceph kernel api

2014-10-08 Thread Ilya Dryomov
On Wed, Oct 8, 2014 at 2:19 AM, Shawn Edwards wrote: > Are there any docs on what is possible by writing/reading from the rbd > driver's sysfs paths? Is it documented anywhere? > > I've seen at least one blog post: > http://www.sebastien-han.fr/blog/2012/06/24/use-rbd-on-a-client/ about how > you

Re: [ceph-users] Network hardware recommendations

2014-10-08 Thread Christian Balzer
On Wed, 08 Oct 2014 00:45:06 + Scott Laird wrote: > IIRC, one thing to look out for is that there are two ways to do IP over > Infiniband. You can either do IP over Infiniband directly (IPoIB), or > encapsulate Ethernet in Infiniband (EoIB), and then do IP over the fake > Ethernet network. >