[ceph-users] how to using hadoop with ceph

2013-10-10 Thread
hi all, i follow the http://ceph.com/docs/master/cephfs/hadoop/ to use hadoop with ceph. I bulid a ceph cluster at 192.168.58.129 , which include 1 mon ,1 mds and 2 osds. and then I insall a hadoop at 192.168.58.132。 both ceph cluster and hadoop cluster works well. First of all ,I

Re: [ceph-users] Radosgw Admin REST API

2013-10-10 Thread Alexis GÜNST HORN
Hello again, In fact, I have still a problem... I can do GET requests without problems, but when I try to do a PUT request, to create a user, then I get a "Code: AccessDenied" return. I do it with this code : def user_create (uid, acess_key, secret_key, email): url = 'http://%s/admin/use

Re: [ceph-users] Ceph / RadosGW deployment questions

2013-10-10 Thread Guang
Thanks Yehuda very much for the response. > In theory you could have either, however, the preferred mode of > installation is with having FastCgiExternalServer and manually running > the radosgw. How many radosgw daemons should be launched in such case, should it equal to the number of httpd proc

Re: [ceph-users] radosgw keystone authtoken caching still not working with 0.67.4

2013-10-10 Thread Darren Birkett
Is anyone else using keystone authentication with radosgw? Anyone having any luck getting the authtoken caching working? - Darren On 8 October 2013 10:17, Darren Birkett wrote: > Hi All, > > What's the best way to try and track down why this isn't working for us? > It doesn't seem that there

Re: [ceph-users] retrieving usage information via admin API

2013-10-10 Thread Matt Thompson
Hi All, The admin API is working fine for me (I've tested adding users, etc.) however when I attempt to query (GET) /admin/usage?format=json I simply receive the following back: {"entries":[],"summary":[]} Is there some configuration I need to make to radosgw or elsewhere to capture and or retur

Re: [ceph-users] About Ceph SSD and HDD strategy

2013-10-10 Thread Kyle Bader
It's hard to comment on how your experience could be made better without more information about your configuration and how your testing. Anything along the lines of what LSI controller model, PCI-E bus speed, number of expander cables, drive type, number of SSDs and whether the SSDs were connected

Re: [ceph-users] retrieving usage information via admin API

2013-10-10 Thread Mike O'Toole
Yes, you need to enabling usage logging in your ceph.conf ... something like: rgw enable usage log = truergw usage log tick interval = 30rgw usage log flush threshold = 1024rgw usage max shards = 32rgw usage max user shards = 1 You can find more info here: http://ceph.com/docs/master/radosgw/conf

Re: [ceph-users] Expanding ceph cluster by adding more OSDs

2013-10-10 Thread Kyle Bader
I've contracted and expanded clusters by up to a rack of 216 OSDs - 18 nodes, 12 drives each. New disks are configured with a CRUSH weight of 0 and I slowly add weight (0.1 to 0.01 increments), wait for the cluster to become active+clean and then add more weight. I was expanding after contraction

[ceph-users] use ceph-rest-api without rados-gw

2013-10-10 Thread su kucherova
Hi I want to use ceph-rest-api.But i dont want to use rados-gw I have not setup rados-gw.I have osd+mon+msd. I get an error while starting the ceph-rest-api.I dont know what i am missing abt ceph-rest-api. Can we not use ceph-rest-api to manage osd+mon+msd. Only? ceph-rest-api -c /etc/ceph/ceph.c

Re: [ceph-users] Dumpling ceph.conf looks different

2013-10-10 Thread su kucherova
So let me understand again. Ceph doesnt need [osd]\[msd] details in ceph.conf. Ceph can detect it automatically.(not sure how exactly i guess looking into /var/lib/ceph/osd folder) Suppose I give it in ceph.conf can I still continue to use ceph-deploy? _

Re: [ceph-users] Dumpling ceph.conf looks different

2013-10-10 Thread Gregory Farnum
On Thursday, October 10, 2013, su kucherova wrote: > So let me understand again. > Ceph doesnt need [osd]\[msd] details in ceph.conf. Ceph can detect it > automatically.(not sure how exactly i guess looking into /var/lib/ceph/osd > folder) > Yeah. Suppose I give it in ceph.conf can I still con

Re: [ceph-users] show warning when make source code of ceph

2013-10-10 Thread Gregory Farnum
Yeah, that write return value warning is only in cuttlefish, but it shouldn't cause any trouble. The second one is still around, but it's not a code warning so much as gcc going "ack, that's a lot of work!" And it's not a problem, either. :) -Greg On Wednesday, October 9, 2013, yy-nm wrote: > war

Re: [ceph-users] how to using hadoop with ceph

2013-10-10 Thread Noah Watkins
On Thu, Oct 10, 2013 at 12:27 AM, 鹏 wrote: > > First of all ,I install ceph at 192.168.58.132 , tar -zxvf > ceph-0.6.2.tar.gz ./configure make make install ; does this mean "The > native Ceph file system client must be installed on each participating node > in the Hadoop cluster" .Shoul

Re: [ceph-users] how to using hadoop with ceph

2013-10-10 Thread Noah Watkins
On Thu, Oct 10, 2013 at 7:29 AM, Noah Watkins wrote: > hadoop cluster. You do not need to run any Ceph dameons, but it is > common to run them together for data locality. If you are building Woah, my wording here is terrible. What I meant to say is that you don't necessarily need to run hadoop a

Re: [ceph-users] use ceph-rest-api without rados-gw

2013-10-10 Thread Sébastien Han
Hi, Did you create a key for your user ceph? Sébastien Han Cloud Engineer "Always give 100%. Unless you're giving blood.” Phone: +33 (0)1 49 70 99 72 Mail: sebastien@enovance.com Address : 10, rue de la Victoire - 75009 Paris Web : www.enovance.com - Twitter : @enovance On October 10,

Re: [ceph-users] kernel: [ 8773.432358] libceph: osd1 192.168.0.131:6803 socket error on read

2013-10-10 Thread Gregory Farnum
(Sorry for the delayed response, this was in my spam folder!) Has this issue persisted? Are you using the stock 13.04 kernel? Can you describe your setup a little more clearly? It sounds like maybe you're using CephFS now and were using rbd before; is that right? What data did you move, when, and

[ceph-users] (no subject)

2013-10-10 Thread 何晓波
发自我的 Windows Phone ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] recovering from unhealthy state

2013-10-10 Thread Gregory Farnum
On Wed, Oct 9, 2013 at 10:19 PM, Kees Bos wrote: > Hi, > > > I've managed to get cepth in a unhealthy state, from which it will not > recover automatically. I've done some 'ceph osd out X' and stopped > ceph-osd processes before the rebalancing was completed. (All in a test > environment :-) ) > >

Re: [ceph-users] recovering from unhealthy state

2013-10-10 Thread Kees Bos
On Thu, 2013-10-10 at 18:24 +0200, Gregory Farnum wrote: > On Wed, Oct 9, 2013 at 10:19 PM, Kees Bos wrote: > > Hi, > > > > > > I've managed to get cepth in a unhealthy state, from which it will not > > recover automatically. I've done some 'ceph osd out X' and stopped > > ceph-osd processes befor

Re: [ceph-users] recovering from unhealthy state

2013-10-10 Thread Gregory Farnum
On Thu, Oct 10, 2013 at 10:40 AM, Kees Bos wrote: > On Thu, 2013-10-10 at 18:24 +0200, Gregory Farnum wrote: >> On Wed, Oct 9, 2013 at 10:19 PM, Kees Bos wrote: >> > Hi, >> > >> > >> > I've managed to get cepth in a unhealthy state, from which it will not >> > recover automatically. I've done som

Re: [ceph-users] kernel: [ 8773.432358] libceph: osd1 192.168.0.131:6803 socket error on read

2013-10-10 Thread Gregory Farnum
Okay. As a quick guess you probably used a CRUSH placement option with your new pools that wasn't supported by the old kernel, although it might have been something else. I suspect that you'll find FUSE works better for you anyway as long as you can use it — faster updates from us to you. ;) -Greg

Re: [ceph-users] Radosgw Admin REST API

2013-10-10 Thread Derek Yarnell
Yes, there is something afoot, I guess i hadn't gotten far enough to try creating users just yet. I have these caps, "caps": [ { "type": "buckets", "perm": "*"}, { "type": "metadata", "perm": "*"}, { "type": "usage", "perm": "*"}, { "t

Re: [ceph-users] Radosgw Admin REST API

2013-10-10 Thread Matt Thompson
Accidentally replied to Alexis directly, but this works for me: url = 'http://%s/admin/user?uid=someuser&display-name=somename&format=json' % server r = requests.put(url, auth=S3Auth(aws_key, secret, server)) Note that "display-name" is a required parameter. -Matt On Thu, Oct 10, 2013 at 7:26

Re: [ceph-users] retrieving usage information via admin API

2013-10-10 Thread Matt Thompson
Thanks Mike, "rgw enable usage log = true" did it. On Thu, Oct 10, 2013 at 12:50 PM, Mike O'Toole wrote: > > Yes, you need to enabling usage logging in your ceph.conf ... something > like: > > rgw enable usage log = true > rgw usage log tick interval = 30 > rgw usage log flush threshold = 1024 >

Re: [ceph-users] Radosgw Admin REST API

2013-10-10 Thread Derek Yarnell
On 10/10/13 2:36 PM, Matt Thompson wrote: > Accidentally replied to Alexis directly, but this works for me: > > url = > 'http://%s/admin/user?uid=someuser&display-name=somename&format=json' % > server > r = requests.put(url, auth=S3Auth(aws_key, secret, server)) > > Note that "display-name" is a

[ceph-users] osds and gateway not coming up on restart

2013-10-10 Thread Mike O'Toole
So I took a power hit today and after coming back up 3 of my osds and my radosgw are not coming back up. The logs show no clue as to what may have happened. When I manually try to restart the gateway I see the following in the logs: 2013-10-10 16:04:23.166046 7f8480d9a700 2 RGWDataChangesLog:

Re: [ceph-users] osds and gateway not coming up on restart

2013-10-10 Thread Wido den Hollander
On 10/10/2013 10:43 PM, Mike O'Toole wrote: So I took a power hit today and after coming back up 3 of my osds and my radosgw are not coming back up. The logs show no clue as to what may have happened. When I manually try to restart the gateway I see the following in the logs: 2013-10-10 16:04:

[ceph-users] cephforum.com

2013-10-10 Thread james
Hello! Anyone else think a web forum for ceph could work? I'm thinking simple vbulletin or phpBB site. To me it seems this would increase accessibility to the great info (&minds) on here... but obviously it would need those great minds to work :) Happy to host... Cheers :) __

Re: [ceph-users] cephforum.com

2013-10-10 Thread Mark Nelson
On 10/10/2013 03:49 PM, ja...@peacon.co.uk wrote: Hello! Anyone else think a web forum for ceph could work? I'm thinking simple vbulletin or phpBB site. Better watch out for the new vbulletin vulnerability! To me it seems this would increase accessibility to the great info (&minds) on here

Re: [ceph-users] cephforum.com

2013-10-10 Thread Wido den Hollander
On 10/10/2013 10:49 PM, ja...@peacon.co.uk wrote: Hello! Anyone else think a web forum for ceph could work? I'm thinking simple vbulletin or phpBB site. To me it seems this would increase accessibility to the great info (&minds) on here... but obviously it would need those great minds to work

Re: [ceph-users] osds and gateway not coming up on restart

2013-10-10 Thread Wido den Hollander
On 10/10/2013 11:01 PM, Mike O'Toole wrote: I created them with ceph-deploy and there are no OSD entries in the ceph.conf. Trying to start them that way doesnt work. (bringing discussion back to the list) Are you sure there is no logging? Because there should be in /var/log/ceph Wido >

Re: [ceph-users] osds and gateway not coming up on restart

2013-10-10 Thread Mike O'Toole
Sorry, I didn't mean it logged nothing, I just saw no clues that were apparent to me.Subsequent restart attempts log nothing. Here are the last few lines 2013-10-10 15:19:49.832776 7f94567ac700 10 -- 10.10.2.202:6809/15169 reaper deleted pipe 0x1e045002013-10-10 15:19:49.832785 7f94

Re: [ceph-users] osds and gateway not coming up on restart

2013-10-10 Thread Mike O'Toole
I think I just figured out the issue. When I installed ceph I had already prepared the three partitions for these OSDS and had fstab entries for mounting them. When I first did this install I didn't realize that ceph would mount the file system for you. I removed the entries and everything ca

Re: [ceph-users] cephforum.com

2013-10-10 Thread Joao Eduardo Luis
On 10/10/2013 09:55 PM, Wido den Hollander wrote: On 10/10/2013 10:49 PM, ja...@peacon.co.uk wrote: Hello! Anyone else think a web forum for ceph could work? I'm thinking simple vbulletin or phpBB site. To me it seems this would increase accessibility to the great info (&minds) on here... but

[ceph-users] WARNING: read region map

2013-10-10 Thread Jefferson Alcantara
hi , I need help , when I try run radosgw-admin metadata list I `m receive the mensage 0 WARNING: cannot read region mapanybody know what is this ? Regards att[] ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/lis

[ceph-users] error radosgw service

2013-10-10 Thread Jefferson Alcantara
I need help for my radosgw machine , when I try start the service radosgw I receive equal bellow : /etc/init.d/radosgw start Starting client.radosgw.napsao-nix-qa-ceph-s301... 2013-10-10 20:34:14.047159 7f5dfc20b780 -1 WARNING: libcurl doesn't support curl_multi_wait() 2013-10-10 20:34:14.047173

Re: [ceph-users] WARNING: read region map

2013-10-10 Thread Yehuda Sadeh
The region map has not been initialized. Try running: $ radosgw-admin region-map update Yehuda On Thu, Oct 10, 2013 at 4:34 PM, Jefferson Alcantara wrote: > hi , > > > I need help , when I try run radosgw-admin metadata list I `m receive the > mensage 0 WARNING: cannot read region mapanyb

Re: [ceph-users] error radosgw service

2013-10-10 Thread Yehuda Sadeh
On Thu, Oct 10, 2013 at 4:38 PM, Jefferson Alcantara wrote: > I need help for my radosgw machine , when I try start the service radosgw I > receive equal bellow : > > /etc/init.d/radosgw start > Starting client.radosgw.napsao-nix-qa-ceph-s301... > 2013-10-10 20:34:14.047159 7f5dfc20b780 -1 WARNI

Re: [ceph-users] use ceph-rest-api without rados-gw

2013-10-10 Thread Dan Mick
On 10/10/2013 06:41 AM, su kucherova wrote: Hi I want to use ceph-rest-api.But i dont want to use rados-gw I have not setup rados-gw.I have osd+mon+msd. ceph-rest-api has nothing to do with radosgw, so that should be fine. I get an error while starting the ceph-rest-api.I dont know what i a

Re: [ceph-users] kernel: [ 8773.432358] libceph: osd1 192.168.0.131:6803 socket error on read

2013-10-10 Thread Frerot , Jean-Sébastien
Hi, Thx for your reply :) kernel: Linux compute01 3.8.0-31-generic #46-Ubuntu SMP Tue Sep 10 20:03:44 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux So yes I'm using cephfs and was also using rdb at the same time using different pools. My ceph fs was setup 3 months ago and I upgraded it a couple of da

[ceph-users] SSD pool write performance

2013-10-10 Thread Sergey Pimkov
Hello! I'm testing small CEPH pool consists of some SSD drives (without any spinners). Ceph version is 0.67.4. Seems like write performance of this configuration is not so good as possible, when I testing it with small block size (4k). Pool configuration: 2 mons on separated hosts, one host

Re: [ceph-users] kernel: [ 8773.432358] libceph: osd1 192.168.0.131:6803 socket error on read

2013-10-10 Thread Frerot , Jean-Sébastien
Hi, I followed this documentation and didn't specify any CRUSH settings. http://ceph.com/docs/next/rbd/rbd-openstack/ -- Jean-Sébastien Frerot jsfre...@egliseespoir.com 2013/10/10 Gregory Farnum > Okay. As a quick guess you probably used a CRUSH placement option with > your new pools that w