[ceph-users] radosgw Segmentation fault on obj copy

2013-12-02 Thread Dominik Mostowiec
Hi, I have strange problem. Obj copy (0 size) killing radosgw. Head for this file: Content-Type: application/octet-stream Server: Apache/2.2.22 (Ubuntu) ETag: "d41d8cd98f00b204e9800998ecf8427e-0" Last-Modified: 2013-12-01T10:37:15Z rgw log. 2013-12-02 08:18:59.196651 7f5308ff1700 1 == starti

Re: [ceph-users] radosgw Segmentation fault on obj copy

2013-12-02 Thread Dominik Mostowiec
Hi, I found that issue is related with "ETag: -0" (ends -0) This is known bug ? -- Regards Dominik 2013/12/2 Dominik Mostowiec : > Hi, > I have strange problem. > Obj copy (0 size) killing radosgw. > > Head for this file: > Content-Type: application/octet-stream > Server: Apache/2.2.22 (Ubunt

Re: [ceph-users] Nearly full OSDs with very little (apparent) FS utilization

2013-12-02 Thread Miguel Afonso Oliveira
Hi, Sorry for the very late reply. I have been trying a lot of things... On 25/10/13 22:40, Yan, Zheng wrote: On Sat, Oct 26, 2013 at 2:05 AM, Gregory Farnum wrote: Are you sure you're using only CephFS? Do you have any snapshots? -Greg Software Engineer #42 @ http://inktank.com | http://ceph

[ceph-users] qemu-kvm packages for centos

2013-12-02 Thread Darren Birkett
Hi List, Any chance the following will be updated with the latest packages for dumpling/emperor: http://ceph.com/packages/qemu-kvm/centos/x86_64/ Using CentOS 6.4 and dumpling with OpenStack Havana, I am unable to boot from rbd volumes until I install an rbd-ified qemu-kvm. I have grabbed the l

Re: [ceph-users] does ceph-deploy adding of osds automatically update ceph.conf? It seems no...

2013-12-02 Thread Alfredo Deza
On Fri, Nov 29, 2013 at 9:46 AM, Gautam Saxena wrote: > I've got ceph up and running on a 3-node centos 6.4 cluster. However, after > I > > a) set the cluster to nout as follows: ceph osd set noout > b) rebooted 1 node > c) logged into that 1 node, I tried to do: service ceph start osd.12 > > but

[ceph-users] Ceph User Committee Formal Announcement Format

2013-12-02 Thread Regola, Nathan (Contractor)
I'm looking forward to working with everyone involved with the Ceph User Committee (http://wiki.ceph.com/01Planning/02Blueprints/Firefly/Ceph_User_Committee#D etailed_Description). I believe that all of the members of the Ceph User Committee should have received an email from Loic asking them to c

Re: [ceph-users] adding another mon failed

2013-12-02 Thread Alfredo Deza
On Fri, Nov 29, 2013 at 1:24 PM, German Anders wrote: > Hi, i'm having issues while trying to add another monitor to my cluster: > > ceph@ceph-deploy01:~/ceph-cluster$ ceph-deploy mon create ceph-node02 > [ceph_deploy.cli][INFO ] Invoked (1.3.3): /usr/bin/ceph-deploy mon create > ceph-node02 > [

Re: [ceph-users] qemu-kvm packages for centos

2013-12-02 Thread Dan Van Der Ster
Hi, See this one also: http://tracker.ceph.com/issues/6365 But I’m not sure the Inktank patched qemu-kvm is relevant any longer since RedHat just released qemu-kvm-rhev with RBD support. Cheers, Dan On 02 Dec 2013, at 15:36, Darren Birkett wrote: > Hi List, > > Any chance the following will b

Re: [ceph-users] Real size of rbd image

2013-12-02 Thread Stephen Taylor
I had not enabled discard/trim for the filesystem using the image, but I have done so this morning. I doesn't appear to make a difference. The extents I'm seeing reported as "zero" are not actually discarded extents. They are extents that contain data that was added after the ending snapshot I'm

Re: [ceph-users] radosgw Segmentation fault on obj copy

2013-12-02 Thread Yehuda Sadeh
That's unknown bug. I have a guess as to how the original object was created. Can you read the original object, but only copy fails? On Dec 2, 2013 4:53 AM, "Dominik Mostowiec" wrote: > Hi, > I found that issue is related with "ETag: -0" (ends -0) > This is known bug ? > > -- > Regards > Domi

Re: [ceph-users] Adding new OSDs, need to increase PGs?

2013-12-02 Thread Brian Andrus
Yes, I would recommend increasing PGs in your case. The pg_num and pgp_num recommendations are designed to be fairly broad to cover a wide range of different hardware that a ceph user might be utilizing. You basically should be using a number that will ensure data granularity across all your OSDs.

Re: [ceph-users] radosgw Segmentation fault on obj copy

2013-12-02 Thread Dominik Mostowiec
Yes I can read it. Oryginal object is 0 size. Regards Dominik On Dec 2, 2013 6:14 PM, "Yehuda Sadeh" wrote: > That's unknown bug. I have a guess as to how the original object was > created. Can you read the original object, but only copy fails? > On Dec 2, 2013 4:53 AM, "Dominik Mostowiec" > w

[ceph-users] CEPH HA with Ubuntu OpenStack and Highly Available Controller Nodes

2013-12-02 Thread Gary Harris (gharris)
Hi, I have a question about CEP high availability when integrated with OpenStack. Assuming you have all openstack controller nodes in HA mode would your actually have an HA CEPH implementation as well meaning two Primary OSDs or both pointing to the primary? Or, do the client requests get for

Re: [ceph-users] radosgw Segmentation fault on obj copy

2013-12-02 Thread Dominik Mostowiec
Hi, Another file with the same problems: 2013-12-01 11:37:15.556687 7f7891fd3700 1 == starting new request req=0x25406d0 = 2013-12-01 11:37:15.556739 7f7891fd3700 2 req 1314:0.52initializing 2013-12-01 11:37:15.556789 7f7891fd3700 10 s->object=files/192.txt s->bucket=testbucket 2

Re: [ceph-users] CEPH HA with Ubuntu OpenStack and Highly Available Controller Nodes

2013-12-02 Thread Randy Smith
On Mon, Dec 2, 2013 at 9:43 AM, Gary Harris (gharris) wrote: > Hi, I have a question about CEP high availability when integrated with > OpenStack. > > > Assuming you have all openstack controller nodes in HA mode would your > actually > have an HA CEPH implementation as well meaning two Primar

Re: [ceph-users] CEPH HA with Ubuntu OpenStack and Highly Available Controller Nodes

2013-12-02 Thread Mike Dawson
Gary, Yes, Ceph can provide a highly available infrastructure. A few pointers: - IO will stop if anything less than a majority of the monitors are functioning properly. So, run a minimum of 3 Ceph monitors distributed across your data center's failure domains. If you choose to add more monito

Re: [ceph-users] radosgw Segmentation fault on obj copy

2013-12-02 Thread Yehuda Sadeh
By any chance are you uploading empty objects through the multipart upload api? On Mon, Dec 2, 2013 at 12:08 PM, Dominik Mostowiec wrote: > Hi, > Another file with the same problems: > > 2013-12-01 11:37:15.556687 7f7891fd3700 1 == starting new request > req=0x25406d0 = > 2013-12-01 11:3

Re: [ceph-users] radosgw Segmentation fault on obj copy

2013-12-02 Thread Dominik Mostowiec
Yes, this is probably upload empty file. This is the problem? -- Regards Dominik 2013/12/2 Yehuda Sadeh : > By any chance are you uploading empty objects through the multipart upload > api? > > On Mon, Dec 2, 2013 at 12:08 PM, Dominik Mostowiec > wrote: >> Hi, >> Another file with the same pro

[ceph-users] Website related discussions

2013-12-02 Thread Loic Dachary
Hi Patrick, Aaron and Nathan are willing to take an active role in editing http://ceph.com/. During the Ceph Developer Summit, it was suggested that we use ceph-users@ until it's too much noise and only then create a dedicated mailing list. This is a friendly warning that such a noisy conversat

Re: [ceph-users] radosgw Segmentation fault on obj copy

2013-12-02 Thread Yehuda Sadeh
Looks like it. There should be a guard against it (mulitpart upload minimum is 5M). On Mon, Dec 2, 2013 at 12:32 PM, Dominik Mostowiec wrote: > Yes, this is probably upload empty file. > This is the problem? > > -- > Regards > Dominik > > > 2013/12/2 Yehuda Sadeh : >> By any chance are you upload

Re: [ceph-users] Website related discussions

2013-12-02 Thread Patrick McGarry
Hey Loic, That's great! As for ceph-user I was referring more to a way for people to drop content or requests on the community at large. For a chatty discussion about website changes we can just include the interested parties (message to follow) and others can chime in with their desire to help

Re: [ceph-users] radosgw Segmentation fault on obj copy

2013-12-02 Thread Dominik Mostowiec
You're right. S3 api doc: http://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadComplete.html "Err:EntityTooSmall Your proposed upload is smaller than the minimum allowed object size. Each part must be at least 5 MB in size, except the last part." Thanks. This error should be triggered from rad

Re: [ceph-users] qemu-kvm packages for centos

2013-12-02 Thread Alvin Starr
I have been looking at the src rpm qemu-img-0.12.1.2-2.415.el6.x86_64 It looks like it there are a number of comments implying that it supprts rdb but qemu -h returns. Supported formats: raw cow qcow vdi vmdk cloop dmg bochs vpc vvfat qcow2 qed vhdx parallels nbd blkdebug host_cdrom host_flopp

Re: [ceph-users] qemu-kvm packages for centos

2013-12-02 Thread Dan Van Der Ster
You need the rhev package. https://www.mail-archive.com/ceph-users@lists.ceph.com/msg05962.html On Dec 2, 2013 10:18 PM, Alvin Starr wrote: I have been looking at the src rpm qemu-img-0.12.1.2-2.415.el6.x86_64 It looks like it there are a number of comments implying that it supprts rdb but qemu

Re: [ceph-users] radosgw Segmentation fault on obj copy

2013-12-02 Thread Yehuda Sadeh
Actually, I read that differently. It only says that if there's more than 1 part, all parts except for the last one need to be > 5M. Which means that for uploads that are smaller than 5M there should be zero or one parts. On Mon, Dec 2, 2013 at 12:54 PM, Dominik Mostowiec wrote: > You're right. >

Re: [ceph-users] radosgw Segmentation fault on obj copy

2013-12-02 Thread Yehuda Sadeh
I'm having trouble reproducing the issue. What version are you using? Thanks, Yehuda On Mon, Dec 2, 2013 at 2:16 PM, Yehuda Sadeh wrote: > Actually, I read that differently. It only says that if there's more > than 1 part, all parts except for the last one need to be > 5M. Which > means that for

Re: [ceph-users] odd performance graph

2013-12-02 Thread Smart Weblications GmbH - Florian Wiessner
Hi, Am 02.12.2013 04:27, schrieb James Harper: >> Hi, >> The low points are all ~35Mbytes/sec and the high points are all ~60Mbytes/sec. This is very reproducible. >>> >>> It occurred to me that just stopping the OSD's selectively would allow me to >>> see if there was a change when one

Re: [ceph-users] odd performance graph

2013-12-02 Thread Gruher, Joseph R
>I don't know how rbd works inside, but i think ceph rbd here returns zeros >without real osd disk read if the block/sector of the rbd-disk is unused. That >would explain the graph you see. You can try adding a second rbd image and >not format/use it and benchmark this disk, then make a filesystem

Re: [ceph-users] odd performance graph

2013-12-02 Thread Mark Nelson
On 12/02/2013 05:06 PM, Gruher, Joseph R wrote: I don't know how rbd works inside, but i think ceph rbd here returns zeros without real osd disk read if the block/sector of the rbd-disk is unused. That would explain the graph you see. You can try adding a second rbd image and not format/use it an

[ceph-users] Granularity/efficiency of copy-on-write?

2013-12-02 Thread Bill Eldridge
Hi all, We're looking at using Ceph's copy-on-write for a ton of users' replicated cloud image environments, and are wondering how efficient Ceph is for adding user data to base images - is data added in normal 4kB or 64kB sizes, or can you specify block size for volumes (so you can have video

Re: [ceph-users] Nearly full OSDs with very little (apparent) FS utilization

2013-12-02 Thread Yan, Zheng
On Mon, Dec 2, 2013 at 9:03 PM, Miguel Afonso Oliveira wrote: > Hi, > > Sorry for the very late reply. I have been trying a lot of things... > > > On 25/10/13 22:40, Yan, Zheng wrote: >> >> On Sat, Oct 26, 2013 at 2:05 AM, Gregory Farnum wrote: >>> >>> Are you sure you're using only CephFS? Do yo

Re: [ceph-users] issues with Swift and ceph

2013-12-02 Thread Yehuda Sadeh
On Mon, Dec 2, 2013 at 4:39 PM, jheryl williams wrote: > Hello , > > I have been looking all over the net to find a solution to my problem and I > came across one that you helped someone else with. I was wondering if you > can assist me as well. I pretty much created a ceph cluster from the step b

Re: [ceph-users] radosgw Segmentation fault on obj copy

2013-12-02 Thread Dominik Mostowiec
0.56.7 On Dec 2, 2013 11:30 PM, "Yehuda Sadeh" wrote: > I'm having trouble reproducing the issue. What version are you using? > > Thanks, > Yehuda > > On Mon, Dec 2, 2013 at 2:16 PM, Yehuda Sadeh wrote: > > Actually, I read that differently. It only says that if there's more > > than 1 part, all

Re: [ceph-users] radosgw Segmentation fault on obj copy

2013-12-02 Thread Dominik Mostowiec
0.56.7 W dniu poniedziałek, 2 grudnia 2013 użytkownik Yehuda Sadeh napisał: > I'm having trouble reproducing the issue. What version are you using? > > Thanks, > Yehuda > > On Mon, Dec 2, 2013 at 2:16 PM, Yehuda Sadeh > > > wrote: > > Actually, I read that differently. It only says that if there

Re: [ceph-users] radosgw Segmentation fault on obj copy

2013-12-02 Thread Yehuda Sadeh
I see. Do you have backtrace for the crash? On Mon, Dec 2, 2013 at 6:19 PM, Dominik Mostowiec wrote: > 0.56.7 > > W dniu poniedziałek, 2 grudnia 2013 użytkownik Yehuda Sadeh napisał: > >> I'm having trouble reproducing the issue. What version are you using? >> >> Thanks, >> Yehuda >> >> On Mon, D

Re: [ceph-users] Real size of rbd image

2013-12-02 Thread Josh Durgin
On 12/02/2013 08:10 AM, Stephen Taylor wrote: I had not enabled discard/trim for the filesystem using the image, but I have done so this morning. I doesn't appear to make a difference. The extents I'm seeing reported as "zero" are not actually discarded extents. They are extents that contain d

Re: [ceph-users] odd performance graph

2013-12-02 Thread James Harper
> > I also noticed a graph like this once i benchmarked w2k8 guest on ceph with > rbd. > To me it looked like when the space on the drive is used, the throughput is > lower, when the space read by rbd on the drive is unused, the reads are > superfast. > > I don't know how rbd works inside, but i

Re: [ceph-users] radosgw Segmentation fault on obj copy

2013-12-02 Thread Dominik Mostowiec
for another object. http://pastebin.com/VkVAYgwn 2013/12/3 Yehuda Sadeh : > I see. Do you have backtrace for the crash? > > On Mon, Dec 2, 2013 at 6:19 PM, Dominik Mostowiec > wrote: >> 0.56.7 >> >> W dniu poniedziałek, 2 grudnia 2013 użytkownik Yehuda Sadeh napisał: >> >>> I'm having trouble re

Re: [ceph-users] radosgw Segmentation fault on obj copy

2013-12-02 Thread Yehuda Sadeh
I created earlier an issue (6919) and updated it with the relevant issue. This has been fixed in dumpling, although I don't remember hitting the scenario that you did. Was probably hitting it as part of the development work that was done then. In any case I created a branch with the relevant fixes

Re: [ceph-users] optimal setup with 4 x ethernet ports

2013-12-02 Thread Kyle Bader
> Is having two cluster networks like this a supported configuration? Every osd and mon can reach every other so I think it should be. Maybe. If your back end network is a supernet and each cluster network is a subnet of that supernet. For example: Ceph.conf cluster network (supernet): 10.0.0.0/8

Re: [ceph-users] radosgw Segmentation fault on obj copy

2013-12-02 Thread Dominik Mostowiec
Thanks Workaround, don't use multipart when obj size == 0 ? On Dec 3, 2013 6:43 AM, "Yehuda Sadeh" wrote: > I created earlier an issue (6919) and updated it with the relevant > issue. This has been fixed in dumpling, although I don't remember > hitting the scenario that you did. Was probably hitt

Re: [ceph-users] Adding new OSDs, need to increase PGs?

2013-12-02 Thread Robert van Leeuwen
> On 2 dec. 2013, at 18:26, "Brian Andrus" wrote: > Setting your pg_num and pgp_num to say... 1024 would A) increase data > granularity, B) likely lend no noticeable increase to resource consumption, > and C) allow some room for future OSDs two be added while still within range > of acceptab

Re: [ceph-users] radosgw Segmentation fault on obj copy

2013-12-02 Thread Yehuda Sadeh
For bobtail at this point yes. You can try the unofficial version with that fix off the gitbuilder. Another option is to upgrade everything to dumpling. Yehuda On Mon, Dec 2, 2013 at 10:24 PM, Dominik Mostowiec wrote: > Thanks > Workaround, don't use multipart when obj size == 0 ? > > On Dec 3,

[ceph-users] french ceph meetup

2013-12-02 Thread eric mourgaya
Hi, The first unofficial French ceph user meetup took place in Nantes on Friday 29th 2013. There were 8 participants. Yann from the University of Nantes related his use case of ceph. During his presentation we talked about hardware choices, about using SSD and problems he has experienced in his