Re: [ceph-users] OOM-Killer for ceph-osd

2014-04-27 23:58 GMT+02:00 Andrey Korolyov : > Nothing looks wrong, except heartbeat interval which probably should > be smaller due to recovery considerations. Try ``ceph osd tell X heap > release'' and if it will not change memory consumption, file a bug. What should I look for running this ? Se

Re: [ceph-users] SSD journal overload?

Hi, perhaps due IOs from the journal? You can test with iostat (like "iostat -dm 5 sdg"). on debian iostat is in the package sysstat. Udo Am 28.04.2014 07:38, schrieb Indra Pramana: > Hi Craig, > > Good day to you, and thank you for your enquiry. > > As per your suggestion, I have created a 3r

Re: [ceph-users] SSD journal overload?

You what model SSD? Which version of the kernel? 2014-04-28 12:35 GMT+04:00 Udo Lembke : > Hi, > perhaps due IOs from the journal? > You can test with iostat (like "iostat -dm 5 sdg"). > > on debian iostat is in the package sysstat. > > Udo > > Am 28.04.2014 07:38, schrieb Indra Pramana: > > Hi

Re: [ceph-users] cluster_network ignored

2014-04-26 12:06 GMT+02:00 Gandalf Corvotempesta : > I've not defined cluster IPs for each OSD server but only the whole subnet. > Should I define each IP for each OSD ? This is not wrote on docs and > could be tricky to do this in big environments with hundreds of nodes I've added "cluster addr"

Re: [ceph-users] Access denied error

Hi Yehuda, I am using the same above method to call the api and used the way which described in the http://ceph.com/docs/master/radosgw/s3/authentication/#access-control-lists-aclsfor connection. The method in the http://s3.amazonaws.com/doc/s3-developer-guide/RESTAuthentication.html is for genera

[ceph-users] Please provide me rados gateway configuration (rgw.conf) for lighttpd

Hi All, I would like to use lighttpd instead of apache for rados gateway configuration. But i am facing issues with syntax for rgw.conf. Could you please share the details how I can prepare rgw.conf fot lighttpd? Please also suggest version of mod_fastcgi for apache version 2.4.3. Thanks, Srini

[ceph-users] Unable to add CEPH as Primary Storage - libvirt error undefined storage pool type

Hi, I'm trying to add CEPH as Primary Storage, but my libvirt 0.10.2 (CentOS 6.5) does some complaints: - internal error missing backend for pool type 8 Is it possible that the libvirt 0.10.2 (shipped with CentOS 6.5) was not compiled with RBD support ? Can't find how to check this... I'm able

Re: [ceph-users] OOM-Killer for ceph-osd

On 04/28/2014 12:33 PM, Gandalf Corvotempesta wrote: > 2014-04-27 23:58 GMT+02:00 Andrey Korolyov : >> Nothing looks wrong, except heartbeat interval which probably should >> be smaller due to recovery considerations. Try ``ceph osd tell X heap >> release'' and if it will not change memory consumpt

Re: [ceph-users] SSD journal overload?

Hi Udo and Irek, Good day to you, and thank you for your emails. >perhaps due IOs from the journal? >You can test with iostat (like "iostat -dm 5 sdg"). Yes, I have shared the iostat result earlier on this same thread. At times the utilisation of the 2 journal drives will hit 100%, especially wh

Re: [ceph-users] Unable to add CEPH as Primary Storage - libvirt error undefined storage pool type

On 04/28/2014 12:49 PM, Andrija Panic wrote: Hi, I'm trying to add CEPH as Primary Storage, but my libvirt 0.10.2 (CentOS 6.5) does some complaints: - internal error missing backend for pool type 8 Is it possible that the libvirt 0.10.2 (shipped with CentOS 6.5) was not compiled with RBD suppo

Re: [ceph-users] SSD journal overload?

Most likely you need to apply a patch to the kernel. http://www.theirek.com/blog/2014/02/16/patch-dlia-raboty-s-enierghoniezavisimym-keshiem-ssd-diskov 2014-04-28 15:20 GMT+04:00 Indra Pramana : > Hi Udo and Irek, > > Good day to you, and thank you for your emails. > > > >perhaps due IOs from t

Re: [ceph-users] SSD journal overload?

Hi Irek, Thanks for the article. Do you have any other web sources pertaining to the same issue, which is in English? Looking forward to your reply, thank you. Cheers. On Mon, Apr 28, 2014 at 7:40 PM, Irek Fasikhov wrote: > Most likely you need to apply a patch to the kernel. > > > http://ww

Re: [ceph-users] SSD journal overload?

This is my article :). To patch to the kernel (http://www.theirek.com/downloads/code/CMD_FLUSH.diff ). After rebooting, run the following commands: echo temporary write through > /sys/class/scsi_disk//cache_type 2014-04-28 15:44 GMT+04:00 Indra Pramana : > Hi Irek, > > Thanks for the article. Do

Re: [ceph-users] Access denied error

Hi Punit, Le 28 avr. 2014 à 11:55, Punit Dambiwal mailto:hypu...@gmail.com>> a écrit : > Hi Yehuda, > > I am using the same above method to call the api and used the way > which described in the > http://ceph.com/docs/master/radosgw/s3/authentication/#access-control-lists-acls > for connection. T

Re: [ceph-users] Unable to add CEPH as Primary Storage - libvirt error undefined storage pool type

Thank you very much Wido, any suggestion on compiling libvirt with support (I already found a way) or perhaps use some prebuilt , that you would recommend ? Best On 28 April 2014 13:25, Wido den Hollander wrote: > On 04/28/2014 12:49 PM, Andrija Panic wrote: > >> Hi, >> >> I'm trying to add CE

[ceph-users] What happened if rbd lose a block?

What will happened if RBD lose all copied of data-block and I read the block? Context: I want use RDB as main storage with replication factor 1 and drbd for replication on non rbd storage by client side. For example: Computer1: 1. connect rbd as /dev/rbd15 2. use rbd as disk for drbd Computer2:

Re: [ceph-users] What happened if rbd lose a block?

On 04/28/2014 02:35 PM, Timofey Koolin wrote: What will happened if RBD lose all copied of data-block and I read the block? The read to the object will block until a replica comes online to serve it. Remember this with Ceph: "Consistency goes over availability" Context: I want use RDB as ma

Re: [ceph-users] Unable to add CEPH as Primary Storage - libvirt error undefined storage pool type

On 04/28/2014 02:15 PM, Andrija Panic wrote: Thank you very much Wido, any suggestion on compiling libvirt with support (I already found a way) or perhaps use some prebuilt , that you would recommend ? No special suggestions, just make sure you use at least Ceph 0.67.7 I'm not aware of any pr

Re: [ceph-users] Unable to add CEPH as Primary Storage - libvirt error undefined storage pool type

On 28/04/14 14:54, Wido den Hollander wrote: On 04/28/2014 02:15 PM, Andrija Panic wrote: Thank you very much Wido, any suggestion on compiling libvirt with support (I already found a way) or perhaps use some prebuilt , that you would recommend ? No special suggestions, just make sure you us

Re: [ceph-users] packages for Trusty

hello all, to begin with there is no Emperor package for saucy. Emperor for saucy is only rolled through git and based on my experience that can broke ceph cluster to have the test builds rolling in constantly. I don t know why there is a lack on the ceph.com/download section. But the fact th

Re: [ceph-users] Unable to add CEPH as Primary Storage - libvirt error undefined storage pool type

Thanks Dan :) On 28 April 2014 15:02, Dan van der Ster wrote: > > On 28/04/14 14:54, Wido den Hollander wrote: > >> On 04/28/2014 02:15 PM, Andrija Panic wrote: >> >>> Thank you very much Wido, >>> any suggestion on compiling libvirt with support (I already found a way) >>> or perhaps use some

Re: [ceph-users] Unable to add CEPH as Primary Storage - libvirt error undefined storage pool type

Dan, is this maybe just rbd support for kvm package (I already have rbd enabled qemu, qemu-img etc from ceph.com site) I need just libvirt with rbd support ? Thanks On 28 April 2014 15:05, Andrija Panic wrote: > Thanks Dan :) > > > On 28 April 2014 15:02, Dan van der Ster wrote: > >> >> On 28/

Re: [ceph-users] Unable to add CEPH as Primary Storage - libvirt error undefined storage pool type

Indeed. Actually, we didn't need a special libvirt, only the special qemu-kvm-rhev. But we use it via openstack, so I don't know if there is other redhat magic involved. sorry, dan On 28/04/14 15:08, Andrija Panic wrote: Dan, is this maybe just rbd support for kvm package (I already have rb

Re: [ceph-users] Installing ceph without access to the internet

On Sun, Apr 27, 2014 at 7:24 AM, Cedric Lemarchand wrote: > Hi rAn, > > Le 27/04/2014 13:13, rAn rAnn a écrit : > > Thanks all > Im trying to deploy from node1(the admin node) to the new node via the > command " ceph-deploy install node2". > I have coppied the two main repositories (noarc and x86-

[ceph-users] rbd Problem with disk full

Hello, I need rbd kernel module to really delete data on osd related disks. Having a ever growing "hidden data" is not a great solution. Then we can say that first of all we should be able at least manually to strip out the "hidden" data aka the replicas. I use rbd image let say it is 10 TB

Re: [ceph-users] OpenStack Icehouse and ephemeral disks created from image

FYI It’s fixed here: https://review.openstack.org/#/c/90644/1 Sébastien Han Cloud Engineer "Always give 100%. Unless you're giving blood.” Phone: +33 (0)1 49 70 99 72 Mail: sebastien@enovance.com Address : 11 bis, rue Roquépine - 75008 Paris Web : www.enovance.com - Twitter : @eno

[ceph-users] rbd always expending data space problem.

Hello all, recently I get to the conclusion that of a 40 TB of physical space I could use only 16TB before seeing pg stick because osd was too full. The data space used seems to be for ever growing. using ceph osd reweight-by-utilization 103 seems at first to rebalance the osd pg use. Then the

[ceph-users] apple support automated mail ? WTF!?

**Hello, each time I send a mail to the ceph user mailing list I receive an email from apple support?! Is that a joke? Alphe Salas ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-us

Re: [ceph-users] OpenStack Icehouse and ephemeral disks created from image

On 28 April 2014 15:58, Sebastien Han wrote: > FYI It’s fixed here: https://review.openstack.org/#/c/90644/1 I already have this patch and it didn't help. Have it fixed the problem in your cluster? -- Maciej Gałkiewicz Shelly Cloud Sp. z o. o., Co-founder, Sysadmin http://shellycloud.com/, ma

Re: [ceph-users] OpenStack Icehouse and ephemeral disks created from image

Yes yes, just restart cinder-api and cinder-volume. It worked for me. Sébastien Han Cloud Engineer "Always give 100%. Unless you're giving blood.” Phone: +33 (0)1 49 70 99 72 Mail: sebastien@enovance.com Address : 11 bis, rue Roquépine - 75008 Paris Web : www.enovance.com - Twitte

[ceph-users] Data placement algorithm

Hy all, I'm currently interesting in comparing Ceph's block replica placement policy with other placements algorithm. Is it possible to access to the source code of Ceph's placement policy and where can I find it? Thanks a lot. Best regards. CS ___

Re: [ceph-users] Fwd: Access denied error

Hi Yehuda, We are using the same above method to call the api and used the way which described in the http://ceph.com/docs/master/radosgw/s3/authentication/#access-control-lists-acls for connection. The method in the http://s3.amazonaws.com/doc/s3-developer-guide/RESTAuthentication.html is f

Re: [ceph-users] cluster_network ignored

To see where your OSDs and Mon are listening, you have various cmds in Linux, e.g: 'lsof -ni | grep ceph' - you should see one LISTEN line for the monitor, 2 LISTEN lines for the OSDs and a lot of ESTABLISHED lines, which indicate communication between OSDs and OSDs and clients 'netstat -atn | gr

Re: [ceph-users] Data placement algorithm

I think what you want is Ceph's CRUSH algorithm. Source code: https://github.com/ceph/ceph/tree/master/src/crush Paper: http://www.ssrc.ucsc.edu/Papers/weil-sc06.pdf On Mon, Apr 28, 2014 at 5:27 PM, Séguin Cyril wrote: > Hy all, > > I'm currently interesting in comparing Ceph's block replica plac

Re: [ceph-users] Only one OSD log available per node?

It is not. My guess from looking at the time stamps is that maybe you have a log rotation system set up that isn't working properly? -Greg On Sunday, April 27, 2014, Indra Pramana wrote: > Dear all, > > I have multiple OSDs per node (normally 4) and I realised that for all the > nodes that I hav

Re: [ceph-users] Data placement algorithm

Yes it is! Thanks a lot. On 28/04/2014 17:18, xan.peng wrote: I think what you want is Ceph's CRUSH algorithm. Source code: https://github.com/ceph/ceph/tree/master/src/crush Paper: http://www.ssrc.ucsc.edu/Papers/weil-sc06.pdf On Mon, Apr 28, 2014 at 5:27 PM, Séguin Cyril wrote: Hy all, I'

Re: [ceph-users] What happened if rbd lose a block?

Is a setting for change the behavior to return read error instead of block read? I think it is more reasonable behavior because it is similar to bad block on HDD: it can’t be read. Or may be a timeout some seconds, then return read error for the block and other absented blocks in same image/PG.

[ceph-users] CentOS 6 Yum repository broken / tampered with?

Were there any changes to the EL6 yum packages at http://ceph.com/rpm/el6/x86_64/ ? There are a number of files showing a modification date of '25-Apr-2014', but it seems that no one regenerated the repository metadata. This breaks installations using the repository, you'll get errors like th

Re: [ceph-users] apple support automated mail ? WTF!?

I thought that was just me.. I guess someone from apple is subscribed? On 4/28/2014 10:06 AM, Alphe Salas Michels wrote: **Hello, each time I send a mail to the ceph user mailing list I receive an email from apple support?! Is that a joke? Alphe Salas

Re: [ceph-users] apple support automated mail ? WTF!?

Yeah, someone subscribed supp...@apple.com. I just unsubscribed them from the list. Let me know if it shows back up and I'll ban it. Thanks. Best Regards, Patrick McGarry Director, Community || Inktank http://ceph.com || http://inktank.com @scuttlemonkey || @ceph || @inktank On Mon, Apr 2

[ceph-users] [ANN] ceph-deploy 1.5.0 released!

Hi All, There is a new release of ceph-deploy, the easy deployment tool for Ceph. This release comes with a few bug fixes and a few features: * implement `osd list` * add a status check on OSDs when deploying * sync local mirrors to remote hosts when installing * support flags and options set in

Re: [ceph-users] cluster_network ignored

2014-04-28 17:17 GMT+02:00 Kurt Bauer : > What do you mean by "I see all OSDs down"? I mean that my OSDs are detected as down: $ sudo ceph osd tree # id weight type name up/down reweight -1 12.74 root default -2 3.64 host osd13 0 1.82 osd.0 down 0 2 1.82 osd.2 down 0 -3 5.46 host osd12 1 1.82 osd

[ceph-users] Creating a bucket failed

Hello, I've installed Ceph Emperor on my Ubuntu 12.04 server to test many things. Everything was pretty good so far, but now I got a problem (403, AccessDenied) when I try to create a bucket through S3-compatible API. Please read the following information. *Client Information* Computer: Ubuntu

Re: [ceph-users] Creating a bucket failed

This could happen if your client is uses the bucket through subdomain scheme, but the rgw is not resolving it correctly (either rgw_dns_name is misconfigured, or you were accessing it through different host name). Yehuda On Mon, Apr 28, 2014 at 2:02 PM, Seowon Jung wrote: > Hello, > > I've ins

Re: [ceph-users] Creating a bucket failed

Thank you so much for your quick reply. I created a subuser for Swift, but it got the authorization error. Is it related to the same problem? $ swift --verbose -V 1.0 -A http://lab0.coe.hawaii.edu/auth -U admin:swift -K RnelTPTJGc4rt6LlRjF4AnxfJhrLvu4J6+PTUl+s post test Container PUT failed: ht

Re: [ceph-users] Creating a bucket failed

Hello, Le 28/04/2014 23:29, Seowon Jung a écrit : > Thank you so much for your quick reply. I created a subuser for > Swift, but it got the authorization error. Is it related to the same > problem? In the way bucket access via subdomain is specific to S3 and you are now using Swift, I don't thin

Re: [ceph-users] osd_recovery_max_single_start

On Apr 24, 2014, at 10:09 AM, Chad Seys wrote: > Hi David, > Thanks for the reply. > I'm a little confused by OSD versus PGs in the description of the two > options osd_recovery_max_single_start and osd_recovery_max_active . An OSD manages all the PGs in its object store (a subset of all PG

Re: [ceph-users] Creating a bucket failed

Cedric, I created this user as described on the official document $ sudo radosgw-admin subuser create --uid=johndoe --subuser=johndoe:swift --access=full The subuser seems to have a full permission. $ radosgw-admin user info --uid=admin "swift_keys": [ { "user": "admin:swift",

Re: [ceph-users] Slow RBD Benchmark Compared To Direct I/O Test

Dear Christian and all, Anyone can advise? Looking forward to your reply, thank you. Cheers. On Thu, Apr 24, 2014 at 1:51 PM, Indra Pramana wrote: > Hi Christian, > > Good day to you, and thank you for your reply. > > On Wed, Apr 23, 2014 at 11:41 PM, Christian Balzer wrote: > >> > > > Usi

Re: [ceph-users] RBD clone for OpenStack Nova ephemeral volumes

I have decoupled the Nova rbd-ephemeral-clone branch from the multiple-image-location patch, the result can be found at the same location on GitHub as before: https://github.com/angdraug/nova/tree/rbd-ephemeral-clone I will keep rebasing this over Nova master, I also plan to update the rbd-clone-i

Re: [ceph-users] Ceph mds laggy and failed assert in function replay mds/journal.cc

Hi, We had applied the patch and recompile ceph as well as updated the ceph.conf as per suggested, when we re-run ceph-mds we noticed the following: 2014-04-29 10:45:22.260798 7f90b971d700 0 log [WRN] : replayed op client.324186:51366457,12681393 no session for client.324186 2014-04-29 10:45:2

Re: [ceph-users] Only one OSD log available per node?

Hi Greg, The log rotation works fine, it will rotate the logs every day at around 6:50am. However, there are no writes to the files (except for one osd log file) so it will rotate empty files for most of them. -rw-r--r-- 1 root root 313884 Apr 29 12:07 ceph-osd.12.log -rw-r--r-- 1 root root 198

Re: [ceph-users] Only one OSD log available per node?

Are your OSDs actually running? I see that your older logs have more data in them; did you change log rotation from the defaults? On Monday, April 28, 2014, Indra Pramana wrote: > Hi Greg, > > The log rotation works fine, it will rotate the logs every day at around > 6:50am. However, there are n

Re: [ceph-users] SSD journal overload?

Hi Irek, Good day to you, and thank you for your e-mail. Is there a better way other than patching the kernel? I would like to avoid having to compile a custom kernel for my OS. I read that I can disable write-caching on the drive using hdparm: hdparm -W0 /dev/sdf hdparm -W0 /dev/sdg I tested o

Re: [ceph-users] Only one OSD log available per node?

Hi Greg, Yes, all my OSDs are running. 1945 ?Ssl 215:36 /usr/bin/ceph-osd --cluster=ceph -i 12 -f 2090 ?Sl 165:07 /usr/bin/ceph-osd --cluster=ceph -i 15 -f 2100 ?Sl 205:29 /usr/bin/ceph-osd --cluster=ceph -i 13 -f 2102 ?Sl 196:01 /usr/bin/ceph-osd --clus

Re: [ceph-users] Ceph mds laggy and failed assert in function replay mds/journal.cc

On Tue, Apr 29, 2014 at 11:24 AM, Jingyuan Luke wrote: > Hi, > > We had applied the patch and recompile ceph as well as updated the > ceph.conf as per suggested, when we re-run ceph-mds we noticed the > following: > > > 2014-04-29 10:45:22.260798 7f90b971d700 0 log [WRN] : replayed op > client.3