Re: [ceph-users] Multiple journals and an OSD on one SSD doable?

me=60 --time_based --group_reporting --name=journal-test write: io=1507.4MB, bw=25723KB/s, iops=6430, runt= 60007msec Also tested a Micron 550 we had sitting around and it maxed out at 2.5mb/s, both results conflict with the chart Regards, Cameron Scrace Infrastructure Engineer Mobile +64 22 610 4629

Re: [ceph-users] Multiple journals and an OSD on one SSD doable?

money for the project. Thanks, Cameron Scrace Infrastructure Engineer Mobile +64 22 610 4629 Phone +64 4 462 5085 Email cameron.scr...@solnet.co.nz Solnet Solutions Limited Level 12, Solnet House 70 The Terrace, Wellington 6011 PO Box 397, Wellington 6140 www.solnet.co.nz From

Re: [ceph-users] Multiple journals and an OSD on one SSD doable?

be part of the cache pool. 2) Put the spinning disk journals on the OS SSDs and use the 2 1TB SSDs for the cache pool. In both cases the other 4 1TB SSDs will be part of their own tier. Thanks a lot! Cameron Scrace Infrastructure Engineer Mobile +64 22 610 4629 Phone +64 4 462 5085 Email

Re: [ceph-users] Multiple journals and an OSD on one SSD doable?

The other option we were considering was putting the journals on the OS SSDs, they are only 250GB and the rest would be for the OS. Is that a decent option? Thanks! Cameron Scrace Infrastructure Engineer Mobile +64 22 610 4629 Phone +64 4 462 5085 Email cameron.scr...@solnet.co.nz Solnet

[ceph-users] Multiple journals and an OSD on one SSD doable?

performance too much? Thanks, Cameron Scrace Infrastructure Engineer Mobile +64 22 610 4629 Phone +64 4 462 5085 Email cameron.scr...@solnet.co.nz Solnet Solutions Limited Level 12, Solnet House 70 The Terrace, Wellington 6011 PO Box 397, Wellington 6140 www.solnet.co.nz Attention: This email may

[ceph-users] ceph-deploy osd prepare/activate failing with journal on raid device.

seems to think that the journal and OSD are meant to be on the same drive.: http://pastebin.com/CEk8Teys Anyone had this issue before or any suggestions on how to fix it? When I use a non raided device it works perfectly. Thanks, Cameron Scrace Infrastructure Engineer Mobile +64 22 610 4629

Re: [ceph-users] Monitors not reaching quorum. (SELinux off, IPtables off, can see tcp traffic)

It mostly like is the model of switch. In its settings the minimum frame size you can set is 1518, default MTU is 1500, seems the switch wants the 18 byte difference. We are using a pair of Netgear XS712T and bonded pairs of Intel 10-Gigabit X540-AT2 (rev 01) with 3 VLans. Cameron Scrace

Re: [ceph-users] Monitors not reaching quorum. (SELinux off, IPtables off, can see tcp traffic)

The interface MTU has to be 18 or more bytes lower than the switch MTU or it just stops working. As far as I know the monitor communication is not being encapsulated by any SDN. Cameron Scrace Infrastructure Engineer Mobile +64 22 610 4629 Phone +64 4 462 5085 Email cameron.scr

Re: [ceph-users] Monitors not reaching quorum. (SELinux off, IPtables off, can see tcp traffic)

Seems to be something to do with our switch. If the interface MTU is too close to the switch MTU it stops working. Thanks for all your help :) Cameron Scrace Infrastructure Engineer Mobile +64 22 610 4629 Phone +64 4 462 5085 Email cameron.scr...@solnet.co.nz Solnet Solutions Limited Level

Re: [ceph-users] Monitors not reaching quorum. (SELinux off, IPtables off, can see tcp traffic)

Thanks for the links, Jumbo frames are definitely working. Although we had to set the MTU to 8192 because one of the components doesn't support an MTU higher than that. Thanks for the help. Looks like we may just have to deal with jumbo frames being off. Cameron Scrace Infrastru

Re: [ceph-users] Monitors not reaching quorum. (SELinux off, IPtables off, can see tcp traffic)

Setting the MTU to 1500 worked, monitors reach quorum right away. Unfortunately we really want Jumbo Frames to be on, any ideas on how to get ceph to work with them on? Thanks! Cameron Scrace Infrastructure Engineer Mobile +64 22 610 4629 Phone +64 4 462 5085 Email cameron.scr

Re: [ceph-users] Monitors not reaching quorum. (SELinux off, IPtables off, can see tcp traffic)

) and it still times out. Thanks! Cameron Scrace Infrastructure Engineer Mobile +64 22 610 4629 Phone +64 4 462 5085 Email cameron.scr...@solnet.co.nz Solnet Solutions Limited Level 12, Solnet House 70 The Terrace, Wellington 6011 PO Box 397, Wellington 6140 www.solnet.co.nz From: Somnath

[ceph-users] Monitors not reaching quorum. (SELinux off, IPtables off, can see tcp traffic)

"name": "wcm1", "addr": "10.1.226.64:6789\/0" }, { "rank": 1, "name": "wcm2", "addr": "10.1.226.65:6789\/0"