I had a question about one of the points on the slide. Slide 24, last
bullet point, says:
* If you use XFS, don't put your OSD journal as a file on the disk
o Use a separate partition, the first partition!
o We still need to reinstall our whole cluster to re-partition the
OSDs
Do you have any details on that?
I am doing that, mostly because my rados bench test indicated that it
was faster than using a journal partition. Now I'm having time-out
issues with OSD during recovery, when there really shouldn't be any, so
I'm curious.
Thanks for any info.
*Craig Lewis*
Senior Systems Engineer
Office +1.714.602.1309
Email cle...@centraldesktop.com <mailto:cle...@centraldesktop.com>
*Central Desktop. Work together in ways you never thought possible.*
Connect with us Website <http://www.centraldesktop.com/> | Twitter
<http://www.twitter.com/centraldesktop> | Facebook
<http://www.facebook.com/CentralDesktop> | LinkedIn
<http://www.linkedin.com/groups?gid=147417> | Blog
<http://cdblog.centraldesktop.com/>
On 4/1/14 07:18 , Dan Van Der Ster wrote:
Hi,
On 1 Apr 2014 at 15:59:07, Andrey Korolyov (and...@xdel.ru
<mailto:and...@xdel.ru>) wrote:
On 04/01/2014 03:51 PM, Robert Sander wrote:
> On 01.04.2014 13:38, Karol Kozubal wrote:
>
>> I am curious to know what is the largest known ceph production
deployment?
>
> I would assume it is the CERN installation.
>
> Have a look at the slides from Frankfurt Ceph Day:
>
> http://www.slideshare.net/Inktank_Ceph/scaling-ceph-at-cern
>
> Regards
>
Just curious, how CERN guys built the network topology to prevent
possible cluster splits, because split in the middle will cause huge
downtime even for a relatively short split time enough to mark half of
those 1k OSDs as down by remaining MON majority.
The mons are distributed around the data centre, across N switches.
The OSDs are across a few switches --- actually, we could use CRUSH
rules to replicate across switches but didn't do so because of an
(unconfirmed) fear that the uplinks would become a bottleneck.
So a switch or routing outage scenario is clearly a point of failure
where some PGs could become stale, but we've been lucky enough not to
suffer from that yet.
BTW, this 3PB cluster was built to test the scalability of Ceph's
implementation, not because we have 3PB of data to store in Ceph today
(most of the results of those tests are discussed in that
presentation.). And we are currently partitioning this cluster down
into a smaller production instance for Cinder and other instances for
ongoing tests.
BTW#2, I don't think the CERN cluster is the largest. Isn't
DreamHost's bigger?
Cheers, Dan
-- Dan van der Ster || Data & Storage Services || CERN IT Department --
_______________________________________________
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
