10+1 is a bad idea for obvious reasons (not enough coding chunks, you will be offline if even one server is offline).
The real problem is that your 20kb files will be split up into 2kb chunks and the metadata overhead and bluestore min alloc size will eat up your disk space. Paul 2018-07-30 13:44 GMT+02:00 Anton Aleksandrov <an...@aleksandrov.eu>: > Hello community, > > I am building first cluster for project, that hosts millions of small > (from 20kb) and big (up to 10mb) files. Right now we are moving from local > 16tb raid storage to cluster of 12 small machines. We are planning to have > 11 OSD nodes, use erasure coding pool (10+1) and one host for MDS. > > On my local tests I see, that available space decrease unproportionally to > the amount of data copied into cluster. With clean cluster I have, for > example 100gb available space, but after copying 40gb in - size decreases > for about 5-10%. Is that normal? > > Is there any term, that would specify cluster's minimal object size? > > I also have question if having so many small files (current number is > about 50'000'000 files at least) - could have negative impact and where > would be our bottleneck? As we don't have money for SSD, we will have > WAL/DB on separate simple HDD. > > Also - would that help to put Metadata pool on separate disks, away from > Data pool drives for CephFS? > > Regards, > Anton. > > _______________________________________________ > ceph-users mailing list > ceph-users@lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > -- Paul Emmerich Looking for help with your Ceph cluster? Contact us at https://croit.io croit GmbH Freseniusstr. 31h 81247 München www.croit.io Tel: +49 89 1896585 90
_______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
