On 22.05.2014 17:30, Craig Lewis wrote:
On 5/22/14 06:16 , Georg Höllrigl wrote:
I have created one bucket that holds many small files, separated into
different "directories". But whenever I try to acess the bucket, I
only run into some timeout. The timeout is at around 30 - 100 seconds.
This is smaller then the Apache timeout of 300 seconds.
Just so we're all talking about the same things, what does "many small
files" mean to you? Also, how are you separating them into
"directories"? Are you just giving files in the same "directory" the
same leading string, like "dir1_subdir1_filename"?
I can only estimate how many files. ATM I've 25M files on the origin but
only 1/10th has been synced to radosgw. These are distributed throuhg 20
folders, each containing about 2k directories with ~ 100 - 500 files each.
Do you think that's too much in that usecase?
I'm putting about 1M objects, random sizes, in each bucket. I'm not
having problems getting individual files, or uploading new ones. It
does take a long time for s3cmd to list the contents of the bucket. The
only time I get timeouts is when my cluster is very unhealthy.
If you're doing a lot more than that, say 10M or 100M objects, then that
could cause a hot spot on disk. You might be better off taking your
"directories", and putting them in their own bucket.
--
*Craig Lewis*
Senior Systems Engineer
Office +1.714.602.1309
Email cle...@centraldesktop.com <mailto:cle...@centraldesktop.com>
*Central Desktop. Work together in ways you never thought possible.*
Connect with us Website <http://www.centraldesktop.com/> | Twitter
<http://www.twitter.com/centraldesktop> | Facebook
<http://www.facebook.com/CentralDesktop> | LinkedIn
<http://www.linkedin.com/groups?gid=147417> | Blog
<http://cdblog.centraldesktop.com/>
_______________________________________________
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
