Hi Tijn,
I had also created my first CEPH storage cluster almost as you have
created. I had 3 VMs for OSD nodes and 1 VM for Monitor node.
All 3 OSD VMs were having one 10 GB virtual disks. so I faced almost
same problem as you are facing right now.
Then changing disk space from 10 GB to 20 GB solved my problem.
I don't know if dynamic disks will create any problem. But I think
instead of having 6 OSDs you can have 3 OSDs, one OSD per VM and can
increase disk size from 10 GB to 20 GB for 3 OSDs.
I don't know this will solve your problem or not but worthy to try. I
mean 3 OSDs are enough for testing purpose initially.
Regards,
Pratik
On 8/5/2014 12:37 PM, Tijn Buijs wrote:
Hello Pratik,
I'm using virtual disks as OSDs. I prefer virtual disks over
directories because this resembles the production environment a bit
better.
I'm using VirtualBox for virtualisation. The OSDs are dynamic disks,
not pre-allocated, but this shouldn't be a problem, right? I don't
have the diskspace on my iMac to have all 6 OSDs pre-allocated :).
I've made the virtual OSD disks 10 GB each, by the way, so that should
be enough for a first test, imho.
Met vriendelijke groet/With kind regards,
Tijn Buijs
Cloud.nl logo
t...@cloud.nl <mailto:t...@cloud.nl> | T. 0800-CLOUDNL / +31 (0)162
820 000 | F. +31 (0)162 820 001
Cloud.nl B.V. | Minervum 7092D | 4817 ZK Breda | www.cloud.nl
<http://www.cloud.nl>
On 04/08/14 14:51, Pratik Rupala wrote:
Hi,
You mentioned that you have 3 hosts which are VMs. Are you using
simple directories as OSDs or virtual disks as OSDs?
I had same problem few days back where enough space was not available
from OSD for the cluster.
Try to increase the size of disks if you are using virtual disks and
if you are using directories as OSDs then check whether you have
enough space on root device using df -h command on OSD node.
Regards,
Pratik
On 8/4/2014 4:11 PM, Tijn Buijs wrote:
Hi Everybody,
My idea was that maybe I was inpatient or something, so I let my
Ceph cluster running over the weekend. So from friday 15:00 until
now (it is monday morning 11:30 here now) it kept on running. And it
didn't help :). It still needs to create 192 PGs.
I've reinstalled my entier cluster a few times now. I switched over
from CentOS 6.5 to Ubuntu 14.04.1 LTS and back to CentOS again, and
every time I get exactly the same results. The PGs are getting in
the incomplete, stuck inactive, stuk unclean state. What am I doing
wrong? :).
For the moment I'm running with 6 OSDs evenly divided over 3 hosts
(so each host has 2 OSDs). I've only got 1 monitor configured in my
current cluster. I hit some other problem when trying to add monitor
2 and 3 again. And to not complicate things with multiple problems
at the same time I've switched back to only 1 monitor. The cluster
should work that way, right?
To make things clear for everybody, here is the output of ceph
health and ceph -s:
$ ceph health
HEALTH_WARN 192 pgs incomplete; 192 pgs stuck inactive; 192 pgs
stuck unclean
$ ceph -s
cluster 43d5f48b-d034-4f50-bec8-5c4f3ad8276f
health HEALTH_WARN 192 pgs incomplete; 192 pgs stuck inactive;
192 pgs stuck unclean
monmap e1: 1 mons at {ceph-mon1=10.28.28.71:6789/0}, election
epoch 1, quorum 0 ceph-mon1
osdmap e20: 6 osds: 6 up, 6 in
pgmap v40: 192 pgs, 3 pools, 0 bytes data, 0 objects
197 MB used, 30456 MB / 30653 MB avail
192 incomplete
I hope somebody has an idea for me to try :).
Met vriendelijke groet/With kind regards,
Tijn Buijs
Cloud.nl logo
t...@cloud.nl <mailto:t...@cloud.nl> | T. 0800-CLOUDNL / +31 (0)162
820 000 | F. +31 (0)162 820 001
Cloud.nl B.V. | Minervum 7092D | 4817 ZK Breda | www.cloud.nl
<http://www.cloud.nl>
On 31/07/14 17:19, Alfredo Deza wrote:
On Thu, Jul 31, 2014 at 10:36 AM, Tijn Buijs <t...@cloud.nl
<mailto:t...@cloud.nl>> wrote:
Hello everybody,
At cloud.nl <http://cloud.nl> we are going to use Ceph. So I
find it a good idea to get some handson experience with it, so
I can work with it :). So I'm installing a testcluster in a few
VirtualBox machines on my iMac, which runs OS X 10.9.4
offcourse. I know I will get a lousy performance, but that's
not the objective here. The objective is to get some experience
with Ceph, to see how it works.
But I hit an issue during the initial setup of the cluster.
When I'm done installing everything and following the howto's
on ceph.com <http://ceph.com> (the preflight
<http://ceph.com/docs/master/start/quick-start-preflight/> and
the Storage Cluster quick start
<http://ceph.com/docs/master/start/quick-ceph-deploy/>) I need
to run ceph health to see that everything is running perfectly.
But it doesn't run perfectly, I get the following output:
ceph@ceph-admin:~$ ceph health
HEALTH_WARN 192 pgs incomplete; 192 pgs stuck inactive; 192 pgs
stuck unclean
And it stays at this information, it never ever changes. So
everything is really stuck. But I don't know what is stuck
exactly and how I can fix it. Some more info about my cluster:
ceph@ceph-admin:~$ ceph -s
cluster d31586a5-6dd6-454e-8835-0d6d9e204612
health HEALTH_WARN 192 pgs incomplete; 192 pgs stuck
inactive; 192 pgs stuck unclean
monmap e3: 3 mons at
{ceph-mon1=10.28.28.18:6789/0,ceph-mon2=10.28.28.31:6789/0,ceph-mon3=10.28.28.50:6789/0
<http://10.28.28.18:6789/0,ceph-mon2=10.28.28.31:6789/0,ceph-mon3=10.28.28.50:6789/0>},
election epoch 4, quorum 0,1,2 ceph-mon1,ceph-mon2,ceph-mon3
osdmap e25: 6 osds: 6 up, 6 in
pgmap v56: 192 pgs, 3 pools, 0 bytes data, 0 objects
197 MB used, 30455 MB / 30653 MB avail
192 creating+incomplete
I'm running on Ubuntu 14.04.1 LTS Server. I did try to get it
running on CentOS 6.5 too (CentOS 6.5 is my actual distro of
choice, but Ceph has more affinity with Ubuntu, so I tried that
too), but I got exactly the same results.
But because this is my first install of Ceph I don't know the
exact debug commands and stuff. I'm willing to get this
working, but I just don't know how :). Any help is appreciated :).
Did you use ceph-deploy? (the link to the quick start guide makes
me think you did)
If that was the case, did you get any warnings/errors at all?
ceph-deploy is very verbose because some of these things are hard
to debug. Mind sharing that output?
Met vriendelijke groet/With kind regards,
Tijn Buijs
Cloud.nl logo
t...@cloud.nl <mailto:t...@cloud.nl> | T. 0800-CLOUDNL / +31
(0)162 820 000 <tel:%2B31%20%280%29162%20820%20000> | F. +31
(0)162 820 001 <tel:%2B31%20%280%29162%20820%20001>
Cloud.nl B.V. | Minervum 7092D | 4817 ZK Breda | www.cloud.nl
<http://www.cloud.nl>
_______________________________________________
ceph-users mailing list
ceph-users@lists.ceph.com <mailto:ceph-users@lists.ceph.com>
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
