On 3/13/2013 9:31 AM, Greg Farnum wrote:
On Wednesday, March 13, 2013 at 5:52 AM, Ansgar Jazdzewski wrote:
hi,
i added 10 new OSD's to my cluster, after the growth is done, i got:
##########
# ceph -s
health HEALTH_WARN 217 pgs stuck unclean
monmap e4: 2 mons at {a=10.100.217.3:6789/0,b=10.100.217.4:6789/0
(http://10.100.217.3:6789/0,b=10.100.217.4:6789/0)}, election epoch 4, quorum
0,1 a,b
osdmap e1480: 14 osds: 14 up, 14 in
pgmap v8690731: 776 pgs: 559 active+clean, 217 active+remapped; 341 GB data,
685 GB used, 15390 GB / 16075 GB avail
mdsmap e312: 1/1/1 up {0=d=up:active}, 3 up:standby
##########
during the growth some vm was online, with rbd! is that the reason for the warning?
Nope, it's not because you were using the cluster. The "unclean" PGs here are those which are in the
"active+remapped" state. That's actually two states — "active" which is good, because it means they're serving
reads and writes"; "remapped" which means that for some reason the current set of OSDs handling them isn't the set that
CRUSH thinks should be handling them. Given your cluster expansion that probably means that your CRUSH map and rules aren't behaving
themselves and are failing to assign the right number of replicas to those PGs. You can check this by looking at the PG dump. If you
search for "ceph active remapped" it looks to me like you'll get some useful results; you might also just be able to enable
the CRUSH tunables (http://ceph.com/docs/master/rados/operations/crush-map/#tunables).
I experienced this (stuck active+remapped) frequently with the stock
0.41 apt-get/Ubuntu version of ceph. Less so with Bobtail.
Jeff Anderson-Lee
John, this is becoming a more common problem; we should generate some more
targeted documentation around it. :)
-Greg
Software Engineer #42 @ http://inktank.com | http://ceph.com
_______________________________________________
ceph-users mailing list
[email protected]
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________
ceph-users mailing list
[email protected]
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
