On Apr 3, 2013, at 10:58 AM, Gregory Farnum <g...@inktank.com> wrote:
> On Wed, Apr 3, 2013 at 9:45 AM, John Nielsen <li...@jnielsen.net> wrote: >> On Apr 1, 2013, at 3:33 PM, Gregory Farnum <g...@inktank.com> wrote: >> >>>> On Mon, Apr 1, 2013 at 2:16 PM, Sam Lang <sl...@inktank.com> wrote: >>>>> On Mon, Apr 1, 2013 at 5:59 AM, Papaspyrou, Alexander >>>>> <papaspy...@adesso-mobile.de> wrote: >>>>> 1. So far, I understand that OSD ids have to be numeric, nothing else in >>>>> there. What I couldn't find is whether they really have to start at 0 or >>>>> 1, >>>>> and whether I need to increase them stepwise? The background of my >>>>> question >>>>> is automation: It would make our life much simpler if I could use some >>>>> calculated value for the OSD id (say, the node's decimal IP address and >>>>> some >>>>> local ID for the disk), because we could then statically assign them >>>>> without >>>>> having to know what other OSDs already exist. >>>> >>>> They need to start from 0. The max_osd value maintained by the >>>> monitor bounds the number of possible osds, keeps the size of the >>>> osdmap down, etc. >>> >>> Note that using the "ceph osd create" command gives you an ID to use >>> these days, and that's where you should be deriving these from. >>> There's no support at all for making up your own OSD IDs; sorry. >> >> NOW you tell me.. :) I have a small cluster where I am using the first digit >> of the OSD to indicate which server it is in and the second to indicate >> which disk in that server it uses. Aside from occasional spurious entries >> (for the missing digits) in the CRUSH map it seems to work fine. >> >> Can you elaborate on why this might be Bad? I'm planning and beginning to >> deploy a larger cluster and was going to use four-digit OSD id's (two for >> the server, two for the disk). What problems should I expect if I do that? >> >> How hard would it be to support making up one's own OSD ID's? >> >> I would also suggest that this be better documented on the website. The >> 5-minute quickstart says nothing about ID constraints and offers only 0 and >> 1 as examples. The "ceph-conf" page says only that "the instance ID for an >> OSD is always numeric". The "add-or-rm-osds" page mentions "ceph osd create" >> and its ability to set a "UUID" automatically, but only after several steps >> that require you to already know the "osd-number". > > The problem is that these IDs are used as indexes into arrays, and > people creating their own tends to lead to large and sparse arrays > which actually become expensive to handle for a variety of reasons. > Divorcing the "name" and "id" has been a thing in the back of our > minds for a while but has never made it all the way to something we > want to work on right now. :/ Until we support that we're pretty > unlikely to support making up one's own IDs — we used to do so and it > got people into so much trouble that we turned it off and just rely on > monitor allocation at this point. It's possible the docs are out of > date on this subject but they need not to be (John :). Thanks for clarifying! Yes, having names instead of ID numbers for OSD's would be superior to just handling sparse indexes efficiently. Consider this a vote for that feature, but for now I'll do without. JN _______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
