Re: [ceph-users] Calamari build in vagrants

[Steffen Winther]

Dan Mick  writes:

> 
> 0cbcfbaa791baa3ee25c4f1a135f005c1d568512 on the 1.2.3 branch has the
> change to yo 1.1.0.  I've just cherry-picked that to v1.3 and master.
Thanks, but sorry for my newbie question :)
How/where do I find this ID from the 1.2.3 branch?
on git I guess... will look there...

meanwhile I managed to built with precise64 vagrant
and locally installed yo@1.0.6, see my 
comments on this blog:
http://karan-mj.blogspot.fi/2014/09/ceph-calamari-survival-guide.html


___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] ceph-osd - No Longer Creates osd.X upon Launch - Bug ?

[Mykola Golub]

On Thu, Feb 05, 2015 at 08:33:39AM -0700, Ron Allred wrote:
> Hello,
> 
> The latest ceph-osd in Firefly v0.80.8, no longer auto creates its osd.X
> entry, in the osd map, which it was assigned via ceph.conf.
> 
> I am very aware documentation states "ceph osd create", can do this job,
> but this command only assigns the next sequential osd.X number.  This is
> highly undesirable.  For _years_, we have assigned number ranges to each
> OSD server for an organized multi-tier (SSD / SAS / SATA) crush map.
> (leaving gaps in osd numbering, naturally.)  Skipping 'ceph osd create'
> entirely.
> 
> We are now facing a problem that an OSD remove+replace, now can't use
> it's former osd.X ID.  Making a huge mess of documentation, number
> patterning, and disk labeling.
> 
> Is there a work-around to forcefully create an osd.X number??

The "ceph osd create" could be extended to have OSD ID as a second
optional argument (the first is already used for uuid).
 
  ceph osd create  

The command would succeed only if the ID were not in use.

Ron, would this work for you?

I have a patch as a proof of concept:

https://github.com/trociny/ceph/compare/wip-osd_create

-- 
Mykola Golub
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Having problem to start Radosgw

[Francois Lafont]

Hi,

Le 15/02/2015 08:37, B L a écrit :

> this is the resulting output after adding “-n client.radosgw.gateway” : 
> https://gist.github.com/anonymous/f16701d6cacc8911620f 
> 
> 
> I can see one problem only in the above output: -1 Couldn't init storage 
> provider (RADOS) .. please check the output, probably you can find something 
> useful

I can see: "librados: client.radosgw.gateway authentication error
(1) Operation not permitted".

Maybe a problem with the rights of your radosgw account?
Here is the keyring file of my radosgw.gw1 account that
works well:

~$ sudo cat /etc/ceph/ceph.client.radosgw.gw1.keyring 
[sudo] password for flaf: 
### This file is managed by Puppet, don't edit it ###

[client.radosgw.gw1]
  key = XX==
  caps mon = "allow rwx"
  caps osd = "allow rwx"

HTH

-- 
François Lafont
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[ceph-users] arm cluster install

[hp cre]

Hello all,  I'm currently studying the possibility of creating a small ceph
cluster on arm nodes.

The reasonably priced boards I found (like the banana pi/pro, Orange
pi/pro/h3, etc..) most have either dual core or quad core Allwinner chips
and 1GB RAM. They also use a micro sd card for os and a sata drive
connection.

Has anyone ever tried a deployment like this? I want to know if that
hardware is enough to create a usable cluster.

I dont want loads of storage,  just want to end up with a responsive os and
a 10 gb storage pool.

Beyond the basic minimum hardware listed on the ceph documents,  i haven't
found much info on this.  The pilot implementation with startup company
calxeda was on 64-bit arm boards,  not 32-bit,  and they were a custom
spec.

Any ideas welcome.  Thanks.
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[ceph-users] initially conf calamari to know about my Ceph cluster(s)

[Steffen Winther]

Trying to figure out how to initially configure calamari clients to know about 
my
Ceph Cluster(s) when such aint install through ceph.deploy
but through Proxmox pveceph.

Assume I possible need to copy some client admin keys and
configure my MON hosts somehow, any pointers to doc on this?

___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] ceph-osd - No Longer Creates osd.X upon Launch - Bug ?

[Sage Weil]

On Sun, 15 Feb 2015, Mykola Golub wrote:
> On Thu, Feb 05, 2015 at 08:33:39AM -0700, Ron Allred wrote:
> > Hello,
> > 
> > The latest ceph-osd in Firefly v0.80.8, no longer auto creates its osd.X
> > entry, in the osd map, which it was assigned via ceph.conf.
> > 
> > I am very aware documentation states "ceph osd create", can do this job,
> > but this command only assigns the next sequential osd.X number.  This is
> > highly undesirable.  For _years_, we have assigned number ranges to each
> > OSD server for an organized multi-tier (SSD / SAS / SATA) crush map.
> > (leaving gaps in osd numbering, naturally.)  Skipping 'ceph osd create'
> > entirely.
> > 
> > We are now facing a problem that an OSD remove+replace, now can't use
> > it's former osd.X ID.  Making a huge mess of documentation, number
> > patterning, and disk labeling.
> > 
> > Is there a work-around to forcefully create an osd.X number??
> 
> The "ceph osd create" could be extended to have OSD ID as a second
> optional argument (the first is already used for uuid).
>  
>   ceph osd create  
> 
> The command would succeed only if the ID were not in use.
> 
> Ron, would this work for you?
> 
> I have a patch as a proof of concept:
> 
> https://github.com/trociny/ceph/compare/wip-osd_create

This looks reasonable to me!

Do you mind adding a few test cases in qa/workunits/cephtool/test.sh to go 
along with it?

Usual disclaimer: we discourage getting creative with the osd ids because 
they are allocated as an *array* in memory, so skipping entries consumes 
some extra memory.. this can become significant if there are large 
gaps and/or clusters are large.

Thanks!
sage

___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] ceph-osd - No Longer Creates osd.X upon Launch - Bug ?

[Gregory Farnum]

On Sun, Feb 15, 2015 at 5:39 PM, Sage Weil  wrote:
> On Sun, 15 Feb 2015, Mykola Golub wrote:
>> On Thu, Feb 05, 2015 at 08:33:39AM -0700, Ron Allred wrote:
>> > Hello,
>> >
>> > The latest ceph-osd in Firefly v0.80.8, no longer auto creates its osd.X
>> > entry, in the osd map, which it was assigned via ceph.conf.
>> >
>> > I am very aware documentation states "ceph osd create", can do this job,
>> > but this command only assigns the next sequential osd.X number.  This is
>> > highly undesirable.  For _years_, we have assigned number ranges to each
>> > OSD server for an organized multi-tier (SSD / SAS / SATA) crush map.
>> > (leaving gaps in osd numbering, naturally.)  Skipping 'ceph osd create'
>> > entirely.
>> >
>> > We are now facing a problem that an OSD remove+replace, now can't use
>> > it's former osd.X ID.  Making a huge mess of documentation, number
>> > patterning, and disk labeling.
>> >
>> > Is there a work-around to forcefully create an osd.X number??
>>
>> The "ceph osd create" could be extended to have OSD ID as a second
>> optional argument (the first is already used for uuid).
>>
>>   ceph osd create  
>>
>> The command would succeed only if the ID were not in use.
>>
>> Ron, would this work for you?
>>
>> I have a patch as a proof of concept:
>>
>> https://github.com/trociny/ceph/compare/wip-osd_create
>
> This looks reasonable to me!
>
> Do you mind adding a few test cases in qa/workunits/cephtool/test.sh to go
> along with it?
>
> Usual disclaimer: we discourage getting creative with the osd ids because
> they are allocated as an *array* in memory, so skipping entries consumes
> some extra memory.. this can become significant if there are large
> gaps and/or clusters are large.

These options used to exist and were removed quite deliberately. I
don't remember the entire conversation at this point but we'll need to
find and address the concerns raised then before reintroducing the
ability to explicitly set OSD IDs. IIRC I was on the losing end of
this, because it's definitely behavior we should be offering to
admins, but the issues were significant enough we had to eliminate the
option. Methods of preserving the user-facing utility like adding OSD
names were deemed too difficult to implement. :(

(I think it largely had to do with serious issues over the
availability and location of data when OSDs disappear, but new ones
with the same ID are present. And what you do when somebody then
resurrects the original OSDs. But there might have been other things
too.)

-Greg
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] ceph-osd - No Longer Creates osd.X upon Launch - Bug ?

[Sage Weil]

On Sun, 15 Feb 2015, Gregory Farnum wrote:
> On Sun, Feb 15, 2015 at 5:39 PM, Sage Weil  wrote:
> > On Sun, 15 Feb 2015, Mykola Golub wrote:
> >> On Thu, Feb 05, 2015 at 08:33:39AM -0700, Ron Allred wrote:
> >> > Hello,
> >> >
> >> > The latest ceph-osd in Firefly v0.80.8, no longer auto creates its osd.X
> >> > entry, in the osd map, which it was assigned via ceph.conf.
> >> >
> >> > I am very aware documentation states "ceph osd create", can do this job,
> >> > but this command only assigns the next sequential osd.X number.  This is
> >> > highly undesirable.  For _years_, we have assigned number ranges to each
> >> > OSD server for an organized multi-tier (SSD / SAS / SATA) crush map.
> >> > (leaving gaps in osd numbering, naturally.)  Skipping 'ceph osd create'
> >> > entirely.
> >> >
> >> > We are now facing a problem that an OSD remove+replace, now can't use
> >> > it's former osd.X ID.  Making a huge mess of documentation, number
> >> > patterning, and disk labeling.
> >> >
> >> > Is there a work-around to forcefully create an osd.X number??
> >>
> >> The "ceph osd create" could be extended to have OSD ID as a second
> >> optional argument (the first is already used for uuid).
> >>
> >>   ceph osd create  
> >>
> >> The command would succeed only if the ID were not in use.
> >>
> >> Ron, would this work for you?
> >>
> >> I have a patch as a proof of concept:
> >>
> >> https://github.com/trociny/ceph/compare/wip-osd_create
> >
> > This looks reasonable to me!
> >
> > Do you mind adding a few test cases in qa/workunits/cephtool/test.sh to go
> > along with it?
> >
> > Usual disclaimer: we discourage getting creative with the osd ids because
> > they are allocated as an *array* in memory, so skipping entries consumes
> > some extra memory.. this can become significant if there are large
> > gaps and/or clusters are large.
> 
> These options used to exist and were removed quite deliberately. I
> don't remember the entire conversation at this point but we'll need to
> find and address the concerns raised then before reintroducing the
> ability to explicitly set OSD IDs. IIRC I was on the losing end of
> this, because it's definitely behavior we should be offering to
> admins, but the issues were significant enough we had to eliminate the
> option. Methods of preserving the user-facing utility like adding OSD
> names were deemed too difficult to implement. :(
> 
> (I think it largely had to do with serious issues over the
> availability and location of data when OSDs disappear, but new ones
> with the same ID are present. And what you do when somebody then
> resurrects the original OSDs. But there might have been other things
> too.)

The part I remember was just that 'ceph osd create ' wasn't a use and 
idempotent command.  I don't think reusing ids is the problem, though if 
it is then it is still a problem since osd create will re-use the first 
available id.  I think all this option lets us do that we didn't before is 
leave gaps in the id space?

sage
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] ceph-osd - No Longer Creates osd.X upon Launch - Bug ?

[Mykola Golub]

On Sun, Feb 15, 2015 at 06:24:45PM -0800, Sage Weil wrote:
> On Sun, 15 Feb 2015, Gregory Farnum wrote:
> > On Sun, Feb 15, 2015 at 5:39 PM, Sage Weil  wrote:
> > > On Sun, 15 Feb 2015, Mykola Golub wrote:

> > >> https://github.com/trociny/ceph/compare/wip-osd_create
> > >
> > > This looks reasonable to me!
> > >
> > > Do you mind adding a few test cases in qa/workunits/cephtool/test.sh to go
> > > along with it?

Will do. Thanks.

> > >
> > > Usual disclaimer: we discourage getting creative with the osd ids because
> > > they are allocated as an *array* in memory, so skipping entries consumes
> > > some extra memory.. this can become significant if there are large
> > > gaps and/or clusters are large.
> > 
> > These options used to exist and were removed quite deliberately. I
> > don't remember the entire conversation at this point but we'll need to
> > find and address the concerns raised then before reintroducing the
> > ability to explicitly set OSD IDs. IIRC I was on the losing end of
> > this, because it's definitely behavior we should be offering to
> > admins, but the issues were significant enough we had to eliminate the
> > option. Methods of preserving the user-facing utility like adding OSD
> > names were deemed too difficult to implement. :(
> > 
> > (I think it largely had to do with serious issues over the
> > availability and location of data when OSDs disappear, but new ones
> > with the same ID are present. And what you do when somebody then
> > resurrects the original OSDs. But there might have been other things
> > too.)
> 
> The part I remember was just that 'ceph osd create ' wasn't a use and 
> idempotent command.  I don't think reusing ids is the problem, though if 
> it is then it is still a problem since osd create will re-use the first 
> available id.  I think all this option lets us do that we didn't before is 
> leave gaps in the id space?

I think so too -- leaving gaps should be the only difference comparing
to what we already have. I was wandering though if I might need to do
something with those nonexistent IDs in the gap, e.g. pre-allocating
them explicitly with some flags combination. It looks like I don't...

-- 
Mykola Golub
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com