Hi,

Thanks for the replies. Likely will not choose this method but wanted to
make sure that it was a good technical reason rather than just a "best
practice". I did not quite think of "conntracker" at the time so this is a
good one to consider.

Thanks

Pieter

On 12 November 2014 14:30, Haomai Wang <haomaiw...@gmail.com> wrote:

> Actually, our production cluster(up to ten) all are that ceph-osd ran
> on compute-node(KVM).
>
> The primary action is that you need to constrain the cpu and memory.
> For example, you can alloc a ceph cpu-set and memory group, let
> ceph-osd run with it within limited cores and memory.
>
> The another risk is the network. Because compute-node and ceph-osd
> shared the same kernel network stack, it exists some risks that VM may
> ran out of network resources such as conntracker in netfilter
> framework.
>
> On Wed, Nov 12, 2014 at 10:23 PM, Mark Nelson <mark.nel...@inktank.com>
> wrote:
> > Technically there's no reason it shouldn't work, but it does complicate
> > things.  Probably the biggest worry would be that if something bad
> happens
> > on the compute side (say it goes nuts with network or memory transfers)
> it
> > could slow things down enough that OSDs start failing heartbeat checks
> > causing ceph to go into recovery and maybe cause a vicious cycle of
> > nastiness.
> >
> > You can mitigate some of this with cgroups and try to dedicate specific
> > sockets and memory banks to Ceph/Compute, but we haven't done a lot of
> > testing yet afaik.
> >
> > Mark
> >
> >
> > On 11/12/2014 07:45 AM, Pieter Koorts wrote:
> >>
> >> Hi,
> >>
> >> A while back on a blog I saw mentioned that Ceph should not be run on
> >> compute nodes and in the general sense should be on dedicated hardware.
> >> Does this really still apply?
> >>
> >> An example, if you have nodes comprised of
> >>
> >> 16+ cores
> >> 256GB+ RAM
> >> Dual 10GBE Network
> >> 2+8 OSD (SSD log + HDD store)
> >>
> >> I understand that Ceph can use a lot of IO and CPU in some cases but if
> >> the nodes are powerful enough does it not make it an option to run
> >> compute and storage on the same hardware to either increase density of
> >> compute or save money on additional hardware?
> >>
> >> What are the reasons for not running Ceph on the Compute nodes.
> >>
> >> Thanks
> >>
> >> Pieter
> >>
> >>
> >> _______________________________________________
> >> ceph-users mailing list
> >> ceph-users@lists.ceph.com
> >> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
> >>
> >
> > _______________________________________________
> > ceph-users mailing list
> > ceph-users@lists.ceph.com
> > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>
>
>
> --
> Best Regards,
>
> Wheat
> _______________________________________________
> ceph-users mailing list
> ceph-users@lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>
_______________________________________________
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Reply via email to