Re: [ceph-users] Prioritize Heartbeat packets

2015-03-09 Thread Robert LeBlanc
Jian, Thanks for the clarification. I'll mark traffic destined for the monitors as well. We are getting ready to put our first cluster into production. If you are interested we will be testing the heartbeat priority to see if we can saturate the network (not an easy task for 40 Gb) and keep the cl

Re: [ceph-users] Prioritize Heartbeat packets

2015-03-09 Thread Jian Wen
Only OSD calls set_socket_priority(). See https://github.com/ceph/ceph/pull/3353 On Tue, Mar 10, 2015 at 3:36 AM, Robert LeBlanc wrote: > I've found commit 9b9a682fe035c985e416ee1c112fa58f9045a27c and I see > that when 'osd heartbeat use min delay socket = true' it will mark the > packet with DS

Re: [ceph-users] Prioritize Heartbeat packets

2015-03-09 Thread Robert LeBlanc
I've found commit 9b9a682fe035c985e416ee1c112fa58f9045a27c and I see that when 'osd heartbeat use min delay socket = true' it will mark the packet with DSCP CS6. Based on the setting of the socket in msg/simple/Pipe.cc is it possible that this can apply to both OSD and monitor? I don't understand t

Re: [ceph-users] Prioritize Heartbeat packets

2015-03-07 Thread Daniel Swarbrick
Judging by the commit, this ought to do the trick: osd heartbeat use min delay socket = true On 07/03/15 01:20, Robert LeBlanc wrote: I see that Jian Wen has done work on this for 0.94. I tried looking through the code to see if I can figure out how to configure this new option, but it all went

Re: [ceph-users] Prioritize Heartbeat packets

2015-03-06 Thread Robert LeBlanc
I see that Jian Wen has done work on this for 0.94. I tried looking through the code to see if I can figure out how to configure this new option, but it all went over my head pretty quick. Can I get a brief summary on how to set the priority of heartbeat packets or where to look in the code to fig

Re: [ceph-users] Prioritize Heartbeat packets

2014-08-28 Thread Daniel Swarbrick
On 28/08/14 02:56, Sage Weil wrote: > I seem to remember someone telling me there were hooks/hints you could > call that would tag either a socket or possibly data on that socket with a > label for use by iptables and such.. but I forget what it was. > Something like setsockopt() SO_MARK?

Re: [ceph-users] Prioritize Heartbeat packets

2014-08-27 Thread Robert LeBlanc
Interesting concept. What if this was extended to an external message bus system like RabbitMQ, ZeroMQ, etc? Robert LeBlanc Sent from a mobile device please excuse any typos. On Aug 27, 2014 7:34 PM, "Matt W. Benjamin" wrote: > Hi, > > I wasn't thinking of an interface to mark sockets directly

Re: [ceph-users] Prioritize Heartbeat packets

2014-08-27 Thread Matt W. Benjamin
Hi, I wasn't thinking of an interface to mark sockets directly (didn't know one existed at the socket interface), rather something we might maintain, perhaps a query interface on the server, or perhaps DBUS, etc. Matt - "Sage Weil" wrote: > On Wed, 27 Aug 2014, Matt W. Benjamin wrote: >

Re: [ceph-users] Prioritize Heartbeat packets

2014-08-27 Thread Sage Weil
On Wed, 27 Aug 2014, Matt W. Benjamin wrote: > > - "Sage Weil" wrote: > > > > What would be best way for us to mark which sockets are heartbeat > > related? > > Is there some setsockopt() type call we should be using, or should we > > > > perhaps use a different port range for heartbeat t

Re: [ceph-users] Prioritize Heartbeat packets

2014-08-27 Thread Matt W. Benjamin
- "Sage Weil" wrote: > > What would be best way for us to mark which sockets are heartbeat > related? > Is there some setsockopt() type call we should be using, or should we > > perhaps use a different port range for heartbeat traffic? Would be be plausible to have hb messengers identify

Re: [ceph-users] Prioritize Heartbeat packets

2014-08-27 Thread Robert LeBlanc
On Wed, Aug 27, 2014 at 4:15 PM, Sage Weil wrote: > On Wed, 27 Aug 2014, Robert LeBlanc wrote: > > I'm looking for a way to prioritize the heartbeat traffic higher than the > > storage and replication traffic. I would like to keep the ceph.conf as > > simple as possible by not adding the individu

Re: [ceph-users] Prioritize Heartbeat packets

2014-08-27 Thread Sage Weil
On Wed, 27 Aug 2014, Robert LeBlanc wrote: > I'm looking for a way to prioritize the heartbeat traffic higher than the > storage and replication traffic. I would like to keep the ceph.conf as > simple as possible by not adding the individual osd IP addresses and ports, > but it looks like the liste