Re: 10GE TOR port buffers (was Re: 10G switch recommendaton)

2012-01-28 Thread Saku Ytti
On (2012-01-28 21:53 +0900), Masataka Ohta wrote: > 1.5MB @ 100Mbps is 120ms, which is prohibitively lengthy > even as BE. > > The solution is to have less number of classes. The solution is to per class define max queue size, so user with fewer queues configured will not use all available buff

Re: 10GE TOR port buffers (was Re: 10G switch recommendaton)

2012-01-28 Thread Masataka Ohta
Saku Ytti wrote: >>> And if switch does support QoS but operator configures only BE, and >>> operator does not limit BE queue size, operator will see buffer bloat, >> >> 1.5MB @ 10Gbps is only 1.2ms, which is not buffer bloat. > > You can't buffer these in ingress or you risk HOLB issue, you must

Re: 10GE TOR port buffers (was Re: 10G switch recommendaton)

2012-01-28 Thread Saku Ytti
On (2012-01-28 21:06 +0900), Masataka Ohta wrote: > The required amount of memory is merely 150KB. Assuming we don't support jumbo frames and switch cannot queue sub packet sizes (normally they can't but VXR at least has 512B cell concept, so tx-ring is packet size agnostic, but this is just PA-A

Re: 10GE TOR port buffers (was Re: 10G switch recommendaton)

2012-01-28 Thread Masataka Ohta
Saku Ytti wrote: > No, you're not crazy. If your core is higher rate than your customer, then > you need at minimum serialization delay difference of buffering. > If core is 10G and access 100M, you need buffer for minimum of 100 packets, > to handle the single 10G incoming, without any extra buff

Re: 10GE TOR port buffers (was Re: 10G switch recommendaton)

2012-01-28 Thread Saku Ytti
On (2012-01-27 22:40 +0100), bas wrote: > But do you generally agree that "the market" has a requirement for a > deep-buffer TOR switch? > > Or am I crazy for thinking that my customers need such a solution? No, you're not crazy. If your core is higher rate than your customer, then you need at m

Re: 10GE TOR port buffers (was Re: 10G switch recommendaton)

2012-01-27 Thread Łukasz Bromirski
On 2012-01-28 01:00, Joel jaeggli wrote: C NSP has been full with threads about appalling microburst performance of the 6500 for years.. And people who care have been using something other than a c6500 for years. it's a 15 year old architecture, and it's had a pretty good run, but it's 2012. An

Re: 10GE TOR port buffers (was Re: 10G switch recommendaton)

2012-01-27 Thread Leo Bicknell
In a message written on Fri, Jan 27, 2012 at 04:00:36PM -0800, Joel jaeggli wrote: > And people who care have been using something other than a c6500 for > years. it's a 15 year old architecture, and it's had a pretty good run, > but it's 2012. One of the frustrating things, which the c6500 embod

RE: 10GE TOR port buffers (was Re: 10G switch recommendaton)

2012-01-27 Thread George Bonser
ons a crap shoot, at best. From: Nick Hilliard Sent: Friday, January 27, 2012 4:51 PM To: bas Cc: George Bonser; nanog Subject: Re: 10GE TOR port buffers (was Re: 10G switch recommendaton) In the case of a tor switch, the use case is typically in a situation where you're not downsteppin

RE: 10GE TOR port buffers (was Re: 10G switch recommendaton)

2012-01-27 Thread George Bonser
> > It is also possible and in fact easy to have enough to accumulate > latency in places where you should be discarding packets earlier. > > I'd rather not be in either situation, but in the later I can police my > way out of it. That is why I added the "it depends on the end to end application

Re: 10GE TOR port buffers (was Re: 10G switch recommendaton)

2012-01-27 Thread Nick Hilliard
On 27 Jan 2012, at 23:08, bas wrote: > Im my (our) busines model _is_ the internet connectivity... > We could give the customer double the port capacity, if they were > willing to pay, but in real life they do not care... > > While all respondents replies hold truth a (technial business) logic. >

Re: 10GE TOR port buffers (was Re: 10G switch recommendaton)

2012-01-27 Thread Joel jaeggli
On 1/27/12 15:40 , bas wrote: > Hi All, > > On Sat, Jan 28, 2012 at 12:32 AM, Joel jaeggli wrote: >> On 1/27/12 14:53 , bas wrote: >>> While I agree _again_! >>> >>> It does not explain why TOR boxes have little buffers and chassis box >>> have many. >> >> you need purportionally more buf

Re: 10GE TOR port buffers (was Re: 10G switch recommendaton)

2012-01-27 Thread bas
Hi All, On Sat, Jan 28, 2012 at 12:32 AM, Joel jaeggli wrote: > On 1/27/12 14:53 , bas wrote: >> While I agree _again_! >> >> It does not explain why TOR boxes have little buffers and chassis box >> have many. > > you need purportionally more buffer when you need to drain 16 x 10 gig > in

Re: 10GE TOR port buffers (was Re: 10G switch recommendaton)

2012-01-27 Thread Joel jaeggli
On 1/27/12 15:01 , George Bonser wrote: > > >> -Original Message- From: bas Sent: Friday, January 27, 2012 >> 2:54 PM To: George Bonser Subject: Re: 10GE TOR port buffers (was >> Re: 10G switch recommendaton) >> >> While I agree _again_! >>

Re: 10GE TOR port buffers (was Re: 10G switch recommendaton)

2012-01-27 Thread Joel jaeggli
On 1/27/12 14:53 , bas wrote: > While I agree _again_! > > It does not explain why TOR boxes have little buffers and chassis box > have many. you need purportionally more buffer when you need to drain 16 x 10 gig into 4 x 10Gig then when you're trying to drain 10Gb/s into 2 x 1Gb/s there

Re: 10GE TOR port buffers (was Re: 10G switch recommendaton)

2012-01-27 Thread bas
Hi, > The margin on a top-of-rack switch is very low.  48 port gige with > 10GE uplinks are basically commodity boxes, with plenty of competition. > Saving $100 on the bill of materials by cutting out some buffer > makes the box more competitive when it's at a $2k price point. The list of 10GE TO

Re: 10GE TOR port buffers (was Re: 10G switch recommendaton)

2012-01-27 Thread bas
On Sat, Jan 28, 2012 at 12:01 AM, George Bonser wrote: > Going to 4 10G aggregated uplinks instead of 2 might get you a much better > performance boost than increasing buffers. > But it really depends on the end to end application. Also these TOR boxes go to my (more expensive ASR9K and MX) boxe

Re: 10GE TOR port buffers (was Re: 10G switch recommendaton)

2012-01-27 Thread bas
Hi, On Fri, Jan 27, 2012 at 11:54 PM, George Bonser wrote: >> >> My customers want to buffer 10 to 24 * 10GE in a 1 or 2 10GE uplinks to >> do this they need some buffers >> >> Bas > > It might be cheaper for them to go to 3 or 4 10G uplinks than to replace all > their switch hardware. Im m

Re: 10GE TOR port buffers (was Re: 10G switch recommendaton)

2012-01-27 Thread Leo Bicknell
In a message written on Fri, Jan 27, 2012 at 11:30:14PM +0100, bas wrote: > While your reasoning holds truth it does not explain why the expensive > chassis solution (good) makes my customers happy, and the cheaper TOR > solution makes my customers unhappy. > > Bufferbloat does not matter to t

RE: 10GE TOR port buffers (was Re: 10G switch recommendaton)

2012-01-27 Thread George Bonser
> -Original Message- > From: bas > Sent: Friday, January 27, 2012 2:54 PM > To: George Bonser > Subject: Re: 10GE TOR port buffers (was Re: 10G switch recommendaton) > > While I agree _again_! > > It does not explain why TOR boxes have little buffers

Re: 10GE TOR port buffers (was Re: 10G switch recommendaton)

2012-01-27 Thread bas
While I agree _again_! It does not explain why TOR boxes have little buffers and chassis box have many. On Fri, Jan 27, 2012 at 11:36 PM, George Bonser wrote: >> >> Buffers in most network gear is bad, don't do it. >> > > +1 > > I'm amazed at how many will spend money on switches with mo

RE: 10GE TOR port buffers (was Re: 10G switch recommendaton)

2012-01-27 Thread George Bonser
> > Buffers in most network gear is bad, don't do it. > +1 I'm amazed at how many will spend money on switches with more buffering but won't take steps to ease the congestion. Part of the reason is trying to convince non-technical people that packet loss in and of itself doesn't have to be

Re: 10GE TOR port buffers (was Re: 10G switch recommendaton)

2012-01-27 Thread bas
Hi, On Fri, Jan 27, 2012 at 10:52 PM, Leo Bicknell wrote: > In a message written on Fri, Jan 27, 2012 at 10:40:03PM +0100, bas wrote: >> But do you generally agree that "the market" has a requirement for a >> deep-buffer TOR switch? >> >> Or am I crazy for thinking that my customers need such a s

Re: 10GE TOR port buffers (was Re: 10G switch recommendaton)

2012-01-27 Thread Leo Bicknell
In a message written on Fri, Jan 27, 2012 at 10:40:03PM +0100, bas wrote: > But do you generally agree that "the market" has a requirement for a > deep-buffer TOR switch? > > Or am I crazy for thinking that my customers need such a solution? You're crazy. :) You need to google "bufferbloat", whi

Re: 10GE TOR port buffers (was Re: 10G switch recommendaton)

2012-01-27 Thread bas
On Fri, Jan 27, 2012 at 5:55 PM, Saku Ytti wrote: > On (2012-01-27 17:35 +0100), bas wrote: > But generally nice list, especially the 10GE fixed config looked realistic, > sometimes I wish we'd have 'dpreview' style page for routers and switches, > especially now with dozen or more vendors selling

RE: 10GE TOR port buffers (was Re: 10G switch recommendaton)

2012-01-27 Thread Tom Ammon
The HP6600 is a store and forward, not a cut-through. The HP reps that I have dealt with seem to be pretty open to sharing architecture drawings of their stuff, so I bet you could probably get your hands on the same one that I have. Their NDA is a mutual disclosure, though, so that might make th

Re: 10GE TOR port buffers (was Re: 10G switch recommendaton)

2012-01-27 Thread Saku Ytti
On (2012-01-27 17:35 +0100), bas wrote: > Chassis: > Juniper EX8200-8XS512MB/10GE > Cisco WS-X6708-10GE 32MB/10GE (or 24MB) > Cisco N7K-M132XP-12 36MB/10GE > Arista DCS-7548S-LC 48MB/10GE > Brocade BR-MLX-10Gx8-X128MB/10GE (not sure) > > 1GE aggr