Re: few big monolithic PEs vs many small PEs

2019-07-02 Thread Mark Tinka
On 27/Jun/19 21:41, James Bensley wrote: > > > Large boxes like the MX2020, ASR9922, NCS6K, etc. these can only reasonably > be used as P nodes in my opinion. The NCS6000 was always designed as a core router to replace the CRS. We just haven't seen the need for one since the CRS-X we run we o

Re: few big monolithic PEs vs many small PEs

2019-06-29 Thread Mark Tinka
On 28/Jun/19 10:35, adamv0...@netconsultings.com wrote: > If the PEs are sufficiently small I'd even go further as to L3VPNs-PE vs > L2VPNs-PE services etc..., it's mostly because of streamlined/simplified hw > and code certification testing. > But as with all the decentralize-centralize swi

RE: few big monolithic PEs vs many small PEs

2019-06-28 Thread adamv0025
> Mark Tinka > Sent: Thursday, June 27, 2019 4:31 PM > > > > On 27/Jun/19 14:48, James Bensley wrote: > > > That to me is a simple scenario, and it can be mapped with a > > dependency tree. But in my experience, and maybe it's just me, things > > are usually a lot more complicated than this. Th

RE: few big monolithic PEs vs many small PEs

2019-06-28 Thread adamv0025
Hi James, > From: James Bensley > Sent: Thursday, June 27, 2019 1:48 PM > > On Thu, 27 Jun 2019 at 12:46, wrote: > > > > > From: James Bensley > > > Sent: Thursday, June 27, 2019 9:56 AM > > > > > > One experience I have made is that when there is an outage on a > > > large PE, even when it st

Re: few big monolithic PEs vs many small PEs

2019-06-27 Thread Mark Tinka
On 28/Jun/19 01:23, Mike Hammett wrote: > I've ran into many providers where they had routers in the top 10 or > 15 markets...  and that was it. If you wanted a connection in South > Bend or Indianapolis or New Orleans or Ohio or...  you were backhauled > potentially hundreds of miles to a nearb

Re: few big monolithic PEs vs many small PEs

2019-06-27 Thread Mike Hammett
Big routers also mean they're a lot more expensive. You have to squeeze more life out of them because they cost you hundreds of thousands of dollars. You run them longer than you really should. If you run more, smaller, $20k or $30k routers, you'll replace them on a more reasonable cycle.

Re: few big monolithic PEs vs many small PEs

2019-06-27 Thread Mike Hammett
I've ran into many providers where they had routers in the top 10 or 15 markets... and that was it. If you wanted a connection in South Bend or Indianapolis or New Orleans or Ohio or... you were backhauled potentially hundreds of miles to a nearby big market. More smaller POPs reduces the tro

Re: few big monolithic PEs vs many small PEs

2019-06-27 Thread James Bensley
On 27 June 2019 16:26:03 BST, Mark Tinka wrote: > > >On 27/Jun/19 10:58, James Bensley wrote: > >> Hi Adam, >> >> Over the years I have been bitten multiple times by having fewer big >> routers with either far too many services/customers connected to them >> or too much traffic going through th

Re: few big monolithic PEs vs many small PEs

2019-06-27 Thread James Bensley
On 27 June 2019 16:31:27 BST, Mark Tinka wrote: > > >On 27/Jun/19 14:48, James Bensley wrote: > >> That to me is a simple scenario, and it can be mapped with a >> dependency tree. But in my experience, and maybe it's just me, things >> are usually a lot more complicated than this. The root cause i

Re: few big monolithic PEs vs many small PEs

2019-06-27 Thread Mark Tinka
On 27/Jun/19 14:48, James Bensley wrote: > That to me is a simple scenario, and it can be mapped with a > dependency tree. But in my experience, and maybe it's just me, things > are usually a lot more complicated than this. The root cause is > probably a bad design introducing too much complexi

Re: few big monolithic PEs vs many small PEs

2019-06-27 Thread Mark Tinka
On 27/Jun/19 14:03, adamv0...@netconsultings.com wrote: > I believe it would, for a time, but it would require SW upgrade -testing > etc.. even newer SW in itself gave us better resource management and > performance optimizations. > However even with powerful CP and streamlined SW we'd be sti

Re: few big monolithic PEs vs many small PEs

2019-06-27 Thread Mark Tinka
On 27/Jun/19 10:58, James Bensley wrote: > Hi Adam, > > Over the years I have been bitten multiple times by having fewer big > routers with either far too many services/customers connected to them > or too much traffic going through them. These days I always go for > more smaller/more routers t

Re: few big monolithic PEs vs many small PEs

2019-06-27 Thread James Bensley
On Thu, 27 Jun 2019 at 12:46, wrote: > > > From: James Bensley > > Sent: Thursday, June 27, 2019 9:56 AM > > > > One experience I have made is that when there is an outage on a large PE, > > even when it still has spare capacity, is that the business impact can be > > too > > much to handle (the

Re: few big monolithic PEs vs many small PEs

2019-06-27 Thread James Bensley
On Wed, 19 Jun 2019 at 21:23, wrote: > > Hi folks, > > Recently I ran into a peculiar situation where we had to cap couple of PE > even though merely a half of the rather big chassis was populated with > cards, reason being that the central RE/RP was not able to cope with the > combined number of

RE: few big monolithic PEs vs many small PEs

2019-06-27 Thread adamv0025
> From: Mark Tinka > Sent: Friday, June 21, 2019 1:27 PM > > > > On 21/Jun/19 10:32, adamv0...@netconsultings.com wrote: > > > So this particular case, the major POPs, is actually where we ran into the > problem of RE/RP becoming full (too many VRFs/Routes/BGP sessions) > halfway through the c

RE: few big monolithic PEs vs many small PEs

2019-06-27 Thread adamv0025
> From: James Bensley > Sent: Thursday, June 27, 2019 9:56 AM > > One experience I have made is that when there is an outage on a large PE, > even when it still has spare capacity, is that the business impact can be too > much to handle (the support desk is overwhelmed, customers become irate > i

Re: few big monolithic PEs vs many small PEs

2019-06-21 Thread Bryan Holloway
On 6/21/19 10:01 AM, Aaron Gould wrote: I was reading this and thought, planet earth is a single point of failure. ...but, I guess we build and design and connect as much redundancy (logic, hw, sw, power) as the customer requires and pays for and that we can truly accomplish. -Aaron

Re: few big monolithic PEs vs many small PEs

2019-06-21 Thread Anderson, Charles R
On Fri, Jun 21, 2019 at 09:01:38AM -0500, Aaron Gould wrote: > I was reading this and thought, planet earth is a single point of failure. > > ...but, I guess we build and design and connect as much redundancy (logic, > hw, sw, power) as the customer requires and pays for and that we can

RE: few big monolithic PEs vs many small PEs

2019-06-21 Thread Aaron Gould
I was reading this and thought, planet earth is a single point of failure. ...but, I guess we build and design and connect as much redundancy (logic, hw, sw, power) as the customer requires and pays for and that we can truly accomplish. -Aaron

Re: few big monolithic PEs vs many small PEs

2019-06-21 Thread Mike Hammett
ll MPLS routers. - Mike Hammett Intelligent Computing Solutions Midwest Internet Exchange The Brothers WISP - Original Message - From: "Tarko Tikan" To: adamv0...@netconsultings.com, nanog@nanog.org Sent: Friday, June 21, 2019 2:51:20 AM Subject: Re: few big monolit

Re: few big monolithic PEs vs many small PEs

2019-06-21 Thread Mark Tinka
On 21/Jun/19 10:32, adamv0...@netconsultings.com wrote: > Well yes but if say I compare just a single line-card cost to a standalone > fixed-format 1RU router with a similar capacity -the card will always be > cheaper and then as I'll start adding cards on the left-hand side of the > equatio

Re: few big monolithic PEs vs many small PEs

2019-06-21 Thread Mark Tinka
On 21/Jun/19 10:46, adamv0...@netconsultings.com wrote: > I'd actually like to hear more on that if you don't mind. What part, Juniper's Ethernet switching portfolio? > You actually haven't answered the question I'm afraid :) > So would you connect the Juniper now Arista aggregation switch t

RE: few big monolithic PEs vs many small PEs

2019-06-21 Thread adamv0025
> From: Mark Tinka > Sent: Friday, June 21, 2019 9:07 AM > > > > On 21/Jun/19 09:36, adamv0...@netconsultings.com wrote: > > > And indeed there are cases where we connect customers directly on to > > the PEs, but then it's somehow ok for a line-card to be part of just a > > single chassis (or a

RE: few big monolithic PEs vs many small PEs

2019-06-21 Thread adamv0025
Hey Mark, > From: Mark Tinka > Sent: Thursday, June 20, 2019 3:27 PM > > On 19/Jun/19 22:22, adamv0...@netconsultings.com wrote: > > > Yes it will cost a bit more (router is more expensive than a LC) > > I found the reverse to be true... chassis' are cheap. Line cards are costly. > Well yes but

Re: few big monolithic PEs vs many small PEs

2019-06-21 Thread Mark Tinka
On 21/Jun/19 09:36, adamv0...@netconsultings.com wrote: > And indeed there are cases where we connect customers directly on to > the PEs, but then it's somehow ok for a line-card to be part of just a > single chassis (or a PE). We'd typically do this for very high-speed ports (100Gbps), as it

Re: few big monolithic PEs vs many small PEs

2019-06-21 Thread Tarko Tikan
hey, So what is the primary goal of us using the aggregation/access layer? It's to achieve better utilization of the expensive router ports right? (hence called aggregation) I'm in the eyeball business so saving router ports is not a primary concern. Aggregation exists to aggregate downstre

RE: few big monolithic PEs vs many small PEs

2019-06-21 Thread adamv0025
Hey, > From: Tarko Tikan > Sent: Thursday, June 20, 2019 8:28 AM > > hey, > > > For availability I think it is best approach to do many small edge > > devices. > > This is also great for planned maintenance. ISSU has not really worked out for > any of the vendors and with two small devices you

Re: few big monolithic PEs vs many small PEs

2019-06-21 Thread Saku Ytti
On Fri, 21 Jun 2019 at 10:09, wrote: > Just on the human cockups though, we're putting more and more automation in > to help address the problem of human imperfections. With automation we break far far less often, far far more. MTTR is also increased due to skill rot, in CLI jockey network you

RE: few big monolithic PEs vs many small PEs

2019-06-21 Thread adamv0025
Hey Saku, > From: Saku Ytti > Sent: Thursday, June 20, 2019 7:04 AM > > On Wed, 19 Jun 2019 at 23:25, wrote: > > > The conclusion I came to was that *currently the best approach would > > be to use several medium to small(fixed) PEs to replace a big > > monolithic chasses based system. > > Fo

Re: few big monolithic PEs vs many small PEs

2019-06-20 Thread Mark Tinka
On 19/Jun/19 22:22, adamv0...@netconsultings.com wrote: > Yes it will cost a bit more (router is more expensive than a LC) I found the reverse to be true... chassis' are cheap. Line cards are costly. > > Would like to hear what are your thoughts on this conundrum. So this depends on where

Re: few big monolithic PEs vs many small PEs

2019-06-20 Thread Tarko Tikan
hey, For availability I think it is best approach to do many small edge devices. This is also great for planned maintenance. ISSU has not really worked out for any of the vendors and with two small devices you can upgrade them independently. Great for aggregation, enables you to dual-home

Re: few big monolithic PEs vs many small PEs

2019-06-19 Thread Saku Ytti
On Wed, 19 Jun 2019 at 23:25, wrote: > The conclusion I came to was that *currently the best approach would be to > use several medium to small(fixed) PEs to replace a big monolithic chasses > based system. For availability I think it is best approach to do many small edge devices. Because softw

Re: few big monolithic PEs vs many small PEs

2019-06-19 Thread i3D.net - Martijn Schmidt via NANOG
Hi Adam, Depends on how big of a router you need for your "small PE". Taking Juniper as an example, the MX204 is pretty unbeatable cost wise if you can make do with its 4*QSFP28 & 8*SFP+ interfaces. There's a very big gap between the MX204 and the first chassis based router in the MX lineup, ev