Re: Managing IOS Configuration Snippets
For a large install I set up a solution that might help. I utilized a Mediawiki install and its API to create, update and pull the configuration on many IOS devices. A wiki page for the host name was dynamically created and the configuration was placed there daily or hourly. This allowed support to review the configuration and advise customers quicker. Additional hacks for updating the devices via the wiki were used. The goal was transparency for the support team and the side effect was wiki page history showing what day and what lines changed. As mentioned the answer to your question would likely make a good article. On Wed, Feb 26, 2014 at 3:22 PM, Ryan Shea wrote: > Howdy network operator cognoscenti, > > I'd love to hear your creative and workable solutions for a way to track > in-line the configuration revisions you have on your cisco-like devices. > Let me clearify/frame: > > You have a set of tested/approved configurations for your routers which use > IOS style configuration. These configurations of course are always refined > and updated. You break these pieces of configuration into logical sections, > for example a configuration file for NTP configuration, a file for control > plane filter and store these in some revision control system. Put aside for > the moment whether this is a reasonable way to comprehend deployed > configurations. What methods do some of you use to know which version of a > configuration you have deployed to a given router for auditing and update > purposes? Remarks are a convenient way to do this for ACLs - but I don't > have similar mechanics for top level configurations. About a decade ago I > thought I'd be super clever and encode versioning information into the snmp > location - but that is just awful and there is a much better way everyone > is using, right? Flexible commenting on other vendors/platforms make this a > bit easier. > > Assume that this version encoding perfectly captures what is on the router > and that no person is monkeying with the config... version 77 of the > control plane filter is the same everywhere. -- ~ Andrew "lathama" Latham lath...@gmail.com http://lathama.net ~
Re: Managing IOS Configuration Snippets
Wow, this sounds fantastic! Have any code you can share? Cheers, Harry On Feb 27, 2014 6:52 AM, Andrew Latham wrote: > > For a large install I set up a solution that might help. I utilized a > Mediawiki install and its API to create, update and pull the > configuration on many IOS devices. A wiki page for the host name was > dynamically created and the configuration was placed there daily or > hourly. This allowed support to review the configuration and advise > customers quicker. Additional hacks for updating the devices via the > wiki were used. The goal was transparency for the support team and the > side effect was wiki page history showing what day and what lines > changed. As mentioned the answer to your question would likely make a > good article. > > On Wed, Feb 26, 2014 at 3:22 PM, Ryan Shea wrote: > > Howdy network operator cognoscenti, > > > > I'd love to hear your creative and workable solutions for a way to track > > in-line the configuration revisions you have on your cisco-like devices. > > Let me clearify/frame: > > > > You have a set of tested/approved configurations for your routers which use > > IOS style configuration. These configurations of course are always refined > > and updated. You break these pieces of configuration into logical sections, > > for example a configuration file for NTP configuration, a file for control > > plane filter and store these in some revision control system. Put aside for > > the moment whether this is a reasonable way to comprehend deployed > > configurations. What methods do some of you use to know which version of a > > configuration you have deployed to a given router for auditing and update > > purposes? Remarks are a convenient way to do this for ACLs - but I don't > > have similar mechanics for top level configurations. About a decade ago I > > thought I'd be super clever and encode versioning information into the snmp > > location - but that is just awful and there is a much better way everyone > > is using, right? Flexible commenting on other vendors/platforms make this a > > bit easier. > > > > Assume that this version encoding perfectly captures what is on the router > > and that no person is monkeying with the config... version 77 of the > > control plane filter is the same everywhere. > > > > -- > ~ Andrew "lathama" Latham lath...@gmail.com http://lathama.net ~ >
Re: Managing IOS Configuration Snippets
Rancid with the git plugin can be used to attain pretty much the exact same thing a lot more easily, if you're after an existing implementation of it. Cheers, Paul On 2/27/2014 午後 09:44, Harry Hoffman wrote: Wow, this sounds fantastic! Have any code you can share? Cheers, Harry On Feb 27, 2014 6:52 AM, Andrew Latham wrote: For a large install I set up a solution that might help. I utilized a Mediawiki install and its API to create, update and pull the configuration on many IOS devices. A wiki page for the host name was dynamically created and the configuration was placed there daily or hourly. This allowed support to review the configuration and advise customers quicker. Additional hacks for updating the devices via the wiki were used. The goal was transparency for the support team and the side effect was wiki page history showing what day and what lines changed. As mentioned the answer to your question would likely make a good article. On Wed, Feb 26, 2014 at 3:22 PM, Ryan Shea wrote: Howdy network operator cognoscenti, I'd love to hear your creative and workable solutions for a way to track in-line the configuration revisions you have on your cisco-like devices. Let me clearify/frame: You have a set of tested/approved configurations for your routers which use IOS style configuration. These configurations of course are always refined and updated. You break these pieces of configuration into logical sections, for example a configuration file for NTP configuration, a file for control plane filter and store these in some revision control system. Put aside for the moment whether this is a reasonable way to comprehend deployed configurations. What methods do some of you use to know which version of a configuration you have deployed to a given router for auditing and update purposes? Remarks are a convenient way to do this for ACLs - but I don't have similar mechanics for top level configurations. About a decade ago I thought I'd be super clever and encode versioning information into the snmp location - but that is just awful and there is a much better way everyone is using, right? Flexible commenting on other vendors/platforms make this a bit easier. Assume that this version encoding perfectly captures what is on the router and that no person is monkeying with the config... version 77 of the control plane filter is the same everywhere. -- ~ Andrew "lathama" Latham lath...@gmail.com http://lathama.net ~
Re: Managing IOS Configuration Snippets
On (2014-02-26 17:37 -0500), Robert Drake wrote: > Consider looking at Tail-F's NCS, which according to marketing > presentations appears to do everything I want right now. I'd like > to believe them but I don't have any money so I can't test it out. > :) Tail-F is probably least bad option out there. In configuration management, this is super easy: DB => Template => Network This is super hard: Network => DB The first one keeps all platform specific logic in flat ascii files filled with variables from template. When you introduce new product, feature, vendor to network, you only add new ascii templates, extremely easy, no platform-specific logic in DB. The second one every little change in network, requires parser changes trying to model it back to DB. This is not sustainable. We can kid ourselves that NetCONF/YANG will solve this, but they won't. SNMP is old technology, when new feature comes to vendor, it may take _years_ before MIB comes. There is no reason to suspect you will be able to get feature out via NetCONF just because it is there. And if you can't do it 100% then you have to write parser which can understand it. You only need the second one, in case 100% is not from DB. But it is actually trivial to produce 100% from DB. You don't want DB to model base configuration, that's lot of work for no gain, that'll come from template or at most DB vendor-specific-blob. Then after you push configuration from DB to network, you immediately collect configuration and create relation of DB-config 2 network-config, now you can keep ensuring network has correct config. If it does not have, you don't know why not, you can't fix the error itself, but you can repovision whole box, so you do get configuration conformance check, it's just very crude. But the alternative, trying to understand network config, is just never ending path to to pain. If someone is going to do it, model it to python or ruby ORM and put it in github so others can contribute and we don't need to do it alone. -- ++ytti
Re: Managing IOS Configuration Snippets
On Thu, Feb 27, 2014 at 8:58 AM, Saku Ytti wrote: > On (2014-02-26 17:37 -0500), Robert Drake wrote: > > > Consider looking at Tail-F's NCS, which according to marketing > > presentations appears to do everything I want right now. I'd like > > to believe them but I don't have any money so I can't test it out. > > :) > > Tail-F is probably least bad option out there. > > In configuration management, this is super easy: > > DB => Template => Network > > This is super hard: > > Network => DB > > > The first one keeps all platform specific logic in flat ascii files filled > with variables from template. > When you introduce new product, feature, vendor to network, you only add > new > ascii templates, extremely easy, no platform-specific logic in DB. > > The second one every little change in network, requires parser changes > trying > to model it back to DB. This is not sustainable. We can kid ourselves that > NetCONF/YANG will solve this, but they won't. SNMP is old technology, when > new > feature comes to vendor, it may take _years_ before MIB comes. There is no > reason to suspect you will be able to get feature out via NetCONF just > because > it is there. And if you can't do it 100% then you have to write parser > which > can understand it. > > You only need the second one, in case 100% is not from DB. But it is > actually > trivial to produce 100% from DB. You don't want DB to model base > configuration, that's lot of work for no gain, that'll come from template > or > at most DB vendor-specific-blob. > Then after you push configuration from DB to network, you immediately > collect > configuration and create relation of DB-config 2 network-config, now you > can > keep ensuring network has correct config. If it does not have, you don't > know > why not, you can't fix the error itself, but you can repovision whole box, > so > you do get configuration conformance check, it's just very crude. > > But the alternative, trying to understand network config, is just never > ending > path to to pain. If someone is going to do it, model it to python or ruby > ORM > and put it in github so others can contribute and we don't need to do it > alone. > > -- > ++ytti > > Agree with this. We started out with rancid, quickly moved to a homebrew scp and git backed system with webgit/cgit as the user interface. If you are lucky your network equipment supports "advanced features" like ssh keys. If not, you might be stuck using sshpass to ease config collection. Built a config parsing system that would decompose monolithic configs into configlet files. Md5sum the file and use as part of the filename. You can then see "version" information for parts of the config tree. Quickly realized that maintaining this system is a full time job, due to the advanced status of network equipment software... Now looking at Tail-F NCS. Demo is impressive. I'm hopeful. Stating the obvious: the software running on most network equipment is of poor quality. The tools to manage this are a combination of high quality engineers and homebrew tools. Vendor tools are of a similar quality to the equipment software. I'd like to think "SDN" is an attempt to improve this, but I have my doubts. -- Tim:>
Re: Managing IOS Configuration Snippets
On Thu, Feb 27, 2014 at 9:50 AM, Ryan Shea wrote: > A couple more thoughts, regarding > > Network => DB > > I completely agree that trying to use the network config itself as the > authority for what we intend to be on a device is not the right long-term > approach. There is still a problem with Network => DB that I see. Assuming > you have *many* devices, that may or may not be up at a given time, or may > be in various stages of turn-up / burn-in / decom it is expected that a > config change will not successfully make it to all devices. There are other > timing issues, like a config built for a device being turned up, followed > by a push of an update to all devices that "succeeds", followed by the > final turn-up of this device. Even if you have a fancy config pushing > engine, let's just take as a given that you'll need to scrub through your > rancid-git backups to determine what needs to be updated. > > Regarding the MD5 approach, let's also think that configlets could have > "no" commands in them. In the NTP example I had before, if we wanted to > remove an NTP server the configlet would need the "no" version, but the > rancid backup obviously would not have this. I'm not trying to work a unit > test assertion framework here either. Some vendors have more robust > commenting, and this can be quite convenient for explicitly stating what > was pushed to the device. What are you using in your network... banner, > snmp-location, hope, prayer? > We don't do this, but the only flexible commenting in IOS style configs is ACLs. You could have an ACL that contains remarks only, and include version information: ip access-list CFG-VER remark CFG-VER-NTP 1.0.3 remark CFG-VER-VTY 4.3.2 end You could break this into individual ACLs if you prefer: ip access-list CFG-VER-NTP remark CFG-VER-NTP 1.0.3 end ip access-list CFG-VER-VTY remark CFG-VER-VTY 4.3.2 end Seems ridiculous, but that is the sorry state of the network OS. -- Tim:>
Re: Managing IOS Configuration Snippets
On (2014-02-27 09:50 -0500), Ryan Shea wrote: > Regarding the MD5 approach, let's also think that configlets could have > "no" commands in them. In the NTP example I had before, if we wanted to For DB => Template => Network it's to me very easy, but yes, each template you make must have anti-template version. So let's say you have NTP model, which may contain some access restriction information, NTP version, NTP peers. When you apply this model to device, then some platform specific ntp template is called. If you remove this from device, you need to call 'anti' version of the template. Very simple and easy. You also wondered how do I know which version of config network device has, this is hard problem. To know exactly what is wrong and how to address just that. If you can relax requirement to know if configuration is correct or incorrect it becomes trivial. But fixing incorrect is either full reprovision of new config (at least in IOS and JunOS not a problem, won't break the unchanged bits). Or you have human resolve it (of course as custom dictates first you punish the responsible severely but swiftly) -- ++ytti
Re: Managing IOS Configuration Snippets
On 2/26/14, 16:22 , Ryan Shea wrote: Howdy network operator cognoscenti, I'd love to hear your creative and workable solutions for a way to track in-line the configuration revisions you have on your cisco-like devices. ... Assume that this version encoding perfectly captures what is on the router and that no person is monkeying with the config... version 77 of the control plane filter is the same everywhere. At a previous job, our roll-your-own solution was a template based system(*) generating full configs; all the version history for template sections, per-router local tweaks, and generated results was kept in RCS, and the actual last-configured version, plus any incremental changes, was kept in the login banner. So at login you'd see something like: blah blah authorized users blah blah $Id: routername-confg,v 1.23 2013/08/20 03:07:16 username Exp INCR: 1.2,1.3,1.4,1.5,1.6 and that version tracking made its way through to rancid for easy offline auditing. This made it nice and easy to tell when and what had been updated, though it still would take a couple steps to identify what exact subsections had been changed over time (since the incremental version tags were specific deltas in per-device configurations. You could probably do it in a more global way too - git commit ids, maybe? - but you also don't want to make the version string too wordy either). -e * based on ftp://ftp.cac.washington.edu/pub/config-generator/, but substantially enhanced beyond the last public domain version. I know I'd be really happy if the current version was ever open-sourced...
Re: Managing IOS Configuration Snippets
On Thu, Feb 27, 2014 at 10:34 PM, Erik Muller wrote: > At a previous job, our roll-your-own solution was a template based system(*) > generating full configs; all the version history for template sections, > per-router local tweaks, and generated results was kept in RCS, and the > actual last-configured version, plus any incremental changes, was kept in > the login banner. This has been around for several years now - http://sourceforge.net/projects/cisco-conf-rep/ -- Suresh Ramasubramanian (ops.li...@gmail.com)
RE: Managing IOS Configuration Snippets
Along those same lines, we've been using alias exec for the same thing for a while: Alias exec NTP 6500_NTP_V1.0.1 Alias exec bgp 6500_peer_V2.0.0 Thanks, Chuck -Original Message- From: Tim Durack [mailto:tdur...@gmail.com] Sent: Thursday, February 27, 2014 11:50 AM To: Ryan Shea Cc: nanog@nanog.org Subject: Re: Managing IOS Configuration Snippets On Thu, Feb 27, 2014 at 9:50 AM, Ryan Shea wrote: > A couple more thoughts, regarding > > Network => DB > > I completely agree that trying to use the network config itself as the > authority for what we intend to be on a device is not the right > long-term approach. There is still a problem with Network => DB that I > see. Assuming you have *many* devices, that may or may not be up at a > given time, or may be in various stages of turn-up / burn-in / decom > it is expected that a config change will not successfully make it to > all devices. There are other timing issues, like a config built for a > device being turned up, followed by a push of an update to all devices > that "succeeds", followed by the final turn-up of this device. Even if > you have a fancy config pushing engine, let's just take as a given > that you'll need to scrub through your rancid-git backups to determine what needs to be updated. > > Regarding the MD5 approach, let's also think that configlets could > have "no" commands in them. In the NTP example I had before, if we > wanted to remove an NTP server the configlet would need the "no" > version, but the rancid backup obviously would not have this. I'm not > trying to work a unit test assertion framework here either. Some > vendors have more robust commenting, and this can be quite convenient > for explicitly stating what was pushed to the device. What are you > using in your network... banner, snmp-location, hope, prayer? > We don't do this, but the only flexible commenting in IOS style configs is ACLs. You could have an ACL that contains remarks only, and include version information: ip access-list CFG-VER remark CFG-VER-NTP 1.0.3 remark CFG-VER-VTY 4.3.2 end You could break this into individual ACLs if you prefer: ip access-list CFG-VER-NTP remark CFG-VER-NTP 1.0.3 end ip access-list CFG-VER-VTY remark CFG-VER-VTY 4.3.2 end Seems ridiculous, but that is the sorry state of the network OS. -- Tim:>
Re: Managing IOS Configuration Snippets
On 2/27/14, 12:21 , Suresh Ramasubramanian wrote:
This has been around for several years now -
http://sourceforge.net/projects/cisco-conf-rep/
But that's just archiving, like rancid, right? Still doesn't have any
correlation to the template-management side of things. While having the
backups makes it easy to check for simple things ("do all my routers have
the right syslog host set?"), OP's question is about tracking what versions
of templates may have been applied to routers; if there's any complex logic
(like, "are all active customer routes on this device included in the bcp38
acl on the upstream interface") or site-specific things, that can get a lot
harder to audit without the metadata on how the configuration got there.
-e
Re: Managing IOS Configuration Snippets
A lot of template management discussion focusses on using the network
configs as the canonical model of the network.
Storing the network model in the DB (whatever form that takes) is much
more sane.
There is the brownfields issue of populating that database and then
building device state from there, but once that's done a lot of the
problems go away.
A solution like rancid/tail-f then simply becomes the mechanics to
push your device state to the devices.
Some good stuff here:
https://www.nanog.org/meetings/nanog44/presentations/Monday/Gill_programatic_N44.pdf
--Simon
On 27 February 2014 09:46, Erik Muller wrote:
> On 2/27/14, 12:21 , Suresh Ramasubramanian wrote:
>>
>> This has been around for several years now -
>> http://sourceforge.net/projects/cisco-conf-rep/
>
>
> But that's just archiving, like rancid, right? Still doesn't have any
> correlation to the template-management side of things. While having the
> backups makes it easy to check for simple things ("do all my routers have
> the right syslog host set?"), OP's question is about tracking what versions
> of templates may have been applied to routers; if there's any complex logic
> (like, "are all active customer routes on this device included in the bcp38
> acl on the upstream interface") or site-specific things, that can get a lot
> harder to audit without the metadata on how the configuration got there.
> -e
>
>
Re: Managing IOS Configuration Snippets
On 27 February 2014 10:39, Ryan Shea wrote: > Very cool, thanks Erik. I can think of many ways to encode version > metadata. Probably best to be somewhere in between overly verbose (full > version $Id / date / author for every config chunk) and being unreadable > (base64 encoded gzip of unique configlet identifiers and versions). > Updating a banner feels a bit easier when you are pushing a full > device-specific configuration from a templating system. Regardless of where > it is stored, keeping the metadata in one of these fields (banner for > example) means that checking the contents of the banner configlet now > requires slightly more logic - which is fine. > > Chuck, interesting use of alias. > > Simon, completely agree that the network itself should not be the intent > store. The real focus here is when your intent is in a DB/templating system > thingy, how do you operationally ensure that intent matches reality. Again, > with many devices going through upgrades, disabled/unreachable devices, new > devices, pre-configured devices. The intent pusher is not blindly and > constantly pushing to all devices, and it's likely not safe to do that. Absolutely. Expect/SNMP/config scraping is a solution here. It's tedious and boring to write the hooks, but it's not impossible. A solution like tail-f is a much better solution here. My personal preference would be to just push/pull JSON off the devices. I think there are two separate components here (which are often conflated): the mechanics to push/pull from devices into a data structure, and the network database to work with those data structures. There's a place for both device level models and network level models. --Simon > > > On Thu, Feb 27, 2014 at 12:04 PM, Erik Muller wrote: > >> On 2/26/14, 16:22 , Ryan Shea wrote: >> >>> Howdy network operator cognoscenti, >>> >>> I'd love to hear your creative and workable solutions for a way to track >>> in-line the configuration revisions you have on your cisco-like devices. >>> >> ... >> >> >>> Assume that this version encoding perfectly captures what is on the router >>> and that no person is monkeying with the config... version 77 of the >>> control plane filter is the same everywhere. >>> >> >> At a previous job, our roll-your-own solution was a template based >> system(*) generating full configs; all the version history for template >> sections, per-router local tweaks, and generated results was kept in RCS, >> and the actual last-configured version, plus any incremental changes, was >> kept in the login banner. >> So at login you'd see something like: >> >> blah blah authorized users blah blah >> $Id: routername-confg,v 1.23 2013/08/20 03:07:16 username Exp >> INCR: 1.2,1.3,1.4,1.5,1.6 >> >> and that version tracking made its way through to rancid for easy offline >> auditing. This made it nice and easy to tell when and what had been >> updated, though it still would take a couple steps to identify what exact >> subsections had been changed over time (since the incremental version tags >> were specific deltas in per-device configurations. You could probably do >> it in a more global way too - git commit ids, maybe? - but you also don't >> want to make the version string too wordy either). >> >> -e >> >> * based on ftp://ftp.cac.washington.edu/pub/config-generator/, but >> substantially enhanced beyond the last public domain version. I know I'd >> be really happy if the current version was ever open-sourced... >> >>
Re: Managing IOS Configuration Snippets
On 27 Feb 2014, at 12:46, Erik Muller wrote: > On 2/27/14, 12:21 , Suresh Ramasubramanian wrote: >> This has been around for several years now - >> http://sourceforge.net/projects/cisco-conf-rep/ > > But that's just archiving, like rancid, right? This is not any kind of sensible answer to the original question, but the general approach “give ops people a shell on a box with a rancid repository, encourage them to write scripts that do stuff” has the potential to cause all kinds of good things to happen faster than the time taken to organise a conference call to discuss requirements gathering for a “production” system. Examples, as ever: http://www.nanog.org/meetings/nanog26/presentations/stephen.pdf ftp://ftp.isc.org/isc/toolmakers/ (It’s not pretty. But it’s not supposed to be pretty.) Joe
Re: Managing IOS Configuration Snippets
On 2/27/14, 15:52 , Joe Abley wrote: This is not any kind of sensible answer to the original question, but the general approach “give ops people a shell on a box with a rancid repository, encourage them to write scripts that do stuff” has the potential to cause all kinds of good things to happen faster than the time taken to organise a conference call to discuss requirements gathering for a “production” system. +1000. And that applies equally to the backend. I have yet to meet a fancy, integrated, database-driven configuration management system that can beat a bunch of flat files and a few perl scripts. Hackability of a system can be a definite virtue here. -e
Hat - bcp38.info - Storm Center Diary
Hat, A reader suggested I reach out to you, he thought you might like a simple graphic I put together on the Storm Center Diary post. Talked about BCP38 today. Email me off list and I will send it. ~Richard signature.asc Description: Message signed with OpenPGP using GPGMail
Re: Managing IOS Configuration Snippets
Definitely. Depends what form the database takes - I don't think SQL is the right answer here. Sticking with flat files and perl scripts as much as possible is good guidance. I'm biased, but I'd go with Python: http://www.youtube.com/watch?v=EGK5jjyUBCQ --Simon On 27 February 2014 13:05, Erik Muller wrote: > On 2/27/14, 15:52 , Joe Abley wrote: >> >> This is not any kind of sensible answer to the original question, but >> the general approach "give ops people a shell on a box with a rancid >> repository, encourage them to write scripts that do stuff" has the >> potential to cause all kinds of good things to happen faster than the >> time taken to organise a conference call to discuss requirements >> gathering for a "production" system. > > > +1000. And that applies equally to the backend. I have yet to meet a > fancy, integrated, database-driven configuration management system that can > beat a bunch of flat files and a few perl scripts. Hackability of a system > can be a definite virtue here. > -e > >
Re: Managing IOS Configuration Snippets
Putting aside the fact that snippets aren’t a good way to conceptualize deployed router code, my gut still tells me to question the question here. The first is does this stuff change often enough to warrant a fancy versioning solution? I have yet to see NTP deployed in a different way than when I first learned to configure it. Next, when it does change how often is it not rolled out to every router. If NTP or CPP or SNMP or some other administrative option were configured differently across my network I would want to audit it and fix not version control. What if some of the configs don’t match the defined versions? It may be better to create standard templates and version them in SVN or GIT and then use config backups to track which devices have the standard configs. There are some for pay tools that can search for certain statements on various boxes and either alert or remediate when differences are found. On Feb 26, 2014, at 4:22 PM, Ryan Shea wrote: > Howdy network operator cognoscenti, > > I'd love to hear your creative and workable solutions for a way to track > in-line the configuration revisions you have on your cisco-like devices. > Let me clearify/frame: > > You have a set of tested/approved configurations for your routers which use > IOS style configuration. These configurations of course are always refined > and updated. You break these pieces of configuration into logical sections, > for example a configuration file for NTP configuration, a file for control > plane filter and store these in some revision control system. Put aside for > the moment whether this is a reasonable way to comprehend deployed > configurations. What methods do some of you use to know which version of a > configuration you have deployed to a given router for auditing and update > purposes? Remarks are a convenient way to do this for ACLs - but I don't > have similar mechanics for top level configurations. About a decade ago I > thought I'd be super clever and encode versioning information into the snmp > location - but that is just awful and there is a much better way everyone > is using, right? Flexible commenting on other vendors/platforms make this a > bit easier. > > Assume that this version encoding perfectly captures what is on the router > and that no person is monkeying with the config... version 77 of the > control plane filter is the same everywhere.
Re: Filter NTP traffic by packet size?
On Feb 26, 2014, at 12:44 PM, Brandon Galbraith wrote: > On Wed, Feb 26, 2014 at 6:56 AM, Keegan Holley wrote: > > More politely stated, it’s not the responsibility of the operator to decide > > what belongs on the network and what doesn’t. Users can run any services > > that’s not illegal or even reuse ports for other applications. That being > > said commonly exploited ports (TCP 25 for example) are often blocked. This > > is usually done to block or protect an application though not to single out > > a particular port number. > > Don't most residential ISPs already block port 25 outbound? > http://www.postcastserver.com/help/Port_25_Blocking.aspx > > Blocking chargen at the edge doesn't seem to be outside of the realm of > possibilities. As I said, SMTP is blocked because it’s the default port for a commonly run and often misconfigured application. Blocking the chargen port is definitely reasonable, but it’s not a popular application. Most people use the port as an clever non-default port for some other service like ssh.
Re: Managing ACL exceptions (was Re: Filter NTP traffic by packet size?)
It depends on how many customers you have and what sort of contract you have with them if any. A significant amount of attack traffic comes from residential networks where a “one-size-fits-all” policy is definitely best. On Feb 26, 2014, at 4:01 PM, Jay Ashworth wrote: > - Original Message - >> From: "Brandon Galbraith" > >> On Wed, Feb 26, 2014 at 6:56 AM, Keegan Holley >> wrote: >>> More politely stated, it’s not the responsibility of the operator to >>> decide what belongs on the network and what doesn’t. Users can run any >>> services that’s not illegal or even reuse ports for other >>> applications. > >> Blocking chargen at the edge doesn't seem to be outside of the realm >> of possibilities. > > All of these conversations are variants of "how easy is it to set up a > default ACL for loops, and then manage exceptions to it?". > > Assuming your gear permits it, I don't personally see all that much > Bad Actorliness in setting a relatively tight bidirectional ACL for > Random Edge Customers, and opening up -- either specific ports, or > just "to a less-/un-filtered ACL" on specific request. > > The question is -- as it is with BCP38 -- *can the edge gear handle it*? > > And if not: why not? (Protip: because buyers of that gear aren't > agitating for it) > > Cheers, > -- jra > -- > Jay R. Ashworth Baylink > j...@baylink.com > Designer The Things I Think RFC 2100 > Ashworth & Associates http://www.bcp38.info 2000 Land Rover DII > St Petersburg FL USA BCP38: Ask For It By Name! +1 727 647 1274 >
Verizon FIOS IPv6?
My strategy, should I remember it tomorrow: We have a business-class FIOS connection where I work and a static IP as well. At least three people who work here have FIOS at home. I've read rumors about business class customers who really work their phone sex getting native ipv6, and I also heard somethin about static ip's. So I'll try that, and also mention that "we're transitioning our employees who remote in from home to FIOS but we'd like ipv6 for ... VPN purposes, NAT traversal, etc ..." I mean, that should get them a little wet right? I have a bit of a hairbrained theory that the reason ISP's have stagnated on ipv6 has to do with relationship between capitalism and scarcity. Having a limited quantity of anything makes it more valuable. Why wouldn't that apply to IP's?
congestion between Cogent and CenturyLink
Hello, We send periodic 10-15Mbps bursts of traffic to a business partner and it appears to transition from Cogent to Century Link in Atlanta. During the day performance is normal and latency appears acceptable on a trace route. 12 ms 13 ms 12 ms te0-6-1-7.rcr21.msp01.atlas.cogentco.com [38.88.188.41] 26 ms 26 ms 27 ms be2410.ccr22.ord01.atlas.cogentco.com [154.54.7.229] 43 ms 44 ms 44 ms be2099.ccr22.atl01.atlas.cogentco.com [154.54.28.74] 44 ms 44 ms 44 ms be2051.ccr21.atl02.atlas.cogentco.com [154.54.0.162] 43 ms 42 ms 43 ms qwest.atl02.atlas.cogentco.com [154.54.13.30] 49 ms 49 ms 50 ms min-edge-10.inet.qwest.net [205.171.128.154] But after hours latency spikes and throughput drops to less than 1Mbps. te8-3.ccr01.msp01.atlas.cogentco.com (38.88.188.41) 4.603 ms te0-0-0-19.mpd22.ord01.atlas.cogentco.com (154.54.1.230) 17.838 ms be2099.ccr22.atl01.atlas.cogentco.com (154.54.28.74) 33.156 ms be2051.ccr21.atl02.atlas.cogentco.com (154.54.0.162) 36.449 ms qwest.atl02.atlas.cogentco.com (154.54.13.30) 89.583 ms min-edge-10.inet.qwest.net (205.171.128.150) 89.806 ms Century Link stated that Cogent is oversubscribing the link, and that they've requested Cogent resolve the problem, but that action has yet to be taken. I've tried reaching out to Cogent but as we're not a direct customer they wouldn't provide assistance. Has anyone else seen similar issues? Thanks, Aidan
RE: Verizon FIOS IPv6?
Good luck. We've been bitching at our sales rep for years, as we've added circuits, and haven't gotten even empty promises; just the same endless Verizon BS about "it's being tested in select markets" although no one has ever been able to prove that to be the case. You definitely get static IP's on business connections; that's just a matter of how much you pay and how many you need. David -Original Message- From: Tristan Lear [mailto:trissypi...@gmail.com] Sent: Thursday, February 27, 2014 1:45 AM To: nanog@nanog.org Subject: Verizon FIOS IPv6? My strategy, should I remember it tomorrow: We have a business-class FIOS connection where I work and a static IP as well. At least three people who work here have FIOS at home. I've read rumors about business class customers who really work their phone sex getting native ipv6, and I also heard somethin about static ip's. So I'll try that, and also mention that "we're transitioning our employees who remote in from home to FIOS but we'd like ipv6 for ... VPN purposes, NAT traversal, etc ..." I mean, that should get them a little wet right? I have a bit of a hairbrained theory that the reason ISP's have stagnated on ipv6 has to do with relationship between capitalism and scarcity. Having a limited quantity of anything makes it more valuable. Why wouldn't that apply to IP's?
Re: Verizon FIOS IPv6?
I echo the 'good luck' and ditto on the experience. There's a lot of people anxious to get IPv6 on FIOS, but there seems to be precious little movement over there. * David Hubbard (dhubb...@dino.hostasaurus.com) wrote: > Good luck. We've been bitching at our sales rep for years, as we've added > circuits, and haven't gotten even empty promises; just the same endless > Verizon BS about "it's being tested in select markets" although no one has > ever been able to prove that to be the case. You definitely get static IP's > on business connections; that's just a matter of how much you pay and how > many you need. > > David > > -Original Message- > From: Tristan Lear [mailto:trissypi...@gmail.com] > Sent: Thursday, February 27, 2014 1:45 AM > To: nanog@nanog.org > Subject: Verizon FIOS IPv6? > > My strategy, should I remember it tomorrow: > > We have a business-class FIOS connection where I work and a static IP as > well. At least three people who work here have FIOS at home. I've read rumors > about business class customers who really work their phone sex getting native > ipv6, and I also heard somethin about static ip's. So I'll try that, and also > mention that "we're transitioning our employees who remote in from home to > FIOS but we'd like ipv6 for ... VPN purposes, NAT traversal, etc ..." I mean, > that should get them a little wet right? > > I have a bit of a hairbrained theory that the reason ISP's have stagnated on > ipv6 has to do with relationship between capitalism and scarcity. Having a > limited quantity of anything makes it more valuable. Why wouldn't that apply > to IP's? > > signature.asc Description: Digital signature
Re: Verizon FIOS IPv6?
On Thu, Feb 27, 2014 at 09:18:08PM -0500, Stephen Frost wrote: > I echo the 'good luck' and ditto on the experience. > > There's a lot of people anxious to get IPv6 on FIOS, but there seems to > be precious little movement over there. > > * David Hubbard (dhubb...@dino.hostasaurus.com) wrote: > > Good luck. We've been bitching at our sales rep for years, as we've added > > circuits, and haven't gotten even empty promises; just the same endless > > Verizon BS about "it's being tested in select markets" although no one has > > ever been able to prove that to be the case. You definitely get static > > IP's on business connections; that's just a matter of how much you pay and > > how many you need. > > > > David Another ditto :) I think they are Defnitely milking their highway robbery IPV4 allocation costs. Confidence is low for IPV6 from FIOS anytime soon. -- Bryan G. Seitz
Re: Verizon FIOS IPv6?
On Thu, Feb 27, 2014 at 9:18 PM, Stephen Frost wrote: > I echo the 'good luck' and ditto on the experience. > > There's a lot of people anxious to get IPv6 on FIOS, but there seems to > be precious little movement over there. > it really is just an embarrassment :( perhaps shame will work to motivate them instead? > * David Hubbard (dhubb...@dino.hostasaurus.com) wrote: >> Good luck. We've been bitching at our sales rep for years, as we've added >> circuits, and haven't gotten even empty promises; just the same endless >> Verizon BS about "it's being tested in select markets" although no one has >> ever been able to prove that to be the case. You definitely get static IP's >> on business connections; that's just a matter of how much you pay and how >> many you need. >> >> David >> >> -Original Message- >> From: Tristan Lear [mailto:trissypi...@gmail.com] >> Sent: Thursday, February 27, 2014 1:45 AM >> To: nanog@nanog.org >> Subject: Verizon FIOS IPv6? >> >> My strategy, should I remember it tomorrow: >> >> We have a business-class FIOS connection where I work and a static IP as >> well. At least three people who work here have FIOS at home. I've read >> rumors about business class customers who really work their phone sex >> getting native ipv6, and I also heard somethin about static ip's. So I'll >> try that, and also mention that "we're transitioning our employees who >> remote in from home to FIOS but we'd like ipv6 for ... VPN purposes, NAT >> traversal, etc ..." I mean, that should get them a little wet right? >> >> I have a bit of a hairbrained theory that the reason ISP's have stagnated on >> ipv6 has to do with relationship between capitalism and scarcity. Having a >> limited quantity of anything makes it more valuable. Why wouldn't that apply >> to IP's? >> >>
Re: Verizon FIOS IPv6?
* Christopher Morrow (morrowc.li...@gmail.com) wrote: > On Thu, Feb 27, 2014 at 9:18 PM, Stephen Frost wrote: > > There's a lot of people anxious to get IPv6 on FIOS, but there seems to > > be precious little movement over there. > > it really is just an embarrassment :( Oh, I agree, and the old UUNET folks (whose side of that house has had this done for, uh, forever...) should really take ownership and scream bloody murder at the FIOS people to either get their $h1t together or get out of the way. > perhaps shame will work to motivate them instead? It hasn't worked thus far, though I admit, I've been tempted more than once to call them and threaten that I'm going to switch to Comcast, if it wasn't such a laughable idea. :/ Thanks, Stephen signature.asc Description: Digital signature
Re: congestion between Cogent and CenturyLink
With cogent? Now you will be asking us if the Pope is really Catholic :) On 28-Feb-2014 7:43 AM, "Aidan Scheller" wrote: > Hello, > > > > We send periodic 10-15Mbps bursts of traffic to a business partner and it > appears to transition from Cogent to Century Link in Atlanta. During the > day performance is normal and latency appears acceptable on a trace route. > > > > 12 ms 13 ms 12 ms te0-6-1-7.rcr21.msp01.atlas.cogentco.com [38.88.188.41] > > 26 ms 26 ms 27 ms be2410.ccr22.ord01.atlas.cogentco.com [154.54.7.229] > > 43 ms 44 ms 44 ms be2099.ccr22.atl01.atlas.cogentco.com [154.54.28.74] > > 44 ms 44 ms 44 ms be2051.ccr21.atl02.atlas.cogentco.com [154.54.0.162] > > 43 ms 42 ms 43 ms qwest.atl02.atlas.cogentco.com [154.54.13.30] > > 49 ms 49 ms 50 ms min-edge-10.inet.qwest.net [205.171.128.154] > > > > But after hours latency spikes and throughput drops to less than 1Mbps. > > > > te8-3.ccr01.msp01.atlas.cogentco.com (38.88.188.41) 4.603 ms > > te0-0-0-19.mpd22.ord01.atlas.cogentco.com (154.54.1.230) 17.838 ms > > be2099.ccr22.atl01.atlas.cogentco.com (154.54.28.74) 33.156 ms > > be2051.ccr21.atl02.atlas.cogentco.com (154.54.0.162) 36.449 ms > > qwest.atl02.atlas.cogentco.com (154.54.13.30) 89.583 ms > > min-edge-10.inet.qwest.net (205.171.128.150) 89.806 ms > > > > Century Link stated that Cogent is oversubscribing the link, and that > they've requested Cogent resolve the problem, but that action has yet to be > taken. I've tried reaching out to Cogent but as we're not a direct > customer they wouldn't provide assistance. > > > > Has anyone else seen similar issues? > > > > Thanks, > > > > Aidan >
Re: Verizon FIOS IPv6?
On Thu, Feb 27, 2014 at 9:48 PM, Stephen Frost wrote: > * Christopher Morrow (morrowc.li...@gmail.com) wrote: >> On Thu, Feb 27, 2014 at 9:18 PM, Stephen Frost wrote: >> > There's a lot of people anxious to get IPv6 on FIOS, but there seems to >> > be precious little movement over there. >> >> it really is just an embarrassment :( > > Oh, I agree, and the old UUNET folks (whose side of that house has had > this done for, uh, forever...) should really take ownership and scream > bloody murder at the FIOS people to either get their $h1t together or > get out of the way. most of them did this, like 5+ yrs ago :( when they stopped being listened to, they left. (most of them) > >> perhaps shame will work to motivate them instead? > > It hasn't worked thus far, though I admit, I've been tempted more than > once to call them and threaten that I'm going to switch to Comcast, if > it wasn't such a laughable idea. :/ > > Thanks, > > Stephen
Re: Managing IOS Configuration Snippets
On Thu, Feb 27, 2014 at 8:38 PM, Keegan Holley wrote: > Putting aside the fact that snippets aren't a good way to conceptualize > deployed router code, my gut still tells me to question the question here. > The first is does this stuff change often enough to warrant a fancy > versioning solution? I have yet to see NTP deployed in a different way than > when I first learned to configure it. Next, when it does change how often is > it not rolled out to every router. If NTP or CPP or SNMP or some other > administrative option were configured differently across my sure, so you're saying that a large bit (maybe) of the router config is 'one size fits all' and 'never changes' where 'never' is really 'very infrequently'. sure, agreed... but there are parts of the config that do change more frequently (depending on the network perhaps)... how do you go about seeing which version / setup is deployed EXCEPT by building a home-grown 'config parser' and seeing that 'what is deployed matches mostly what I have in my config store for this router/class-of-router/network' ? It's a shame that vendors of network equipment don't have to manage large networks of their own equipment under constrained opex environments (no fair comparing contracted work where you bill for time + materials, that's the wrong incentive set)... I bet that'd get them to fix stuff up right quick. network I would want to audit it and fix not version control. What if some of the configs don't match the defined versions? It may be better to create standard templates and version them in SVN or GIT and then use config backups to track which devices have the standard configs. There are some for pay tools that can search for certain statements on various boxes and either alert or remediate when differences are found. > > > On Feb 26, 2014, at 4:22 PM, Ryan Shea wrote: > >> Howdy network operator cognoscenti, >> >> I'd love to hear your creative and workable solutions for a way to track >> in-line the configuration revisions you have on your cisco-like devices. >> Let me clearify/frame: >> >> You have a set of tested/approved configurations for your routers which use >> IOS style configuration. These configurations of course are always refined >> and updated. You break these pieces of configuration into logical sections, >> for example a configuration file for NTP configuration, a file for control >> plane filter and store these in some revision control system. Put aside for >> the moment whether this is a reasonable way to comprehend deployed >> configurations. What methods do some of you use to know which version of a >> configuration you have deployed to a given router for auditing and update >> purposes? Remarks are a convenient way to do this for ACLs - but I don't >> have similar mechanics for top level configurations. About a decade ago I >> thought I'd be super clever and encode versioning information into the snmp >> location - but that is just awful and there is a much better way everyone >> is using, right? Flexible commenting on other vendors/platforms make this a >> bit easier. >> >> Assume that this version encoding perfectly captures what is on the router >> and that no person is monkeying with the config... version 77 of the >> control plane filter is the same everywhere. > >
Re: congestion between Cogent and CenturyLink
+1, which semi-large eyeball does Cogent NOT have capacity problems to? On 2/28/2014 午前 11:55, Suresh Ramasubramanian wrote: With cogent? Now you will be asking us if the Pope is really Catholic :) On 28-Feb-2014 7:43 AM, "Aidan Scheller" wrote: Hello, We send periodic 10-15Mbps bursts of traffic to a business partner and it appears to transition from Cogent to Century Link in Atlanta. During the day performance is normal and latency appears acceptable on a trace route. 12 ms 13 ms 12 ms te0-6-1-7.rcr21.msp01.atlas.cogentco.com [38.88.188.41] 26 ms 26 ms 27 ms be2410.ccr22.ord01.atlas.cogentco.com [154.54.7.229] 43 ms 44 ms 44 ms be2099.ccr22.atl01.atlas.cogentco.com [154.54.28.74] 44 ms 44 ms 44 ms be2051.ccr21.atl02.atlas.cogentco.com [154.54.0.162] 43 ms 42 ms 43 ms qwest.atl02.atlas.cogentco.com [154.54.13.30] 49 ms 49 ms 50 ms min-edge-10.inet.qwest.net [205.171.128.154] But after hours latency spikes and throughput drops to less than 1Mbps. te8-3.ccr01.msp01.atlas.cogentco.com (38.88.188.41) 4.603 ms te0-0-0-19.mpd22.ord01.atlas.cogentco.com (154.54.1.230) 17.838 ms be2099.ccr22.atl01.atlas.cogentco.com (154.54.28.74) 33.156 ms be2051.ccr21.atl02.atlas.cogentco.com (154.54.0.162) 36.449 ms qwest.atl02.atlas.cogentco.com (154.54.13.30) 89.583 ms min-edge-10.inet.qwest.net (205.171.128.150) 89.806 ms Century Link stated that Cogent is oversubscribing the link, and that they've requested Cogent resolve the problem, but that action has yet to be taken. I've tried reaching out to Cogent but as we're not a direct customer they wouldn't provide assistance. Has anyone else seen similar issues? Thanks, Aidan
Re: congestion between Cogent and CenturyLink
* Paul S. (cont...@winterei.se) wrote: > +1, which semi-large eyeball does Cogent NOT have capacity problems to? Soon, Comcast... Given what's going on w/ them and Netflix. Thanks, Stephen signature.asc Description: Digital signature
Re: congestion between Cogent and CenturyLink
I'm already seeing a huge improvement to Comcast after Netflix moved a lot of traffic off of the ports. On Feb 27, 2014, at 22:21, Stephen Frost wrote: > * Paul S. (cont...@winterei.se) wrote: >> +1, which semi-large eyeball does Cogent NOT have capacity problems to? > > Soon, Comcast... Given what's going on w/ them and Netflix. > >Thanks, > >Stephen
Re: congestion between Cogent and CenturyLink
I saw the same effect after the Netflix peering started. http://imgur.com/a/aVFAS On Thu, Feb 27, 2014 at 10:22 PM, Jason Canady wrote: > I'm already seeing a huge improvement to Comcast after Netflix moved a lot > of traffic off of the ports. > > > On Feb 27, 2014, at 22:21, Stephen Frost wrote: > > > * Paul S. (cont...@winterei.se) wrote: > >> +1, which semi-large eyeball does Cogent NOT have capacity problems to? > > > > Soon, Comcast... Given what's going on w/ them and Netflix. > > > >Thanks, > > > >Stephen > >
Re: Verizon FIOS IPv6?
- Original Message - > From: "Christopher Morrow" > On Thu, Feb 27, 2014 at 9:48 PM, Stephen Frost > wrote: > > * Christopher Morrow (morrowc.li...@gmail.com) wrote: > >> On Thu, Feb 27, 2014 at 9:18 PM, Stephen Frost > >> wrote: > >> > There's a lot of people anxious to get IPv6 on FIOS, but there > >> > seems to be precious little movement over there. > >> > >> it really is just an embarrassment :( > > > > Oh, I agree, and the old UUNET folks (whose side of that house has had > > this done for, uh, forever...) should really take ownership and scream > > bloody murder at the FIOS people to either get their $h1t together > > or get out of the way. > > most of them did this, like 5+ yrs ago :( > when they stopped being listened to, they left. (most of them) > >> perhaps shame will work to motivate them instead? > > > > It hasn't worked thus far, though I admit, I've been tempted more than > > once to call them and threaten that I'm going to switch to Comcast, if > > it wasn't such a laughable idea. :/ Well, for the record, I don't expect anything at all out of Vzn, since they got it made illegal for munis to own fiber in all or part of 19 states, at least in large part on the basis of promises that they wouldn't cherry pick in their FiOS rollouts... and then went and did just that; it's public record they did their last deployment in 2010, and they have no plans to do any more. Unless, of couse, some muni or Google moves into somewhere to roll out; *then* they'll be right there on your doorstep applying for trenching permits. Cheers, -- jra -- Jay R. Ashworth Baylink j...@baylink.com Designer The Things I Think RFC 2100 Ashworth & Associates http://www.bcp38.info 2000 Land Rover DII St Petersburg FL USA BCP38: Ask For It By Name! +1 727 647 1274
Re: Verizon FIOS IPv6?
On Thu, 27 Feb 2014, Bryan Seitz wrote: On Thu, Feb 27, 2014 at 09:18:08PM -0500, Stephen Frost wrote: I echo the 'good luck' and ditto on the experience. There's a lot of people anxious to get IPv6 on FIOS, but there seems to be precious little movement over there. I've been fighting this battle for as long as I've had FIOS (about a year and a half), with no end in sight. Front-line reps don't know the situation, and I don't fault them for that. Getting a hold of anyone who comment with anything approaching authority has been impossible. In the meantime, I will continue using my tunnel through HE, which works great (kudos to HE). jms
Re: Verizon FIOS IPv6?
On Thu, 27 Feb 2014, Tristan Lear wrote: We have a business-class FIOS connection where I work and a static IP as well. At least three people who work here have FIOS at home. I've read rumors about business class customers who really work their phone sex getting native ipv6, and I also heard somethin about static ip's. So I'll try that, and also mention that "we're transitioning our employees who remote in from home to FIOS but we'd like ipv6 for ... VPN purposes, NAT traversal, etc ..." I mean, that should get them a little wet right? Not likely. Verizon is a very expensive date, so you *really* have to open the wallet to make that kind of impression, and by that point, you're working with VZ Enterprise, which is what used to be UUNET, where v6 is easy to get, so the point ends up being moot. I have a bit of a hairbrained theory that the reason ISP's have stagnated on ipv6 has to do with relationship between capitalism and scarcity. Having a limited quantity of anything makes it more valuable. Why wouldn't that apply to IP's? I doubt it's anything quite so nefarious, though VZ trying to figure out how to monetize their IPv6 rollout is certainly a possibility. I've heard all sorts of BS answers as to why there is no v6 for FIOS, such as: 1. "We're having problems getting it to work on our set-top boxes." So go dual-stack and let the set-top boxes stay v4 until the problem gets worked out. VZ has already stated that dual-stack is the way thry're going to do it. 2. "We have plenty of IPv4 space." Perhaps today, yes, but that misses the point entirely. jms
