Shaun R. wrote: Your message is intoxicable ;)
> I worked for a company in the past that had a very large flat network. > The network consisted of two /20's (255.255.240.0) that were configured > on a 7206 npe-300 router that connected to a bunch of catalyst 2924 > switches (the old school ones). I still use them, and they work great! > Everything was on vlan1. The company > was a small hosting company that provided mainly dedicated servers. > This company was constantly having problems with what i called broadcast > attacks. The network graphs would show traffic on all interfaces spike > and normally the 100mbit uplink between the switches would saturate and > the network would die. From that experience i took my time to design > and deploy my network to be as correct as possible. Out of curiosity, did your experience find that the issues were related to actual broadcast problems? > I put each customer > on there own vlan with there own subnet carved out. My 3750 stack is my > access/core and i have 7206-VXR-npe-g2's for borders (bgp/ospf). Every > edge switch is uplinked twice with gigE (2gbit of bw) and customers are > uplinked normally at 100mbit. For years this was fine and worked great > but when deploying our own servers i always found myself kicking out a > new vlan and subnet. I wasnt sure if it was needed being that it was > our own servers (our own servers meaning that we managed them, customer > do not have admin/root access). Again, out of pure curiosity, why did you do it this way? Do you manage ACLs per server? We're a small op that provides access, hosting and colo, and I'm wondering why you'd adopt this strategy. > Then came virtual server hosting. With VPS Hosting we have one physical > server (a host) that we carve out a /26 for and assign it to it's own > vlan. We've done this for a few years now and it's worked fine but it's > also kind of caused problems. One problem is that some hosts needs more > ips than other hosts. We end up with some hosts having 20 ips free in > there subnet while other hosts have none and need another allocation > assigned to them. Also, we cannot move a customer from one host to > another with out making the customer change ip address's. I'm but a rookie, but it appears as though some design research regarding IP assignment strategies may be beneficial. > For a while > now i've been wanting to just combine all the VPS hosts into one vlan > and carve them out /24's as needed. Then each host could just get a ip > from that pool and when that pool started to become depleated i could > assign another /24. Let the big boys criticize me here... what I've done is push our 'hosting' arm to the outside of the edge of our network. The 'hosting/colo' acts as a client premise. Even though it resides within our primary PoP, it connects to the network the same way that a client aggregation router does. I use iBGP from the 'hosting' routers to the edge routers in order to provide 'in-house' redundancy. Then, I allow the 'hosting' routers to advertise whatever IP blocks that they need. I have certain (relatively) large prefixes dedicated to hosting that are reserved, but can be (and sometimes are) re-purposed in a heartbeat because of the dynamic setup. In your statement above, to "assign another /24" would allow you to re-purpose prefixes for the pool, and use them on a slice-by-slice basis. Either way, the comment that I'm currently quoting appears more sound than anything else so far. > Another problem that company i worked for had was that they where > calculating bandwidth usage off the 2924 network interfaces. The > problem with this we later found was that ARP/Broadcast traffic ended up > being a huge amount added to there bill at the end of the month. I want > to say that each customer had around 4-6GB of transfer tacked onto there > bandwidth usage. ...relative. I'd like to know what you used to do your billing, and who authorized the billing. With that kind of potential over-billing, a job at your employer could mean a massive pay raise :) > So what i'm really asking is... > 1. When should i really cut out a new vlan for a server or group of > servers for my own use (meaning the customer doesnt have admin > privileges to the machine)? Whenever you deem it necessary. > 2. Was the problem with the large network that they didnt cut the /20 > into smaller subnets or was the problem that they didnt cut them into > smaller subnets and put them into there own vlans? It's a matter of implementation. Whether you look at it as a subnet or a vlan, it doesn't matter. When I first got our /21 from ARIN, I had to do some serious reading. My primary objective was how to allow the migration of clients without having to make them renumber. (During which time, I renumbered our entire network twice, once from MCI IP's, and from our then 'new' upstream IP's). After that, I slowly started to learn (via experience and MUCH reading) that renumbering IP's wasn't the only 'renumbering'. We also want to have no client impact with prefix lengths (subnet mask) and default gateway addresses. Only experience will tell you what went wrong, and where, given much feedback from diagnostics and other troubleshooting. (I've found that seeing this on numerous 'small' networks, it comes naturally. When dealing with a network larger than what I'm used to, the experience in troubleshooting/documenting paves the way). > 3. Say i combine all the VPS Hosts, 1000 Virtual servers in 1 vlan, > with say 15 /24's... Is this ok? ...you stated above that you suspected broadcast issues, but now you want to put... nevermind. No, it's not ok. > how is this compared to say having 25 > vlans/subnets with each pysical host in one of them? Because technically and logically, each VLAN is it's own broadcast domain. It means that my server can scream to everyone in it's VLAN (broadcast domain) all it wants, but it can't affect any other host in any other VLAN. It can only craft a well-designed broadcast attack on the network that isn't protecting against it, but only affect a limited number of connected nodes. Otherwise, put all hosts in one VLAN, a single problem on a single host will surely take down your entire network. (with cascading and escalating detriment). > Anything else i should be worried about here? Your sanity. I'd advise that you hire someone who has had the experience of designing an infrastructure the size of yours, at least as a consultant. Steve ps. feedback welcome on anything that I wrote...
smime.p7s
Description: S/MIME Cryptographic Signature
_______________________________________________ cisco-nsp mailing list [email protected] https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
