Hi Derick, Changing the transport didn't help, R9 still creates the VPN label for the interface regardless of the transport-address. I am just not sure if R9 is at fault or R1 is.
On Fri, Sep 4, 2009 at 3:03 PM, Derick Winkworth <[email protected]> wrote: > Does changing the transport address to the interface have any effect? > > Rick Mur wrote: > > Weird! Still a bug in the CSC code. > > Great workaround though! This is very good for your preparation :-) > > > > > > -- > > Regards, > > > > Rick Mur > > CCIE2 #21946 (R&S / Service Provider) > > Sr. Support Engineer IPexpert, Inc. > > URL: http://www.IPexpert.com > > > > On 4 sep 2009, at 18:47, Bryan Bartik wrote: > > > >> Well I think Antoine was on to something...and I found another > >> workaround :-) > >> > >> I did a packet capture in Dynamips and just before the session > >> drops, the TCP/LDP keepalive between R1 and R9 loop between each > >> other. I get about 254 packets in under a sec and I can see the TTL > >> decrement on each one! This happens every minute and eventually the > >> timeout expires because these keepalive are not received and > >> processed locally by R9 (thus every 3 minutes it drops). R1 is > >> sending the keepalive as a labeled packet even though R9 is directly > >> connected, I think it should be an imp-null label. According to the > >> capture R1 is putting label 20 on the keepalive. Here I verified it: > >> > >> R1#sho mpls ldp bindings > >> lib entry: 123.1.9.0/24, rev 2 > >> local binding: label: imp-null > >> remote binding: lsr: 123.1.9.9:0, label: 20 > >> > >> R9#sho mpls forwarding-table labels 20 > >> Local Outgoing Prefix Bytes Label Outgoing Next > >> Hop > >> Label Label or VC or Tunnel Id Switched interface > >> 20 Pop Label 123.1.9.0/24[V] <http://123.1.9.0/24%5BV%5D> > 3358266 Se1/2 > >> point2point > >> R9# > >> > >> R1 is learning the label 20 and using that to send packets onto the > >> directly connected network. R9 receives it, pops it and sends it > >> back out of the interface instead of receiving it locally. (I am > >> using the interface as ldp transport address for this example). It's > >> hard to tell who is at fault here! > >> > >> In order for the session to actually come up again, all the learned > >> labels have to get flushed so the TCP exchange can happen. Once it > >> does, TCP handshake and label exchange occurs without any label > >> encapsulation. R9 then advertises the vrf subnet (connected to R1) > >> in a label mapping message with a label of 20 (verified with > >> capture). This label is now used by R1 and any further TCP exchange > >> uses that label and packets loop. It's funny to see it happen, > >> because as soon as R9 sends that label mapping, R1 uses the label in > >> the ACK! However this does not cause any problems at this point, > >> because R1 also sends an unlabeled ACK as well, completeing the > >> exchange (weird). > >> > >> To fix this, I have configured the following on R1: > >> > >> access-list 1 deny 123.1.9.0 0.0.0.255 > >> access-list 1 permit any > >> mpls ldp neighbor 123.1.9.9 labels accept 1 > >> > >> On R3 I did similar: > >> > >> access-list 1 deny 123.3.8.0 0.0.0.255 > >> access-list 1 permit any > >> mpls ldp neighbor 123.3.8.8 labels accept 1 > >> > >> It's been up for more than 3 minutes: > >> > >> R1#sho mpls ldp neighbor > >> Peer LDP Ident: 123.1.9.9:0; Local LDP Ident 123.123.123.1:0 > >> TCP connection: 123.1.9.9.11032 - 123.1.9.1.646 > >> State: Oper; Msgs sent/rcvd: 13/13; Downstream > >> Up time: 00:04:55 > >> LDP discovery sources: > >> Serial1/0, Src IP addr: 123.1.9.9 > >> Addresses bound to peer LDP Ident: > >> 123.1.9.9 > >> > >> R3#sho mpls ldp neighbor > >> Peer LDP Ident: 123.3.8.8:0; Local LDP Ident 123.123.123.3:0 > >> TCP connection: 123.3.8.8.11036 - 123.3.8.3.646 > >> State: Oper; Msgs sent/rcvd: 15/15; Downstream > >> Up time: 00:06:13 > >> LDP discovery sources: > >> Serial1/0, Src IP addr: 123.3.8.8 > >> Addresses bound to peer LDP Ident: > >> 123.3.8.8 > >> > >> Can't believe that is they way it should be designed (I still think > >> bug), but for now it is stable :-) > >> > >> thanks all, > >> > >> On Fri, Sep 4, 2009 at 8:47 AM, Rick Mur <[email protected]> wrote: > >> True, the CSC code can be buggy sometimes. As this 3 minute marker > >> is definitely a bug. I recall I also had some issues where it worked > >> only some period of time, so maybe I ran into that same issue. Still > >> it's sometimes buggy and I don't think that will happen on your lab. > >> > >> > >> -- > >> > >> Regards, > >> > >> Rick Mur > >> CCIE2 #21946 (R&S / Service Provider) > >> Sr. Support Engineer IPexpert, Inc. > >> URL: http://www.IPexpert.com > >> > >> On 4 sep 2009, at 14:11, Bryan Bartik wrote: > >> > >>> Thanks Guys, I am using 7200s in dynamips with 12.2S code. In fact, > >>> this is a mock set up of vol 2 lab 4, I just took everything else > >>> out of the equation. I started from scratch just doing the MPLS VPN > >>> scenarios. Even R2 is not doing anything (ospf or ldp) right now. I > >>> am also just using physical interfaces on my frame relay cloud :) I > >>> don't see the client tagging the packets but I could look deeper > >>> into this. It's just amazing that is every 3 minutes pretty much on > >>> the dot! I wonder if it's some type of CSC bug having to do with > >>> MPLS on the VRF interface...I will do some more testing and let you > >>> know. > >>> > >>> On Fri, Sep 4, 2009 at 3:22 AM, Antonie Henning - MWEB < > [email protected] > >>>> wrote: > >>> Had a similar issue. I narrowed it down to point to point frame- > >>> relay subinterface with csc igp (ospf) and ldp enabled. > >>> > >>> What I saw was a loop. The client would tag packets to the carrier > >>> for the directly connected subnet. I was expecting to see it pop > >>> the label: > >>> > >>> R2(config-subif)#do sh ip cef 123.2.6.6 > >>> 123.2.6.0/24 > >>> attached to Serial4/0.206 label 614 > >>> > >>> The 614 label then sends the packet back to the client and a loop > >>> forms: > >>> > >>> R2(config)#do trace 123.2.6.6 > >>> > >>> Type escape sequence to abort. > >>> Tracing the route to 123.2.6.6 > >>> > >>> 1 123.2.6.6 [MPLS: Label 614 Exp 0] 8 msec 32 msec 12 msec > >>> 2 123.2.6.2 20 msec 40 msec 40 msec > >>> 3 123.2.6.6 [MPLS: Label 614 Exp 0] 44 msec 24 msec 44 msec > >>> 4 123.2.6.2 20 msec 48 msec 64 msec > >>> 5 123.2.6.6 [MPLS: Label 614 Exp 0] 40 msec 24 msec 40 msec > >>> 6 123.2.6.2 48 msec 72 msec 60 msec > >>> 7 123.2.6.6 [MPLS: Label 614 Exp 0] 60 msec 36 msec 64 msec > >>> 8 123.2.6.2 56 msec 60 msec 88 msec > >>> 9 123.2.6.6 [MPLS: Label 614 Exp 0] 60 msec 108 msec 60 msec > >>> > >>> Changing the frame-relay config to use the main interface solved > >>> the problem. > >>> > >>> Hth > >>> 21500.net > >>> > >>> -----Original Message----- > >>> From: [email protected] [mailto:[email protected]] On > >>> Behalf Of Francisco Baena > >>> Sent: 04 September 2009 08:11 AM > >>> To: 'Bryan Bartik'; [email protected]; > [email protected] > >>> Subject: RE: Carrier Supporting Carrier (CSC) - LDP peering between > >>> SPs drops every 3 minutes > >>> > >>> Make it two. I had the same problem with vol II - Lab 4 (I think), > >>> but from > >>> R2 to R6. > >>> > >>> At that point I blamed on dynamips being buggy as all the routing/ > >>> MPLS > >>> tables seem fine. > >>> > >>> I look forward to a resolution on this. The interesting thing is > >>> that when I > >>> shut down the connection from R1 to R2, the problem went away, so > >>> it sounds > >>> like an IGP issue. However even making a Sham link between r6 and > >>> r9 and > >>> increasing the OSPF cost from R1 to R2 (to ensure AS200 was the > >>> exit point), > >>> made no difference. > >>> > >>> I would say that a possible workaround could be to make the R1-R9 LDP > >>> sessions targeted, but obviously what we all want to know is what > >>> the heck > >>> happened there in the first place. > >>> > >>> During my testing I disabled MPLS TE too in AS200, just in case, > >>> but no > >>> cigar.... > >>> > >>> Cheers, > >>> Francisco > >>> http://www.linkedin.com/in/fbaena > >>> > >>> > >>> -----Original Message----- > >>> From: [email protected] [mailto:[email protected]] On > >>> Behalf Of > >>> Bryan Bartik > >>> Sent: 04 September 2009 05:14 > >>> To: [email protected]; [email protected] > >>> Subject: Carrier Supporting Carrier (CSC) - LDP peering between SPs > >>> drops > >>> every 3 minutes > >>> > >>> Hello, > >>> > >>> I am running an inter-as + CSC scenario with OSPF and MPLS between > >>> the SPs. > >>> The LDP peering is dropping exactly every 3 minutes and coming back > >>> up. > >>> Connectivity is fine throughout the VPN when the session is up. I > >>> don't see > >>> any route flapping in the debugs. > >>> > >>> AS65123[R3]---------[R8]AS100---------AS200[R9]---------[R1]AS65123 > >>> > >>> AS100 and AS200 have an inter-as VPN supporting the 2nd level carrier > >>> AS65123. > >>> R9 and R8 have vrf interfaces connected to R1 and R3 respectively. > >>> R1 has an LDP/OSPF peering with R9 (in the vrf) in AS100 > >>> R3 has an LDP/OSPF peering with R8 (in the vrf) in AS200 > >>> > >>> Both LDP sessions are bouncing on queue every 3 minutes! Here is R8 > >>> for > >>> example: > >>> > >>> R8# > >>> *Sep 3 21:50:49.575: %LDP-5-NBRCHG: LDP Neighbor 123.123.123.3:0 > >>> is DOWN > >>> *Sep 3 21:50:59.475: %LDP-5-NBRCHG: LDP Neighbor 123.123.123.3:0 > >>> is UP > >>> *Sep 3 21:53:59.491: %LDP-5-NBRCHG: LDP Neighbor 123.123.123.3:0 > >>> is DOWN > >>> *Sep 3 21:54:09.291: %LDP-5-NBRCHG: LDP Neighbor 123.123.123.3:0 > >>> is UP > >>> *Sep 3 21:57:09.311: %LDP-5-NBRCHG: LDP Neighbor 123.123.123.3:0 > >>> is DOWN > >>> *Sep 3 21:57:19.331: %LDP-5-NBRCHG: LDP Neighbor 123.123.123.3:0 > >>> is UP > >>> *Sep 3 22:00:19.351: %LDP-5-NBRCHG: LDP Neighbor 123.123.123.3:0 > >>> is DOWN > >>> *Sep 3 22:00:29.239: %LDP-5-NBRCHG: LDP Neighbor 123.123.123.3:0 > >>> is UP > >>> > >>> Now just before the session dies I get the following from "debug > >>> mpls ldp > >>> transport events interface x/x". This is from R9: > >>> > >>> R9# > >>> *Sep 3 22:09:27.295: ldp: Send ldp hello; Serial1/2, src/dst > >>> 123.1.9.9/224.0.0.2, inst_id 0 > >>> *Sep 3 22:09:28.787: ldp: Rcvd ldp hello; Serial1/2, from > >>> 123.1.9.1 ( > >>> 123.123.123.1:0), intf_id 0, opt 0xC > >>> *Sep 3 22:09:32.127: ldp: Send ldp hello; Serial1/2, src/dst > >>> 123.1.9.9/224.0.0.2, inst_id 0 > >>> *Sep 3 22:09:33.239: ldp: Rcvd ldp hello; Serial1/2, from > >>> 123.1.9.1 ( > >>> 123.123.123.1:0), intf_id 0, opt 0xC > >>> *Sep 3 22:09:36.111: ldp: Send ldp hello; Serial1/2, src/dst > >>> 123.1.9.9/224.0.0.2, inst_id 0 > >>> *Sep 3 22:09:36.907: tagcon: Session KeepAlive timer expired, peer > >>> 123.123.123.1:0 (pp 0x64027BD8) > >>> *Sep 3 22:09:36.911: ldp: Close LDP transport conn for adj > >>> 0x63486648 > >>> *Sep 3 22:09:36.915: ldp: Closing ldp conn 123.1.9.9:646 <-> > >>> 123.123.123.1:11015, adj 0x63486648 > >>> *Sep 3 22:09:36.919: ldp: Adj 0x63486648; state set to closed > >>> *Sep 3 22:09:36.923: %LDP-5-NBRCHG: LDP Neighbor 123.123.123.1:0 > >>> is DOWN > >>> > >>> Even though I am sending and receiving hellos, I am getting the > >>> session > >>> keepalive timer expired. Any ideas? > >>> > >>> Thanks! > >>> -- > >>> Bryan Bartik > >>> CCIE #23707 (R&S), CCNP > >>> Sr. Support Engineer - IPexpert, Inc. > >>> URL: http://www.IPexpert.com > >>> > >>> _____________________________________________________________________ > >>> Subscription information: http://www.groupstudy.com/list/comserv.html > >>> > >>> No virus found in this incoming message. > >>> Checked by AVG - www.avg.com > >>> Version: 8.5.409 / Virus Database: 270.13.76/2343 - Release Date: > >>> 09/03/09 > >>> 05:50:00 > >>> > >>> _____________________________________________________________________ > >>> Subscription information: http://www.groupstudy.com/list/comserv.html > >>> Connect with South Africa s leading Internet Service Provider and > >>> discover the magic of the Internet and all its possibilities. > >>> Call 08600 32000 or click here(http://www.mweb.co.za/productsservices/ > >>> ) for more. > >>> > >>> MWEB :-) CONNECT AND YOU CAN. > >>> > >>> This electronic communication and the attached file(s) are subject > >>> to a disclaimer which can be accessed on the following link: > >>> Disclaimer - or copy the following URL into your browser - > > http://www.mweb.co.za/disclaimer > >>> . > >>> If you are unable to view the disclaimer, please contact > [email protected] > >>> for a copy. > >>> > >>> > >>> > >>> -- > >>> Bryan Bartik > >>> CCIE #23707 (R&S), CCNP > >>> Sr. Support Engineer - IPexpert, Inc. > >>> URL: http://www.IPexpert.com > >>> _______________________________________________ > >>> For more information regarding industry leading CCIE Lab training, > >>> please visit www.ipexpert.com > >> > >> > >> > >> -- > >> Bryan Bartik > >> CCIE #23707 (R&S), CCNP > >> Sr. Support Engineer - IPexpert, Inc. > >> URL: http://www.IPexpert.com > > > > _____________________________________________________________________ > > Subscription information: http://www.groupstudy.com/list/comserv.html > > > > > > ------------------------------------------------------------------------ > > > > > > No virus found in this incoming message. > > Checked by AVG - www.avg.com > > Version: 8.5.409 / Virus Database: 270.13.76/2345 - Release Date: > 09/04/09 05:51:00 > > > > -- Bryan Bartik CCIE #23707 (R&S), CCNP Sr. Support Engineer - IPexpert, Inc. URL: http://www.IPexpert.com
_______________________________________________ For more information regarding industry leading CCIE Lab training, please visit www.ipexpert.com
