Jo, That's what it appears. Because it is in a VRF, it creates that VPN label as it would to advertise to a regular PE. It advertises that same label when the LDP peering comes up to R1. No split-horizon in LDP! R9 doesn't know any better...but I think R1 should! I am still going to dig for some more information, maybe query TAC. I'll let you know if I find anything else.
On Fri, Sep 4, 2009 at 11:49 AM, Jo Knight <[email protected]> wrote: > I am seeing the same thing on this IOS: > > Cisco IOS Software, 7200 Software (C7200-K91P-M), Version 12.2(25)S9, > RELEASE SOFTWARE (fc1) > > 00:27:36: %LDP-5-NBRCHG: LDP Neighbor 150.1.1.1:0 is DOWN > 00:27:43: %LDP-5-NBRCHG: LDP Neighbor 150.1.1.1:0 is UP > 00:30:43: %LDP-5-NBRCHG: LDP Neighbor 150.1.1.1:0 is DOWN > 00:30:50: %LDP-5-NBRCHG: LDP Neighbor 150.1.1.1:0 is UP > 00:33:50: %LDP-5-NBRCHG: LDP Neighbor 150.1.1.1:0 is DOWN > 00:34:00: %LDP-5-NBRCHG: LDP Neighbor 150.1.1.1:0 is UP > 00:37:00: %LDP-5-NBRCHG: LDP Neighbor 150.1.1.1:0 is DOWN > 00:37:11: %LDP-5-NBRCHG: LDP Neighbor 150.1.1.1:0 is UP > 00:40:11: %LDP-5-NBRCHG: LDP Neighbor 150.1.1.1:0 is DOWN > > > I implemented the workaround and it seems stable now. > > Is it happening due to ldp being configured on an interface within a VRF in > this case? > > Good spot though - thanks. > > Jo > > 2009/9/4 Bryan Bartik <[email protected]> > > Well I think Antoine was on to something...and I found another workaround >> :-) >> >> I did a packet capture in Dynamips and just before the session drops, the >> TCP/LDP keepalive between R1 and R9 loop between each other. I get about 254 >> packets in under a sec and I can see the TTL decrement on each one! This >> happens every minute and eventually the timeout expires because these >> keepalive are not received and processed locally by R9 (thus every 3 minutes >> it drops). R1 is sending the keepalive as a labeled packet even though R9 is >> directly connected, I think it should be an imp-null label. According to the >> capture R1 is putting label 20 on the keepalive. Here I verified it: >> >> R1#sho mpls ldp bindings >> lib entry: 123.1.9.0/24, rev 2 >> local binding: label: imp-null >> remote binding: lsr: 123.1.9.9:0, label: 20 >> >> R9#sho mpls forwarding-table labels 20 >> Local Outgoing Prefix Bytes Label Outgoing Next >> Hop >> Label Label or VC or Tunnel Id Switched >> interface >> 20 Pop Label 123.1.9.0/24[V] <http://123.1.9.0/24%5BV%5D> >> 3358266 Se1/2 point2point >> R9# >> >> R1 is learning the label 20 and using that to send packets onto the >> directly connected network. R9 receives it, pops it and sends it back out of >> the interface instead of receiving it locally. (I am using the interface as >> ldp transport address for this example). It's hard to tell who is at fault >> here! >> >> In order for the session to actually come up again, all the learned labels >> have to get flushed so the TCP exchange can happen. Once it does, TCP >> handshake and label exchange occurs without any label encapsulation. R9 then >> advertises the vrf subnet (connected to R1) in a label mapping message with >> a label of 20 (verified with capture). This label is now used by R1 and any >> further TCP exchange uses that label and packets loop. It's funny to see it >> happen, because as soon as R9 sends that label mapping, R1 uses the label in >> the ACK! However this does not cause any problems at this point, because R1 >> also sends an unlabeled ACK as well, completeing the exchange (weird). >> >> To fix this, I have configured the following on R1: >> >> access-list 1 deny 123.1.9.0 0.0.0.255 >> access-list 1 permit any >> mpls ldp neighbor 123.1.9.9 labels accept 1 >> >> On R3 I did similar: >> >> access-list 1 deny 123.3.8.0 0.0.0.255 >> access-list 1 permit any >> mpls ldp neighbor 123.3.8.8 labels accept 1 >> >> It's been up for more than 3 minutes: >> >> R1#sho mpls ldp neighbor >> Peer LDP Ident: 123.1.9.9:0; Local LDP Ident 123.123.123.1:0 >> TCP connection: 123.1.9.9.11032 - 123.1.9.1.646 >> State: Oper; Msgs sent/rcvd: 13/13; Downstream >> Up time: 00:04:55 >> LDP discovery sources: >> Serial1/0, Src IP addr: 123.1.9.9 >> Addresses bound to peer LDP Ident: >> 123.1.9.9 >> >> R3#sho mpls ldp neighbor >> Peer LDP Ident: 123.3.8.8:0; Local LDP Ident 123.123.123.3:0 >> TCP connection: 123.3.8.8.11036 - 123.3.8.3.646 >> State: Oper; Msgs sent/rcvd: 15/15; Downstream >> Up time: 00:06:13 >> LDP discovery sources: >> Serial1/0, Src IP addr: 123.3.8.8 >> Addresses bound to peer LDP Ident: >> 123.3.8.8 >> >> Can't believe that is they way it should be designed (I still think bug), >> but for now it is stable :-) >> >> thanks all, >> >> >> On Fri, Sep 4, 2009 at 8:47 AM, Rick Mur <[email protected]> wrote: >> >>> True, the CSC code can be buggy sometimes. As this 3 minute marker is >>> definitely a bug. I recall I also had some issues where it worked only some >>> period of time, so maybe I ran into that same issue. Still it's sometimes >>> buggy and I don't think that will happen on your lab. >>> >>> -- >>> Regards, >>> >>> Rick Mur >>> CCIE2 #21946 (R&S / Service Provider) >>> Sr. Support Engineer – IPexpert, Inc. >>> URL: http://www.IPexpert.com >>> >>> On 4 sep 2009, at 14:11, Bryan Bartik wrote: >>> >>> Thanks Guys, I am using 7200s in dynamips with 12.2S code. In fact, this >>> is a mock set up of vol 2 lab 4, I just took everything else out of the >>> equation. I started from scratch just doing the MPLS VPN scenarios. Even R2 >>> is not doing anything (ospf or ldp) right now. I am also just using physical >>> interfaces on my frame relay cloud :) I don't see the client tagging the >>> packets but I could look deeper into this. It's just amazing that is every 3 >>> minutes pretty much on the dot! I wonder if it's some type of CSC bug having >>> to do with MPLS on the VRF interface...I will do some more testing and let >>> you know. >>> >>> On Fri, Sep 4, 2009 at 3:22 AM, Antonie Henning - MWEB < >>> [email protected]> wrote: >>> >>>> Had a similar issue. I narrowed it down to point to point frame-relay >>>> subinterface with csc igp (ospf) and ldp enabled. >>>> >>>> What I saw was a loop. The client would tag packets to the carrier for >>>> the directly connected subnet. I was expecting to see it pop the label: >>>> >>>> R2(config-subif)#do sh ip cef 123.2.6.6 >>>> 123.2.6.0/24 >>>> attached to Serial4/0.206 label 614 >>>> >>>> The 614 label then sends the packet back to the client and a loop forms: >>>> >>>> R2(config)#do trace 123.2.6.6 >>>> >>>> Type escape sequence to abort. >>>> Tracing the route to 123.2.6.6 >>>> >>>> 1 123.2.6.6 [MPLS: Label 614 Exp 0] 8 msec 32 msec 12 msec >>>> 2 123.2.6.2 20 msec 40 msec 40 msec >>>> 3 123.2.6.6 [MPLS: Label 614 Exp 0] 44 msec 24 msec 44 msec >>>> 4 123.2.6.2 20 msec 48 msec 64 msec >>>> 5 123.2.6.6 [MPLS: Label 614 Exp 0] 40 msec 24 msec 40 msec >>>> 6 123.2.6.2 48 msec 72 msec 60 msec >>>> 7 123.2.6.6 [MPLS: Label 614 Exp 0] 60 msec 36 msec 64 msec >>>> 8 123.2.6.2 56 msec 60 msec 88 msec >>>> 9 123.2.6.6 [MPLS: Label 614 Exp 0] 60 msec 108 msec 60 msec >>>> >>>> Changing the frame-relay config to use the main interface solved the >>>> problem. >>>> >>>> Hth >>>> 21500.net >>>> >>>> -----Original Message----- >>>> From: [email protected] [mailto:[email protected]] On Behalf Of >>>> Francisco Baena >>>> Sent: 04 September 2009 08:11 AM >>>> To: 'Bryan Bartik'; [email protected]; [email protected] >>>> Subject: RE: Carrier Supporting Carrier (CSC) - LDP peering between SPs >>>> drops every 3 minutes >>>> >>>> Make it two. I had the same problem with vol II - Lab 4 (I think), but >>>> from >>>> R2 to R6. >>>> >>>> At that point I blamed on dynamips being buggy as all the routing/MPLS >>>> tables seem fine. >>>> >>>> I look forward to a resolution on this. The interesting thing is that >>>> when I >>>> shut down the connection from R1 to R2, the problem went away, so it >>>> sounds >>>> like an IGP issue. However even making a Sham link between r6 and r9 and >>>> increasing the OSPF cost from R1 to R2 (to ensure AS200 was the exit >>>> point), >>>> made no difference. >>>> >>>> I would say that a possible workaround could be to make the R1-R9 LDP >>>> sessions targeted, but obviously what we all want to know is what the >>>> heck >>>> happened there in the first place. >>>> >>>> During my testing I disabled MPLS TE too in AS200, just in case, but no >>>> cigar.... >>>> >>>> Cheers, >>>> Francisco >>>> http://www.linkedin.com/in/fbaena >>>> >>>> >>>> -----Original Message----- >>>> From: [email protected] [mailto:[email protected]] On Behalf Of >>>> Bryan Bartik >>>> Sent: 04 September 2009 05:14 >>>> To: [email protected]; [email protected] >>>> Subject: Carrier Supporting Carrier (CSC) - LDP peering between SPs >>>> drops >>>> every 3 minutes >>>> >>>> Hello, >>>> >>>> I am running an inter-as + CSC scenario with OSPF and MPLS between the >>>> SPs. >>>> The LDP peering is dropping exactly every 3 minutes and coming back up. >>>> Connectivity is fine throughout the VPN when the session is up. I don't >>>> see >>>> any route flapping in the debugs. >>>> >>>> AS65123[R3]---------[R8]AS100---------AS200[R9]---------[R1]AS65123 >>>> >>>> AS100 and AS200 have an inter-as VPN supporting the 2nd level carrier >>>> AS65123. >>>> R9 and R8 have vrf interfaces connected to R1 and R3 respectively. >>>> R1 has an LDP/OSPF peering with R9 (in the vrf) in AS100 >>>> R3 has an LDP/OSPF peering with R8 (in the vrf) in AS200 >>>> >>>> Both LDP sessions are bouncing on queue every 3 minutes! Here is R8 for >>>> example: >>>> >>>> R8# >>>> *Sep 3 21:50:49.575: %LDP-5-NBRCHG: LDP Neighbor 123.123.123.3:0 is >>>> DOWN >>>> *Sep 3 21:50:59.475: %LDP-5-NBRCHG: LDP Neighbor 123.123.123.3:0 is UP >>>> *Sep 3 21:53:59.491: %LDP-5-NBRCHG: LDP Neighbor 123.123.123.3:0 is >>>> DOWN >>>> *Sep 3 21:54:09.291: %LDP-5-NBRCHG: LDP Neighbor 123.123.123.3:0 is UP >>>> *Sep 3 21:57:09.311: %LDP-5-NBRCHG: LDP Neighbor 123.123.123.3:0 is >>>> DOWN >>>> *Sep 3 21:57:19.331: %LDP-5-NBRCHG: LDP Neighbor 123.123.123.3:0 is UP >>>> *Sep 3 22:00:19.351: %LDP-5-NBRCHG: LDP Neighbor 123.123.123.3:0 is >>>> DOWN >>>> *Sep 3 22:00:29.239: %LDP-5-NBRCHG: LDP Neighbor 123.123.123.3:0 is UP >>>> >>>> Now just before the session dies I get the following from "debug mpls >>>> ldp >>>> transport events interface x/x". This is from R9: >>>> >>>> R9# >>>> *Sep 3 22:09:27.295: ldp: Send ldp hello; Serial1/2, src/dst >>>> 123.1.9.9/224.0.0.2, inst_id 0 >>>> *Sep 3 22:09:28.787: ldp: Rcvd ldp hello; Serial1/2, from 123.1.9.1 ( >>>> 123.123.123.1:0), intf_id 0, opt 0xC >>>> *Sep 3 22:09:32.127: ldp: Send ldp hello; Serial1/2, src/dst >>>> 123.1.9.9/224.0.0.2, inst_id 0 >>>> *Sep 3 22:09:33.239: ldp: Rcvd ldp hello; Serial1/2, from 123.1.9.1 ( >>>> 123.123.123.1:0), intf_id 0, opt 0xC >>>> *Sep 3 22:09:36.111: ldp: Send ldp hello; Serial1/2, src/dst >>>> 123.1.9.9/224.0.0.2, inst_id 0 >>>> *Sep 3 22:09:36.907: tagcon: Session KeepAlive timer expired, peer >>>> 123.123.123.1:0 (pp 0x64027BD8) >>>> *Sep 3 22:09:36.911: ldp: Close LDP transport conn for adj 0x63486648 >>>> *Sep 3 22:09:36.915: ldp: Closing ldp conn 123.1.9.9:646 <-> >>>> 123.123.123.1:11015, adj 0x63486648 >>>> *Sep 3 22:09:36.919: ldp: Adj 0x63486648; state set to closed >>>> *Sep 3 22:09:36.923: %LDP-5-NBRCHG: LDP Neighbor 123.123.123.1:0 is >>>> DOWN >>>> >>>> Even though I am sending and receiving hellos, I am getting the session >>>> keepalive timer expired. Any ideas? >>>> >>>> Thanks! >>>> -- >>>> Bryan Bartik >>>> CCIE #23707 (R&S), CCNP >>>> Sr. Support Engineer - IPexpert, Inc. >>>> URL: http://www.IPexpert.com >>>> >>>> _____________________________________________________________________ >>>> Subscription information: http://www.groupstudy.com/list/comserv.html >>>> >>>> No virus found in this incoming message. >>>> Checked by AVG - www.avg.com >>>> Version: 8.5.409 / Virus Database: 270.13.76/2343 - Release Date: >>>> 09/03/09 >>>> 05:50:00 >>>> >>>> _____________________________________________________________________ >>>> Subscription information: http://www.groupstudy.com/list/comserv.html >>>> Connect with South Africa’s leading Internet Service Provider and >>>> discover the magic of the Internet and all its possibilities. >>>> Call 08600 32000 or click here(http://www.mweb.co.za/productsservices/) >>>> for more. >>>> >>>> MWEB :-) CONNECT AND YOU CAN. >>>> >>>> This electronic communication and the attached file(s) are subject to a >>>> disclaimer which can be accessed on the following link: >>>> Disclaimer - or copy the following URL into your browser - >>>> http://www.mweb.co.za/disclaimer. >>>> If you are unable to view the disclaimer, please contact [email protected] >>>> a copy. >>>> >>> >>> >>> >>> -- >>> Bryan Bartik >>> CCIE #23707 (R&S), CCNP >>> Sr. Support Engineer - IPexpert, Inc. >>> URL: http://www.IPexpert.com >>> _______________________________________________ >>> For more information regarding industry leading CCIE Lab training, please >>> visit www.ipexpert.com >>> >>> >>> >> >> >> -- >> Bryan Bartik >> CCIE #23707 (R&S), CCNP >> Sr. Support Engineer - IPexpert, Inc. >> URL: http://www.IPexpert.com >> >> _______________________________________________ >> For more information regarding industry leading CCIE Lab training, please >> visit www.ipexpert.com >> >> > -- Bryan Bartik CCIE #23707 (R&S), CCNP Sr. Support Engineer - IPexpert, Inc. URL: http://www.IPexpert.com
_______________________________________________ For more information regarding industry leading CCIE Lab training, please visit www.ipexpert.com
