Weird! Still a bug in the CSC code.
Great workaround though! This is very good for your preparation :-)


-- Regards,

Rick Mur
CCIE2 #21946 (R&S / Service Provider)
Sr. Support Engineer – IPexpert, Inc.
URL: http://www.IPexpert.com

On 4 sep 2009, at 18:47, Bryan Bartik wrote:

Well I think Antoine was on to something...and I found another workaround :-)

I did a packet capture in Dynamips and just before the session drops, the TCP/LDP keepalive between R1 and R9 loop between each other. I get about 254 packets in under a sec and I can see the TTL decrement on each one! This happens every minute and eventually the timeout expires because these keepalive are not received and processed locally by R9 (thus every 3 minutes it drops). R1 is sending the keepalive as a labeled packet even though R9 is directly connected, I think it should be an imp-null label. According to the capture R1 is putting label 20 on the keepalive. Here I verified it:

R1#sho mpls ldp bindings
  lib entry: 123.1.9.0/24, rev 2
        local binding:  label: imp-null
        remote binding: lsr: 123.1.9.9:0, label: 20

R9#sho mpls forwarding-table labels 20
Local Outgoing Prefix Bytes Label Outgoing Next Hop
Label  Label or VC   or Tunnel Id      Switched      interface
20 Pop Label 123.1.9.0/24[V] 3358266 Se1/2 point2point
R9#

R1 is learning the label 20 and using that to send packets onto the directly connected network. R9 receives it, pops it and sends it back out of the interface instead of receiving it locally. (I am using the interface as ldp transport address for this example). It's hard to tell who is at fault here!

In order for the session to actually come up again, all the learned labels have to get flushed so the TCP exchange can happen. Once it does, TCP handshake and label exchange occurs without any label encapsulation. R9 then advertises the vrf subnet (connected to R1) in a label mapping message with a label of 20 (verified with capture). This label is now used by R1 and any further TCP exchange uses that label and packets loop. It's funny to see it happen, because as soon as R9 sends that label mapping, R1 uses the label in the ACK! However this does not cause any problems at this point, because R1 also sends an unlabeled ACK as well, completeing the exchange (weird).

To fix this, I have configured the following on R1:

access-list 1 deny   123.1.9.0 0.0.0.255
access-list 1 permit any
mpls ldp neighbor 123.1.9.9 labels accept 1

On R3 I did similar:

access-list 1 deny   123.3.8.0 0.0.0.255
access-list 1 permit any
mpls ldp neighbor 123.3.8.8 labels accept 1

It's been up for more than 3 minutes:

R1#sho mpls ldp neighbor
    Peer LDP Ident: 123.1.9.9:0; Local LDP Ident 123.123.123.1:0
        TCP connection: 123.1.9.9.11032 - 123.1.9.1.646
        State: Oper; Msgs sent/rcvd: 13/13; Downstream
        Up time: 00:04:55
        LDP discovery sources:
          Serial1/0, Src IP addr: 123.1.9.9
        Addresses bound to peer LDP Ident:
          123.1.9.9

R3#sho mpls ldp neighbor
    Peer LDP Ident: 123.3.8.8:0; Local LDP Ident 123.123.123.3:0
        TCP connection: 123.3.8.8.11036 - 123.3.8.3.646
        State: Oper; Msgs sent/rcvd: 15/15; Downstream
        Up time: 00:06:13
        LDP discovery sources:
          Serial1/0, Src IP addr: 123.3.8.8
        Addresses bound to peer LDP Ident:
          123.3.8.8

Can't believe that is they way it should be designed (I still think bug), but for now it is stable :-)

thanks all,

On Fri, Sep 4, 2009 at 8:47 AM, Rick Mur <[email protected]> wrote:
True, the CSC code can be buggy sometimes. As this 3 minute marker is definitely a bug. I recall I also had some issues where it worked only some period of time, so maybe I ran into that same issue. Still it's sometimes buggy and I don't think that will happen on your lab.


--

Regards,

Rick Mur
CCIE2 #21946 (R&S / Service Provider)
Sr. Support Engineer – IPexpert, Inc.
URL: http://www.IPexpert.com

On 4 sep 2009, at 14:11, Bryan Bartik wrote:

Thanks Guys, I am using 7200s in dynamips with 12.2S code. In fact, this is a mock set up of vol 2 lab 4, I just took everything else out of the equation. I started from scratch just doing the MPLS VPN scenarios. Even R2 is not doing anything (ospf or ldp) right now. I am also just using physical interfaces on my frame relay cloud :) I don't see the client tagging the packets but I could look deeper into this. It's just amazing that is every 3 minutes pretty much on the dot! I wonder if it's some type of CSC bug having to do with MPLS on the VRF interface...I will do some more testing and let you know.

On Fri, Sep 4, 2009 at 3:22 AM, Antonie Henning - MWEB <[email protected] > wrote: Had a similar issue. I narrowed it down to point to point frame- relay subinterface with csc igp (ospf) and ldp enabled.

What I saw was a loop. The client would tag packets to the carrier for the directly connected subnet. I was expecting to see it pop the label:

R2(config-subif)#do sh ip cef 123.2.6.6
123.2.6.0/24
 attached to Serial4/0.206 label 614

The 614 label then sends the packet back to the client and a loop forms:

R2(config)#do trace 123.2.6.6

Type escape sequence to abort.
Tracing the route to 123.2.6.6

 1 123.2.6.6 [MPLS: Label 614 Exp 0] 8 msec 32 msec 12 msec
 2 123.2.6.2 20 msec 40 msec 40 msec
 3 123.2.6.6 [MPLS: Label 614 Exp 0] 44 msec 24 msec 44 msec
 4 123.2.6.2 20 msec 48 msec 64 msec
 5 123.2.6.6 [MPLS: Label 614 Exp 0] 40 msec 24 msec 40 msec
 6 123.2.6.2 48 msec 72 msec 60 msec
 7 123.2.6.6 [MPLS: Label 614 Exp 0] 60 msec 36 msec 64 msec
 8 123.2.6.2 56 msec 60 msec 88 msec
 9 123.2.6.6 [MPLS: Label 614 Exp 0] 60 msec 108 msec 60 msec

Changing the frame-relay config to use the main interface solved the problem.

Hth
21500.net

-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Francisco Baena
Sent: 04 September 2009 08:11 AM
To: 'Bryan Bartik'; [email protected]; [email protected]
Subject: RE: Carrier Supporting Carrier (CSC) - LDP peering between SPs drops every 3 minutes

Make it two. I had the same problem with vol II - Lab 4 (I think), but from
R2 to R6.

At that point I blamed on dynamips being buggy as all the routing/ MPLS
tables seem fine.

I look forward to a resolution on this. The interesting thing is that when I shut down the connection from R1 to R2, the problem went away, so it sounds like an IGP issue. However even making a Sham link between r6 and r9 and increasing the OSPF cost from R1 to R2 (to ensure AS200 was the exit point),
made no difference.

I would say that a possible workaround could be to make the R1-R9 LDP
sessions targeted, but obviously what we all want to know is what the heck
happened there in the first place.

During my testing I disabled MPLS TE too in AS200, just in case, but no
cigar....

Cheers,
Francisco
http://www.linkedin.com/in/fbaena


-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of
Bryan Bartik
Sent: 04 September 2009 05:14
To: [email protected]; [email protected]
Subject: Carrier Supporting Carrier (CSC) - LDP peering between SPs drops
every 3 minutes

Hello,

I am running an inter-as + CSC scenario with OSPF and MPLS between the SPs. The LDP peering is dropping exactly every 3 minutes and coming back up. Connectivity is fine throughout the VPN when the session is up. I don't see
any route flapping in the debugs.

AS65123[R3]---------[R8]AS100---------AS200[R9]---------[R1]AS65123

AS100 and AS200 have an inter-as VPN supporting the 2nd level carrier
AS65123.
R9 and R8 have vrf interfaces connected to R1 and R3 respectively.
R1 has an LDP/OSPF peering with R9 (in the vrf) in AS100
R3 has an LDP/OSPF peering with R8 (in the vrf) in AS200

Both LDP sessions are bouncing on queue every 3 minutes! Here is R8 for
example:

R8#
*Sep 3 21:50:49.575: %LDP-5-NBRCHG: LDP Neighbor 123.123.123.3:0 is DOWN *Sep 3 21:50:59.475: %LDP-5-NBRCHG: LDP Neighbor 123.123.123.3:0 is UP *Sep 3 21:53:59.491: %LDP-5-NBRCHG: LDP Neighbor 123.123.123.3:0 is DOWN *Sep 3 21:54:09.291: %LDP-5-NBRCHG: LDP Neighbor 123.123.123.3:0 is UP *Sep 3 21:57:09.311: %LDP-5-NBRCHG: LDP Neighbor 123.123.123.3:0 is DOWN *Sep 3 21:57:19.331: %LDP-5-NBRCHG: LDP Neighbor 123.123.123.3:0 is UP *Sep 3 22:00:19.351: %LDP-5-NBRCHG: LDP Neighbor 123.123.123.3:0 is DOWN *Sep 3 22:00:29.239: %LDP-5-NBRCHG: LDP Neighbor 123.123.123.3:0 is UP

Now just before the session dies I get the following from "debug mpls ldp
transport events interface x/x". This is from R9:

R9#
*Sep  3 22:09:27.295: ldp: Send ldp hello; Serial1/2, src/dst
123.1.9.9/224.0.0.2, inst_id 0
*Sep 3 22:09:28.787: ldp: Rcvd ldp hello; Serial1/2, from 123.1.9.1 (
123.123.123.1:0), intf_id 0, opt 0xC
*Sep  3 22:09:32.127: ldp: Send ldp hello; Serial1/2, src/dst
123.1.9.9/224.0.0.2, inst_id 0
*Sep 3 22:09:33.239: ldp: Rcvd ldp hello; Serial1/2, from 123.1.9.1 (
123.123.123.1:0), intf_id 0, opt 0xC
*Sep  3 22:09:36.111: ldp: Send ldp hello; Serial1/2, src/dst
123.1.9.9/224.0.0.2, inst_id 0
*Sep  3 22:09:36.907: tagcon: Session KeepAlive timer expired, peer
123.123.123.1:0 (pp 0x64027BD8)
*Sep 3 22:09:36.911: ldp: Close LDP transport conn for adj 0x63486648
*Sep  3 22:09:36.915: ldp: Closing ldp conn 123.1.9.9:646 <->
123.123.123.1:11015, adj 0x63486648
*Sep  3 22:09:36.919: ldp: Adj 0x63486648; state set to closed
*Sep 3 22:09:36.923: %LDP-5-NBRCHG: LDP Neighbor 123.123.123.1:0 is DOWN

Even though I am sending and receiving hellos, I am getting the session
keepalive timer expired. Any ideas?

Thanks!
--
Bryan Bartik
CCIE #23707 (R&S), CCNP
Sr. Support Engineer - IPexpert, Inc.
URL: http://www.IPexpert.com

_____________________________________________________________________
Subscription information: http://www.groupstudy.com/list/comserv.html

No virus found in this incoming message.
Checked by AVG - www.avg.com
Version: 8.5.409 / Virus Database: 270.13.76/2343 - Release Date: 09/03/09
05:50:00

_____________________________________________________________________
Subscription information: http://www.groupstudy.com/list/comserv.html
Connect with South Africa’s leading Internet Service Provider and discover the magic of the Internet and all its possibilities. Call 08600 32000 or click here(http://www.mweb.co.za/productsservices/ ) for more.

MWEB :-)  CONNECT AND YOU CAN.

This electronic communication and the attached file(s) are subject to a disclaimer which can be accessed on the following link: Disclaimer - or copy the following URL into your browser - http://www.mweb.co.za/disclaimer . If you are unable to view the disclaimer, please contact [email protected] for a copy.



--
Bryan Bartik
CCIE #23707 (R&S), CCNP
Sr. Support Engineer - IPexpert, Inc.
URL: http://www.IPexpert.com
_______________________________________________
For more information regarding industry leading CCIE Lab training, please visit www.ipexpert.com




--
Bryan Bartik
CCIE #23707 (R&S), CCNP
Sr. Support Engineer - IPexpert, Inc.
URL: http://www.IPexpert.com

_______________________________________________
For more information regarding industry leading CCIE Lab training, please visit 
www.ipexpert.com

Reply via email to