On Tue, 23 Apr 2013, keith scott wrote:
> After changing the following line on our edge Firewalls PC.conf the Centos
> server that was unusable is now usable. I've done another tcp dump and
> there are still lot's of TCP ACT DUP's but not as many as there were
> before,
>
> match on $ExtIf scrub (random-id min-ttl 64 set-tos lowdelay reassemble
> tcp max-mss 1472) label "Scrubbing"
>
> to...
>
> match in on $ExtIf scrub (random-id min-ttl 64 set-tos lowdelay
> reassemble tcp max-mss 1472) label "Scrubbing"
>
> I will have to do some reading so see exactly why the above rule is causing
> issue with Centos VM's but for now everything seems back to normal :>)
My guess is that you previously did not have "reassemble tcp" enabled.
Generally speaking, you will not want to enable "reassemble tcp" if you're
talking to certain non-RFC1323 compliant hosts since the PAWS checks will
potentially result in stalled TCP connections.
> On Tue, Apr 23, 2013 at 12:11 AM, Keith <ke...@scott-land.net> wrote:
> > Hi, we recently switched our squid server from a OBSD server on VMware a
> > Centos server on XEN but there appears to be an issue somewhere between
> > the centos server and our OBSD Routers (DMZ) or our external OBSD
> > firewalls.
> >
> > If I log into the Centos server and run either wget or curl to an
> > exnternal http server I get a kind of random 1 in 3 chance or it working
> > or taking upto 30 seconds to complete. I've run tcpdump on the Centos box
> > and on the router and have imported the results into wireshare and they
> > both show lots of TCP Dup ACK's as shown below.
> >
> > We don't have any issues with any of our other servers that are also on
> > the same lan as this squid server so I think it's either a Centos,
> > Centos/Xen, or a OBSD issue. does anyone have any ideas what might be
> > going on here ?
> >
> > This dump was captured on our OBSD router.
> >
> > No. Time Source Destination Protocol Length
> > Info 3917 2.797310 10.0.0.X 20.0.0.X TCP 74
> > 35247
> >
> > > http [SYN] Seq=0 Win=14600 Len=0 MSS=1460 SACK_PERM=1 TSval=2936085
> >
> > TSecr=0 WS=64
> > 3922 2.799411 10.0.0.X 20.0.0.X TCP 66
> > 35247
> >
> > > http [ACK] Seq=1 Ack=1 Win=14656 Len=0 TSval=2936087 TSecr=0
> >
> > 3923 2.799543 10.0.0.X 20.0.0.X HTTP 175 GET
> > / HTTP/1.0
> > 3926 2.801331 10.0.0.X 20.0.0.X TCP 66 [TCP
> > Dup ACK 3923#1] 35247 > http [ACK] Seq=110 Ack=1 Win=14656 Len=0
> > TSval=2936089 TSecr=0
> > 3927 2.801333 10.0.0.X 20.0.0.X TCP 66 [TCP
> > Dup ACK 3923#2] 35247 > http [ACK] Seq=110 Ack=1 Win=14656 Len=0
> > TSval=2936089 TSecr=0
> > 3930 2.802423 10.0.0.X 20.0.0.X TCP 66 [TCP
> > Dup ACK 3923#3] 35247 > http [ACK] Seq=110 Ack=1 Win=14656 Len=0
> > TSval=2936090 TSecr=0
> > 3931 2.802425 10.0.0.X 20.0.0.X TCP 66 [TCP
> > Dup ACK 3923#4] 35247 > http [ACK] Seq=110 Ack=1 Win=14656 Len=0
> > TSval=2936090 TSecr=0
> > 4140 3.002585 10.0.0.X 20.0.0.X HTTP 175
> > [TCP Retransmission] GET / HTTP/1.0
> > 4142 3.003391 10.0.0.X 20.0.0.X TCP 66 [TCP
> > Dup ACK 4140#1] 35247 > http [ACK] Seq=110 Ack=1 Win=14656 Len=0
> > TSval=2936291 TSecr=0
> > 4663 3.410632 10.0.0.X 20.0.0.X HTTP 175
> > [TCP Retransmission] GET / HTTP/1.0
> > 4665 3.411451 10.0.0.X 20.0.0.X TCP 66 [TCP
> > Dup ACK 4663#1] 35247 > http [ACK] Seq=110 Ack=1 Win=14656 Len=0
> > TSval=2936699 TSecr=0
> > 5538 4.226611 10.0.0.X 20.0.0.X HTTP 175
> > [TCP Retransmission] GET / HTTP/1.0
> > 5541 4.227445 10.0.0.X 20.0.0.X TCP 66 [TCP
> > Dup ACK 5538#1] 35247 > http [ACK] Seq=110 Ack=1 Win=14656 Len=0
> > TSval=2937515 TSecr=0
> > 9846 5.843961 10.0.0.X 20.0.0.X TCP 66 [TCP
> > Dup ACK 5538#2] 35247 > http [ACK] Seq=110 Ack=1 Win=14656 Len=0
> > TSval=2939132 TSecr=0
> > 9851 5.844811 10.0.0.X 20.0.0.X TCP 66 [TCP
> > Dup ACK 5538#3] 35247 > http [ACK] Seq=110 Ack=1 Win=14656 Len=0
> > TSval=2939133 TSecr=0
> > 9861 5.858633 10.0.0.X 20.0.0.X HTTP 175
> > [TCP Retransmission] GET / HTTP/1.0
> > 9863 5.859432 10.0.0.X 20.0.0.X TCP 66 [TCP
> > Dup ACK 9861#1] 35247 > http [ACK] Seq=110 Ack=1 Win=14656 Len=0
> > TSval=2939147 TSecr=0
> > 14821 9.122718 10.0.0.X 20.0.0.X HTTP 175
> > [TCP Retransmission] GET / HTTP/1.0
> > 14823 9.123526 10.0.0.X 20.0.0.X TCP 66 [TCP
> > Dup ACK 14821#1] 35247 > http [ACK] Seq=110 Ack=1 Win=14656 Len=0
> > TSval=2942411 TSecr=0
> > 17858 11.859699 10.0.0.X 20.0.0.X TCP 66 [TCP
> > Dup ACK 14821#2] 35247 > http [ACK] Seq=110 Ack=1 Win=14656 Len=0
> > TSval=2945148 TSecr=0
> > 17863 11.860531 10.0.0.X 20.0.0.X TCP 66 [TCP
> > Dup ACK 14821#3] 35247 > http [ACK] Seq=110 Ack=1 Win=14656 Len=0
> > TSval=2945148 TSecr=0
> > 25393 15.650790 10.0.0.X 20.0.0.X HTTP 175
> > [TCP Retransmission] GET / HTTP/1.0
> > 25395 15.651626 10.0.0.X 20.0.0.X TCP 66 [TCP
> > Dup ACK 25393#1] 35247 > http [ACK] Seq=110 Ack=1 Win=14656 Len=0
> > TSval=2948939 TSecr=0
> > 45327 23.890899 10.0.0.X 20.0.0.X TCP 66 [TCP
> > Dup ACK 25393#2] 35247 > http [ACK] Seq=110 Ack=1 Win=14656 Len=0
> > TSval=2957178 TSecr=0
> > 48330 25.906963 10.0.0.X 20.0.0.X TCP 74
> > 35248
> >
> > > http [SYN] Seq=0 Win=14600 Len=0 MSS=1460 SACK_PERM=1 TSval=2959194
> >
> > TSecr=0 WS=64
> > 48337 25.908983 10.0.0.X 20.0.0.X TCP 66
> > 35248
> >
> > > http [ACK] Seq=1 Ack=1 Win=14656 Len=0 TSval=2959197 TSecr=0
> >
> > 48338 25.909077 10.0.0.X 20.0.0.X HTTP 175 GET
> > / HTTP/1.0
> > 48342 25.911184 10.0.0.X 20.0.0.X TCP 66 [TCP
> > Dup ACK 48338#1] 35248 > http [ACK] Seq=110 Ack=1 Win=14656 Len=0
> > TSval=2959199 TSecr=0
> > 48343 25.911186 10.0.0.X 20.0.0.X TCP 66 [TCP
> > Dup ACK 48338#2] 35248 > http [ACK] Seq=110 Ack=1 Win=14656 Len=0
> > TSval=2959199 TSecr=0
> > 48346 25.912272 10.0.0.X 20.0.0.X TCP 66 [TCP
> > Dup ACK 48338#3] 35248 > http [ACK] Seq=110 Ack=1 Win=14656 Len=0
> > TSval=2959200 TSecr=0
> > 48347 25.912274 10.0.0.X 20.0.0.X TCP 66 [TCP
> > Dup ACK 48338#4] 35248 > http [ACK] Seq=110 Ack=1 Win=14656 Len=0
> > TSval=2959200 TSecr=0
> > 48788 26.112919 10.0.0.X 20.0.0.X HTTP 175
> > [TCP Retransmission] GET / HTTP/1.0
> > 48794 26.113718 10.0.0.X 20.0.0.X TCP 66 [TCP
> > Dup ACK 48788#1] 35248 > http [ACK] Seq=110 Ack=1 Win=14656 Len=0
> > TSval=2959401 TSecr=0
> > 49385 26.520920 10.0.0.X 20.0.0.X HTTP 175
> > [TCP Retransmission] GET / HTTP/1.0
> > 49387 26.521745 10.0.0.X 20.0.0.X TCP 66 [TCP
> > Dup ACK 49385#1] 35248 > http [ACK] Seq=110 Ack=1 Win=14656 Len=0
> > TSval=2959809 TSecr=0
> > 50594 27.336952 10.0.0.X 20.0.0.X HTTP 175
> > [TCP Retransmission] GET / HTTP/1.0
> > 50596 27.337765 10.0.0.X 20.0.0.X TCP 66 [TCP
> > Dup ACK 50594#1] 35248 > http [ACK] Seq=110 Ack=1 Win=14656 Len=0
> > TSval=2960625 TSecr=0
> > 52574 28.921899 10.0.0.X 20.0.0.X TCP 66 [TCP
> > Dup ACK 50594#2] 35248 > http [ACK] Seq=110 Ack=1 Win=14656 Len=0
> > TSval=2962210 TSecr=0
> > 52576 28.922743 10.0.0.X 20.0.0.X TCP 66 [TCP
> > Dup ACK 50594#3] 35248 > http [ACK] Seq=110 Ack=1 Win=14656 Len=0
> > TSval=2962210 TSecr=0
> > 52639 28.968964 10.0.0.X 20.0.0.X HTTP 175
> > [TCP Retransmission] GET / HTTP/1.0
> > 52641 28.969752 10.0.0.X 20.0.0.X TCP 66 [TCP
> > Dup ACK 52639#1] 35248 > http [ACK] Seq=110 Ack=1 Win=14656 Len=0
> > TSval=2962257 TSecr=0
> > 55547 32.233026 10.0.0.X 20.0.0.X HTTP 175
> > [TCP Retransmission] GET / HTTP/1.0
> > 55549 32.233851 10.0.0.X 20.0.0.X TCP 66 [TCP
> > Dup ACK 55547#1] 35248 > http [ACK] Seq=110 Ack=1 Win=14656 Len=0
> > TSval=2965521 TSecr=0
> > 59833 34.937494 10.0.0.X 20.0.0.X TCP 66 [TCP
> > Dup ACK 55547#2] 35248 > http [ACK] Seq=110 Ack=1 Win=14656 Len=0
> > TSval=2968225 TSecr=0
> > 59835 34.938503 10.0.0.X 20.0.0.X TCP 66 [TCP
> > Dup ACK 55547#3] 35248 > http [ACK] Seq=110 Ack=1 Win=14656 Len=0
> > TSval=2968226 TSecr=0
> > 66466 38.761131 10.0.0.X 20.0.0.X HTTP 175
> > [TCP Retransmission] GET / HTTP/1.0
> > 66468 38.761969 10.0.0.X 20.0.0.X TCP 66 [TCP
> > Dup ACK 66466#1] 35248 > http [ACK] Seq=110 Ack=1 Win=14656 Len=0
> > TSval=2972049 TSecr=0
> > 82253 46.859463 10.0.0.X 20.0.0.X TCP 66 [TCP
> > Dup ACK 66466#2] 35248 > http [ACK] Seq=110 Ack=1 Win=14656 Len=0
> > TSval=2980147 TSecr=0
> > 97032 51.906615 10.0.0.X 20.0.0.X TCP 74
> > 35249
> >
> > > http [SYN] Seq=0 Win=14600 Len=0 MSS=1460 SACK_PERM=1 TSval=2985194
> >
> > TSecr=0 WS=64
> > 97034 51.908763 10.0.0.X 20.0.0.X TCP 66
> > 35249
> >
> > > http [ACK] Seq=1 Ack=1 Win=14656 Len=0 TSval=2985196 TSecr=0
> >
> > 97035 51.908849 10.0.0.X 20.0.0.X HTTP 175 GET
> > / HTTP/1.0
> > 97038 51.910955 10.0.0.X 20.0.0.X TCP 66 [TCP
> > Dup ACK 97035#1] 35249 > http [ACK] Seq=110 Ack=1 Win=14656 Len=0
> > TSval=2985198 TSecr=0
> > 97039 51.910957 10.0.0.X 20.0.0.X TCP 66 [TCP
> > Dup ACK 97035#2] 35249 > http [ACK] Seq=110 Ack=1 Win=14656 Len=0
> > TSval=2985198 TSecr=0
> > 97042 51.912054 10.0.0.X 20.0.0.X TCP 66 [TCP
> > Dup ACK 97035#3] 35249 > http [ACK] Seq=110 Ack=1 Win=14656 Len=0
> > TSval=2985199 TSecr=0
> > 97043 51.912056 10.0.0.X 20.0.0.X TCP 66 [TCP
> > Dup ACK 97035#4] 35249 > http [ACK] Seq=110 Ack=1 Win=14656 Len=0
> > TSval=2985199 TSecr=0
> > 97301 52.112305 10.0.0.X 20.0.0.X HTTP 175
> > [TCP Retransmission] GET / HTTP/1.0
> > 97304 52.113105 10.0.0.X 20.0.0.X TCP 66 [TCP
> > Dup ACK 97301#1] 35249 > http [ACK] Seq=110 Ack=1 Win=14656 Len=0
> > TSval=2985400 TSecr=0
> > 97833 52.520290 10.0.0.X 20.0.0.X HTTP 175
> > [TCP Retransmission] GET / HTTP/1.0
> > 97835 52.521102 10.0.0.X 20.0.0.X TCP 66 [TCP
> > Dup ACK 97833#1] 35249 > http [ACK] Seq=110 Ack=1 Win=14656 Len=0
> > TSval=2985808 TSecr=0
> > 99547 53.336306 10.0.0.X 20.0.0.X HTTP 175
> > [TCP Retransmission] GET / HTTP/1.0
> > 99549 53.337113 10.0.0.X 20.0.0.X TCP 66 [TCP
> > Dup ACK 99547#1] 35249 > http [ACK] Seq=110 Ack=1 Win=14656 Len=0
> > TSval=2986624 TSecr=0
> > 103240 54.952961 10.0.0.X 20.0.0.X TCP 66 [TCP
> > Dup ACK 99547#2] 35249 > http [ACK] Seq=110 Ack=1 Win=14656 Len=0
> > TSval=2988240 TSecr=0
> > 103242 54.953807 10.0.0.X 20.0.0.X TCP 66 [TCP
> > Dup ACK 99547#3] 35249 > http [ACK] Seq=110 Ack=1 Win=14656 Len=0
> > TSval=2988241 TSecr=0
> > 103256 54.968334 10.0.0.X 20.0.0.X HTTP 175
> > [TCP Retransmission] GET / HTTP/1.0
> > 103260 54.969125 10.0.0.X 20.0.0.X TCP 66 [TCP
> > Dup ACK 103256#1] 35249 > http [ACK] Seq=110 Ack=1 Win=14656 Len=0
> > TSval=2988256 TSecr=0
> > 109179 58.232391 10.0.0.X 20.0.0.X HTTP 175
> > [TCP Retransmission] GET / HTTP/1.0
> > 109181 58.233424 10.0.0.X 20.0.0.X TCP 66 [TCP
> > Dup ACK 109179#1] 35249 > http [ACK] Seq=110 Ack=1 Win=14656 Len=0
> > TSval=2991521 TSecr=0
> > 114534 60.968474 10.0.0.X 20.0.0.X TCP 66 [TCP
> > Dup ACK 109179#2] 35249 > http [ACK] Seq=110 Ack=1 Win=14656 Len=0
> > TSval=2994256 TSecr=0
> > 114536 60.969325 10.0.0.X 20.0.0.X TCP 66 [TCP
> > Dup ACK 109179#3] 35249 > http [ACK] Seq=110 Ack=1 Win=14656 Len=0
> > TSval=2994256 TSecr=0
> > 122863 64.760491 10.0.0.X 20.0.0.X HTTP 175
> > [TCP Retransmission] GET / HTTP/1.0
> > 122865 64.761316 10.0.0.X 20.0.0.X TCP 66 [TCP
> > Dup ACK 122863#1] 35249 > http [ACK] Seq=110 Ack=1 Win=14656 Len=0
> > TSval=2998048 TSecr=0
> > 140956 72.999821 10.0.0.X 20.0.0.X TCP 66 [TCP
> > Dup ACK 122863#2] 35249 > http [ACK] Seq=110 Ack=1 Win=14656 Len=0
> > TSval=3006287 TSecr=0
> > 149188 77.906440 10.0.0.X 20.0.0.X TCP 74
> > 35250
> >
> > > http [SYN] Seq=0 Win=14600 Len=0 MSS=1460 SACK_PERM=1 TSval=3011193
> >
> > TSecr=0 WS=64
> > 149190 77.908726 10.0.0.X 20.0.0.X TCP 66
> > 35250
> >
> > > http [ACK] Seq=1 Ack=1 Win=14656 Len=0 TSval=3011196 TSecr=0
> >
> > 149191 77.908820 10.0.0.X 20.0.0.X HTTP 175 GET
> > / HTTP/1.0
> > 149194 77.910921 10.0.0.X 20.0.0.X TCP 66
> > 35250
> >
> > > http [ACK] Seq=110 Ack=1449 Win=17536 Len=0 TSval=3011198
> > > TSecr=1465368908
> >
> > 149195 77.910923 10.0.0.X 20.0.0.X TCP 66
> > 35250
> >
> > > http [ACK] Seq=110 Ack=1794 Win=20416 Len=0 TSval=3011198
> > > TSecr=1465368908
> >
> > 149196 77.912997 10.0.0.X 20.0.0.X TCP 66
> > 35250
> >
> > > http [FIN, ACK] Seq=110 Ack=1794 Win=20416 Len=0 TSval=3011200
> >
> > TSecr=1465368908
> > 149199 77.914014 10.0.0.X 20.0.0.X TCP 66
> > 35250
> >
> > > http [ACK] Seq=111 Ack=1795 Win=20416 Len=0 TSval=3011201
> > > TSecr=1465368908
> >
> > Cheers
> > Keith
--
"Action without study is fatal. Study without action is futile."
-- Mary Ritter Beard
