Hi, first, thanks for the quick answer. On our proxy where we currently have this number of entries in the 100k stick table:
echo "show table l1f_loadbalancer" | sudo nc -U /var/run/haproxy.stat # table: l1f_loadbalancer, type: string, size:102400, used:12166 So it currently has 12166 entries If as suggested I dump the sessions like this: echo "show sess all" | sudo nc -U /var/run/haproxy.stat |grep -c "^0x7" 502 So, I only see sessions in the low 100s and these are only from the last couple of seconds, so that's only our current traffic. To me this looks like there are currently no existing sessions which belong to the entries in the stick table and all these entries are kind of orphaned but they are counted internally as still somehow in use so they don't get cleaned up and also can't be dumped manually. Since we fill our table with a string derived from a L7 http header and never talk to the real client directly, there are only IPs from our internal network in the session dump, but I can send you a dump anyway, if that helps. Gernot ----- Original Message ----- > From: "Willy Tarreau" <[email protected]> > To: "Gernot Pörner" <[email protected]> > Cc: "haproxy" <[email protected]> > Sent: Thursday, November 17, 2016 7:17:02 PM > Subject: Re: stick table entries in use > Hi, > > On Thu, Nov 17, 2016 at 04:37:45PM +0000, Gernot Pörner wrote: >> Hi again, >> >> we are running haproxy 1.6.9 for a week now and the problem persists, we had >> some >> 30000 entries in there, all of them should now be timed out but about half of >> them are still in there flagged as "in use" and with a exp of 0: >> >> 0x7faef417d3ec: key=105.106.165.238 use=120 exp=0 gpc0_rate(10000)=0 >> 0x7faef429db1c: key=105.106.172.152 use=171 exp=0 gpc0_rate(10000)=0 >> 0x7faef42f0c2c: key=105.106.177.245 use=235 exp=0 gpc0_rate(10000)=0 >> 0x7faef3fd001c: key=105.106.184.115 use=99 exp=0 gpc0_rate(10000)=0 >> 0x7faef3b2c1fc: key=105.106.21.28 use=40 exp=0 gpc0_rate(10000)=0 >> 0x7faef2c1d89c: key=105.106.3.207 use=5 exp=0 gpc0_rate(10000)=0 >> 0x7faef2ec2d4c: key=105.106.6.240 use=66 exp=0 gpc0_rate(10000)=0 >> 0x7faef406a9ac: key=105.107.105.77 use=15 exp=0 gpc0_rate(10000)=0 >> 0x7faef36b2cdc: key=105.107.117.186 use=7 exp=0 gpc0_rate(10000)=0 >> 0x7faef412a71c: key=105.107.132.95 use=23 exp=0 gpc0_rate(10000)=0 >> 0x7faef3f97d4c: key=105.107.160.117 use=69 exp=0 gpc0_rate(10000)=0 >> 0x7faef41afaec: key=105.107.32.8 use=142 exp=0 gpc0_rate(10000)=0 >> 0x7faef408676c: key=105.107.41.220 use=21 exp=0 gpc0_rate(10000)=0 >> 0x7faef4051c9c: key=105.107.45.99 use=59 exp=0 gpc0_rate(10000)=0 >> 0x7faef216001c: key=105.107.85.185 use=25 exp=0 gpc0_rate(10000)=0 >> 0x7faef2edc4cc: key=105.107.87.236 use=25 exp=0 gpc0_rate(10000)=0 >> >> >> Some of the newer entries which come in do time out as expected. >> A restart of haproxy clears the table but nothing else will do it >> >> Any ideas? > > I forgot about this one already I'm sorry. I've identified a remaining > issue in 1.7 and 1.6.9 by which some connections would not be completely > closed sometimes. Some of them may appear in CLOSE_WAIT on your system, > but given that they disappear after a short time you may not see them if > the problem happens in bursts. Could you please issue a "show sess all" > on your stats socket and dump the output so that we can see if that could > be the cause ? > > Please note that it *will* reveal potentially sensitive information such > as clients' IP addresses and your servers' IP addresses. So if in doubt, > don't copy the list, or if you want you can filter only the entries that > match the IPs which never expire, that's as you want. > > thanks, > Willy
