Filtered results from show fd for that particular virtual server: 10 : st=0x22(R:pRa W:pRa) ev=0x01(heopI) [lc] cnext=-3 cprev=-2 tmask=0x1 umask=0x0 owner=0x53a5690 iocb=0x59d689(conn_fd_handler) back=0 cflg=0x80243300 fe=virtual-server_front mux=H2 ctx=0x6502860 h2c.st0=2 .err=0 .maxid=17 .lastid=-1 .flg=0x10000 .nbst=0 .nbcs=1 .fctl_cnt=0 .send_cnt=0 .tree_cnt=1 .orph_cnt=0 .sub=0 .dsi=13 .dbuf=0@(nil)+0/0 .msi=-1 .mbuf=0@(nil)+0/0 last_h2s=0x5907040 .id=13 .flg=0x4005 .rxbuf=0@(nil)+0/0 .cs=0x905b1b0 .cs.flg=0x00106a00 .cs.data=0x5d1d228 98 : st=0x22(R:pRa W:pRa) ev=0x01(heopI) [lc] cnext=-1809 cprev=-2 tmask=0x1 umask=0x0 owner=0xa3bb7f0 iocb=0x59d689(conn_fd_handler) back=0 cflg=0x80201300 fe=virtual-server_front mux=H2 ctx=0xa71f310 h2c.st0=3 .err=0 .maxid=0 .lastid=-1 .flg=0x0008 .nbst=0 .nbcs=0 .fctl_cnt=0 .send_cnt=0 .tree_cnt=0 .orph_cnt=0 .sub=0 .dsi=3 .dbuf=16384@0x5873f10+61/16384 .msi=-1 .mbuf=0@(nil)+0/0 184 : st=0x05(R:PrA W:pra) ev=0x01(heopI) [lC] cnext=-3 cprev=-2 tmask=0xffffffffffffffff umask=0x0 owner=0x23eb040 iocb=0x57e662(listener_accept) l.st=RDY fe=virtual-server_front 660 : st=0x22(R:pRa W:pRa) ev=0x11(HeopI) [lc] cnext=-3 cprev=-2 tmask=0x1 umask=0x0 owner=0x533d6e0 iocb=0x59d689(conn_fd_handler) back=0 cflg=0x80243300 fe=virtual-server_front mux=H2 ctx=0x8031b90 h2c.st0=2 .err=0 .maxid=49 .lastid=-1 .flg=0x10000 .nbst=0 .nbcs=1 .fctl_cnt=0 .send_cnt=0 .tree_cnt=1 .orph_cnt=0 .sub=0 .dsi=49 .dbuf=0@(nil)+0/0 .msi=-1 .mbuf=0@(nil)+0/0 last_h2s=0x70f1e80 .id=31 .flg=0x4005 .rxbuf=0@(nil)+0/0 .cs=0x6f373d0 .cs.flg=0x00106a00 .cs.data=0x56bb788 699 : st=0x22(R:pRa W:pRa) ev=0x11(HeopI) [lc] cnext=-87 cprev=-2 tmask=0x1 umask=0x0 owner=0x6694b60 iocb=0x59d689(conn_fd_handler) back=0 cflg=0x80243300 fe=virtual-server_front mux=H2 ctx=0x56e7b00 h2c.st0=2 .err=0 .maxid=111 .lastid=-1 .flg=0x10000 .nbst=0 .nbcs=1 .fctl_cnt=0 .send_cnt=0 .tree_cnt=1 .orph_cnt=0 .sub=0 .dsi=111 .dbuf=0@(nil)+0/0 .msi=-1 .mbuf=0@(nil)+0/0 last_h2s=0x5931bf0 .id=47 .flg=0x4105 .rxbuf=0@(nil)+0/0 .cs=0x5943120 .cs.flg=0x00106a00 .cs.data=0x77af4c8 970 : st=0x22(R:pRa W:pRa) ev=0x01(heopI) [lc] cnext=-3 cprev=-2 tmask=0x1 umask=0x0 owner=0x67684b0 iocb=0x59d689(conn_fd_handler) back=0 cflg=0x80243300 fe=virtual-server_front mux=H2 ctx=0x5c90c30 h2c.st0=2 .err=0 .maxid=125 .lastid=-1 .flg=0x10000 .nbst=0 .nbcs=1 .fctl_cnt=0 .send_cnt=0 .tree_cnt=1 .orph_cnt=0 .sub=0 .dsi=125 .dbuf=0@(nil)+0/0 .msi=-1 .mbuf=0@(nil)+0/0 last_h2s=0x7ac8650 .id=43 .flg=0x4005 .rxbuf=0@(nil)+0/0 .cs=0x7901a20 .cs.flg=0x00106a00 .cs.data=0x882c388 1282 : st=0x22(R:pRa W:pRa) ev=0x11(HeopI) [lc] cnext=-3 cprev=-2 tmask=0x1 umask=0x0 owner=0x6f23720 iocb=0x59d689(conn_fd_handler) back=0 cflg=0x80243300 fe=virtual-server_front mux=H2 ctx=0x6090cf0 h2c.st0=2 .err=0 .maxid=129 .lastid=-1 .flg=0x10000 .nbst=0 .nbcs=1 .fctl_cnt=0 .send_cnt=0 .tree_cnt=1 .orph_cnt=0 .sub=0 .dsi=129 .dbuf=0@(nil)+0/0 .msi=-1 .mbuf=0@(nil)+0/0 last_h2s=0x5cc0890 .id=17 .flg=0x4005 .rxbuf=0@(nil)+0/0 .cs=0x64d33f0 .cs.flg=0x00106a00 .cs.data=0x639a3e8 3041 : st=0x22(R:pRa W:pRa) ev=0x11(HeopI) [lc] cnext=-955 cprev=-2 tmask=0x1 umask=0x0 owner=0x6de8980 iocb=0x59d689(conn_fd_handler) back=0 cflg=0x80243300 fe=virtual-server_front mux=H2 ctx=0x5beca10 h2c.st0=2 .err=0 .maxid=89 .lastid=-1 .flg=0x10000 .nbst=0 .nbcs=1 .fctl_cnt=0 .send_cnt=0 .tree_cnt=1 .orph_cnt=0 .sub=0 .dsi=89 .dbuf=0@(nil)+0/0 .msi=-1 .mbuf=0@(nil)+0/0 last_h2s=0x82f5900 .id=15 .flg=0x4005 .rxbuf=0@(nil)+0/0 .cs=0x7e027a0 .cs.flg=0x00106a00 .cs.data=0x6e5d398
wt., 30 kwi 2019 o 08:31 Maciej Zdeb <mac...@zdeb.pl> napisał(a): > Forgot to attach information about HAProxy (for tracing the issue I > compiled it with debug symbols and without optimizations): > > haproxy -vv > HA-Proxy version 1.9.7 2019/04/25 - https://haproxy.org/ > Build options : > TARGET = linux2628 > CPU = generic > CC = gcc > CFLAGS = -O0 -g -fno-strict-aliasing -Wdeclaration-after-statement > -fwrapv -Wno-unused-label -Wno-sign-compare -Wno-unused-parameter > -Wno-old-style-declaration -Wno-ignored-qualifiers -Wno-clobbered > -Wno-missing-field-initializers -Wtype-limits -DIP_BIND_ADDRESS_NO_PORT=24 > OPTIONS = USE_GETADDRINFO=1 USE_ZLIB=1 USE_REGPARM=1 USE_DL=1 > USE_OPENSSL=1 USE_LUA=1 USE_PCRE=1 USE_PCRE_JIT=1 > > Default settings : > maxconn = 2000, bufsize = 16384, maxrewrite = 1024, maxpollevents = 200 > > Built with OpenSSL version : OpenSSL 1.1.1b 26 Feb 2019 > Running on OpenSSL version : OpenSSL 1.1.1b 26 Feb 2019 > OpenSSL library supports TLS extensions : yes > OpenSSL library supports SNI : yes > OpenSSL library supports : TLSv1.0 TLSv1.1 TLSv1.2 TLSv1.3 > Built with Lua version : Lua 5.3.5 > Built with transparent proxy support using: IP_TRANSPARENT > IPV6_TRANSPARENT IP_FREEBIND > Built with zlib version : 1.2.8 > Running on zlib version : 1.2.8 > Compression algorithms supported : identity("identity"), > deflate("deflate"), raw-deflate("deflate"), gzip("gzip") > Built with PCRE version : 8.43 2019-02-23 > Running on PCRE version : 8.43 2019-02-23 > PCRE library supports JIT : yes > Encrypted password support via crypt(3): yes > Built with multi-threading support. > > Available polling systems : > epoll : pref=300, test result OK > poll : pref=200, test result OK > select : pref=150, test result OK > Total: 3 (3 usable), will use epoll. > > Available multiplexer protocols : > (protocols marked as <default> cannot be specified using 'proto' keyword) > h2 : mode=HTTP side=FE > h2 : mode=HTX side=FE|BE > <default> : mode=HTX side=FE|BE > <default> : mode=TCP|HTTP side=FE|BE > > Available filters : > [SPOE] spoe > [COMP] compression > [CACHE] cache > [TRACE] trace > > wt., 30 kwi 2019 o 08:16 Maciej Zdeb <mac...@zdeb.pl> napisał(a): > >> Hi, >> >> I'm returning with problem similar as in thread "[1.9.6] One of haproxy >> processes using 100% CPU". >> >> Thanks to Olivier hard work, some issues where fixed but still not all of >> them. :( Currently it is much harder to trigger and it occurs on https >> virtual server and not tcp one. I'm observing this problem for HAProxy >> 1.9.7, process starts using 100% cpu and admin socket is still responsive. >> >> Attached gdb session. Please let me know if you need some more info from >> gdb, unfortunately I'm not gdb expert and I'm not sure what to look for. It >> is production server, however I can keep it for a couple hours isolated >> from production traffic. >> >> Pasting some anonymized info from show sess, please note that it was >> dumped on 30/Apr/2019:06:53:54 (before dump, traffic was switched to >> another server) but some connections are from 29/Apr/2019 and persist in >> that strange state. >> >> CC: Oliver :) >> >> socat /var/run/haproxy/haproxy1.sock - <<< "show sess all" >> 0x56bb4f0: [29/Apr/2019:12:31:37.186578] id=14500574 proto=tcpv4 >> source=C.C.C.C:56567 >> flags=0x44e, conn_retries=2, srv_conn=0x24db060, pend_pos=(nil) >> waiting=0 >> frontend=virtual-server_front (id=45 mode=http), listener=? (id=1) >> addr=V.V.V.V:443 >> backend=V.V.V.V:443_back (id=46 mode=http) addr=S.S.S.S:57654 >> server=slot_9_3 (id=78) addr=B.B.B.B:31160 >> task=0x6ec4070 (state=0x00 nice=0 calls=2 exp=<NEVER> tmask=0x1 >> age=18h22m) >> txn=0x57eadf0 flags=0x88003000 meth=1 status=-1 req.st=MSG_DONE rsp.st >> =MSG_RPBEFORE >> req.f=0x0c blen=0 chnk=0 next=0 >> rsp.f=0x00 blen=0 chnk=0 next=0 >> si[0]=0x56bb788 (state=EST flags=0x48008 endp0=CS:0x6f373d0 exp=<NEVER> >> et=0x000 sub=0) >> si[1]=0x56bb7c8 (state=CON flags=0x111 endp1=CS:0x53a9250 exp=<PAST> >> et=0x008 sub=2) >> co0=0x533d6e0 ctrl=tcpv4 xprt=SSL mux=H2 data=STRM >> target=LISTENER:0x23eb040 >> flags=0x80243300 fd=660 fd.state=22 fd.cache=0 updt=0 fd.tmask=0x1 >> cs=0x6f373d0 csf=0x00106a00 ctx=0x70f1e80 >> co1=0x91263b0 ctrl=tcpv4 xprt=RAW mux=PASS data=STRM >> target=SERVER:0x24db060 >> flags=0x00403370 fd=51 fd.state=22 fd.cache=0 updt=0 fd.tmask=0x1 >> cs=0x53a9250 csf=0x00000200 ctx=(nil) >> req=0x56bb500 (f=0x4cc00000 an=0x8000 pipe=0 tofwd=0 total=1610) >> an_exp=<NEVER> rex=<NEVER> wex=<NEVER> >> buf=0x56bb508 data=0x58ba660 o=1831 p=1831 req.next=0 i=0 size=16384 >> res=0x56bb560 (f=0x80000000 an=0x0 pipe=0 tofwd=0 total=0) >> an_exp=<NEVER> rex=<NEVER> wex=<NEVER> >> buf=0x56bb568 data=(nil) o=0 p=0 rsp.next=0 i=0 size=0 >> 0x639a150: [29/Apr/2019:12:31:53.231220] id=14501038 proto=tcpv4 >> source=C2.C2.C2.C2:30107 >> flags=0x44e, conn_retries=2, srv_conn=0x24cdad0, pend_pos=(nil) >> waiting=0 >> frontend=virtual-server_front (id=45 mode=http), listener=? (id=1) >> addr=V.V.V.V:443 >> backend=V.V.V.V:443_back (id=46 mode=http) addr=S2.S2.S2.S2:8760 >> server=slot_9_2 (id=61) addr=B.B.B.B:31160 >> task=0x6625ab0 (state=0x00 nice=0 calls=2 exp=<NEVER> tmask=0x1 >> age=18h22m) >> txn=0x5af9a10 flags=0x88003000 meth=1 status=-1 req.st=MSG_DONE rsp.st >> =MSG_RPBEFORE >> req.f=0x0c blen=0 chnk=0 next=0 >> rsp.f=0x00 blen=0 chnk=0 next=0 >> si[0]=0x639a3e8 (state=EST flags=0x48008 endp0=CS:0x64d33f0 exp=<NEVER> >> et=0x000 sub=0) >> si[1]=0x639a428 (state=CON flags=0x111 endp1=CS:0x5d681c0 exp=<PAST> >> et=0x008 sub=2) >> co0=0x6f23720 ctrl=tcpv4 xprt=SSL mux=H2 data=STRM >> target=LISTENER:0x23eb040 >> flags=0x80243300 fd=1282 fd.state=22 fd.cache=0 updt=0 fd.tmask=0x1 >> cs=0x64d33f0 csf=0x00106a00 ctx=0x5cc0890 >> co1=0x6039c60 ctrl=tcpv4 xprt=RAW mux=PASS data=STRM >> target=SERVER:0x24cdad0 >> flags=0x00403370 fd=577 fd.state=22 fd.cache=0 updt=0 fd.tmask=0x1 >> cs=0x5d681c0 csf=0x00000200 ctx=0x5d681e0 >> req=0x639a160 (f=0x4cc00000 an=0x8000 pipe=0 tofwd=0 total=1137) >> an_exp=<NEVER> rex=<NEVER> wex=<NEVER> >> buf=0x639a168 data=0x5408ef0 o=1349 p=1349 req.next=0 i=0 size=16384 >> res=0x639a1c0 (f=0x80000000 an=0x0 pipe=0 tofwd=0 total=0) >> an_exp=<NEVER> rex=<NEVER> wex=<NEVER> >> buf=0x639a1c8 data=(nil) o=0 p=0 rsp.next=0 i=0 size=0 >> 0x882c0f0: [29/Apr/2019:12:31:46.503263] id=14503967 proto=tcpv4 source= >> 5.173.79.240:7434 >> flags=0x44e, conn_retries=2, srv_conn=0x24cdad0, pend_pos=(nil) >> waiting=0 >> frontend=virtual-server_front (id=45 mode=http), listener=? (id=1) >> addr=V.V.V.V:443 >> backend=V.V.V.V:443_back (id=46 mode=http) addr=S2.S2.S2.S2:24688 >> server=slot_9_2 (id=61) addr=B.B.B.B:31160 >> task=0x7c13f50 (state=0x00 nice=0 calls=2 exp=<NEVER> tmask=0x1 >> age=18h22m) >> txn=0x5978210 flags=0x88003000 meth=1 status=-1 req.st=MSG_DONE rsp.st >> =MSG_RPBEFORE >> req.f=0x0c blen=0 chnk=0 next=0 >> rsp.f=0x00 blen=0 chnk=0 next=0 >> si[0]=0x882c388 (state=EST flags=0x48008 endp0=CS:0x7901a20 exp=<NEVER> >> et=0x000 sub=0) >> si[1]=0x882c3c8 (state=CON flags=0x111 endp1=CS:0x6ca3a50 exp=<PAST> >> et=0x008 sub=2) >> co0=0x67684b0 ctrl=tcpv4 xprt=SSL mux=H2 data=STRM >> target=LISTENER:0x23eb040 >> flags=0x80243300 fd=970 fd.state=22 fd.cache=0 updt=0 fd.tmask=0x1 >> cs=0x7901a20 csf=0x00106a00 ctx=0x7ac8650 >> co1=0x74cba90 ctrl=tcpv4 xprt=RAW mux=PASS data=STRM >> target=SERVER:0x24cdad0 >> flags=0x00403370 fd=2185 fd.state=22 fd.cache=0 updt=0 fd.tmask=0x1 >> cs=0x6ca3a50 csf=0x00000200 ctx=(nil) >> req=0x882c100 (f=0x4cc00000 an=0x8000 pipe=0 tofwd=0 total=2096) >> an_exp=<NEVER> rex=<NEVER> wex=<NEVER> >> buf=0x882c108 data=0x5f42c30 o=2307 p=2307 req.next=0 i=0 size=16384 >> res=0x882c160 (f=0x80000000 an=0x0 pipe=0 tofwd=0 total=0) >> an_exp=<NEVER> rex=<NEVER> wex=<NEVER> >> buf=0x882c168 data=(nil) o=0 p=0 rsp.next=0 i=0 size=0 >> 0x77af230: [29/Apr/2019:16:05:25.433552] id=26109777 proto=tcpv4 >> source=C3.C3.C3.C3:19974 >> flags=0x4e, conn_retries=2, srv_conn=0x24cb520, pend_pos=(nil) waiting=0 >> frontend=virtual-server_front (id=45 mode=http), listener=? (id=1) >> addr=V.V.V.V:443 >> backend=V.V.V.V:443_back (id=46 mode=http) addr=S2.S2.S2.S2:55296 >> server=slot_6_2 (id=58) addr=B2.B2.B2.B2:31165 >> task=0x54bce70 (state=0x00 nice=0 calls=2 exp=<NEVER> tmask=0x1 >> age=14h48m) >> txn=0x5f57d30 flags=0x88000000 meth=3 status=-1 req.st=MSG_DONE rsp.st >> =MSG_RPBEFORE >> req.f=0x0d blen=66 chnk=0 next=0 >> rsp.f=0x00 blen=0 chnk=0 next=0 >> si[0]=0x77af4c8 (state=EST flags=0x48008 endp0=CS:0x5943120 exp=<NEVER> >> et=0x000 sub=0) >> si[1]=0x77af508 (state=CON flags=0x111 endp1=CS:0x7a69aa0 exp=<PAST> >> et=0x008 sub=2) >> co0=0x6694b60 ctrl=tcpv4 xprt=SSL mux=H2 data=STRM >> target=LISTENER:0x23eb040 >> flags=0x80243300 fd=699 fd.state=22 fd.cache=0 updt=0 fd.tmask=0x1 >> cs=0x5943120 csf=0x00106a00 ctx=0x5931bf0 >> co1=0x6af1500 ctrl=tcpv4 xprt=RAW mux=PASS data=STRM >> target=SERVER:0x24cb520 >> flags=0x00403370 fd=1574 fd.state=22 fd.cache=0 updt=0 fd.tmask=0x1 >> cs=0x7a69aa0 csf=0x00000200 ctx=(nil) >> req=0x77af240 (f=0x4cc00000 an=0x8000 pipe=0 tofwd=0 total=1183) >> an_exp=<NEVER> rex=<NEVER> wex=<NEVER> >> buf=0x77af248 data=0x540cf00 o=1398 p=1398 req.next=0 i=0 size=16384 >> res=0x77af2a0 (f=0x80000000 an=0x0 pipe=0 tofwd=0 total=0) >> an_exp=<NEVER> rex=<NEVER> wex=<NEVER> >> buf=0x77af2a8 data=(nil) o=0 p=0 rsp.next=0 i=0 size=0 >> 0x5d1cf90: [29/Apr/2019:18:54:15.974874] id=35265590 proto=tcpv4 >> source=C4.C4.C4.C4:55010 >> flags=0x44e, conn_retries=2, srv_conn=0x24bb9e0, pend_pos=(nil) >> waiting=0 >> frontend=virtual-server_front (id=45 mode=http), listener=? (id=1) >> addr=V.V.V.V:443 >> backend=V.V.V.V:443_back (id=46 mode=http) addr=10.32.189.89:17092 >> server=slot_3_1 (id=38) addr=B3.B3.B3.B3:31565 >> task=0x9848fb0 (state=0x00 nice=0 calls=2 exp=<NEVER> tmask=0x1 >> age=11h59m) >> txn=0x5540d70 flags=0x88003000 meth=1 status=-1 req.st=MSG_DONE rsp.st >> =MSG_RPBEFORE >> req.f=0x0c blen=0 chnk=0 next=0 >> rsp.f=0x00 blen=0 chnk=0 next=0 >> si[0]=0x5d1d228 (state=EST flags=0x48008 endp0=CS:0x905b1b0 exp=<NEVER> >> et=0x000 sub=0) >> si[1]=0x5d1d268 (state=CON flags=0x111 endp1=CS:0x5474790 exp=<PAST> >> et=0x008 sub=2) >> co0=0x53a5690 ctrl=tcpv4 xprt=SSL mux=H2 data=STRM >> target=LISTENER:0x23eb040 >> flags=0x80243300 fd=10 fd.state=22 fd.cache=0 updt=0 fd.tmask=0x1 >> cs=0x905b1b0 csf=0x00106a00 ctx=0x5907040 >> co1=0x67a6ed0 ctrl=tcpv4 xprt=RAW mux=PASS data=STRM >> target=SERVER:0x24bb9e0 >> flags=0x00403370 fd=1356 fd.state=22 fd.cache=0 updt=0 fd.tmask=0x1 >> cs=0x5474790 csf=0x00000200 ctx=(nil) >> req=0x5d1cfa0 (f=0x4cc00000 an=0x8000 pipe=0 tofwd=0 total=1428) >> an_exp=<NEVER> rex=<NEVER> wex=<NEVER> >> buf=0x5d1cfa8 data=0x5bd1c00 o=1646 p=1646 req.next=0 i=0 size=16384 >> res=0x5d1d000 (f=0x80000000 an=0x0 pipe=0 tofwd=0 total=0) >> an_exp=<NEVER> rex=<NEVER> wex=<NEVER> >> buf=0x5d1d008 data=(nil) o=0 p=0 rsp.next=0 i=0 size=0 >> 0x6e5d100: [29/Apr/2019:21:47:45.668692] id=46668022 proto=tcpv4 >> source=C5.C5.C5.C5:38289 >> flags=0x44e, conn_retries=2, srv_conn=0x24d4be0, pend_pos=(nil) >> waiting=0 >> frontend=virtual-server_front (id=45 mode=http), listener=? (id=1) >> addr=V.V.V.V:443 >> backend=V.V.V.V:443_back (id=46 mode=http) addr=S.S.S.S:22918 >> server=slot_1_3 (id=70) addr=B4.B4.B4.B4:31992 >> task=0x59e1a10 (state=0x00 nice=0 calls=2 exp=<NEVER> tmask=0x1 >> age=9h6m) >> txn=0x92b23d0 flags=0x88003000 meth=1 status=-1 req.st=MSG_DONE rsp.st >> =MSG_RPBEFORE >> req.f=0x0c blen=0 chnk=0 next=0 >> rsp.f=0x00 blen=0 chnk=0 next=0 >> si[0]=0x6e5d398 (state=EST flags=0x48008 endp0=CS:0x7e027a0 exp=<NEVER> >> et=0x000 sub=0) >> si[1]=0x6e5d3d8 (state=CON flags=0x111 endp1=CS:0x6459520 exp=<PAST> >> et=0x008 sub=2) >> co0=0x6de8980 ctrl=tcpv4 xprt=SSL mux=H2 data=STRM >> target=LISTENER:0x23eb040 >> flags=0x80243300 fd=3041 fd.state=22 fd.cache=0 updt=0 fd.tmask=0x1 >> cs=0x7e027a0 csf=0x00106a00 ctx=0x82f5900 >> co1=0xb84e280 ctrl=tcpv4 xprt=RAW mux=PASS data=STRM >> target=SERVER:0x24d4be0 >> flags=0x00403370 fd=1787 fd.state=22 fd.cache=0 updt=0 fd.tmask=0x1 >> cs=0x6459520 csf=0x00000200 ctx=0x6459540 >> req=0x6e5d110 (f=0x4cc00000 an=0x8000 pipe=0 tofwd=0 total=1112) >> an_exp=<NEVER> rex=<NEVER> wex=<NEVER> >> buf=0x6e5d118 data=0x7235570 o=1324 p=1324 req.next=0 i=0 size=16384 >> res=0x6e5d170 (f=0x80000000 an=0x0 pipe=0 tofwd=0 total=0) >> an_exp=<NEVER> rex=<NEVER> wex=<NEVER> >> buf=0x6e5d178 data=(nil) o=0 p=0 rsp.next=0 i=0 size=0 >> 0x680b1b0: [30/Apr/2019:06:53:54.039969] id=62359636 proto=unix_stream >> source=unix:1 >> flags=0x8, conn_retries=0, srv_conn=(nil), pend_pos=(nil) waiting=0 >> frontend=GLOBAL (id=0 mode=tcp), listener=? (id=1) addr=unix:1 >> backend=<NONE> (id=-1 mode=-) >> server=<NONE> (id=-1) >> task=0x67a4f20 (state=0x00 nice=-64 calls=1 exp=<NEVER> tmask=0x1 >> age=0s) >> si[0]=0x680b448 (state=EST flags=0x80008 endp0=CS:0x660fdf0 exp=<NEVER> >> et=0x000 sub=0) >> si[1]=0x680b488 (state=EST flags=0x204018 endp1=APPCTX:0x53a1a30 >> exp=<NEVER> et=0x000 sub=0) >> co0=0x8a5b8f0 ctrl=unix_stream xprt=RAW mux=PASS data=STRM >> target=LISTENER:0x19988b0 >> flags=0x00243300 fd=17 fd.state=22 fd.cache=0 updt=0 fd.tmask=0x1 >> cs=0x660fdf0 csf=0x00001000 ctx=0x660fe10 >> app1=0x53a1a30 st0=7 st1=0 st2=3 applet=<CLI> tmask=0x1, nice=-64, >> calls=1, cpu=0, lat=0 >> req=0x680b1c0 (f=0xc4c220 an=0x0 pipe=0 tofwd=0 total=14) >> an_exp=<NEVER> rex=<NEVER> wex=<NEVER> >> buf=0x680b1c8 data=0x718bca0 o=0 p=0 req.next=0 i=0 size=16384 >> res=0x680b220 (f=0x80008002 an=0x0 pipe=0 tofwd=-1 total=9380) >> an_exp=<NEVER> rex=<NEVER> wex=<NEVER> >> buf=0x680b228 data=0x9d33d90 o=9380 p=9380 rsp.next=0 i=0 size=16384 >> >> socat /var/run/haproxy/haproxy1.sock - <<< "show info" >> Name: HAProxy >> Version: 1.9.7 >> Release_date: 2019/04/25 >> Nbthread: 1 >> Nbproc: 8 >> Process_num: 1 >> Pid: 44782 >> Uptime: 0d 22h54m13s >> Uptime_sec: 82453 >> Memmax_MB: 0 >> PoolAlloc_MB: 9 >> PoolUsed_MB: 9 >> PoolFailed: 0 >> Ulimit-n: 1000000 >> Maxsock: 4001570 >> Maxconn: 2000000 >> Hard_maxconn: 2000000 >> CurrConns: 7 >> CumConns: 16818215 >> CumReq: 62359636 >> MaxSslConns: 0 >> CurrSslConns: 7 >> CumSslConns: 15468203 >> Maxpipes: 0 >> PipesUsed: 0 >> PipesFree: 0 >> ConnRate: 0 >> ConnRateLimit: 0 >> MaxConnRate: 450 >> SessRate: 0 >> SessRateLimit: 0 >> MaxSessRate: 450 >> SslRate: 0 >> SslRateLimit: 0 >> MaxSslRate: 408 >> SslFrontendKeyRate: 0 >> SslFrontendMaxKeyRate: 367 >> SslFrontendSessionReuse_pct: 0 >> SslBackendKeyRate: 0 >> SslBackendMaxKeyRate: 6 >> SslCacheLookups: 5405716 >> SslCacheMisses: 2783916 >> CompressBpsIn: 0 >> CompressBpsOut: 0 >> CompressBpsRateLim: 0 >> ZlibMemUsage: 0 >> MaxZlibMemUsage: 0 >> Tasks: 3748 >> Run_queue: 3 >> Idle_pct: 1 >> node: X >> Stopping: 0 >> Jobs: 898 >> Unstoppable Jobs: 0 >> Listeners: 890 >> ActivePeers: 0 >> ConnectedPeers: 0 >> DroppedLogs: 134 >> BusyPolling: 0 >> >> socat /var/run/haproxy/haproxy1.sock - <<< "show activity" >> thread_id: 0 >> date_now: 1556600105.168009 >> loops: 730017734 >> wake_cache: 220599287 >> wake_tasks: 412569304 >> wake_signal: 0 >> poll_exp: 633168591 >> poll_drop: 19816095 >> poll_dead: 0 >> poll_skip: 0 >> fd_skip: 0 >> fd_lock: 0 >> fd_del: 0 >> conn_dead: 0 >> stream: 144545285 >> empty_rq: 68411039 >> long_rq: 454381935 >> cpust_ms_tot: 116 >> cpust_ms_1s: 0 >> cpust_ms_15s: 0 >> avg_loop_us: 50 >> >> strace >> clock_gettime(CLOCK_THREAD_CPUTIME_ID, {53716, 787194709}) = 0 >> epoll_wait(3, {}, 200, 0) = 0 >> clock_gettime(CLOCK_THREAD_CPUTIME_ID, {53716, 787206841}) = 0 >> close(6) = 0 >> clock_gettime(CLOCK_THREAD_CPUTIME_ID, {53716, 787281934}) = 0 >> epoll_wait(3, {}, 200, 0) = 0 >> clock_gettime(CLOCK_THREAD_CPUTIME_ID, {53716, 787295206}) = 0 >> clock_gettime(CLOCK_THREAD_CPUTIME_ID, {53716, 787357244}) = 0 >> epoll_wait(3, {}, 200, 0) = 0 >> clock_gettime(CLOCK_THREAD_CPUTIME_ID, {53716, 787369184}) = 0 >> clock_gettime(CLOCK_THREAD_CPUTIME_ID, {53716, 787430478}) = 0 >> epoll_wait(3, {}, 200, 0) = 0 >> clock_gettime(CLOCK_THREAD_CPUTIME_ID, {53716, 787442467}) = 0 >> clock_gettime(CLOCK_THREAD_CPUTIME_ID, {53716, 787524492}) = 0 >> epoll_wait(3, {}, 200, 0) = 0 >> clock_gettime(CLOCK_THREAD_CPUTIME_ID, {53716, 787537606}) = 0 >> clock_gettime(CLOCK_THREAD_CPUTIME_ID, {53716, 787603519}) = 0 >> epoll_wait(3, {}, 200, 0) = 0 >> clock_gettime(CLOCK_THREAD_CPUTIME_ID, {53716, 787615975}) = 0 >> clock_gettime(CLOCK_THREAD_CPUTIME_ID, {53716, 787680909}) = 0 >> epoll_wait(3, {}, 200, 0) = 0 >> clock_gettime(CLOCK_THREAD_CPUTIME_ID, {53716, 787693527}) = 0 >> clock_gettime(CLOCK_THREAD_CPUTIME_ID, {53716, 787757296}) = 0 >> epoll_wait(3, {}, 200, 0) = 0 >> clock_gettime(CLOCK_THREAD_CPUTIME_ID, {53716, 787770110}) = 0 >> clock_gettime(CLOCK_THREAD_CPUTIME_ID, {53716, 787833630}) = 0 >> epoll_wait(3, {}, 200, 0) = 0 >> clock_gettime(CLOCK_THREAD_CPUTIME_ID, {53716, 787845869}) = 0 >> clock_gettime(CLOCK_THREAD_CPUTIME_ID, {53716, 787909129}) = 0 >> epoll_wait(3, {}, 200, 0) = 0 >> clock_gettime(CLOCK_THREAD_CPUTIME_ID, {53716, 787921129}) = 0 >> clock_gettime(CLOCK_THREAD_CPUTIME_ID, {53716, 787983756}) = 0 >> epoll_wait(3, {}, 200, 0) = 0 >> clock_gettime(CLOCK_THREAD_CPUTIME_ID, {53716, 787995880}) = 0 >> clock_gettime(CLOCK_THREAD_CPUTIME_ID, {53716, 788058759}) = 0 >> epoll_wait(3, {}, 200, 0) = 0 >> clock_gettime(CLOCK_THREAD_CPUTIME_ID, {53716, 788070708}) = 0 >> clock_gettime(CLOCK_THREAD_CPUTIME_ID, {53716, 788133753}) = 0 >> epoll_wait(3, {}, 200, 0) = 0 >> clock_gettime(CLOCK_THREAD_CPUTIME_ID, {53716, 788145936}) = 0 >> clock_gettime(CLOCK_THREAD_CPUTIME_ID, {53716, 788209051}) = 0 >> epoll_wait(3, {}, 200, 0) = 0 >> clock_gettime(CLOCK_THREAD_CPUTIME_ID, {53716, 788221068}) = 0 >> clock_gettime(CLOCK_THREAD_CPUTIME_ID, {53716, 788284081}) = 0 >> epoll_wait(3, {}, 200, 0) = 0 >> clock_gettime(CLOCK_THREAD_CPUTIME_ID, {53716, 788296179}) = 0 >> clock_gettime(CLOCK_THREAD_CPUTIME_ID, {53716, 788359128}) = 0 >> epoll_wait(3, {}, 200, 0) = 0 >> clock_gettime(CLOCK_THREAD_CPUTIME_ID, {53716, 788371166}) = 0 >> clock_gettime(CLOCK_THREAD_CPUTIME_ID, {53716, 788434304}) = 0 >> epoll_wait(3, {}, 200, 0) = 0 >> clock_gettime(CLOCK_THREAD_CPUTIME_ID, {53716, 788446107}) = 0 >> clock_gettime(CLOCK_THREAD_CPUTIME_ID, {53716, 788509745}) = 0 >> epoll_wait(3, {}, 200, 0) = 0 >> clock_gettime(CLOCK_THREAD_CPUTIME_ID, {53716, 788521850}) = 0 >> clock_gettime(CLOCK_THREAD_CPUTIME_ID, {53716, 788585197}) = 0 >> epoll_wait(3, {}, 200, 0) = 0 >> clock_gettime(CLOCK_THREAD_CPUTIME_ID, {53716, 788597339}) = 0 >> clock_gettime(CLOCK_THREAD_CPUTIME_ID, {53716, 788661289}) = 0 >> epoll_wait(3, {}, 200, 0) = 0 >> clock_gettime(CLOCK_THREAD_CPUTIME_ID, {53716, 788673507}) = 0 >> clock_gettime(CLOCK_THREAD_CPUTIME_ID, {53716, 788736072}) = 0 >> epoll_wait(3, {}, 200, 0) = 0 >> clock_gettime(CLOCK_THREAD_CPUTIME_ID, {53716, 788748221}) = 0 >> clock_gettime(CLOCK_THREAD_CPUTIME_ID, {53716, 788811472}) = 0 >> epoll_wait(3, {}, 200, 0) = 0 >> clock_gettime(CLOCK_THREAD_CPUTIME_ID, {53716, 788823528}) = 0 >> clock_gettime(CLOCK_THREAD_CPUTIME_ID, {53716, 788886604}) = 0 >> epoll_wait(3, {}, 200, 0) = 0 >> clock_gettime(CLOCK_THREAD_CPUTIME_ID, {53716, 788898706}) = 0 >> clock_gettime(CLOCK_THREAD_CPUTIME_ID, {53716, 788960106}) = 0 >> epoll_wait(3, {}, 200, 0) = 0 >> clock_gettime(CLOCK_THREAD_CPUTIME_ID, {53716, 788972268}) = 0 >> clock_gettime(CLOCK_THREAD_CPUTIME_ID, {53716, 789045142}) = 0 >> >> >>
