I can't correlate the two changes, it might be some other external factor. On Thu, Dec 21, 2023 at 2:00 PM Amish <anon.am...@gmail.com> wrote:
> > On 21/12/23 17:55, Francesco Chemolli wrote: > > Hi Amish, > the message you posted really looks like a kernel bug, possibly due to > faulty code, or resulting from a hardware problem. > Nothing squid can do can cause that kind of stack traces in kernel-space. > > A quick search online results in > https://lkml.kernel.org/netdev/20230526163458.2880232-1-eduma...@google.com/T/ > , which looks very much like your message > > Yes even I believe so but I wonder, why only after updating to squid 6.6? > > Regards > > Amish. > > On Thu, Dec 21, 2023 at 1:10 PM Amish <anon.am...@gmail.com> wrote: > >> Hi Alex, >> >> Sorry for late reply. But this is a production system and hence >> debugging is tough. May take few days. >> >> However I noticed that everytime squid hangs (goes dead), the dmesg has >> these errors. >> >> [63567.802188] divide error: 0000 [#1] PREEMPT SMP PTI >> [63567.802215] CPU: 3 PID: 617 Comm: squid Not tainted 6.6.7-arch1-1 #1 >> 4505c4baa0b3d7c4037b0e8f5402626fa360717f >> [63567.802238] Hardware name: Gigabyte Technology Co., Ltd. >> H81M-S/H81M-S, BIOS F2 08/19/2015 >> [63567.802255] RIP: 0010:tcp_rcv_space_adjust+0xbe/0x160 >> [63567.802267] Code: f8 41 89 d0 29 d0 31 d2 49 0f af c2 49 f7 f0 45 8b >> 81 bc 04 00 00 44 0f b6 8b 10 06 00 00 31 d2 49 8d 04 42 48 98 48 c1 e0 >> 08 <49> f7 f1 49 63 d0 >> 48 98 48 39 d0 48 0f 47 c2 39 83 18 01 00 00 7c >> [63567.802287] RSP: 0018:ffffba9f00da3c18 EFLAGS: 00010206 >> [63567.802296] RAX: 0000000004fb2800 RBX: ffff9e124acc1c80 RCX: >> 0000000eccd9dace >> [63567.802304] RDX: 0000000000000000 RSI: 0000000037fa4ae8 RDI: >> 0000000000008349 >> [63567.802314] RBP: ffff9e124acc1c80 R08: 0000000000600000 R09: >> 0000000000000000 >> [63567.802322] R10: 00000000000161d2 R11: ffff9e1346621700 R12: >> 0000000000000000 >> [63567.802331] R13: ffff9e124acc1d58 R14: 0000000000000000 R15: >> ffff9e124acc2214 >> [63567.802340] FS: 00007fc3d627c840(0000) GS:ffff9e1457380000(0000) >> knlGS:0000000000000000 >> [63567.802350] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 >> [63567.802358] CR2: 0000560b7e86d690 CR3: 0000000102f44001 CR4: >> 00000000001706e0 >> [63567.802367] Call Trace: >> [63567.802372] <TASK> >> [63567.802378] ? die+0x36/0x90 >> [63567.802386] ? do_trap+0xda/0x100 >> [63567.802392] ? tcp_rcv_space_adjust+0xbe/0x160 >> [63567.802401] ? do_error_trap+0x6a/0x90 >> [63567.802408] ? tcp_rcv_space_adjust+0xbe/0x160 >> [63567.802415] ? exc_divide_error+0x38/0x50 >> [63567.802423] ? tcp_rcv_space_adjust+0xbe/0x160 >> [63567.802431] ? asm_exc_divide_error+0x1a/0x20 >> [63567.802441] ? tcp_rcv_space_adjust+0xbe/0x160 >> [63567.802449] ? tcp_rcv_space_adjust+0x1a/0x160 >> [63567.802456] tcp_recvmsg_locked+0x2d4/0x960 >> [63567.802464] tcp_recvmsg+0x87/0x1f0 >> [63567.802471] inet6_recvmsg+0x56/0x130 >> [63567.802479] ? __pfx_bpf_lsm_socket_recvmsg+0x10/0x10 >> [63567.802488] ? security_socket_recvmsg+0x44/0x70 >> [63567.802497] sock_recvmsg+0x57/0xd0 >> [63567.802505] sock_read_iter+0x96/0x100 >> [63567.802513] vfs_read+0x303/0x350 >> [63567.802796] ksys_read+0xbb/0xf0 >> [63567.803074] do_syscall_64+0x60/0x90 >> [63567.803346] ? syscall_exit_to_user_mode+0x2b/0x40 >> [63567.803619] ? do_syscall_64+0x6c/0x90 >> [63567.803890] ? syscall_exit_to_user_mode+0x2b/0x40 >> [63567.804163] ? do_syscall_64+0x6c/0x90 >> [63567.804438] ? do_syscall_64+0x6c/0x90 >> [63567.804710] entry_SYSCALL_64_after_hwframe+0x6e/0xd8 >> [63567.804984] RIP: 0033:0x7fc3d5f21531 >> [63567.805271] Code: ff ff eb c3 67 e8 2f c9 01 00 66 2e 0f 1f 84 00 00 >> 00 00 00 0f 1f 44 00 00 f3 0f 1e fa 80 3d 35 ce 0d 00 00 74 13 31 c0 0f >> 05 <48> 3d 00 f0 ff ff >> 77 57 c3 66 0f 1f 44 00 00 48 83 ec 28 48 89 54 >> [63567.805909] RSP: 002b:00007fffa66f6748 EFLAGS: 00000246 ORIG_RAX: >> 0000000000000000 >> [63567.806225] RAX: ffffffffffffffda RBX: 00007fffa66f6830 RCX: >> 00007fc3d5f21531 >> [63567.806542] RDX: 000000000000191a RSI: 000055d106003d80 RDI: >> 00000000000000bd >> [63567.806855] RBP: 000055d10620bc30 R08: 000055d10620bc30 R09: >> 000055d1036a8e78 >> [63567.807163] R10: 000055d1037ad630 R11: 0000000000000246 R12: >> 000000000000191a >> [63567.807472] R13: 000055d10664f410 R14: 000055d106003d80 R15: >> 00007fc3d627c6b8 >> [63567.807768] </TASK> >> [63567.808037] Modules linked in: nfnetlink_queue nf_conntrack_netlink >> xt_connmark xt_mark ip_set_hash_net xt_NFQUEUE xt_set ip_set_hash_ip >> ip_set cls_fw sch_sfq sch_h >> tb nf_nat_ftp nf_conntrack_ftp tls tun cfg80211 rfkill nft_chain_nat >> xt_MASQUERADE xt_nat xt_REDIRECT nf_nat nft_limit xt_limit xt_conntrack >> nf_conntrack nf_defrag_ipv >> 6 nf_defrag_ipv4 xt_multiport xt_tcpudp nft_compat nf_tables libcrc32c >> intel_rapl_msr intel_rapl_common x86_pkg_temp_thermal intel_powerclamp >> snd_hda_codec_realtek cor >> etemp snd_hda_codec_generic ledtrig_audio snd_hda_codec_hdmi kvm_intel >> snd_hda_intel kvm snd_intel_dspcfg snd_intel_sdw_acpi irqbypass >> snd_hda_codec snd_hda_core crct1 >> 0dif_pclmul r8169 crc32_pclmul snd_hwdep polyval_clmulni snd_pcm >> polyval_generic gf128mul snd_timer realtek ax88179_178a spi_nor >> mdio_devres usbnet snd libphy ghash_cl >> mulni_intel mtd sha512_ssse3 at24 soundcore mii mei_pxp mei_hdcp >> sha256_ssse3 sha1_ssse3 iTCO_wdt i2c_i801 mei_me spi_intel_platform >> mousedev aesni_intel intel_pmc_bxt >> spi_intel iTCO_vendor_support >> [63567.808076] i2c_smbus mei crypto_simd lpc_ich cryptd rapl >> intel_cstate intel_uncore psmouse mac_hid pcspkr pkcs8_key_parser fuse >> dm_mod loop nfnetlink ip_tables x_ >> tables ext4 crc32c_generic crc16 mbcache jbd2 i915 serio_raw >> i2c_algo_bit atkbd drm_buddy libps2 ttm vivaldi_fmap intel_gtt xhci_pci >> crc32c_intel drm_display_helper xh >> ci_pci_renesas cec i8042 video serio wmi >> [63567.811333] ---[ end trace 0000000000000000 ]--- >> [63567.811701] RIP: 0010:tcp_rcv_space_adjust+0xbe/0x160 >> [63567.812069] Code: f8 41 89 d0 29 d0 31 d2 49 0f af c2 49 f7 f0 45 8b >> 81 bc 04 00 00 44 0f b6 8b 10 06 00 00 31 d2 49 8d 04 42 48 98 48 c1 e0 >> 08 <49> f7 f1 49 63 d0 >> 48 98 48 39 d0 48 0f 47 c2 39 83 18 01 00 00 7c >> [63567.812848] RSP: 0018:ffffba9f00da3c18 EFLAGS: 00010206 >> [63567.813252] RAX: 0000000004fb2800 RBX: ffff9e124acc1c80 RCX: >> 0000000eccd9dace >> [63567.813651] RDX: 0000000000000000 RSI: 0000000037fa4ae8 RDI: >> 0000000000008349 >> [63567.814064] RBP: ffff9e124acc1c80 R08: 0000000000600000 R09: >> 0000000000000000 >> [63567.814470] R10: 00000000000161d2 R11: ffff9e1346621700 R12: >> 0000000000000000 >> [63567.814879] R13: ffff9e124acc1d58 R14: 0000000000000000 R15: >> ffff9e124acc2214 >> [63567.815290] FS: 00007fc3d627c840(0000) GS:ffff9e1457380000(0000) >> knlGS:0000000000000000 >> [63567.815700] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 >> [63567.816121] CR2: 0000560b7e86d690 CR3: 0000000102f44001 CR4: >> 00000000001706e0 >> >> After this the "child" squid process can not be killed, even by systemd, >> even by kill -9. >> >> It appears that squid "main" process (owned by root user) does not hang >> and hence connections on port 8080 are accepted. >> >> But squid child process (owned by proxy user) hangs and never recovers >> after above dmesg. And hence nothing happens and connection times out. >> >> # ps aux |grep squid >> root 612 0.0 0.2 75500 23680 ? Ss Dec20 0:02 >> /usr/bin/squid -f /etc/squid/btnet/squid.btnet.conf --foreground -sYC >> proxy 617 0.0 0.0 0 0 ? D Dec20 1:00 [squid] >> proxy 620 0.0 0.0 12152 7552 ? S Dec20 0:00 >> (security_file_certgen) -s /var/cache/squid/ssl_db -M 4MB >> proxy 621 0.0 0.0 12152 7552 ? S Dec20 0:00 >> (security_file_certgen) -s /var/cache/squid/ssl_db -M 4MB >> proxy 622 0.0 0.0 11888 5504 ? S Dec20 0:00 >> (security_file_certgen) -s /var/cache/squid/ssl_db -M 4MB >> proxy 623 0.0 0.0 11888 5632 ? S Dec20 0:00 >> (security_file_certgen) -s /var/cache/squid/ssl_db -M 4MB >> proxy 624 0.0 0.0 11888 5632 ? S Dec20 0:00 >> (security_file_certgen) -s /var/cache/squid/ssl_db -M 4MB >> proxy 643 0.0 0.0 6116 3200 ? S Dec20 0:01 >> (logfile-daemon) /var/log/squid/access.log >> >> # telnet 127.0.0.1 8080 >> Trying 127.0.0.1... >> Connected to 127.0.0.1. >> Escape character is '^]'. >> ^] >> telnet> c >> >> # curl -x 127.0.0.1:8080 --max-time 30 https://google.com >> curl: (28) Connection timed out after 30002 milliseconds >> >> # kill 612 617 >> >> Ran kill 5 times, nothing happened. All squid processes remained. >> >> # kill -9 612 617 >> >> Ran once, and all squid processes except 617 (child squid process) got >> killed >> >> # ps aux |grep squid >> proxy 617 0.0 0.0 0 0 ? D Dec20 1:00 [squid] >> root 90764 0.0 0.0 6552 2560 pts/5 S+ 17:36 0:00 grep >> --color=auto squid >> >> Can above information be of any help? >> >> Thanks and regards, >> >> Amish >> >> On 19/12/23 20:46, Alex Rousskov wrote: >> > On 2023-12-18 22:29, Amish wrote: >> >> On 19/12/23 01:14, Alex Rousskov wrote: >> >>> On 2023-12-18 09:35, Amish wrote: >> >>> >> >>>> I use Arch Linux and today I updated squid from squid 5.7 to squid >> >>>> 6.6. >> >>> >> >>> > Dec 18 13:01:24 mumbai squid[604]: kick abandoning conn199 >> >>> >> >>> I do not know whether the above problem is the primary problem in >> >>> your setup, but it is a red flag. Transactions on the same >> >>> connection may get stuck after that message; it is essentially a >> >>> Squid bug. >> >>> >> >>> I am not sure at all, but this bug might be related to Bug 5187 >> >>> workaround that went into Squid v6.2 (commit c44cfe7): >> >>> https://bugs.squid-cache.org/show_bug.cgi?id=5187 >> >>> >> >>> Does Squid accept new TCP connections after it enters what you >> >>> describe as a dead state? For example, does "telnet 127.0.0.1 8080" >> >>> establishes a connection if executed on the same machine as Squid? >> >>> >> >> Yes it establishes connection. But I do not know what to do next. >> > >> > This tells us that your Squid is still listening for incoming >> > connections. Most likely, it is not "dead" but running and just unable >> > to make progress with those connections (for yet unknown reasons). >> > That information is helpful but not sufficient (for me) to solve the >> > problem you are describing. >> > >> > The next step that I would recommend is to collect debugging >> > information from the running process and share a pointer to the >> > corresponding compressed cache.log file: >> > >> > * Ideally, start collection when Squid starts and reproduce the >> > problem while collecting full debugging information: >> > http://wiki.squid-cache.org/SquidFaq/BugReporting#full-debug-output >> > >> > * If you have to, start collection after Squid is already in bad state >> > and just before you use telnet or browser to tickle Squid: >> > >> http://wiki.squid-cache.org/SquidFaq/BugReporting#debugging-a-single-transaction >> > >> > >> > Do not use any secret information (e.g., production certificate keys) >> > for these tests (unless you are going to share the logs privately with >> > those you trust). >> > >> > Do not downgrade to v5 for these tests. >> > >> > >> > HTH, >> > >> > Alex. >> > >> > >> >> Browser showed "Connection timed out" message. But I believe >> >> browser's also connected but nothing happened afterwards. >> >> >> >>> >> >>> > kill -9 does nothing >> >>> >> >>> Is it possible that you are trying to kill the wrong process? You >> >>> should be killing this process AFAICT: >> >>> >> >>> > root 601 0.0 0.2 73816 22528 ? Ss 12:59 0:02 >> >>> > /usr/bin/squid -f /etc/squid/btnet/squid.btnet.conf --foreground >> -sYC >> >> >> >> I did not clarify but all processes needed SIGKILL and vanished >> >> except the Dead squid process which still remained. >> >> >> >> # systemctl stop squid >> >> >> >> Dec 19 08:46:38 mumbai systemd[1]: squid.service: State >> >> 'stop-sigterm' timed out. Killing. >> >> Dec 19 08:46:38 mumbai systemd[1]: squid.service: Killing process 601 >> >> (squid) with signal SIGKILL. >> >> Dec 19 08:46:38 mumbai systemd[1]: squid.service: Killing process 604 >> >> (squid) with signal SIGKILL. >> >> Dec 19 08:46:38 mumbai systemd[1]: squid.service: Killing process 607 >> >> (security_file_c) with signal SIGKILL. >> >> Dec 19 08:46:38 mumbai systemd[1]: squid.service: Killing process 608 >> >> (security_file_c) with signal SIGKILL. >> >> Dec 19 08:46:38 mumbai systemd[1]: squid.service: Killing process 609 >> >> (security_file_c) with signal SIGKILL. >> >> Dec 19 08:46:38 mumbai systemd[1]: squid.service: Killing process 610 >> >> (security_file_c) with signal SIGKILL. >> >> Dec 19 08:46:38 mumbai systemd[1]: squid.service: Killing process 611 >> >> (security_file_c) with signal SIGKILL. >> >> Dec 19 08:46:38 mumbai systemd[1]: squid.service: Killing process 622 >> >> (log_file_daemon) with signal SIGKILL. >> >> Dec 19 08:46:38 mumbai systemd[1]: squid.service: Main process >> >> exited, code=killed, status=9/KILL >> >> Dec 19 08:46:38 mumbai systemd[1]: squid.service: Killing process 604 >> >> (squid) with signal SIGKILL. >> >> >> >> Waited for 2 minutes for squid to stop then pressed ctrl-c to >> >> systemctl stop squid command. >> >> >> >> As you can see in last line shows that attempt was made to kill DEAD >> >> process with PID 604. >> >> >> >> # ps aux |grep squid >> >> proxy 604 0.0 0.0 0 0 ? D Dec18 0:03 >> [squid] >> >> >> >> Now only DEAD squid process remains. >> >> >> >> What next? Should I downgrade to 5.9 and check? >> >> >> >> Regards >> >> >> >> Amish >> >> >> >>> Alex. >> >>> >> >>> >> >>>> After the update from 5.7 to 6.6, squid starts but then reaches >> >>>> Dead state in a minute or two. >> >>>> >> >>>> # ps aux | grep squid >> >>>> root 601 0.0 0.2 73816 22528 ? Ss 12:59 0:02 >> >>>> /usr/bin/squid -f /etc/squid/btnet/squid.btnet.conf --foreground -sYC >> >>>> proxy 604 0.0 0.0 0 0 ? D 12:59 0:03 >> >>>> [squid] >> >>>> proxy 607 0.0 0.0 11976 7424 ? S 12:59 0:00 >> >>>> (security_file_certgen) -s /var/cache/squid/ssl_db -M 4MB >> >>>> proxy 608 0.0 0.0 11976 7168 ? S 12:59 0:00 >> >>>> (security_file_certgen) -s /var/cache/squid/ssl_db -M 4MB >> >>>> proxy 609 0.0 0.0 11712 5632 ? S 12:59 0:00 >> >>>> (security_file_certgen) -s /var/cache/squid/ssl_db -M 4MB >> >>>> proxy 610 0.0 0.0 11712 5376 ? S 12:59 0:00 >> >>>> (security_file_certgen) -s /var/cache/squid/ssl_db -M 4MB >> >>>> proxy 611 0.0 0.0 11712 5504 ? S 12:59 0:00 >> >>>> (security_file_certgen) -s /var/cache/squid/ssl_db -M 4MB >> >>>> proxy 622 0.0 0.0 6116 3200 ? S 12:59 0:00 >> >>>> (logfile-daemon) /var/log/squid/access.log >> >>>> >> >>>> And then all requests get stuck. Notice the D (dead) state of squid. >> >>>> >> >>>> I use multiple ports for multiple purposes. (It all worked fine in >> >>>> squid 5.7) >> >>>> >> >>>> Dec 18 12:59:10 mumbai squid[601]: Starting Authentication on port >> >>>> [::]:3128 >> >>>> Dec 18 12:59:10 mumbai squid[601]: Disabling Authentication on port >> >>>> [::]:3128 (interception enabled) >> >>>> Dec 18 12:59:10 mumbai squid[601]: Starting Authentication on port >> >>>> [::]:8081 >> >>>> Dec 18 12:59:10 mumbai squid[601]: Disabling Authentication on port >> >>>> [::]:8081 (interception enabled) >> >>>> Dec 18 12:59:12 mumbai squid[601]: Starting Authentication on port >> >>>> [::]:8082 >> >>>> Dec 18 12:59:12 mumbai squid[601]: Disabling Authentication on port >> >>>> [::]:8082 (interception enabled) >> >>>> Dec 18 12:59:12 mumbai squid[601]: Starting Authentication on port >> >>>> [::]:8083 >> >>>> Dec 18 12:59:12 mumbai squid[601]: Disabling Authentication on port >> >>>> [::]:8083 (interception enabled) >> >>>> Dec 18 12:59:13 mumbai squid[601]: Starting Authentication on port >> >>>> [::]:8084 >> >>>> Dec 18 12:59:13 mumbai squid[601]: Disabling Authentication on port >> >>>> [::]:8084 (interception enabled) >> >>>> Dec 18 12:59:13 mumbai squid[601]: Starting Authentication on port >> >>>> [::]:3136 >> >>>> Dec 18 12:59:13 mumbai squid[601]: Disabling Authentication on port >> >>>> [::]:3136 (interception enabled) >> >>>> Dec 18 12:59:13 mumbai squid[601]: Starting Authentication on port >> >>>> [::]:3137 >> >>>> Dec 18 12:59:13 mumbai squid[601]: Disabling Authentication on port >> >>>> [::]:3137 (interception enabled) >> >>>> ... >> >>>> Dec 18 12:59:29 mumbai squid[604]: Adaptation support is on >> >>>> Dec 18 12:59:29 mumbai squid[604]: Accepting NAT intercepted HTTP >> >>>> Socket connections at conn19 local=[::]:3128 remote=[::] FD 27 >> >>>> flags=41 >> >>>> listening port: 3128 >> >>>> Dec 18 12:59:29 mumbai squid[604]: Accepting SSL bumped HTTP Socket >> >>>> connections at conn21 local=[::]:8080 remote=[::] FD 28 flags=9 >> >>>> listening port: 8080 >> >>>> Dec 18 12:59:29 mumbai squid[604]: Accepting NAT intercepted SSL >> >>>> bumped HTTPS Socket connections at conn23 local=[::]:8081 >> >>>> remote=[::] FD 29 flags=41 >> >>>> listening port: 8081 >> >>>> Dec 18 12:59:29 mumbai squid[604]: Accepting SSL bumped HTTP Socket >> >>>> connections at conn25 local=[::]:8092 remote=[::] FD 30 flags=9 >> >>>> listening port: 8092 >> >>>> Dec 18 12:59:29 mumbai systemd[1]: Started Squid Web Proxy Server. >> >>>> Dec 18 12:59:29 mumbai squid[604]: Accepting SSL bumped HTTP Socket >> >>>> connections at conn27 local=[::]:8093 remote=[::] FD 31 flags=9 >> >>>> listening port: 8093 >> >>>> Dec 18 12:59:29 mumbai squid[604]: Accepting SSL bumped HTTP Socket >> >>>> connections at conn29 local=[::]:8094 remote=[::] FD 32 flags=9 >> >>>> listening port: 8094 >> >>>> Dec 18 12:59:29 mumbai squid[604]: Accepting NAT intercepted SSL >> >>>> bumped HTTPS Socket connections at conn31 local=[::]:8082 >> >>>> remote=[::] FD 33 flags=41 >> >>>> listening port: 8082 >> >>>> Dec 18 12:59:29 mumbai squid[604]: Accepting NAT intercepted SSL >> >>>> bumped HTTPS Socket connections at conn33 local=[::]:8083 >> >>>> remote=[::] FD 34 flags=41 >> >>>> listening port: 8083 >> >>>> Dec 18 12:59:29 mumbai squid[604]: Accepting NAT intercepted SSL >> >>>> bumped HTTPS Socket connections at conn35 local=[::]:8084 >> >>>> remote=[::] FD 35 flags=41 >> >>>> listening port: 8084 >> >>>> Dec 18 12:59:29 mumbai squid[604]: Accepting NAT intercepted HTTP >> >>>> Socket connections at conn37 local=[::]:3136 remote=[::] FD 36 >> >>>> flags=41 >> >>>> listening port: 3136 >> >>>> Dec 18 12:59:29 mumbai squid[604]: Accepting NAT intercepted HTTP >> >>>> Socket connections at conn39 local=[::]:3137 remote=[::] FD 37 >> >>>> flags=41 >> >>>> listening port: 3137 >> >>>> >> >>>> And then following errors came: >> >>>> >> >>>> >> >>>> Dec 18 12:59:45 mumbai squid[604]: ERROR: failure while accepting a >> >>>> TLS connection on conn41 local=192.168.0.1:8080 >> >>>> remote=192.168.0.111:53867 FD 12 flags=1: SQUID_TLS >> >>>> _ERR_ACCEPT+TLS_LIB_ERR=A000416+TLS_IO_ERR=1 >> >>>> current master transaction: >> >>>> master53 >> >>>> Dec 18 12:59:45 mumbai squid[604]: ERROR: failure while accepting a >> >>>> TLS connection on conn42 local=192.168.0.1:8080 >> >>>> remote=192.168.0.111:53868 FD 14 flags=1: SQUID_TLS >> >>>> _ERR_ACCEPT+TLS_LIB_ERR=A000416+TLS_IO_ERR=1 >> >>>> current master transaction: >> >>>> master53 >> >>>> Dec 18 12:59:45 mumbai squid[604]: ERROR: failure while accepting a >> >>>> TLS connection on conn43 local=192.168.0.1:8080 >> >>>> remote=192.168.0.111:53869 FD 16 flags=1: SQUID_TLS >> >>>> _ERR_ACCEPT+TLS_LIB_ERR=A000416+TLS_IO_ERR=1 >> >>>> current master transaction: >> >>>> master57 >> >>>> Dec 18 12:59:45 mumbai squid[604]: ERROR: failure while accepting a >> >>>> TLS connection on conn44 local=192.168.0.1:8080 >> >>>> remote=192.168.0.111:53870 FD 12 flags=1: SQUID_TLS >> >>>> _ERR_ACCEPT+TLS_LIB_ERR=A000416+TLS_IO_ERR=1 >> >>>> current master transaction: >> >>>> master57 >> >>>> Dec 18 12:59:56 mumbai squid[604]: ERROR: failure while accepting a >> >>>> TLS connection on conn62 local=192.168.0.1:8080 >> >>>> remote=192.168.0.111:53887 FD 12 flags=1: SQUID_TLS >> >>>> _ERR_ACCEPT+TLS_LIB_ERR=A000416+TLS_IO_ERR=1 >> >>>> current master transaction: >> >>>> master95 >> >>>> Dec 18 12:59:59 mumbai squid[604]: ERROR: failure while accepting a >> >>>> TLS connection on conn64 local=192.168.0.1:8080 >> >>>> remote=192.168.0.111:53888 FD 12 flags=1: SQUID_TLS >> >>>> _ERR_ACCEPT+TLS_LIB_ERR=A000416+TLS_IO_ERR=1 >> >>>> current master transaction: >> >>>> master99 >> >>>> Dec 18 13:00:02 mumbai squid[604]: ERROR: failure while accepting a >> >>>> TLS connection on conn65 local=192.168.0.1:8080 >> >>>> remote=192.168.0.178:56115 FD 12 flags=1: SQUID_TLS >> >>>> _ERR_ACCEPT+TLS_LIB_ERR=A000418+TLS_IO_ERR=1 >> >>>> current master transaction: >> >>>> master53 >> >>>> Dec 18 13:01:24 mumbai squid[604]: kick abandoning conn199 >> >>>> local=192.168.0.1:8093 remote=192.168.0.101:52211 FD 52 flags=1 >> >>>> connection: conn199 >> >>>> local=192.168.0.1:8093 remote=192.168.0.101:52211 FD 52 flags=1 >> >>>> Dec 18 13:01:45 mumbai squid[604]: ERROR: failure while accepting a >> >>>> TLS connection on conn240 local=192.168.0.1:8093 >> >>>> remote=192.168.0.111:53931 FD 48 flags=1: SQUID_TL >> >>>> S_ERR_ACCEPT+TLS_LIB_ERR=A000416+TLS_IO_ERR=1 >> >>>> current master transaction: >> >>>> master314 >> >>>> >> >>>> >> >>>> After this point there is nothing in systemd journal (via: >> >>>> journalctl -f -u squid) and same lines are in cache.log. >> >>>> >> >>>> Squid got stuck (DEAD state) at 13:01 and right now it 19:26 (6 >> >>>> hours passed) and squid is still in dead state. >> >>>> >> >>>> kill -9 or kill -ALRM or -HUP also does nothing. >> >>>> >> >>>> So to restart squid - I will need to restart whole system. >> >>>> >> >>>> I have sslbump directives but it is not really applied. >> >>>> >> >>>> #NOTE: nosslbump_ips below contains 192.168.0.0/24 (whole LAN) so >> >>>> effectively there is no SSL bump after step1. >> >>>> >> >>>> acl nosslbump_ips src 192.168.0.0/24 >> >>>> ssl_bump splice ssl_step1 nosslbump_ips >> >>>> ssl_bump peek ssl_step1 >> >>>> ssl_bump splice nosslbump_domains >> >>>> ssl_bump stare sslbump_domains >> >>>> ssl_bump splice ssl_step2 >> >>>> ssl_bump bump all >> >>>> >> >>>> >> >>>> Any idea? If anything changed from 5.7 to 6.6 that may cause this >> >>>> behaviour? >> >>>> >> >>>> Looking at changelog: >> >>>> >> >>>> Bug 5256: Intercepting port fails to accept >> >>>> https://bugs.squid-cache.org/show_bug.cgi?id=5256 >> >>>> >> >>>> Bug 5154: Do not open IPv6 sockets when IPv6 is disabled >> >>>> https://bugs.squid-cache.org/show_bug.cgi?id=5154 >> >>>> >> >>>> Not sure if above two bug FIXES (in between v5.7 to v6.6) are >> >>>> related to my issue. >> >>>> >> >>>> I ran netstat: >> >>>> >> >>>> # netstat -ntlp >> >>>> Active Internet connections (only servers) >> >>>> Proto Recv-Q Send-Q Local Address Foreign Address >> >>>> State PID/Program name >> >>>> ... >> >>>> tcp6 33 0 :::3137 :::* LISTEN - >> >>>> tcp6 0 0 :::3136 :::* LISTEN - >> >>>> tcp6 4 0 :::3128 :::* LISTEN - >> >>>> tcp6 0 0 :::8081 :::* LISTEN - >> >>>> tcp6 0 0 :::8080 :::* LISTEN - >> >>>> tcp6 0 0 :::8083 :::* LISTEN - >> >>>> tcp6 0 0 :::8082 :::* LISTEN - >> >>>> tcp6 0 0 :::8084 :::* LISTEN - >> >>>> tcp6 4097 0 :::8093 :::* LISTEN - >> >>>> tcp6 0 0 :::8092 :::* LISTEN - >> >>>> tcp6 0 0 :::8094 :::* LISTEN - >> >>>> ... >> >>>> >> >>>> I do not have IPv6 enabled, yet there are 33 and 4097 numbers in >> >>>> Recv-Q and also no process/PID owns these ports. >> >>>> >> >>>> Same IPv4 ports are not shown in use by netstat, so only IPv6 ports >> >>>> remain open, that too orphaned! >> >>>> >> >>>> So what is happening? >> >>>> >> >>>> Any idea to solve or any workaround? >> >>>> >> >>>> Thank you, >> >>>> >> >>>> Amish. >> >> _______________________________________________ >> >> squid-users mailing list >> >> squid-users@lists.squid-cache.org >> >> https://lists.squid-cache.org/listinfo/squid-users >> > >> _______________________________________________ >> squid-users mailing list >> squid-users@lists.squid-cache.org >> https://lists.squid-cache.org/listinfo/squid-users >> > > > -- > Francesco > > -- Francesco
_______________________________________________ squid-users mailing list squid-users@lists.squid-cache.org https://lists.squid-cache.org/listinfo/squid-users
