[Kernel-packages] [Bug 2018960] Re: linux-image-5.4.0-149-generic (regression): 0 at net/core/stream.c:212 sk_stream_kill_queues+0xcf/0xe0

2023-06-12 Thread Tom Söderlund
Seeing this on U18 based gateway nodes running 4.15.0-212-generic.

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux-signed in Ubuntu.
https://bugs.launchpad.net/bugs/2018960

Title:
  linux-image-5.4.0-149-generic (regression): 0 at net/core/stream.c:212
  sk_stream_kill_queues+0xcf/0xe0

Status in linux-signed package in Ubuntu:
  Confirmed
Status in linux-signed-kvm package in Ubuntu:
  Confirmed

Bug description:
  After upgrading and rebooting this Ubuntu 20.04 LTS server (Ubuntu
  Focal), I noticed that it was suddenly getting a bunch of kernel log
  (dmesg) reports like:

  WARNING: CPU: 4 PID: 0 at net/core/stream.c:212
  sk_stream_kill_queues+0xcf/0xe0

  while investigating I determined that it is currently running the
  focal-proposed kernel (linux-image-5.4.0-149-generic), which it turns
  out was enabled for this server (clearly it seemed like a good idea at
  the time).

  I'm not expecting focal-proposed to be fixed as if it were a release
  package, but since I couldn't find any reports on Launchpad I figured
  I should let y'all know this focal-proposed package could do with some
  additional work before it's actually released :-)

  There have been at least 80 such reports in the last 5 hours since the
  server was rebooted, differing only by the CPU core and the process
  reported, although it seems the last one was a couple of hours ago, so
  I guess it's traffic dependent/timing dependent.

  ewen@naosr620:~$ uptime
   16:27:32 up  5:19,  1 user,  load average: 0.08, 0.14, 0.06
  ewen@naosr620:~$ dmesg -t | grep WARNING | sed 's/CPU: [0-9]*/CPU: N/; s/PID: 
[0-9]*/PID: N/;' | uniq -c
   88 WARNING: CPU: N PID: N at net/core/stream.c:212 
sk_stream_kill_queues+0xcf/0xe0
  ewen@naosr620:~$ 

  Ubuntu Release:

  ewen@naosr620:~$ lsb_release -rd
  Description:  Ubuntu 20.04.6 LTS
  Release:  20.04
  ewen@naosr620:~$ 

  
  Kernel/package version affected:

  ewen@naosr620:~$ uname -a
  Linux naosr620 5.4.0-149-generic #166-Ubuntu SMP Tue Apr 18 16:51:45 UTC 2023 
x86_64 x86_64 x86_64 GNU/Linux
  ewen@naosr620:~$ dpkg -l | grep linux-image | grep 149
  ii  linux-image-5.4.0-149-generic  5.4.0-149.166  
   amd64Signed kernel image generic
  ii  linux-image-generic5.4.0.149.147  
   amd64Generic Linux kernel image
  ewen@naosr620:~$ apt-cache policy linux-image-5.4.0-149-generic 
  linux-image-5.4.0-149-generic:
Installed: 5.4.0-149.166
Candidate: 5.4.0-149.166
Version table:
   *** 5.4.0-149.166 500
  500 https://mirror.fsmg.org.nz/ubuntu focal-proposed/main amd64 
Packages
  100 /var/lib/dpkg/status
  ewen@naosr620:~$ apt-cache policy linux-image-generic
  linux-image-generic:
Installed: 5.4.0.149.147
Candidate: 5.4.0.149.147
Version table:
   *** 5.4.0.149.147 500
  500 https://mirror.fsmg.org.nz/ubuntu focal-proposed/main amd64 
Packages
  100 /var/lib/dpkg/status
   5.4.0.148.146 500
  500 https://mirror.fsmg.org.nz/ubuntu focal-updates/main amd64 
Packages
  500 https://mirror.fsmg.org.nz/ubuntu focal-security/main amd64 
Packages
   5.4.0.26.32 500
  500 https://mirror.fsmg.org.nz/ubuntu focal/main amd64 Packages
  ewen@naosr620:~$ 
  ewen@naosr620:~$ apt-cache show linux-image-5.4.0-149-generic | grep Source:
  Source: linux-signed
  ewen@naosr620:~$ 

  
  Full example dmesg, including stack trace (they all seem to be WARNINGs, and 
other than filling dmesg / system logs the system "appears to be running okay", 
so I'm not going to rush another reboot now -- near end of business day):

  ewen@naosr620:~$ date
  Tue 09 May 2023 16:34:56 NZST
  ewen@naosr620:~$ dmesg -T | tail -100 | grep -B 150 "end trace" | grep -A 999 
"cut here"
  [Tue May  9 14:21:18 2023] [ cut here ]
  [Tue May  9 14:21:18 2023] WARNING: CPU: 10 PID: 0 at net/core/stream.c:212 
sk_stream_kill_queues+0xcf/0xe0
  [Tue May  9 14:21:18 2023] Modules linked in: mpt3sas raid_class 
scsi_transport_sas mptctl mptbase vhost_net vhost tap ip6t_REJECT 
nf_reject_ipv6 ip6table_mangle ip6table_nat ip6table_raw nf_log_ipv6 xt_recent 
ipt_REJECT nf_reject_ipv4 xt_hashlimit xt_addrtype xt_multiport xt_comment 
xt_conntrack xt_mark iptable_mangle xt_MASQUERADE iptable_nat xt_CT xt_tcpudp 
iptable_raw nfnetlink_log xt_NFLOG nf_log_ipv4 nf_log_common xt_LOG nf_nat_tftp 
nf_nat_snmp_basic nf_conntrack_snmp nf_nat_sip nf_nat_pptp nf_nat_irc 
ebtable_filter nf_nat_h323 ebtables nf_nat_ftp nf_nat_amanda ts_kmp 
ip6table_filter nf_conntrack_amanda nf_nat ip6_tables nf_conntrack_sane 
nf_conntrack_tftp nf_conntrack_sip nf_conntrack_pptp nf_conntrack_netlink 
nfnetlink nf_conntrack_netbios_ns nf_conntrack_broadcast nf_conntrack_irc 
nf_conntrack_h323 nf_conntrack_ftp nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 
iptable_filter bpfilter dell_rbu nls_iso8859_1 ipmi_ssif input_l

[Kernel-packages] [Bug 2018960] Re: linux-image-5.4.0-149-generic (regression): 0 at net/core/stream.c:212 sk_stream_kill_queues+0xcf/0xe0

2023-07-10 Thread Tom Söderlund
Saw this on U18 based nodes in gateway roles running
4.15.0-{211,212}-generic. It seems to be fixed in 4.15.0-213-generic.

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux-signed in Ubuntu.
https://bugs.launchpad.net/bugs/2018960

Title:
  linux-image-5.4.0-149-generic (regression): 0 at net/core/stream.c:212
  sk_stream_kill_queues+0xcf/0xe0

Status in linux-signed package in Ubuntu:
  Fix Released
Status in linux-signed-kvm package in Ubuntu:
  Fix Released

Bug description:
  After upgrading and rebooting this Ubuntu 20.04 LTS server (Ubuntu
  Focal), I noticed that it was suddenly getting a bunch of kernel log
  (dmesg) reports like:

  WARNING: CPU: 4 PID: 0 at net/core/stream.c:212
  sk_stream_kill_queues+0xcf/0xe0

  while investigating I determined that it is currently running the
  focal-proposed kernel (linux-image-5.4.0-149-generic), which it turns
  out was enabled for this server (clearly it seemed like a good idea at
  the time).

  I'm not expecting focal-proposed to be fixed as if it were a release
  package, but since I couldn't find any reports on Launchpad I figured
  I should let y'all know this focal-proposed package could do with some
  additional work before it's actually released :-)

  There have been at least 80 such reports in the last 5 hours since the
  server was rebooted, differing only by the CPU core and the process
  reported, although it seems the last one was a couple of hours ago, so
  I guess it's traffic dependent/timing dependent.

  ewen@naosr620:~$ uptime
   16:27:32 up  5:19,  1 user,  load average: 0.08, 0.14, 0.06
  ewen@naosr620:~$ dmesg -t | grep WARNING | sed 's/CPU: [0-9]*/CPU: N/; s/PID: 
[0-9]*/PID: N/;' | uniq -c
   88 WARNING: CPU: N PID: N at net/core/stream.c:212 
sk_stream_kill_queues+0xcf/0xe0
  ewen@naosr620:~$ 

  Ubuntu Release:

  ewen@naosr620:~$ lsb_release -rd
  Description:  Ubuntu 20.04.6 LTS
  Release:  20.04
  ewen@naosr620:~$ 

  
  Kernel/package version affected:

  ewen@naosr620:~$ uname -a
  Linux naosr620 5.4.0-149-generic #166-Ubuntu SMP Tue Apr 18 16:51:45 UTC 2023 
x86_64 x86_64 x86_64 GNU/Linux
  ewen@naosr620:~$ dpkg -l | grep linux-image | grep 149
  ii  linux-image-5.4.0-149-generic  5.4.0-149.166  
   amd64Signed kernel image generic
  ii  linux-image-generic5.4.0.149.147  
   amd64Generic Linux kernel image
  ewen@naosr620:~$ apt-cache policy linux-image-5.4.0-149-generic 
  linux-image-5.4.0-149-generic:
Installed: 5.4.0-149.166
Candidate: 5.4.0-149.166
Version table:
   *** 5.4.0-149.166 500
  500 https://mirror.fsmg.org.nz/ubuntu focal-proposed/main amd64 
Packages
  100 /var/lib/dpkg/status
  ewen@naosr620:~$ apt-cache policy linux-image-generic
  linux-image-generic:
Installed: 5.4.0.149.147
Candidate: 5.4.0.149.147
Version table:
   *** 5.4.0.149.147 500
  500 https://mirror.fsmg.org.nz/ubuntu focal-proposed/main amd64 
Packages
  100 /var/lib/dpkg/status
   5.4.0.148.146 500
  500 https://mirror.fsmg.org.nz/ubuntu focal-updates/main amd64 
Packages
  500 https://mirror.fsmg.org.nz/ubuntu focal-security/main amd64 
Packages
   5.4.0.26.32 500
  500 https://mirror.fsmg.org.nz/ubuntu focal/main amd64 Packages
  ewen@naosr620:~$ 
  ewen@naosr620:~$ apt-cache show linux-image-5.4.0-149-generic | grep Source:
  Source: linux-signed
  ewen@naosr620:~$ 

  
  Full example dmesg, including stack trace (they all seem to be WARNINGs, and 
other than filling dmesg / system logs the system "appears to be running okay", 
so I'm not going to rush another reboot now -- near end of business day):

  ewen@naosr620:~$ date
  Tue 09 May 2023 16:34:56 NZST
  ewen@naosr620:~$ dmesg -T | tail -100 | grep -B 150 "end trace" | grep -A 999 
"cut here"
  [Tue May  9 14:21:18 2023] [ cut here ]
  [Tue May  9 14:21:18 2023] WARNING: CPU: 10 PID: 0 at net/core/stream.c:212 
sk_stream_kill_queues+0xcf/0xe0
  [Tue May  9 14:21:18 2023] Modules linked in: mpt3sas raid_class 
scsi_transport_sas mptctl mptbase vhost_net vhost tap ip6t_REJECT 
nf_reject_ipv6 ip6table_mangle ip6table_nat ip6table_raw nf_log_ipv6 xt_recent 
ipt_REJECT nf_reject_ipv4 xt_hashlimit xt_addrtype xt_multiport xt_comment 
xt_conntrack xt_mark iptable_mangle xt_MASQUERADE iptable_nat xt_CT xt_tcpudp 
iptable_raw nfnetlink_log xt_NFLOG nf_log_ipv4 nf_log_common xt_LOG nf_nat_tftp 
nf_nat_snmp_basic nf_conntrack_snmp nf_nat_sip nf_nat_pptp nf_nat_irc 
ebtable_filter nf_nat_h323 ebtables nf_nat_ftp nf_nat_amanda ts_kmp 
ip6table_filter nf_conntrack_amanda nf_nat ip6_tables nf_conntrack_sane 
nf_conntrack_tftp nf_conntrack_sip nf_conntrack_pptp nf_conntrack_netlink 
nfnetlink nf_conntrack_netbios_ns nf_conntrack_broadcast nf_conntrack_irc 
nf_conntrack_h323 nf_conntrack_ftp nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 
ip