I can confirm that the problem is indeed not fully fixed.
@electricdaemon said:
> Test kernel posted fixes crash but has another bug with unkillable
stuck defunct docker-proxy service causing more issues. Bug is not
solved. Tested on Linux AWS Lightsail instance.
And that's the problem that I'm seeing as well. Still gathering data for
a bug report.
# uname -a
Linux ip-10-0-69-193 5.13.0-1029-aws #32~20.04.1-Ubuntu SMP Thu Jun 9 13:03:13
UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
root@ip-10-0-69-193:/opt/myapp8/myappserv/int# docker-compose up -d
Creating network "myappserv-int_default" with the default driver
Creating myapp-migrator-int ... done
Creating myapp-dealer-int ...
Creating myapp-offer-int ...
Creating myapp-customer-int ...
Creating myapp-customer-int ... error
Creating myapp-dealer-int ... done
Creating myapp-offer-int ... done
: port is already allocated
ERROR: for customer Cannot start service customer: driver failed programming
external connectivity on endpoint myapp8-customer-int
(fe4112364528b0e7d192c793929c579e8a81af715118c8f83ad7e65e7397f3be): Bind for
0.0.0.0:9001 failed: port is already allocated
ERROR: Encountered errors while bringing up the project.
root@ip-10-0-69-193:/opt/myapp8/myappserv/int# docker-compose down
Stopping myapp8-offer-int ... done
Stopping myapp8-dealer-int ... done
Removing myapp8-customer-int ... done
Removing myapp8-offer-int ... done
Removing myapp8-dealer-int ... done
Removing myapp8-migrator-int ... done
Removing network myappserv-int_default
root@ip-10-0-69-193:/opt/myapp8/myappserv/int# docker-compose up -d
Creating network "myappserv-int_default" with the default driver
Creating myapp8-migrator-int ... done
Creating myapp8-offer-int ...
Creating myapp8-customer-int ...
Creating myapp8-customer-int ... error
WARNING: Host is already in use by another container
Creating myapp8-offer-int ... done
ERROR: for myapp8-customer-int Cannot start service customer: driver failed
programming external connectivity on endpoint myapp8-customer-int
(72fc08854cd278e63cd3234e7fb03c08cb045efdcfb9e42075a1250d893645d5): Bind for
0.0.0.0:9001 failed
Creating myapp8-dealer-int ... done
ERROR: for customer Cannot start service customer: driver failed programming
external connectivity on endpoint myapp8-customer-int
(72fc08854cd278e63cd3234e7fb03c08cb045efdcfb9e42075a1250d893645d5): Bind for
0.0.0.0:9001 failed: port is already allocated
ERROR: Encountered errors while bringing up the project.
# docker-compose config
services:
customer:
container_name: myapp8-customer-int
depends_on:
migrator:
condition: service_completed_successfully
image: reg.mydomain.tld/myapp8/customer:430d4ca
ports:
- published: 9001
target: 9001
restart: always
dealer:
container_name: myapp8-dealer-int
depends_on:
migrator:
condition: service_completed_successfully
image: reg.mydomain.tld/myapp8/dealer:430d4ca
ports:
- published: 9002
target: 9002
restart: always
migrator:
container_name: myapp8-migrator-int
image: reg.mydomain.tld/myapp8/migrator:430d4ca
offer:
container_name: myapp8-offer-int
depends_on:
migrator:
condition: service_completed_successfully
image: reg.mydomain.tld/myapp8/offer:430d4ca
ports:
- published: 9003
target: 9003
restart: always
version: '3'
# netstat -tulpn | egrep "(Foreign|docker-proxy)"
Proto Recv-Q Send-Q Local Address Foreign Address State
PID/Program name
tcp 0 0 0.0.0.0:9001 0.0.0.0:* LISTEN
1090/docker-proxy
tcp 0 0 0.0.0.0:9002 0.0.0.0:* LISTEN
4519/docker-proxy
tcp 0 0 0.0.0.0:9003 0.0.0.0:* LISTEN
4539/docker-proxy
tcp 0 0 0.0.0.0:8055 0.0.0.0:* LISTEN
1037/docker-proxy
tcp 0 0 0.0.0.0:19000 0.0.0.0:* LISTEN
997/docker-proxy
tcp 0 0 127.0.0.1:40000 0.0.0.0:* LISTEN
1077/docker-proxy
tcp 0 0 0.0.0.0:8065 0.0.0.0:* LISTEN
1016/docker-proxy
tcp6 0 0 :::9001 :::* LISTEN
1096/docker-proxy
tcp6 0 0 :::9002 :::* LISTEN
4525/docker-proxy
tcp6 0 0 :::9003 :::* LISTEN
4544/docker-proxy
tcp6 0 0 :::8055 :::* LISTEN
1043/docker-proxy
tcp6 0 0 :::19000 :::* LISTEN
1003/docker-proxy
tcp6 0 0 :::8065 :::* LISTEN
1022/docker-proxy
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux-aws-5.13 in Ubuntu.
https://bugs.launchpad.net/bugs/1977919
Title:
Docker container creation causes kernel oops on linux-aws
5.13.0.1028.31~20.04.22
Status in linux-aws-5.13 package in Ubuntu:
Confirmed
Status in linux-azure-5.13 package in Ubuntu:
Confirmed
Status in linux-gcp-5.13 package in Ubuntu:
Confirmed
Status in linux-intel-iotg-5.15 package in Ubuntu:
Confirmed
Status in linux-oracle-5.13 package in Ubuntu:
Confirmed
Status in linux-aws-5.13 source package in Focal:
Fix Committed
Status in linux-azure-5.13 source package in Focal:
Fix Committed
Status in linux-gcp-5.13 source package in Focal:
Fix Committed
Status in linux-intel-iotg-5.15 source package in Focal:
Won't Fix
Status in linux-oracle-5.13 source package in Focal:
Fix Committed
Bug description:
Running the attached script on the latest AWS AMI for Ubuntu 20.04, I
get a kernel panic and hard reset of the node.
[ 12.314552] VFS: Close: file count is 0
[ 12.351090] ------------[ cut here ]------------
[ 12.351093] kernel BUG at include/linux/fs.h:3104!
[ 12.355272] invalid opcode: 0000 [#1] SMP PTI
[ 12.358963] CPU: 1 PID: 863 Comm: sed Not tainted 5.13.0-1028-aws
#31~20.04.1-Ubuntu
[ 12.366241] Hardware name: Amazon EC2 m5.large/, BIOS 1.0 10/16/2017
[ 12.371130] RIP: 0010:__fput+0x247/0x250
[ 12.374897] Code: 00 48 85 ff 0f 84 8b fe ff ff f6 c7 40 0f 85 82 fe ff ff
e8 ab 38 00 00 e9 78 fe ff ff 4c 89 f7 e8 2e 88 02 00 e9 b5 fe ff ff <0f> 0b 0f
1f 80 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 53 31 db 48
[ 12.389075] RSP: 0018:ffffb50280d9fd88 EFLAGS: 00010246
[ 12.393425] RAX: 0000000000000000 RBX: 00000000000a801d RCX:
ffff9152e0716000
[ 12.398679] RDX: ffff9152cf075280 RSI: 0000000000000001 RDI:
0000000000000000
[ 12.403879] RBP: ffffb50280d9fdb0 R08: 0000000000000001 R09:
ffff9152dfcba2c8
[ 12.409102] R10: ffffb50280d9fd88 R11: ffff9152d04e9d10 R12:
ffff9152d04e9d00
[ 12.414333] R13: ffff9152dfcba2c8 R14: ffff9152cf0752a0 R15:
ffff9152dfc2e180
[ 12.419533] FS: 0000000000000000(0000) GS:ffff9153ea900000(0000)
knlGS:0000000000000000
[ 12.426937] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 12.431506] CR2: 0000556cf30250a8 CR3: 00000000bce10006 CR4:
00000000007706e0
[ 12.436716] DR0: 0000000000000000 DR1: 0000000000000000 DR2:
0000000000000000
[ 12.441941] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7:
0000000000000400
[ 12.447170] PKRU: 55555554
[ 12.450355] Call Trace:
[ 12.453408] <TASK>
[ 12.456296] ____fput+0xe/0x10
[ 12.459633] task_work_run+0x70/0xb0
[ 12.463157] do_exit+0x37b/0xaf0
[ 12.466570] do_group_exit+0x43/0xb0
[ 12.470142] __x64_sys_exit_group+0x18/0x20
[ 12.473989] do_syscall_64+0x61/0xb0
[ 12.477565] ? exit_to_user_mode_prepare+0x9b/0x1c0
[ 12.481734] ? do_user_addr_fault+0x1d0/0x650
[ 12.485665] ? irqentry_exit_to_user_mode+0x9/0x20
[ 12.489790] ? irqentry_exit+0x19/0x30
[ 12.493443] ? exc_page_fault+0x8f/0x170
[ 12.497199] ? asm_exc_page_fault+0x8/0x30
[ 12.501013] entry_SYSCALL_64_after_hwframe+0x44/0xae
[ 12.505289] RIP: 0033:0x7f80d42a1bd6
[ 12.508868] Code: Unable to access opcode bytes at RIP 0x7f80d42a1bac.
[ 12.513783] RSP: 002b:00007ffe924f9ed8 EFLAGS: 00000246 ORIG_RAX:
00000000000000e7
[ 12.520897] RAX: ffffffffffffffda RBX: 00007f80d45a4740 RCX:
00007f80d42a1bd6
[ 12.526115] RDX: 0000000000000000 RSI: 000000000000003c RDI:
0000000000000000
[ 12.531328] RBP: 0000000000000000 R08: 00000000000000e7 R09:
fffffffffffffe98
[ 12.536484] R10: 00007f80d3d422a0 R11: 0000000000000246 R12:
00007f80d45a4740
[ 12.541687] R13: 0000000000000002 R14: 00007f80d45ad708 R15:
0000000000000000
[ 12.546916] </TASK>
[ 12.549829] Modules linked in: xt_conntrack xt_MASQUERADE
nf_conntrack_netlink nfnetlink xfrm_user xfrm_algo xt_addrtype iptable_filter
iptable_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 libcrc32c
bpfilter br_netfilter bridge stp llc aufs overlay nls_iso8859_1 dm_multipath
scsi_dh_rdac scsi_dh_emc scsi_dh_alua crct10dif_pclmul ppdev crc32_pclmul
ghash_clmulni_intel aesni_intel crypto_simd psmouse cryptd parport_pc
input_leds parport ena serio_raw sch_fq_codel ipmi_devintf ipmi_msghandler msr
drm ip_tables x_tables autofs4
[ 12.583913] ---[ end trace 77367fed4d782aa4 ]---
[ 12.587963] RIP: 0010:__fput+0x247/0x250
[ 12.591729] Code: 00 48 85 ff 0f 84 8b fe ff ff f6 c7 40 0f 85 82 fe ff ff
e8 ab 38 00 00 e9 78 fe ff ff 4c 89 f7 e8 2e 88 02 00 e9 b5 fe ff ff <0f> 0b 0f
1f 80 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 53 31 db 48
[ 12.605796] RSP: 0018:ffffb50280d9fd88 EFLAGS: 00010246
[ 12.610166] RAX: 0000000000000000 RBX: 00000000000a801d RCX:
ffff9152e0716000
[ 12.615417] RDX: ffff9152cf075280 RSI: 0000000000000001 RDI:
0000000000000000
[ 12.620635] RBP: ffffb50280d9fdb0 R08: 0000000000000001 R09:
ffff9152dfcba2c8
[ 12.625878] R10: ffffb50280d9fd88 R11: ffff9152d04e9d10 R12:
ffff9152d04e9d00
[ 12.631121] R13: ffff9152dfcba2c8 R14: ffff9152cf0752a0 R15:
ffff9152dfc2e180
[ 12.636358] FS: 0000000000000000(0000) GS:ffff9153ea900000(0000)
knlGS:0000000000000000
[ 12.643770] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 12.648355] CR2: 0000556cf30250a8 CR3: 00000000bce10006 CR4:
00000000007706e0
[ 12.653610] DR0: 0000000000000000 DR1: 0000000000000000 DR2:
0000000000000000
[ 12.658843] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7:
0000000000000400
[ 12.664076] PKRU: 55555554
[ 12.667279] Fixing recursive fault but reboot is needed!
This errors occurs on:
ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220607
(ami-04f23e7f9aab5eab8)
# dpkg -s linux-aws
Package: linux-aws
Status: install ok installed
Priority: optional
Section: kernel
Installed-Size: 12
Maintainer: Ubuntu Kernel Team <[email protected]>
Architecture: amd64
Source: linux-meta-aws-5.13
Version: 5.13.0.1028.31~20.04.22
Provides: kernel-testing--linux-aws-5.13--full--aws,
kernel-testing--linux-aws-5.13--full--preferred
Depends: linux-image-aws (= 5.13.0.1028.31~20.04.22), linux-headers-aws (=
5.13.0.1028.31~20.04.22)
Description: Complete Linux kernel for Amazon Web Services (AWS) systems.
This package will always depend on the latest complete Linux kernel available
for Amazon Web Services (AWS) systems.
But it works fine on:
ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220606
(ami-078c065e38be7296e)
# dpkg -s linux-aws
Package: linux-aws
Status: install ok installed
Priority: optional
Section: kernel
Installed-Size: 12
Maintainer: Ubuntu Kernel Team <[email protected]>
Architecture: amd64
Source: linux-meta-aws-5.13
Version: 5.13.0.1025.27~20.04.20
Provides: kernel-testing--linux-aws-5.13--full--aws,
kernel-testing--linux-aws-5.13--full--preferred
Depends: linux-image-aws (= 5.13.0.1025.27~20.04.20), linux-headers-aws (=
5.13.0.1025.27~20.04.20)
Description: Complete Linux kernel for Amazon Web Services (AWS) systems.
This package will always depend on the latest complete Linux kernel available
for Amazon Web Services (AWS) systems.
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux-aws-5.13/+bug/1977919/+subscriptions
--
Mailing list: https://launchpad.net/~kernel-packages
Post to : [email protected]
Unsubscribe : https://launchpad.net/~kernel-packages
More help : https://help.launchpad.net/ListHelp
