** Description changed: + [ Impact ] + Customer reported that this issue happens randomly during their automated regression runs. It's not consistent and hard to reproduce. They have shared a coredump: Program terminated with signal SIGSEGV, Segmentation fault. #0 0x0000555962aa1c34 in link_drop_foreign_config (link=link@entry=0x555963583f30) at ../src/network/networkd-link.c:2741 2741 ../src/network/networkd-link.c: No such file or directory. (gdb) bt #0 0x0000555962aa1c34 in link_drop_foreign_config (link=link@entry=0x555963583f30) at ../src/network/networkd-link.c:2741 #1 0x0000555962aa233d in link_carrier_lost.lto_priv.328 (link=<optimized out>, link=<optimized out>) at ../src/network/networkd-link.c:3462 #2 0x0000555962a8e9b2 in link_update (m=0x5559635702c0, link=<optimized out>) at ../src/network/networkd-link.c:3698 #3 manager_rtnl_process_link (rtnl=<optimized out>, message=0x5559635702c0, userdata=<optimized out>) at ../src/network/networkd-manager.c:713 #4 0x0000555962a48a16 in process_match (m=0x5559635702c0, rtnl=0x55596355d990) at ../src/libsystemd/sd-netlink/sd-netlink.c:388 #5 process_running (ret=0x0, rtnl=0x55596355d990) at ../src/libsystemd/sd-netlink/sd-netlink.c:418 #6 sd_netlink_process (rtnl=0x55596355d990, ret=ret@entry=0x0) at ../src/libsystemd/sd-netlink/sd-netlink.c:452 #7 0x0000555962a48cb3 in time_callback (s=<optimized out>, usec=<optimized out>, userdata=<optimized out>) at ../src/libsystemd/sd-netlink/sd-netlink.c:759 #8 0x0000555962a4dbae in source_dispatch (s=s@entry=0x55596355dce0) at ../src/libsystemd/sd-event/sd-event.c:2311 #9 0x0000555962a4de2a in sd_event_dispatch (e=<optimized out>, e@entry=0x55596355bf70) at ../src/libsystemd/sd-event/sd-event.c:2663 #10 0x0000555962a4dfb9 in sd_event_run (e=<optimized out>, e@entry=0x55596355bf70, timeout=timeout@entry=18446744073709551615) at ../src/libsystemd/sd-event/sd-event.c:2723 #11 0x0000555962a4e1fb in sd_event_loop (e=<optimized out>) at ../src/libsystemd/sd-event/sd-event.c:2744 #12 0x0000555962a223d6 in main (argc=<optimized out>, argv=<optimized out>) at ../src/network/networkd.c:158 - The following patch fixes the issue: - https://github.com/systemd/systemd/commit/b1b0b42e48303134731e017a108c6c334ef5f4c8 - I was able simulate it to happen on gdb, by breaking once the manager is - setup, getting the loopback device and running link_drop_foreign_config - on the loopback device: + This can be fixed by simply null checking link->network before doing anything with it. Upstream has fixed it on this commit: https://github.com/systemd/systemd/commit/b1b0b42e48303134731e017a108c6c334ef5f4c8 + + [ Test Plan ] + + Since this issue is hard to reproduce, I was able trigger it on gdb, by + adding a breakpoint once the manager is setup, getting the loopback + device and running link_drop_foreign_config on the loopback device: (gdb) b src/network/networkd.c:150 Breakpoint 1 at 0x2f38f: file ../src/network/networkd.c, line 150. (gdb) r Starting program: /lib/systemd/systemd-networkd [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". eth0: IPv6 successfully enabled eth0: Gained IPv6LL Breakpoint 1, main (argc=<optimized out>, argv=<optimized out>) at ../src/network/networkd.c:152 152 log_info("Enumeration completed"); (gdb) p *m->links $1 = {b = {hash_ops = 0x55555578f230 <trivial_hash_ops>, {indirect = {storage = 0x2e, hash_key = "\200\376yUUU\000\000\001\000\000\000\000\000\000", n_entries = 1434074992, n_buckets = 21845, idx_lowest_entry = 256, _pad = "\000\000"}, direct = { storage = ".\000\000\000\000\000\000\000\200\376yUUU\000\000\001\000\000\000\000\000\000\000p?zUUU\000\000\000\001\000\000\000\000"}}, type = HASHMAP_TYPE_PLAIN, has_indirect = false, n_direct_entries = 2, from_pool = true}} (gdb) set $mylink = (Link **) malloc(sizeof(Link *)) (gdb) call link_get(m, 1, $mylink) $2 = 0 (gdb) p *$mylink $3 = (Link *) 0x5555557a3f70 (gdb) p **$mylink $4 = {manager = 0x555555796940, n_ref = 2, ifindex = 1, master_ifindex = 0, ifname = 0x5555557a0060 "lo", kind = 0x0, iftype = 772, state_file = 0x55555579c670 "/run/systemd/netif/links/1", mac = {ether_addr_octet = "\000\000\000\000\000"}, ipv6ll_address = {__in6_u = { __u6_addr8 = '\000' <repeats 15 times>, __u6_addr16 = {0, 0, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, mtu = 65536, udev_device = 0x0, flags = 65609, kernel_operstate = 0 '\000', network = 0x0, state = LINK_STATE_UNMANAGED, operstate = LINK_OPERSTATE_CARRIER, address_messages = 0, address_label_messages = 0, route_messages = 0, routing_policy_rule_messages = 0, routing_policy_rule_remove_messages = 0, enslaving = 0, addresses = 0x0, addresses_foreign = 0x555555796120, routes = 0x0, routes_foreign = 0x555555796180, addresses_configured = false, addresses_ready = false, dhcp_client = 0x0, dhcp_lease = 0x0, lease_file = 0x55555579c620 "/run/systemd/netif/leases/1", original_mtu = 0, dhcp4_messages = 0, dhcp4_configured = false, dhcp6_configured = false, ndisc_messages = 0, ndisc_configured = false, ipv4ll = 0x0, ipv4ll_address = false, ipv4ll_route = false, static_routes_configured = false, routing_policy_rules_configured = false, setting_mtu = false, setting_genmode = false, ipv6_mtu_set = false, pool_addresses = 0x0, dhcp_server = 0x0, ndisc = 0x0, ndisc_rdnss = 0x0, ndisc_dnssl = 0x0, radv = 0x0, dhcp6_client = 0x0, rtnl_extended_attrs = true, lldp = 0x0, lldp_file = 0x55555579c6d0 "/run/systemd/netif/lldp/1", lldp_tx_fast = 0, lldp_emit_event_source = 0x0, bound_by_links = 0x0, bound_to_links = 0x0} (gdb) call link_drop_foreign_config(*$mylink) Program received signal SIGSEGV, Segmentation fault. 0x00005555554aec34 in link_drop_foreign_config (link=0x5555557a3f70) at ../src/network/networkd-link.c:2741 warning: Source file is more recent than executable. 2741 } else if (FLAGS_SET(link->network->keep_configuration, KEEP_CONFIGURATION_STATIC)) The program being debugged was signaled while in a function called from GDB. GDB remains in the frame where the signal was received. To change this behavior use "set unwindonsignal on". Evaluation of the expression containing the function (link_drop_foreign_config) will be abandoned. When the function is done executing, GDB will silently stop. - I then tested with the patch applied and it runs fine: + After applying the patch, the issue does not happen: (gdb) call link_drop_foreign_config(*$mylink) $10 = 0 + + [ Where problems could occur ] + + This fix implements a simple null check on a pointer before accessing + its contents, so it does not impose any risks of regressions.
-- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/2024864 Title: systemd-networkd crashes with SIGSEGV Status in systemd: Fix Released Status in Ubuntu Pro: In Progress Status in Ubuntu Pro 18.04 series: In Progress Status in systemd package in Ubuntu: Fix Released Bug description: [ Impact ] Customer reported that this issue happens randomly during their automated regression runs. It's not consistent and hard to reproduce. They have shared a coredump: Program terminated with signal SIGSEGV, Segmentation fault. #0 0x0000555962aa1c34 in link_drop_foreign_config (link=link@entry=0x555963583f30) at ../src/network/networkd-link.c:2741 2741 ../src/network/networkd-link.c: No such file or directory. (gdb) bt #0 0x0000555962aa1c34 in link_drop_foreign_config (link=link@entry=0x555963583f30) at ../src/network/networkd-link.c:2741 #1 0x0000555962aa233d in link_carrier_lost.lto_priv.328 (link=<optimized out>, link=<optimized out>) at ../src/network/networkd-link.c:3462 #2 0x0000555962a8e9b2 in link_update (m=0x5559635702c0, link=<optimized out>) at ../src/network/networkd-link.c:3698 #3 manager_rtnl_process_link (rtnl=<optimized out>, message=0x5559635702c0, userdata=<optimized out>) at ../src/network/networkd-manager.c:713 #4 0x0000555962a48a16 in process_match (m=0x5559635702c0, rtnl=0x55596355d990) at ../src/libsystemd/sd-netlink/sd-netlink.c:388 #5 process_running (ret=0x0, rtnl=0x55596355d990) at ../src/libsystemd/sd-netlink/sd-netlink.c:418 #6 sd_netlink_process (rtnl=0x55596355d990, ret=ret@entry=0x0) at ../src/libsystemd/sd-netlink/sd-netlink.c:452 #7 0x0000555962a48cb3 in time_callback (s=<optimized out>, usec=<optimized out>, userdata=<optimized out>) at ../src/libsystemd/sd-netlink/sd-netlink.c:759 #8 0x0000555962a4dbae in source_dispatch (s=s@entry=0x55596355dce0) at ../src/libsystemd/sd-event/sd-event.c:2311 #9 0x0000555962a4de2a in sd_event_dispatch (e=<optimized out>, e@entry=0x55596355bf70) at ../src/libsystemd/sd-event/sd-event.c:2663 #10 0x0000555962a4dfb9 in sd_event_run (e=<optimized out>, e@entry=0x55596355bf70, timeout=timeout@entry=18446744073709551615) at ../src/libsystemd/sd-event/sd-event.c:2723 #11 0x0000555962a4e1fb in sd_event_loop (e=<optimized out>) at ../src/libsystemd/sd-event/sd-event.c:2744 #12 0x0000555962a223d6 in main (argc=<optimized out>, argv=<optimized out>) at ../src/network/networkd.c:158 This can be fixed by simply null checking link->network before doing anything with it. Upstream has fixed it on this commit: https://github.com/systemd/systemd/commit/b1b0b42e48303134731e017a108c6c334ef5f4c8 [ Test Plan ] Since this issue is hard to reproduce, I was able trigger it on gdb, by adding a breakpoint once the manager is setup, getting the loopback device and running link_drop_foreign_config on the loopback device: (gdb) b src/network/networkd.c:150 Breakpoint 1 at 0x2f38f: file ../src/network/networkd.c, line 150. (gdb) r Starting program: /lib/systemd/systemd-networkd [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". eth0: IPv6 successfully enabled eth0: Gained IPv6LL Breakpoint 1, main (argc=<optimized out>, argv=<optimized out>) at ../src/network/networkd.c:152 152 log_info("Enumeration completed"); (gdb) p *m->links $1 = {b = {hash_ops = 0x55555578f230 <trivial_hash_ops>, {indirect = {storage = 0x2e, hash_key = "\200\376yUUU\000\000\001\000\000\000\000\000\000", n_entries = 1434074992, n_buckets = 21845, idx_lowest_entry = 256, _pad = "\000\000"}, direct = { storage = ".\000\000\000\000\000\000\000\200\376yUUU\000\000\001\000\000\000\000\000\000\000p?zUUU\000\000\000\001\000\000\000\000"}}, type = HASHMAP_TYPE_PLAIN, has_indirect = false, n_direct_entries = 2, from_pool = true}} (gdb) set $mylink = (Link **) malloc(sizeof(Link *)) (gdb) call link_get(m, 1, $mylink) $2 = 0 (gdb) p *$mylink $3 = (Link *) 0x5555557a3f70 (gdb) p **$mylink $4 = {manager = 0x555555796940, n_ref = 2, ifindex = 1, master_ifindex = 0, ifname = 0x5555557a0060 "lo", kind = 0x0, iftype = 772, state_file = 0x55555579c670 "/run/systemd/netif/links/1", mac = {ether_addr_octet = "\000\000\000\000\000"}, ipv6ll_address = {__in6_u = { __u6_addr8 = '\000' <repeats 15 times>, __u6_addr16 = {0, 0, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, mtu = 65536, udev_device = 0x0, flags = 65609, kernel_operstate = 0 '\000', network = 0x0, state = LINK_STATE_UNMANAGED, operstate = LINK_OPERSTATE_CARRIER, address_messages = 0, address_label_messages = 0, route_messages = 0, routing_policy_rule_messages = 0, routing_policy_rule_remove_messages = 0, enslaving = 0, addresses = 0x0, addresses_foreign = 0x555555796120, routes = 0x0, routes_foreign = 0x555555796180, addresses_configured = false, addresses_ready = false, dhcp_client = 0x0, dhcp_lease = 0x0, lease_file = 0x55555579c620 "/run/systemd/netif/leases/1", original_mtu = 0, dhcp4_messages = 0, dhcp4_configured = false, dhcp6_configured = false, ndisc_messages = 0, ndisc_configured = false, ipv4ll = 0x0, ipv4ll_address = false, ipv4ll_route = false, static_routes_configured = false, routing_policy_rules_configured = false, setting_mtu = false, setting_genmode = false, ipv6_mtu_set = false, pool_addresses = 0x0, dhcp_server = 0x0, ndisc = 0x0, ndisc_rdnss = 0x0, ndisc_dnssl = 0x0, radv = 0x0, dhcp6_client = 0x0, rtnl_extended_attrs = true, lldp = 0x0, lldp_file = 0x55555579c6d0 "/run/systemd/netif/lldp/1", lldp_tx_fast = 0, lldp_emit_event_source = 0x0, bound_by_links = 0x0, bound_to_links = 0x0} (gdb) call link_drop_foreign_config(*$mylink) Program received signal SIGSEGV, Segmentation fault. 0x00005555554aec34 in link_drop_foreign_config (link=0x5555557a3f70) at ../src/network/networkd-link.c:2741 warning: Source file is more recent than executable. 2741 } else if (FLAGS_SET(link->network->keep_configuration, KEEP_CONFIGURATION_STATIC)) The program being debugged was signaled while in a function called from GDB. GDB remains in the frame where the signal was received. To change this behavior use "set unwindonsignal on". Evaluation of the expression containing the function (link_drop_foreign_config) will be abandoned. When the function is done executing, GDB will silently stop. After applying the patch, the issue does not happen: (gdb) call link_drop_foreign_config(*$mylink) $10 = 0 [ Where problems could occur ] This fix implements a simple null check on a pointer before accessing its contents, so it does not impose any risks of regressions. To manage notifications about this bug go to: https://bugs.launchpad.net/systemd/+bug/2024864/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
