Example 2: Nov 06 15:34:38 <hostname> kernel: ------------[ cut here ]------------ Nov 06 15:34:38 <hostname> kernel: NETDEV WATCHDOG: eno1 (tg3): transmit queue 0 timed out Nov 06 15:34:38 <hostname> kernel: WARNING: CPU: 1 PID: 0 at net/sched/sch_generic.c:467 dev_watchdog+0x24f/0x260 Nov 06 15:34:38 <hostname> kernel: Modules linked in: rpcsec_gss_krb5 usblp binfmt_misc dm_multipath scsi_dh_rdac scsi_dh_emc scsi_dh_alua ipmi_ssif intel_rapl_msr intel_rapl_common x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm hpilo rapl intel_cstate acpi_ip> Nov 06 15:34:38 <hostname> kernel: ghash_clmulni_intel cryptd drm psmouse libahci xhci_pci tg3 lpc_ich xhci_pci_renesas Nov 06 15:34:38 <hostname> kernel: CPU: 1 PID: 0 Comm: swapper/1 Not tainted 5.11.0-38-generic #42~20.04.1-Ubuntu Nov 06 15:34:38 <hostname> kernel: Hardware name: HP ProLiant MicroServer Gen8, BIOS J06 04/04/2019 Nov 06 15:34:38 <hostname> kernel: RIP: 0010:dev_watchdog+0x24f/0x260 Nov 06 15:34:38 <hostname> kernel: Code: 07 78 fd ff eb ab 4c 89 ff c6 05 33 22 ee 00 01 e8 26 3c fa ff 44 89 e9 4c 89 fe 48 c7 c7 f0 42 c9 8f 48 89 c2 e8 cf 5a 16 00 <0f> 0b eb 8c 66 66 2e 0f 1f 84 00 00 00 00 00 66 90 0f 1f 44 00 00 Nov 06 15:34:38 <hostname> kernel: RSP: 0018:ffffad17002c4e88 EFLAGS: 00010282 Nov 06 15:34:38 <hostname> kernel: RAX: 0000000000000000 RBX: ffff93cd93e7bd00 RCX: 0000000000000000 Nov 06 15:34:38 <hostname> kernel: RDX: ffff93d07aa68a20 RSI: ffff93d07aa58ac0 RDI: 0000000000000300 Nov 06 15:34:38 <hostname> kernel: RBP: ffffad17002c4eb8 R08: ffff93d07aa58ac0 R09: ffffad17002c4c48 Nov 06 15:34:38 <hostname> kernel: R10: 0000000000cb9cd0 R11: 0000000000cb9db8 R12: 0000000000000005 Nov 06 15:34:38 <hostname> kernel: R13: 0000000000000000 R14: ffff93cd94af04c0 R15: ffff93cd94af0000 Nov 06 15:34:38 <hostname> kernel: FS: 0000000000000000(0000) GS:ffff93d07aa40000(0000) knlGS:0000000000000000 Nov 06 15:34:38 <hostname> kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Nov 06 15:34:38 <hostname> kernel: CR2: 00007f41920eed38 CR3: 00000000be610001 CR4: 00000000001706e0 Nov 06 15:34:38 <hostname> kernel: Call Trace: Nov 06 15:34:38 <hostname> kernel: <IRQ> Nov 06 15:34:38 <hostname> kernel: ? pfifo_fast_enqueue+0x150/0x150 Nov 06 15:34:38 <hostname> kernel: call_timer_fn+0x2e/0x100 Nov 06 15:34:38 <hostname> kernel: __run_timers.part.0+0x1e0/0x250 Nov 06 15:34:38 <hostname> kernel: ? lapic_next_deadline+0x2c/0x40 Nov 06 15:34:38 <hostname> kernel: ? clockevents_program_event+0x8f/0xe0 Nov 06 15:34:38 <hostname> kernel: run_timer_softirq+0x2a/0x50 Nov 06 15:34:38 <hostname> kernel: __do_softirq+0xe0/0x29b Nov 06 15:34:38 <hostname> kernel: asm_call_irq_on_stack+0x12/0x20 Nov 06 15:34:38 <hostname> kernel: </IRQ> Nov 06 15:34:38 <hostname> kernel: do_softirq_own_stack+0x3d/0x50 Nov 06 15:34:38 <hostname> kernel: irq_exit_rcu+0xa4/0xb0 Nov 06 15:34:38 <hostname> kernel: sysvec_apic_timer_interrupt+0x3d/0x90 Nov 06 15:34:38 <hostname> kernel: asm_sysvec_apic_timer_interrupt+0x12/0x20 Nov 06 15:34:38 <hostname> kernel: RIP: 0010:cpuidle_enter_state+0xdf/0x350 Nov 06 15:34:38 <hostname> kernel: Code: ff e8 95 a8 77 ff 80 7d d7 00 74 17 9c 58 0f 1f 44 00 00 f6 c4 02 0f 85 52 02 00 00 31 ff e8 c8 52 7e ff fb 66 0f 1f 44 00 00 <45> 85 ff 0f 88 e5 00 00 00 49 63 c7 4c 2b 75 c8 48 8d 14 40 48 8d Nov 06 15:34:38 <hostname> kernel: RSP: 0018:ffffad1700097e48 EFLAGS: 00000246 Nov 06 15:34:38 <hostname> kernel: RAX: ffff93d07aa6c380 RBX: 0000000000000004 RCX: 000000000000001f Nov 06 15:34:38 <hostname> kernel: RDX: 0000000000000000 RSI: 0000000037c75227 RDI: 0000000000000000 Nov 06 15:34:38 <hostname> kernel: RBP: ffffad1700097e80 R08: 00027d42516aa349 R09: 0000000000000e3b Nov 06 15:34:38 <hostname> kernel: R10: ffff93d07aa6afa4 R11: ffff93d07aa6af84 R12: ffff93d07aa77600 Nov 06 15:34:38 <hostname> kernel: R13: ffffffff9004d8e0 R14: 00027d42516aa349 R15: 0000000000000004 Nov 06 15:34:38 <hostname> kernel: ? cpuidle_enter_state+0xbb/0x350 Nov 06 15:34:38 <hostname> kernel: cpuidle_enter+0x2e/0x40 Nov 06 15:34:38 <hostname> kernel: call_cpuidle+0x23/0x40 Nov 06 15:34:38 <hostname> kernel: do_idle+0x1df/0x260 Nov 06 15:34:38 <hostname> kernel: cpu_startup_entry+0x20/0x30 Nov 06 15:34:38 <hostname> kernel: start_secondary+0x11f/0x160 Nov 06 15:34:38 <hostname> kernel: secondary_startup_64_no_verify+0xc2/0xcb Nov 06 15:34:38 <hostname> kernel: ---[ end trace 28a5883990ef746a ]--- Nov 06 15:34:38 <hostname> kernel: tg3 0000:03:00.0 eno1: transmit timed out, resetting Nov 06 15:34:39 <hostname> kernel: tg3 0000:03:00.0 eno1: 0x00000000: 0x165f14e4, 0x00100446, 0x02000000, 0x00800010 Nov 06 15:34:39 <hostname> kernel: tg3 0000:03:00.0 eno1: 0x00000010: 0xfabf000c, 0x00000000, 0xfabe000c, 0x00000000 Nov 06 15:34:39 <hostname> kernel: tg3 0000:03:00.0 eno1: 0x00000020: 0xfabd000c, 0x00000000, 0x00000000, 0x2133103c Nov 06 15:34:39 <hostname> kernel: tg3 0000:03:00.0 eno1: 0x00000030: 0x00000000, 0x00000048, 0x00000000, 0x0000010a Nov 06 15:34:39 <hostname> kernel: tg3 0000:03:00.0 eno1: 0x00000040: 0x00000000, 0x1a000000, 0xc8035001, 0x64002008 Nov 06 15:34:39 <hostname> kernel: tg3 0000:03:00.0 eno1: 0x00000050: 0x00005803, 0x00000000, 0x0086a005, 0x00000000 Nov 06 15:34:39 <hostname> kernel: tg3 0000:03:00.0 eno1: 0x00000060: 0x00000000, 0x00000000, 0xf0000298, 0x00380081 Nov 06 15:34:39 <hostname> kernel: tg3 0000:03:00.0 eno1: 0x00000070: 0x000710b0, 0xf50eacfb, 0x00000000, 0x00000000 Nov 06 15:34:39 <hostname> kernel: tg3 0000:03:00.0 eno1: 0x00000080: 0x165f14e4, 0x800000f8, 0x00000000, 0x00000e6b Nov 06 15:34:39 <hostname> kernel: tg3 0000:03:00.0 eno1: 0x00000090: 0x00000000, 0x00000000, 0x00000000, 0x000005ab Nov 06 15:34:39 <hostname> kernel: tg3 0000:03:00.0 eno1: 0x000000a0: 0x8010ac11, 0x00000004, 0x00001004, 0x00020010 Nov 06 15:34:39 <hostname> kernel: tg3 0000:03:00.0 eno1: 0x000000b0: 0x10008d81, 0x00105406, 0x0004cc22, 0x10120040 Nov 06 15:34:39 <hostname> kernel: tg3 0000:03:00.0 eno1: 0x000000d0: 0x0000001f, 0x00000000, 0x00000000, 0x00000001 Nov 06 15:34:39 <hostname> kernel: tg3 0000:03:00.0 eno1: 0x000000f0: 0x00000000, 0x05720000, 0x00000000, 0x0023e9b9 Nov 06 15:34:39 <hostname> kernel: tg3 0000:03:00.0 eno1: 0x00000100: 0x13c10001, 0x00100000, 0x00100000, 0x00000020 Nov 06 15:34:39 <hostname> kernel: tg3 0000:03:00.0 eno1: 0x00000110: 0x00002000, 0x00000000, 0x000000a0, 0x04000001 Nov 06 15:34:39 <hostname> kernel: tg3 0000:03:00.0 eno1: 0x00000120: 0x0000000f, 0x03070000, 0x00000000, 0x00000000 Nov 06 15:34:39 <hostname> kernel: tg3 0000:03:00.0 eno1: 0x00000130: 0x00000000, 0x00000000, 0x00000000, 0x15010003 Nov 06 15:34:39 <hostname> kernel: tg3 0000:03:00.0 eno1: 0x00000140: 0xa99a6830, 0x00003464, 0x00000000, 0x00000000 Nov 06 15:34:39 <hostname> kernel: tg3 0000:03:00.0 eno1: 0x00000150: 0x16010004, 0x00000000, 0x0007811b, 0x00000000 Nov 06 15:34:39 <hostname> kernel: tg3 0000:03:00.0 eno1: 0x00000160: 0x00010002, 0x00000000, 0x00000000, 0x00000000 Nov 06 15:34:39 <hostname> kernel: tg3 0000:03:00.0 eno1: 0x00000170: 0x00000000, 0x800000ff, 0x00000000, 0x00000000 Nov 06 15:34:39 <hostname> kernel: tg3 0000:03:00.0 eno1: 0x00000200: 0x00000000, 0x1a000000, 0x00000000, 0x95000000 Nov 06 15:34:39 <hostname> kernel: tg3 0000:03:00.0 eno1: 0x00000210: 0x00000000, 0x72000000, 0x00000000, 0x00000001 Nov 06 15:34:39 <hostname> kernel: tg3 0000:03:00.0 eno1: 0x00000220: 0x00000000, 0x00000001, 0x00000000, 0x00000000 Nov 06 15:34:39 <hostname> kernel: tg3 0000:03:00.0 eno1: 0x00000260: 0x00000000, 0x00000000, 0x00000000, 0x000005ab Nov 06 15:34:39 <hostname> kernel: tg3 0000:03:00.0 eno1: 0x00000280: 0x00000000, 0x00000e6b, 0x00000000, 0x00000678 Nov 06 15:34:39 <hostname> kernel: tg3 0000:03:00.0 eno1: 0x00000300: 0x00000000, 0x00000156, 0x00000000, 0x00000000 Nov 06 15:34:39 <hostname> kernel: tg3 0000:03:00.0 eno1: 0x00000400: 0x18e04808, 0x00400000, 0x00001000, 0x00000880 Nov 06 15:34:39 <hostname> kernel: tg3 0000:03:00.0 eno1: 0x00000410: 0x00003464, 0xa99a6830, 0x00003464, 0xa99a6830 Nov 06 15:34:39 <hostname> kernel: tg3 0000:03:00.0 eno1: 0x00000420: 0x00003464, 0xa99a6830, 0x00003464, 0xa99a6830 Nov 06 15:34:39 <hostname> kernel: tg3 0000:03:00.0 eno1: 0x00000430: 0x00000400, 0x00000000, 0x0000010c, 0x000005f2 Nov 06 15:34:39 <hostname> kernel: tg3 0000:03:00.0 eno1: 0x00000440: 0x00000000, 0x00000000, 0x00000000, 0x082a3800 Nov 06 15:34:39 <hostname> kernel: tg3 0000:03:00.0 eno1: 0x00000450: 0x00000001, 0x00008000, 0x00000000, 0x00000102 Nov 06 15:34:39 <hostname> kernel: tg3 0000:03:00.0 eno1: 0x00000460: 0x00000008, 0x00002620, 0x01ff0002, 0x00000000 Nov 06 15:34:39 <hostname> kernel: tg3 0000:03:00.0 eno1: 0x00000470: 0xa0040010, 0x00000000, 0x30000000, 0x50004042 Nov 06 15:34:39 <hostname> kernel: tg3 0000:03:00.0 eno1: 0x00000480: 0x42000000, 0x7fffffff, 0x06000004, 0x7fffffff Nov 06 15:34:39 <hostname> kernel: tg3 0000:03:00.0 eno1: 0x00000500: 0x00000008, 0x00000002, 0x00000000, 0x00000000 Nov 06 15:34:39 <hostname> kernel: tg3 0000:03:00.0 eno1: 0x00000540: 0x00003464, 0xa99a6833, 0x00000000, 0x00000000 Nov 06 15:34:39 <hostname> kernel: tg3 0000:03:00.0 eno1: 0x00000590: 0x00e00000, 0x00000000, 0x00000000, 0x00000000 Nov 06 15:34:39 <hostname> kernel: tg3 0000:03:00.0 eno1: 0x000005b0: 0x00000000, 0x00000008, 0x00000000, 0x00000000 Nov 06 15:34:39 <hostname> kernel: tg3 0000:03:00.0 eno1: 0x000005c0: 0xbf6f6b18, 0xfe117de7, 0x00000000, 0x00000000 Nov 06 15:34:39 <hostname> kernel: tg3 0000:03:00.0 eno1: 0x00000600: 0xffffffff, 0x00f80011, 0x00000000, 0x00001f04 Nov 06 15:34:39 <hostname> kernel: tg3 0000:03:00.0 eno1: 0x00000610: 0xffffffff, 0x00000000, 0x07c00004, 0x00000000 Nov 06 15:34:39 <hostname> kernel: tg3 0000:03:00.0 eno1: 0x00000620: 0x00000040, 0x00000000, 0x00000000, 0x00000000 Nov 06 15:34:39 <hostname> kernel: tg3 0000:03:00.0 eno1: 0x00000630: 0x01010101, 0x01010101, 0x01010101, 0x01010101 Nov 06 15:34:39 <hostname> kernel: tg3 0000:03:00.0 eno1: 0x00000640: 0x01010101, 0x01010101, 0x01010101, 0x01010101 Nov 06 15:34:39 <hostname> kernel: tg3 0000:03:00.0 eno1: 0x00000650: 0x01010101, 0x01010101, 0x01010101, 0x01010101 Nov 06 15:34:39 <hostname> kernel: tg3 0000:03:00.0 eno1: 0x00000660: 0x01010101, 0x01010101, 0x01010101, 0x01010101 Nov 06 15:34:39 <hostname> kernel: tg3 0000:03:00.0 eno1: 0x00000670: 0x68faca10, 0x0bbe005d, 0xa1c32caa, 0x96de1e36 Nov 06 15:34:39 <hostname> kernel: tg3 0000:03:00.0 eno1: 0x00000680: 0xfcd453ff, 0x9e1be86d, 0x32240e0e, 0x9daa8a1a Nov 06 15:34:39 <hostname> kernel: tg3 0000:03:00.0 eno1: 0x00000690: 0x00346869, 0xeb74b14f, 0x00000000, 0x00000000 Nov 06 15:34:39 <hostname> kernel: tg3 0000:03:00.0 eno1: 0x000006c0: 0x00000000, 0x00000000, 0x04000000, 0x00000000 Nov 06 15:34:39 <hostname> kernel: tg3 0000:03:00.0 eno1: 0x000006d0: 0x00000001, 0x00000000, 0x00000000, 0x00000000 Nov 06 15:34:39 <hostname> kernel: tg3 0000:03:00.0 eno1: 0x00000800: 0x00000000, 0xffffffff, 0x00000000, 0x00000000 Nov 06 15:34:39 <hostname> kernel: tg3 0000:03:00.0 eno1: 0x00000810: 0x00000000, 0xffffffff, 0x00000000, 0x00000000 Nov 06 15:34:39 <hostname> kernel: tg3 0000:03:00.0 eno1: 0x00000820: 0x00000000, 0x00000000, 0xffffffff, 0x00000000 Nov 06 15:34:39 <hostname> kernel: tg3 0000:03:00.0 eno1: 0x00000830: 0x00000000, 0xffffffff, 0xffffffff, 0xffffffff
-- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1950046 Title: tg3 transmit timed out, resetting Status in linux package in Ubuntu: New Bug description: Hi, since earlier this year (sorry, I don't have an exact date) I started to encounter regular network hangs on multiple HPE servers that I manage. The hangs occur on high network load and are followed by a message like "tg3 0000:02:00.0 eno1: transmit timed out, resetting". The problem already started on the original 20.04 kernel and still occurs on the current HWE kernel. Affected machines are HPE ProLiant ML30 Gen9, DL20 Gen9, and Microserver Gen8. The frequency of the problem seems to increase as time passes. There is a long standing upstream ticket at https://bugzilla.kernel.org/show_bug.cgi?id=12877. I'll post log messages in the comments. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1950046/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
