Hi,
Every hour we are seeing OVS crash with the following message
ovs-vsctl --version
ovs-vsctl (Open vSwitch) 2.17.6
DB Schema 8.3.0
2023-10-27T00:33:47.163Z|00006|util(handler17)|EMER|../include/openvswitch/ofpbuf.h:194:
assertion offset + size <= b->size failed in ofpbuf_at_assert()
2023-10-27T01:33:47.277Z|00003|util(handler17)|EMER|../include/openvswitch/ofpbuf.h:194:
assertion offset + size <= b->size failed in ofpbuf_at_assert()
2023-10-27T02:33:47.391Z|00003|util(handler17)|EMER|../include/openvswitch/ofpbuf.h:194:
assertion offset + size <= b->size failed in ofpbuf_at_assert()
2023-10-27T03:33:47.503Z|00004|util(handler17)|EMER|../include/openvswitch/ofpbuf.h:194:
assertion offset + size <= b->size failed in ofpbuf_at_assert()
2023-10-27T04:33:47.613Z|00006|util(handler17)|EMER|../include/openvswitch/ofpbuf.h:194:
assertion offset + size <= b->size failed in ofpbuf_at_assert()
2023-10-27T05:33:47.729Z|00003|util(handler17)|EMER|../include/openvswitch/ofpbuf.h:194:
assertion offset + size <= b->size failed in ofpbuf_at_assert()
2023-10-27T06:33:47.841Z|00006|util(handler17)|EMER|../include/openvswitch/ofpbuf.h:194:
assertion offset + size <= b->size failed in ofpbuf_at_assert()
2023-10-27T07:33:47.885Z|00002|util(handler17)|EMER|../include/openvswitch/ofpbuf.h:194:
assertion offset + size <= b->size failed in ofpbuf_at_assert()
2023-10-27T08:33:47.981Z|00002|util(handler17)|EMER|../include/openvswitch/ofpbuf.h:194:
assertion offset + size <= b->size failed in ofpbuf_at_assert()
2023-10-27T09:33:48.040Z|00002|util(handler17)|EMER|../include/openvswitch/ofpbuf.h:194:
assertion offset + size <= b->size failed in ofpbuf_at_assert()
2023-10-27T10:33:48.112Z|00002|util(handler17)|EMER|../include/openvswitch/ofpbuf.h:194:
assertion offset + size <= b->size failed in ofpbuf_at_assert()
2023-10-27T11:33:48.228Z|00001|util(handler17)|EMER|../include/openvswitch/ofpbuf.h:194:
assertion offset + size <= b->size failed in ofpbuf_at_assert()
2023-10-27T12:33:48.249Z|00003|util(handler17)|EMER|../include/openvswitch/ofpbuf.h:194:
assertion offset + size <= b->size failed in ofpbuf_at_assert()
2023-10-27T13:33:48.273Z|00001|util(handler17)|EMER|../include/openvswitch/ofpbuf.h:194:
assertion offset + size <= b->size failed in ofpbuf_at_assert()
2023-10-27T14:33:48.315Z|00005|util(handler17)|EMER|../include/openvswitch/ofpbuf.h:194:
assertion offset + size <= b->size failed in ofpbuf_at_assert()
2023-10-27T15:33:48.433Z|00002|util(handler17)|EMER|../include/openvswitch/ofpbuf.h:194:
assertion offset + size <= b->size failed in ofpbuf_at_assert()
2023-10-27T16:33:48.488Z|00002|util(handler17)|EMER|../include/openvswitch/ofpbuf.h:194:
assertion offset + size <= b->size failed in ofpbuf_at_assert()
2023-10-27T17:33:48.514Z|00002|util(handler17)|EMER|../include/openvswitch/ofpbuf.h:194:
assertion offset + size <= b->size failed in ofpbuf_at_assert()
2023-10-27T18:33:48.630Z|00003|util(handler17)|EMER|../include/openvswitch/ofpbuf.h:194:
assertion offset + size <= b->size failed in ofpbuf_at_assert()
2023-10-27T19:33:48.744Z|00001|util(handler17)|EMER|../include/openvswitch/ofpbuf.h:194:
assertion offset + size <= b->size failed in ofpbuf_at_assert()
2023-10-27T20:33:48.857Z|00001|util(handler17)|EMER|../include/openvswitch/ofpbuf.h:194:
assertion offset + size <= b->size failed in ofpbuf_at_assert()
2023-10-27T21:33:48.973Z|00001|util(handler17)|EMER|../include/openvswitch/ofpbuf.h:194:
assertion offset + size <= b->size failed in ofpbuf_at_assert()
2023-10-27T22:33:49.093Z|00956|util(handler17)|EMER|../include/openvswitch/ofpbuf.h:194:
assertion offset + size <= b->size failed in ofpbuf_at_assert()
gdb shows the following
Using host libthread_db library "/lib64/libthread_db.so.1".
Core was generated by `ovs-vswitchd unix:/var/run/openvswitch/db.sock
-vconsole:emer -vsyslog:err -vfi'.
Program terminated with signal SIGABRT, Aborted.
#0 0x00007f1b7e54854c in __pthread_kill_implementation () from /lib64/libc.so.6
[Current thread is 1 (Thread 0x7f1b6fbfd640 (LWP 1617989))]
(gdb) bt
#0 0x00007f1b7e54854c in __pthread_kill_implementation () from /lib64/libc.so.6
#1 0x00007f1b7e4fbce6 in raise () from /lib64/libc.so.6
#2 0x00007f1b7e4cf7f3 in abort () from /lib64/libc.so.6
#3 0x000055eb8de11d74 in ovs_abort_valist (err_no=err_no@entry=0,
format=format@entry=0x55eb8e01ca18 "%s: assertion %s failed in %s()",
args=args@entry=0x7f1b6fbad030) at ../lib/util.c:444
#4 0x000055eb8de1c335 in vlog_abort_valist (module_=<optimized out>,
message=0x55eb8e01ca18 "%s: assertion %s failed in %s()",
args=args@entry=0x7f1b6fbad030) at ../lib/vlog.c:1249
#5 0x000055eb8de1c3da in vlog_abort (
module=module@entry=0x55eb8e36a4a0 <this_module.lto_priv>,
message=message@entry=0x55eb8e01ca18 "%s: assertion %s failed in %s()")
at ../lib/vlog.c:1263
#6 0x000055eb8de1c41b in ovs_assert_failure (where=<optimized out>,
function=<optimized out>, condition=<optimized out>) at ../lib/util.c:86
#7 0x000055eb8de555b8 in ofpbuf_at_assert (b=0x7f1b3001cb70, b=0x7f1b3001cb70,
offset=16, size=20) at ../include/openvswitch/ofpbuf.h:194
#8 tc_replace_flower (id=<optimized out>, flower=<optimized out>)
--Type <RET> for more, q to quit, c to continue without paging--
at ../lib/tc.c:3285
#9 0x000055eb8de4c5c8 in netdev_tc_flow_put (netdev=0x55eb968e5e50,
match=<optimized out>, actions=<optimized out>, actions_len=<optimized out>,
ufid=<optimized out>, info=<optimized out>, stats=<optimized out>)
at ../lib/netdev-offload-tc.c:2181
#10 0x000055eb8de31078 in netdev_flow_put (stats=<optimized out>,
info=0x7f1b6fbb5710, ufid=<optimized out>, act_len=<optimized out>,
actions=<optimized out>, match=0x7f1b6fbb5910, netdev=0x55eb968e5e50)
at ../lib/netdev-offload.c:259
#11 parse_flow_put (put=0x7f1b6fbb7c30, dpif=<optimized out>)
at ../lib/dpif-netlink.c:2297
#12 try_send_to_netdev (op=0x7f1b6fbb7c28, dpif=<optimized out>)
at ../lib/dpif-netlink.c:2384
#13 dpif_netlink_operate (dpif_=<optimized out>, ops=0x7f1b6fbb6800,
n_ops=<optimized out>, offload_type=DPIF_OFFLOAD_AUTO)
at ../lib/dpif-netlink.c:2455
#14 0x000055eb8dd3adda in dpif_operate (dpif=0x55eb968cab10, ops=0x7f1b6fbb6800,
n_ops=3, offload_type=<optimized out>) at ../lib/dpif.c:1372
--Type <RET> for more, q to quit, c to continue without paging--
#15 0x000055eb8dce6a6b in handle_upcalls (n_upcalls=<optimized out>,
upcalls=0x7f1b6fbdb430, udpif=0x55eb968983d0)
at ../ofproto/ofproto-dpif-upcall.c:1677
#16 recv_upcalls (handler=handler@entry=0x55eb96931558)
at ../ofproto/ofproto-dpif-upcall.c:915
#17 0x000055eb8dce6c44 in udpif_upcall_handler (arg=0x55eb96931558)
at ../ofproto/ofproto-dpif-upcall.c:815
#18 0x000055eb8dddab13 in ovsthread_wrapper (aux_=<optimized out>)
at ../lib/ovs-thread.c:422
#19 0x00007f1b7e546802 in start_thread () from /lib64/libc.so.6
#20 0x00007f1b7e4e6450 in clone3 () from /lib64/libc.so.6
(gdb) q
Attached a core dump file
Can anyone help here?
core.ovs-vswitchd.984.e5ad44fdff9d4094a14ea0434...
<https://drive.google.com/file/d/1G5OfoJygSHK_tfhLaFTn4kG_OjSBE1h5/view?usp=drive_web>
_______________________________________________
discuss mailing list
[email protected]
https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
