I'll repeat myself. We haven't observed that behavior before, when
we've created and destroyed many interfaces, or at least this is the
first time it's been reported to me. Can you figure out why
ovs-vswitchd is trying to commit so many transactions? It would be
better for it to do avoid doing that. Maybe we can do something to
coalesce transactions or to skip some of them.
On Wed, Apr 24, 2013 at 12:34:31PM +0000, pengyi Peng(Yi) wrote:
> Yes, after backlog comes back to zero, memory is freed, and the memory of
> ovs-vswitchd stops growing.
>
> However, if stream_send always returns -EAGAIN, the messages are held: "The
> send function will not block. If no bytes can be immediately accepted for
> transmission, it returns -EAGAIN immediately."
> In this situation, more messages waits to be sent,more memory is alloced, the
> memory of ovs-vswitchd will keep growing until messages are sent successfully
> and memory is freed.
>
> There is such a possibility indeed.
> 1. create a bridge (ovs-vsctl add-br br-0).
> 2.create a port in br-0 connected to physical interface eth0.
> 3.create two Virtual machines which have 10 virtaual interfaces each.
> 4.create 2000 ports connected to taps created by tunctl command.
>
> After an hour, the memory of ovs-vswitchd is about 310MB. And the backlog is
> over 200 million. I execute "ifconfig eth0 down". Soon, the messages are sent
> successfully and backlog slowly decreases. The ovs-vswitchd momory decreases
> too. If I just let it there and didn't ifconfig eth0 down, I believe the
> memory of ovs-vswitchd will be a horrible size. Can ovs prevent this
> situation? And why stream_send returns -EAGAIN ?
>
> -----????????-----
> ??????: Ben Pfaff [mailto:b...@nicira.com]
> ????????: 2013??4??24?? 1:37
> ??????: pengyi Peng(Yi)
> ????: disc...@openvswitch.org; Liuyongan; Zhangkun (K); dev@openvswitch.org
> ????: Re: ????: ????: ????: [ovs-discuss] [ovs]deleting interfaces connnected
> to ports results in ovs-vswitchd memory growing
>
> On Tue, Apr 23, 2013 at 01:56:17AM +0000, pengyi Peng(Yi) wrote:
> > I find that when ovs-vswitchd memory is abnormally growing, backlog
> > in struct rpc is growing fast too. Almost every time in jsonrpc_run,
> > stream_send returns -EAGAIN, the msg holds and waits next time to
> > send. The backlog pluses the strlen of next msg. So the backlog can
> > be as big as 1228252. But after all ports are deleted, the backlog
> > will slowly decrease and in the end be zero again. However, at this
> > time, with top command, the memory of ovs-vswitchd doesn't change at
> > all. It is still bigger than the memory before all interfaces are
> > deleted.
>
> None of the above is unexpected. In particular, it is normal for
> Unixs implementation of malloc() to hold on to allocated memory even
> after it has been freed.
>
> > And last night, I tested another situation.
> > First I create 200 ports which try to connect to interfaces that don't
> > exist.
> > (ovs-vsctl --timeout=5 --no-wait add-port br-0 tap-no-exist$i
> > vlan_mode=access tag=0 -- set interface tap-no-exist$i type=system)
> >
> > Second, I create 1000 ports connected to interfaces that exist.
> > ( tunctl -t vif$i
> > ifconfig vif$i up
> > ovs-vsctl --timeout=5 --no-wait add-port br-0 vif$i vlan_mode=access tag=0
> > -- set interface vif$i type=system)
> >
> > After all ports finish connecting to interfaces, the memory of
> > ovs-vswitchd is 530832KB. Today it changes to 524444KB. But the
> > backlog in struct rpc is still very big(455532592).
>
> It is unusual to see the backlog grow so high. We haven't observed
> that behavior before when we've created and destroyed many interfaces,
> or at least this is the first time it's been reported to me. Can you
> figure out why ovs-vswitchd is trying to commit so many transactions?
> It would be better for it to do avoid doing that. Maybe we can do
> something to coalesce transactions or to skip some of them.
_______________________________________________
dev mailing list
dev@openvswitch.org
http://openvswitch.org/mailman/listinfo/dev
