I find the reason why ovs commits so many transactions. In bridge_run, ovs-vswitchd would refresh interface statistic if necessary. If the number of ports is big enough such as 500, it will take a long time to check every port's statistic. If statistic changes, ovs- vswitchd will send messages to ovsdb which costs much resource. And if every port's statistic keeps changing, the length and the number of the message will be so big that ovs can't deal with it.
After I delete the call of " iface_refresh_stats(iface)" in bridge_run(void) and complied ovs. ovs-vswitchd won't keep sending messages to ovsdb all the time just cause a little change of ports' statistic. In this situation, I create 500 ports, delete all interfaces at one time. The memory of ovs-vswitchd changes from 6332KB to 7408KB which is much normal. Is there a switch to turn off the collection of statistic of ports? Or if the message of statistic can't be immediately accepted for transmission, and next statistic message of the same port is going to send again, can ovs analyse and free the previous message first? -----邮件原件----- 发件人: Ben Pfaff [mailto:b...@nicira.com] 发送时间: 2013年4月25日 0:51 收件人: pengyi Peng(Yi) 抄送: disc...@openvswitch.org; Liuyongan; Zhangkun (K); dev@openvswitch.org 主题: Re: ????: ????: ????: ????: [ovs-discuss] [ovs]deleting interfaces connnected to ports results in ovs-vswitchd memory growing I'll repeat myself. We haven't observed that behavior before, when we've created and destroyed many interfaces, or at least this is the first time it's been reported to me. Can you figure out why ovs-vswitchd is trying to commit so many transactions? It would be better for it to do avoid doing that. Maybe we can do something to coalesce transactions or to skip some of them. On Wed, Apr 24, 2013 at 12:34:31PM +0000, pengyi Peng(Yi) wrote: > Yes, after backlog comes back to zero, memory is freed, and the memory of > ovs-vswitchd stops growing. > > However, if stream_send always returns -EAGAIN, the messages are held: "The > send function will not block. If no bytes can be immediately accepted for > transmission, it returns -EAGAIN immediately." > In this situation, more messages waits to be sent,more memory is alloced, the > memory of ovs-vswitchd will keep growing until messages are sent successfully > and memory is freed. > > There is such a possibility indeed. > 1. create a bridge (ovs-vsctl add-br br-0). > 2.create a port in br-0 connected to physical interface eth0. > 3.create two Virtual machines which have 10 virtaual interfaces each. > 4.create 2000 ports connected to taps created by tunctl command. > > After an hour, the memory of ovs-vswitchd is about 310MB. And the backlog is > over 200 million. I execute "ifconfig eth0 down". Soon, the messages are sent > successfully and backlog slowly decreases. The ovs-vswitchd momory decreases > too. If I just let it there and didn't ifconfig eth0 down, I believe the > memory of ovs-vswitchd will be a horrible size. Can ovs prevent this > situation? And why stream_send returns -EAGAIN ? > > -----????????----- > ??????: Ben Pfaff [mailto:b...@nicira.com] > ????????: 2013??4??24?? 1:37 > ??????: pengyi Peng(Yi) > ????: disc...@openvswitch.org; Liuyongan; Zhangkun (K); dev@openvswitch.org > ????: Re: ????: ????: ????: [ovs-discuss] [ovs]deleting interfaces connnected > to ports results in ovs-vswitchd memory growing > > On Tue, Apr 23, 2013 at 01:56:17AM +0000, pengyi Peng(Yi) wrote: > > I find that when ovs-vswitchd memory is abnormally growing, backlog > > in struct rpc is growing fast too. Almost every time in jsonrpc_run, > > stream_send returns -EAGAIN, the msg holds and waits next time to > > send. The backlog pluses the strlen of next msg. So the backlog can > > be as big as 1228252. But after all ports are deleted, the backlog > > will slowly decrease and in the end be zero again. However, at this > > time, with top command, the memory of ovs-vswitchd doesn't change at > > all. It is still bigger than the memory before all interfaces are > > deleted. > > None of the above is unexpected. In particular, it is normal for > Unixs implementation of malloc() to hold on to allocated memory even > after it has been freed. > > > And last night, I tested another situation. > > First I create 200 ports which try to connect to interfaces that don't > > exist. > > (ovs-vsctl --timeout=5 --no-wait add-port br-0 tap-no-exist$i > > vlan_mode=access tag=0 -- set interface tap-no-exist$i type=system) > > > > Second, I create 1000 ports connected to interfaces that exist. > > ( tunctl -t vif$i > > ifconfig vif$i up > > ovs-vsctl --timeout=5 --no-wait add-port br-0 vif$i vlan_mode=access tag=0 > > -- set interface vif$i type=system) > > > > After all ports finish connecting to interfaces, the memory of > > ovs-vswitchd is 530832KB. Today it changes to 524444KB. But the > > backlog in struct rpc is still very big(455532592). > > It is unusual to see the backlog grow so high. We haven't observed > that behavior before when we've created and destroyed many interfaces, > or at least this is the first time it's been reported to me. Can you > figure out why ovs-vswitchd is trying to commit so many transactions? > It would be better for it to do avoid doing that. Maybe we can do > something to coalesce transactions or to skip some of them. _______________________________________________ dev mailing list dev@openvswitch.org http://openvswitch.org/mailman/listinfo/dev
