On Thu, May 30, 2013 at 04:54:41PM +0300, Michael S. Tsirkin wrote:
> On Tue, May 28, 2013 at 08:25:56AM -0400, Luiz Capitulino wrote:
> > On Tue, 28 May 2013 06:43:04 +0800
> > Amos Kong <ak...@redhat.com> wrote:
> >
> > > On Mon, May 27, 2013 at 09:24:28AM -0400, Luiz Capitulino wrote:
> > > > On Mon, 27 May 2013 09:10:11 -0400
> > > > Luiz Capitulino <lcapitul...@redhat.com> wrote:
> > > >
> > > > > > We use the QMP event to notify management about the mac changing.
> > > > > >
> > > > > > In this thread, we _wrongly_ considered to use qmp approach to delay
> > > > > > the event for avoiding the flooding.
> > > > > >
> > > > > > eg: monitor_protocol_event_throttle(NIC_RX_FILTER_CHANGED, 1000);
> > > > > >
> > > > > > Now we have a solution (using a flag to turn on/off the notify) to
> > > > > > avoid the
> > > > > > flooding, only emit the event if we have no un-read event.
> > > > > >
> > > > > > If we want to (flag is on) emit the event, we wish the event be
> > > > > > sent ASAP
> > > > > > (so event_throttle isn't needed).
> > > > >
> > > > > Unfortunately this doesn't answer my question. I did understand why
> > > > > you're
> > > > > not using the event throttle API (which is because you don't want to
> > > > > slow down
> > > > > the guest, not the QMP client).
> > > > >
> > > > > My point is whether coupling the event with the query command is
> > > > > really
> > > > > justified or even if it really fixes the problem. Two points:
> > > > >
> > > > > 1. Coupling them is bad design, and will probably strike back, as we
> > > > > plan
> > > > > for a better API for events where events can be disabled
> > > >
> > > > I meant we may in the future, for example, introduce the ability to
> > > > disable
> > > > commands (and events). One could argue that the event w/o a query
> > > > command
> > > > is not that useful, as events can be lost. But loosing an event is one
> > > > thing,
> > > > not having it because it got disabled by a side effect is another.
> > >
> > > event_throttle() couples the event in QMP framework, but we use flags
> > > to disabled the event from real source (emit points/senders).
> > >
> > > If we set the evstate->rate to -1, we can ignore the events in
> > > monitor_protocol_event_queue(), but we could not control the event
> > > emitting of each emit point (each nic).
> > >
> > > > But anyway, my main point in this thread is to make sure we at least
> > > > justify having this coupling. Aren't we optimizing prematurely? Aren't
> > > > we optimizing for a corner case? That's what I want to see answered.
> > >
> > > If it's a corner case, we don't need a general API to disable event.
> >
> > If it's a corner case, it's really worth to fix it?
> >
> > I think that what we need a real world test-case to show us we're
> > doing the right thing.
> >
> > > We can disable this event by a flag, and introduce a new API
> > > if we have same request from other events.
> > >
> > > > > 2. Can you actually show the problem does exist so that we ensure
> > > > > this is
> > > > > not premature optimization? Might be a good idea to have this in
> > > > > the
> > > > > commit log
> > > > >
> > > > > > > (which is to couple the event with the query command) is
> > > > > > > appropriate. We're in user-space already, many things could slow
> > > > > > > the guest down apart from the event generation.
> > > > > > >
> > > > > > > Two questions:
> > > > > > >
> > > > > > > 1. Do we know how slow (or how many packets are actually dropped)
> > > > > > > if the mac is changed too often *and* the event is always
> > > > > > > sent?
> > > > > >
> > > > > > We always disable interface first, then change the macaddr.
> > > > > > But we just have patch to allow guest to change macaddr of
> > > > > > virtio-net when the interface is running.
> > > > > >
> > > > > > | commit 2dcd0cce551983afe2f900125457f10bb5d980ae
> > > > > > | Author: Jiri Pirko <jpi...@redhat.com>
> > > > > > | Date: Tue Dec 11 15:33:56 2012 -0500
> > > > > > |
> > > > > > | [virt] virtio_net: allow to change mac when iface is running
> > > > > >
> > > > > > > 2. Does this solution consider what happens if the QMP client
> > > > > > > does
> > > > > > > respond timely to the event by issuing the query-rx-filter
> > > > > > > command?
> > > > > >
> > > > > > We assume that the QMP client (management) cares about the mac
> > > > > > changing
> > > > > > event, and will query the latest rx-filter state and sync to macvtap
> > > > > > device.
> > > > > >
> > > > > > 1) If QMP client respond timely to the event: that's what we
> > > > > > expected :)
> > > > >
> > > > > Won't this slow down the guest? If not, why?
> > >
> > > If guest changes fx-filter configs frequently & management always query
> > > the
> > > event very timely, this will slow down the guest.
> > >
> > > We should detect & process the abnormal behavior from management.
> >
> > That's not abnormal. Management is doing what it should do.
> >
> > Maybe using the event throttle API can solve the mngt side of the problem,
> > but I still think we need a reproducible test-case to ensure we're doing
> > the right thing.
>
> I agree we should make sure this code is tested.
> It's pretty easy: run ifconfig in a loop in guest.
>
> Amos, did you try this? Probably should otherwise
> we don't really know whether the logic works.
With v4 patch (without using event throttle)
1. continually query rx-filter from monitor
# while true; do echo "info rx-filter" | nc -U /tmp/m; done
2. change mac in guest repeatedly
# while true; do
ifconfig eth1 down; ifconfig eth1 hw ether 12:00:00:00:00:00
ifconfig eth1 down; ifconfig eth1 hw ether 14:00:00:00:00:00
done
One time (down if, change mac, up) takes about 3,500,000 ns in guest
some event will be ignored by qemu. I will try to only query when it
gets NIC_RX_FILTER_CHANGE event from QMP monitor, query ASAP.
==> Resource usage:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
16387 root 20 0 2375m 326m 6684 R 104.2 4.2 8:32.16 qemu-system-x86
loop script takes about 10% guest cpu (1 core), guest is not slow
----
If we don't use the flag (same effect as that management taks 0 ns to
response & complete the query after event comming)
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
4317 root 20 0 2377m 285m 6664 R 103.4 3.7 2:06.82 qemu-system-x86
guest is very slow (no response for the keyboard input), output:
clocksource tsc unstable.
> > > Management (qmp client) always respond timely to the event in the
> > > begining. If guest changes rx-filter very frequently & continuous.
> > > Then we increase the evstate->rate, even disable the event.
> > >
> > > In the normal usecase, we should consider packet losing first (caused by
> > > event delay + the delay is used by management to execute the change)
> > >
> > > ---
> > > btw, currently we could not test in real environment. If related
> > > libvirt work finishes, we can evaluate with real delays, packet
> > > losing, etc.
> > >
> > > The worst condition is we could not accept the delay(packet losing),
> > > we need to consider other solution for mac programming of macvtap.
> > >
> > > > > > 2) If QMP client doesn't respond timely to the event: packets might
> > > > > > drop.
> > > > > > If we change mac when the interface is running, we can accept
> > > > > > trivial
> > > > > > packets dropping.
> > > > > >
> > > > > > For second condition, we need to test in real environment when
> > > > > > libvirt
> > > > > > finishs the work of processing events.
> > >
--
Amos.
