Re: [Qemu-devel] [PATCH 5/6] add visitor for parsing hz[KMG] input string
On Tue, Dec 04, 2012 at 05:34:42PM -0200, Eduardo Habkost wrote:
> From: Igor Mammedov
>
> Signed-off-by: Igor Mammedov
> Acked-by: Andreas Färber
> ---
> qapi/qapi-visit-core.c | 11 +++
> qapi/qapi-visit-core.h | 2 ++
> qapi/string-input-visitor.c | 22 ++
> 3 files changed, 35 insertions(+)
>
> diff --git a/qapi/qapi-visit-core.c b/qapi/qapi-visit-core.c
> index 7a82b63..5c8705e 100644
> --- a/qapi/qapi-visit-core.c
> +++ b/qapi/qapi-visit-core.c
> @@ -311,3 +311,14 @@ void input_type_enum(Visitor *v, int *obj, const char
> *strings[],
> g_free(enum_str);
> *obj = value;
> }
> +
> +void visit_type_freq(Visitor *v, int64_t *obj, const char *name, Error
> **errp)
> +{
> +if (!error_is_set(errp)) {
> +if (v->type_freq) {
> +v->type_freq(v, obj, name, errp);
> +} else {
> +v->type_int(v, obj, name, errp);
> +}
> +}
> +}
> diff --git a/qapi/qapi-visit-core.h b/qapi/qapi-visit-core.h
> index 60aceda..e5e7dd7 100644
> --- a/qapi/qapi-visit-core.h
> +++ b/qapi/qapi-visit-core.h
> @@ -62,6 +62,7 @@ struct Visitor
> void (*type_int64)(Visitor *v, int64_t *obj, const char *name, Error
> **errp);
> /* visit_type_size() falls back to (*type_uint64)() if type_size is
> unset */
> void (*type_size)(Visitor *v, uint64_t *obj, const char *name, Error
> **errp);
> +void (*type_freq)(Visitor *v, int64_t *obj, const char *name, Error
> **errp);
> };
>
> void visit_start_handle(Visitor *v, void **obj, const char *kind,
> @@ -91,5 +92,6 @@ void visit_type_size(Visitor *v, uint64_t *obj, const char
> *name, Error **errp);
> void visit_type_bool(Visitor *v, bool *obj, const char *name, Error **errp);
> void visit_type_str(Visitor *v, char **obj, const char *name, Error **errp);
> void visit_type_number(Visitor *v, double *obj, const char *name, Error
> **errp);
> +void visit_type_freq(Visitor *v, int64_t *obj, const char *name, Error
> **errp);
>
> #endif
> diff --git a/qapi/string-input-visitor.c b/qapi/string-input-visitor.c
> index 497eb9a..74fe395 100644
> --- a/qapi/string-input-visitor.c
> +++ b/qapi/string-input-visitor.c
> @@ -110,6 +110,27 @@ static void parse_start_optional(Visitor *v, bool
> *present,
> *present = true;
> }
>
> +static void parse_type_freq(Visitor *v, int64_t *obj, const char *name,
> +Error **errp)
> +{
> +StringInputVisitor *siv = DO_UPCAST(StringInputVisitor, visitor, v);
> +char *endp = (char *) siv->string;
> +long long val = 0;
> +
> +errno = 0;
If this is for strtosz_suffix_unit(), it looks like this is already
handled internally and can be dropped. Relic from a previous version
that called strtod() directly maybe?
If that's not the case, I think it should be fixed in the called function[s]
rather than for each caller.
> +if (siv->string) {
> +val = strtosz_suffix_unit(siv->string, &endp,
> + STRTOSZ_DEFSUFFIX_B, 1000);
Specifying 1000 as the unit size will make "1M" == 1000^2 as opposed to
1024^2. Is this intentional?
> +}
> +if (!siv->string || val == -1 || *endp) {
> +error_set(errp, QERR_INVALID_PARAMETER_VALUE, name,
> + "a value representable as a non-negative int64");
> +return;
> +}
> +
> +*obj = val;
> +}
> +
> Visitor *string_input_get_visitor(StringInputVisitor *v)
> {
> return &v->visitor;
> @@ -132,6 +153,7 @@ StringInputVisitor *string_input_visitor_new(const char
> *str)
> v->visitor.type_str = parse_type_str;
> v->visitor.type_number = parse_type_number;
> v->visitor.start_optional = parse_start_optional;
> +v->visitor.type_freq = parse_type_freq;
This seems applicable for stuff like -m 1G and potentionally some other
properties. We can make it generic later, but if we do end up re-spinning
consider something like ->type_unit_suffixed_int(). But I'm not against
leaving as is for now since I can't think of a better name for it atm :)
Whatever we call it, based on a recent discussion here:
http://lists.gnu.org/archive/html/qemu-devel/2012-11/msg02872.html
we should have a corresponding implementation in qapi-dealloc-visitor.c.
In this case You can just do:
v->visitor.type_freq = qapi_dealloc_type_int;
There aren't any problems in the current code, we just want to avoid relying on
fallbacks for the dealloc case in general because in some situations the
underlying sizes of the C types don't match and this can cause problems down
the road for the dealloc visitor even though it's okay for other visitor
implementations.
>
> v->string = str;
> return v;
> --
> 1.7.11.7
>
>
Re: [Qemu-devel] [PATCH 5/6] add visitor for parsing hz[KMG] input string
On Wed, Dec 05, 2012 at 05:21:50PM -0200, Eduardo Habkost wrote:
> On Wed, Dec 05, 2012 at 11:52:29AM -0600, mdroth wrote:
> > On Tue, Dec 04, 2012 at 05:34:42PM -0200, Eduardo Habkost wrote:
> [...]
> > > diff --git a/qapi/string-input-visitor.c b/qapi/string-input-visitor.c
> > > index 497eb9a..74fe395 100644
> > > --- a/qapi/string-input-visitor.c
> > > +++ b/qapi/string-input-visitor.c
> > > @@ -110,6 +110,27 @@ static void parse_start_optional(Visitor *v, bool
> > > *present,
> > > *present = true;
> > > }
> > >
> > > +static void parse_type_freq(Visitor *v, int64_t *obj, const char *name,
> > > +Error **errp)
> > > +{
> > > +StringInputVisitor *siv = DO_UPCAST(StringInputVisitor, visitor, v);
> > > +char *endp = (char *) siv->string;
> > > +long long val = 0;
> > > +
> > > +errno = 0;
> >
> > If this is for strtosz_suffix_unit(), it looks like this is already
> > handled internally and can be dropped. Relic from a previous version
> > that called strtod() directly maybe?
> >
> > If that's not the case, I think it should be fixed in the called function[s]
> > rather than for each caller.
> >
> > > +if (siv->string) {
> > > +val = strtosz_suffix_unit(siv->string, &endp,
> > > + STRTOSZ_DEFSUFFIX_B, 1000);
> >
> > Specifying 1000 as the unit size will make "1M" == 1000^2 as opposed to
> > 1024^2. Is this intentional?
>
> I don't know if this is a good idea for a generalx-use visitor, but this is
> the
> current behavior of "-cpu ...,tsc_freq=1M", that we need to keep for
> compatibility, somehow.
>
> >
> > > +}
> > > +if (!siv->string || val == -1 || *endp) {
> > > +error_set(errp, QERR_INVALID_PARAMETER_VALUE, name,
> > > + "a value representable as a non-negative int64");
> > > +return;
> > > +}
> > > +
> > > +*obj = val;
> > > +}
> > > +
> > > Visitor *string_input_get_visitor(StringInputVisitor *v)
> > > {
> > > return &v->visitor;
> > > @@ -132,6 +153,7 @@ StringInputVisitor *string_input_visitor_new(const
> > > char *str)
> > > v->visitor.type_str = parse_type_str;
> > > v->visitor.type_number = parse_type_number;
> > > v->visitor.start_optional = parse_start_optional;
> > > +v->visitor.type_freq = parse_type_freq;
> >
> > This seems applicable for stuff like -m 1G and potentionally some other
> > properties. We can make it generic later, but if we do end up re-spinning
> > consider something like ->type_unit_suffixed_int(). But I'm not against
> > leaving as is for now since I can't think of a better name for it atm :)
>
> I thought the visitor was going to support things like "1GHz", but if it's
> just
> a "suffixed int" with no unit, the name could be changed, I guess.
>
> But we still have the 1000 vs 1024 problem. On the one hand, it would be
> interesting to make make it consistent and use the same base everywhere.
> On the other hand, I assume we have different command-line options using
> different bases and we'll need to keep compatibility.
If we were to map it to a QAPI schema definition at some point, I'd
imagine it looking something like:
{ 'field_name': { 'type': 'suffixed_int', 'unit': 1000 } }
with 'unit' defaulting to 1024 if unspecified. (I have some patches
floating around as part of the QIDL series for doing more descriptive
QAPI definitions)
>
> Must all visitor functions have the
> "function(Visitor *v, obj, const char *name, Error **errp)" signature,
> or can we add additional type-specific arguments? (so we could tell
> the visitor if the default base should be 1000 or 1024)
Visitor functions don't necessarilly have to follow the same basic
signature, so it would be okay to declare it with an extra 'unit' param
and pass that in. We could still hide this behind a visit_type_freq()
wrapper in open-coded visitors as well, I'd just prefer to make the bits
we add to qapi-visit-core.c more general where possible to keep unit
tests and code generation stuff somewhat sane for the core api.
>
> --
> Eduardo
>
Re: [Qemu-devel] [RFC 2/3] virtio-balloon: re-enable balloon stats
On Tue, Dec 04, 2012 at 01:04:47PM -0200, Luiz Capitulino wrote:
> The statistics are now available through device properties via a
> polling mechanism. First, a client has to enable polling, then it
> can query the stats.
>
> The following control properties are introduced:
>
> o stats-polling-interval: a value greater than zero enables polling
>in the specified interval (in seconds). When value equals zero,
>polling is disabled. If polling is already enabled and a value
>greater than zero is written, the polling interval time is changed
>
> o stats-last-update: last stats update timestamp, in seconds.
>
> The following stats properties are introduced:
>
> o stat-swap-in
> o stat-swap-out
> o stat-major-faults
> o stat-minor-faults
> o stat-free-memory
> o stat-total-memory
>
> All values are in bytes. A value of -1 means that the statistic isn't
> available right now.
>
> FIXME: Can balloon_stats_poll_cb(), balloon_stats_set_poll_interval(),
>virtio_balloon_handle_output() can run in parallel?
>
> XXX: Should we return an error instead of -1? Might require a specific
> error. Although this is not exactly a failure...
>
> Signed-off-by: Luiz Capitulino
> ---
>
> NOTE: Anthony suggested having a bool to enable/disable polling, but I prefer
> to let the client specify the polling interval. I can revisit this,
> though.
>
> hw/virtio-balloon.c | 156
> +++-
> 1 file changed, 155 insertions(+), 1 deletion(-)
>
> diff --git a/hw/virtio-balloon.c b/hw/virtio-balloon.c
> index 4398025..06af18f 100644
> --- a/hw/virtio-balloon.c
> +++ b/hw/virtio-balloon.c
> @@ -22,6 +22,8 @@
> #include "virtio-balloon.h"
> #include "kvm.h"
> #include "exec-memory.h"
> +#include "qemu-timer.h"
> +#include "qapi/qapi-visit-core.h"
>
> #if defined(__linux__)
> #include
> @@ -36,6 +38,9 @@ typedef struct VirtIOBalloon
> uint64_t stats[VIRTIO_BALLOON_S_NR];
> VirtQueueElement stats_vq_elem;
> size_t stats_vq_offset;
> +QEMUTimer *stats_timer;
> +int64_t stats_last_update;
> +int64_t stats_poll_interval;
> DeviceState *qdev;
> } VirtIOBalloon;
>
> @@ -53,6 +58,16 @@ static void balloon_page(void *addr, int deflate)
> #endif
> }
>
> +static const char *balloon_stat_names[] = {
> + [VIRTIO_BALLOON_S_SWAP_IN] = "stat-swap-in",
> + [VIRTIO_BALLOON_S_SWAP_OUT] = "stat-swap-out",
> + [VIRTIO_BALLOON_S_MAJFLT] = "stat-major-faults",
> + [VIRTIO_BALLOON_S_MINFLT] = "stat-minor-faults",
> + [VIRTIO_BALLOON_S_MEMFREE] = "stat-free-memory",
> + [VIRTIO_BALLOON_S_MEMTOT] = "stat-total-memory",
> + [VIRTIO_BALLOON_S_NR] = NULL
> +};
> +
> /*
> * reset_stats - Mark all items in the stats array as unset
> *
> @@ -67,6 +82,119 @@ static inline void reset_stats(VirtIOBalloon *dev)
> for (i = 0; i < VIRTIO_BALLOON_S_NR; dev->stats[i++] = -1);
> }
>
> +static bool balloon_stats_supported(const VirtIOBalloon *s)
> +{
> +return s->vdev.guest_features & (1 << VIRTIO_BALLOON_F_STATS_VQ);
> +}
> +
> +static bool balloon_stats_enabled(const VirtIOBalloon *s)
> +{
> +return s->stats_poll_interval > 0;
> +}
> +
> +static void balloon_stats_disable_timer(VirtIOBalloon *s)
> +{
> +if (balloon_stats_enabled(s)) {
> +qemu_del_timer(s->stats_timer);
> +qemu_free_timer(s->stats_timer);
> +s->stats_timer = NULL;
> +s->stats_poll_interval = 0;
> +}
> +}
> +
> +static void balloon_stats_change_timer(VirtIOBalloon *s, int secs)
> +{
> +qemu_mod_timer(s->stats_timer, qemu_get_clock_ms(vm_clock) + secs *
> 1000);
> +}
> +
> +static void balloon_stats_poll_cb(void *opaque)
> +{
> +VirtIOBalloon *s = opaque;
> +
> +virtqueue_push(s->svq, &s->stats_vq_elem, s->stats_vq_offset);
> +virtio_notify(&s->vdev, s->svq);
I think we'll want to add some logic to avoid the potential for
re-pushing an elem we've already processed, as I think that violates the
virtio spec. In the past the monitor blocked us from doing this but with
timer-driven requests I think it's a possibility now.
But the general approach seems sane to me and the code looks to
be in decent shape.
> --
> 1.8.0
>
>
Re: [Qemu-devel] [RFC 3/3] docs: document virtio-balloon stats
On Thu, Dec 06, 2012 at 01:31:09PM -0200, Luiz Capitulino wrote:
> On Thu, 6 Dec 2012 13:24:11 +
> "Daniel P. Berrange" wrote:
>
> > On Tue, Dec 04, 2012 at 01:04:48PM -0200, Luiz Capitulino wrote:
> > > Signed-off-by: Luiz Capitulino
> > > ---
> > > docs/virtio-balloon-stats.txt | 73
> > > +++
> > > 1 file changed, 73 insertions(+)
> > > create mode 100644 docs/virtio-balloon-stats.txt
> > >
> > > diff --git a/docs/virtio-balloon-stats.txt b/docs/virtio-balloon-stats.txt
> > > new file mode 100644
> > > index 000..7e7ddc4
> > > --- /dev/null
> > > +++ b/docs/virtio-balloon-stats.txt
> > > @@ -0,0 +1,73 @@
> > > +virtio balloon memory statistics
> > > +
> > > +
> > > +The virtio balloon driver supports guest memory statistics reporting.
> > > These
> > > +statistics are available to QEMU users as QOM (QEMU Obejct Model) device
> > > +properties via a polling mechanism.
> > > +
> > > +Basically, clients have to enable polling. Then they can query the
> > > available
> > > +statistics.
> > > +
> > > +There are two control properties and six memory statistics from the
> > > guest.
> > > +
> > > +The control properties are:
> > > +
> > > + o stats-polling-interval: a value greater than zero enables polling
> > > + in the specified interval (in seconds). When value equals zero,
> > > + polling is disabled. If polling is already enabled and a value
> > > + greater than zero is written, the polling interval time is changed
> > > +
> > > + o stats-last-update: last stats update timestamp, in seconds
> > > +
> > > +The memory statistics are:
> > > +
> > > + o stat-swap-in
> > > + o stat-swap-out
> > > + o stat-major-faults
> > > + o stat-minor-faults
> > > + o stat-free-memory
> > > + o stat-total-memory
> > > +
> > > +All values are in bytes. A value of -1 means that the statistic isn't
> > > +available right now.
> > > +
> > > +Here are a few examples. The virtio-balloon device is assumed to be in
> > > the
> > > +'/machine/peripheral-anon/device[1]' QOM path.
> > > +
> > > +Enable polling with 2 seconds interval:
> > > +
> > > +{ "execute": "qom-set",
> > > + "arguments": { "path": "/machine/peripheral-anon/device[1]",
> > > + "property": "stats-polling-interval", "value": 2 } }
> > > +
> > > +{ "return": {} }
> > > +
> > > +Change polling to 10 seconds:
> > > +
> > > +{ "execute": "qom-set",
> > > + "arguments": { "path": "/machine/peripheral-anon/device[1]",
> > > + "property": "stats-polling-interval", "value": 10 } }
> > > +
> > > +{ "return": {} }
> > > +
> > > +Get last update timestamp and free memory stat:
> > > +
> > > +{ "execute": "qom-get",
> > > + "arguments": { "path": "/machine/peripheral-anon/device[1]",
> > > + "property": "stats-last-update" } }
> > > +
> > > +{ "return": 1354629634 }
> > > +
> > > +{ "execute": "qom-get",
> > > + "arguments": { "path": "/machine/peripheral-anon/device[1]",
> > > + "property": "stat-free-memory" } }
> > > +
> > > +{ "return": 845115392 }
> > > +
> > > +Disable polling:
> > > +
> > > +{ "execute": "qom-set",
> > > + "arguments": { "path": "/machine/peripheral-anon/device[1]",
> > > + "property": "stats-polling-interval", "value": 0 } }
> > > +
> > > +{ "return": {} }
> >
> >
> > What sort of performance implications are there for enabling polling of
> > virtio stats. Is it the kind of thing that it is reasonable to just
> > enable for all VMs on a 10 second interval, so we'll always have stats
> > available without having to have thought to enable them ahead of time ?
>
> I can't think of any performance implications. Would be nice to get a
> second opinion from the CC'ed people though.
Pushing/popping/processing one vq entry every 10 seconds should be
virtually unnoticeable given that virtio-net/blk do this much more frequently
with much more processing overhead per entry on a relatively idle guest. So
performance-wise, I don't think it's an issue. As to whether or not it
*should* be enabled by default I'm not so sure, but if it actually simplifies
things on that end I'd say it's worth it if the alternatives are
cumbersome.
>
> > eg, the use case I'm wondering is that someone comes along and just
> > runs 'virsh memstats $DOMAIN' and wants to see the latest data
> > right away.
> >
> > I'm not suggesting that libvirt would be actually asking QEMU for the
> > stats every 10 seconds. Only that libvirt tells QEMU to collect them.
> > Then libvirt can just ask for them whenver someone wants them.
>
> Note that once you enable polling, the balloon driver will immediately make
> a request to the guest, that is, it won't wait the specified time interval to
> send the first request.
>
> So, the first call to virsh memstats could start polling and also poll for it
> (although you do need to be prepared for the case where the guest doesn't
> respond).
>
> Also, you could c
Re: [Qemu-devel] [PATCH 2/2] target-i386: use visit_type_unit_suffixed_int() to parse tsc_freq property value
On Thu, Dec 06, 2012 at 10:12:05PM +0100, Igor Mammedov wrote:
> Signed-off-by: Igor Mammedov
Reviewed-by: Michael Roth
> ---
> v2:
>- replace visit_type_freq() with visit_type_unit_suffixed_int()
> in x86_cpuid_set_tsc_freq()
> ---
> target-i386/cpu.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/target-i386/cpu.c b/target-i386/cpu.c
> index c6c2ca0..b7f0aba 100644
> --- a/target-i386/cpu.c
> +++ b/target-i386/cpu.c
> @@ -1195,7 +1195,7 @@ static void x86_cpuid_set_tsc_freq(Object *obj, Visitor
> *v, void *opaque,
> const int64_t max = INT64_MAX;
> int64_t value;
>
> -visit_type_int(v, &value, name, errp);
> +visit_type_unit_suffixed_int(v, &value, name, 1000, errp);
> if (error_is_set(errp)) {
> return;
> }
> --
> 1.7.11.7
>
Re: [Qemu-devel] [PATCH 1/2] qapi: add visitor for parsing int[KMGT] input string
On Thu, Dec 06, 2012 at 10:12:04PM +0100, Igor Mammedov wrote:
> Caller of visit_type_unit_suffixed_int() will have to specify
> value of 'K' suffix via unit argument.
> For Kbytes it's 1024, for Khz it's 1000.
>
> Signed-off-by: Igor Mammedov
Reviewed-by: Michael Roth
> ---
> v2:
> - convert type_freq to type_unit_suffixed_int.
> - provide qapi_dealloc_type_unit_suffixed_int() impl.
> ---
> qapi/qapi-dealloc-visitor.c | 7 +++
> qapi/qapi-visit-core.c | 13 +
> qapi/qapi-visit-core.h | 8
> qapi/string-input-visitor.c | 22 ++
> 4 files changed, 50 insertions(+)
>
> diff --git a/qapi/qapi-dealloc-visitor.c b/qapi/qapi-dealloc-visitor.c
> index 75214e7..57e662c 100644
> --- a/qapi/qapi-dealloc-visitor.c
> +++ b/qapi/qapi-dealloc-visitor.c
> @@ -143,6 +143,12 @@ static void qapi_dealloc_type_enum(Visitor *v, int *obj,
> const char *strings[],
> {
> }
>
> +static void qapi_dealloc_type_unit_suffixed_int(Visitor *v, int64_t *obj,
> +const char *name,
> +const int unit, Error **errp)
> +{
> +}
> +
> Visitor *qapi_dealloc_get_visitor(QapiDeallocVisitor *v)
> {
> return &v->visitor;
> @@ -170,6 +176,7 @@ QapiDeallocVisitor *qapi_dealloc_visitor_new(void)
> v->visitor.type_str = qapi_dealloc_type_str;
> v->visitor.type_number = qapi_dealloc_type_number;
> v->visitor.type_size = qapi_dealloc_type_size;
> +v->visitor.type_unit_suffixed_int = qapi_dealloc_type_unit_suffixed_int;
>
> QTAILQ_INIT(&v->stack);
>
> diff --git a/qapi/qapi-visit-core.c b/qapi/qapi-visit-core.c
> index 7a82b63..dcbc1a9 100644
> --- a/qapi/qapi-visit-core.c
> +++ b/qapi/qapi-visit-core.c
> @@ -311,3 +311,16 @@ void input_type_enum(Visitor *v, int *obj, const char
> *strings[],
> g_free(enum_str);
> *obj = value;
> }
> +
> +void visit_type_unit_suffixed_int(Visitor *v, int64_t *obj, const char *name,
> + const int unit, Error **errp)
> +{
> +if (!error_is_set(errp)) {
> +return;
> +}
> +if (v->type_unit_suffixed_int) {
> +v->type_unit_suffixed_int(v, obj, name, unit, errp);
> +} else {
> +visit_type_int64(v, obj, name, errp);
> +}
> +}
> diff --git a/qapi/qapi-visit-core.h b/qapi/qapi-visit-core.h
> index 60aceda..04e690a 100644
> --- a/qapi/qapi-visit-core.h
> +++ b/qapi/qapi-visit-core.h
> @@ -62,6 +62,12 @@ struct Visitor
> void (*type_int64)(Visitor *v, int64_t *obj, const char *name, Error
> **errp);
> /* visit_type_size() falls back to (*type_uint64)() if type_size is
> unset */
> void (*type_size)(Visitor *v, uint64_t *obj, const char *name, Error
> **errp);
> +/*
> + * visit_unit_suffixed_int() falls back to (*type_int64)()
> + * if type_unit_suffixed_int is unset
> +*/
> +void (*type_unit_suffixed_int)(Visitor *v, int64_t *obj, const char
> *name,
> + const int unit, Error **errp);
> };
>
> void visit_start_handle(Visitor *v, void **obj, const char *kind,
> @@ -91,5 +97,7 @@ void visit_type_size(Visitor *v, uint64_t *obj, const char
> *name, Error **errp);
> void visit_type_bool(Visitor *v, bool *obj, const char *name, Error **errp);
> void visit_type_str(Visitor *v, char **obj, const char *name, Error **errp);
> void visit_type_number(Visitor *v, double *obj, const char *name, Error
> **errp);
> +void visit_type_unit_suffixed_int(Visitor *v, int64_t *obj, const char *name,
> + const int unit, Error **errp);
>
> #endif
> diff --git a/qapi/string-input-visitor.c b/qapi/string-input-visitor.c
> index 497eb9a..d2bd154 100644
> --- a/qapi/string-input-visitor.c
> +++ b/qapi/string-input-visitor.c
> @@ -110,6 +110,27 @@ static void parse_start_optional(Visitor *v, bool
> *present,
> *present = true;
> }
>
> +static void parse_type_unit_suffixed_int(Visitor *v, int64_t *obj,
> + const char *name, const int unit,
> + Error **errp)
> +{
> +StringInputVisitor *siv = DO_UPCAST(StringInputVisitor, visitor, v);
> +char *endp = (char *) siv->string;
> +long long val = 0;
> +
> +if (siv->string) {
> +val = strtosz_suffix_unit(siv->string, &endp,
> + STRTOSZ_DEFSUFFIX_B, unit);
> +}
> +if (!siv->string || val == -1 || *endp) {
> +error_set(errp, QERR_INVALID_PARAMETER_VALUE, name,
> + "a value representable as a non-negative int64");
> +return;
> +}
> +
> +*obj = val;
> +}
> +
> Visitor *string_input_get_visitor(StringInputVisitor *v)
> {
> return &v->visitor;
> @@ -132,6 +153,7 @@ StringInputVisitor *string_input_visitor_new(const char
> *str)
> v->visitor.type_str = parse_type_str;
> v->visitor.type_number = parse_type_number;
> v->visitor.sta
Re: [Qemu-devel] [PATCH v7 2/2] qemu-ga: sample fsfreeze hooks
On Fri, Dec 07, 2012 at 11:55:16AM -0200, Luiz Capitulino wrote:
> On Fri, 07 Dec 2012 17:39:32 +0900
> Tomoki Sekiyama wrote:
>
> > Adds sample hook scripts for --fsfreeze-hook option of qemu-ga.
> > - fsfreeze-hook : execute scripts in fsfreeze-hook.d/
> > - fsfreeze-hook.d/mysql-flush.sh.sample : quiesce MySQL before snapshot
> >
> > Signed-off-by: Tomoki Sekiyama
> > ---
> > .gitignore |1
> > Makefile |2 -
> > scripts/qemu-guest-agent/fsfreeze-hook | 33
> > .../fsfreeze-hook.d/mysql-flush.sh.sample | 55
> >
> > 4 files changed, 90 insertions(+), 1 deletion(-)
> > create mode 100755 scripts/qemu-guest-agent/fsfreeze-hook
> > create mode 100755
> > scripts/qemu-guest-agent/fsfreeze-hook.d/mysql-flush.sh.sample
> >
> > diff --git a/.gitignore b/.gitignore
> > index bd6ba1c..286822d 100644
> > --- a/.gitignore
> > +++ b/.gitignore
> > @@ -93,3 +93,4 @@ cscope.*
> > tags
> > TAGS
> > *~
> > +!scripts/qemu-guest-agent/fsfreeze-hook.d
>
> Why? Do we expect to have *~ files in there?
I think default vim configurations will copy to ~ prior to
editing.
>
> > diff --git a/Makefile b/Makefile
> > index 9ecbcbb..466dcd7 100644
> > --- a/Makefile
> > +++ b/Makefile
> > @@ -248,7 +248,7 @@ clean:
> > # avoid old build problems by removing potentially incorrect old files
> > rm -f config.mak op-i386.h opc-i386.h gen-op-i386.h op-arm.h opc-arm.h
> > gen-op-arm.h
> > rm -f qemu-options.def
> > - find . -name '*.[od]' -exec rm -f {} +
> > + find . -name '*.[od]' -type f -exec rm -f {} +
>
> What does this change have to do with this patch?
Looks like it'll try (and fail, due to lack of -r option to rm) to
delete the fsfreeze-hook.d directory on `make clean`. For in-tree builds
at least.
Re: [Qemu-devel] [PATCH 2/2] target-i386: use visit_type_unit_suffixed_int() to parse tsc_freq property value
On Mon, Dec 10, 2012 at 05:13:45PM +0100, Igor Mammedov wrote:
> On Fri, 7 Dec 2012 18:09:06 -0200
> Eduardo Habkost wrote:
>
> > On Fri, Dec 07, 2012 at 08:00:09PM +0100, Andreas Färber wrote:
> > > Am 06.12.2012 22:12, schrieb Igor Mammedov:
> > > > Signed-off-by: Igor Mammedov
> > > > ---
> > > > v2:
> > > >- replace visit_type_freq() with visit_type_unit_suffixed_int()
> > > > in x86_cpuid_set_tsc_freq()
> > > > ---
> > > > target-i386/cpu.c | 2 +-
> > > > 1 file changed, 1 insertion(+), 1 deletion(-)
> > > >
> > > > diff --git a/target-i386/cpu.c b/target-i386/cpu.c
> > > > index c6c2ca0..b7f0aba 100644
> > > > --- a/target-i386/cpu.c
> > > > +++ b/target-i386/cpu.c
> > > > @@ -1195,7 +1195,7 @@ static void x86_cpuid_set_tsc_freq(Object *obj,
> > > > Visitor *v, void *opaque, const int64_t max = INT64_MAX;
> > > > int64_t value;
> > > >
> > > > -visit_type_int(v, &value, name, errp);
> > > > +visit_type_unit_suffixed_int(v, &value, name, 1000, errp);
> > > > if (error_is_set(errp)) {
> > > > return;
> > > > }
> > >
> > > This trivial usage is fine obviously. But since this series set out to
> > > make things more generic I am missing at least one use case for 1024.
> > > Does nothing like that exist in qdev-properties.c or so already?
> >
> > cutils.c has:
> >
> > int64_t strtosz_suffix(const char *nptr, char **end, const char
> > default_suffix) {
> > return strtosz_suffix_unit(nptr, end, default_suffix, 1024);
> > }
> >
> > $ git grep -w strtosz_suffix
> > [...]
> > qapi/opts-visitor.c:val = strtosz_suffix(opt->str ? opt->str : "",
> > &endptr, qemu-img.c:sval = strtosz_suffix(argv[optind++], &end,
> > STRTOSZ_DEFSUFFIX_B); qemu-img.c:sval = strtosz_suffix(optarg,
> > &end, STRTOSZ_DEFSUFFIX_B);
> >
> > The opts-visitor.c match, in turn, is inside opts_type_size(), that's the
> > ->type_size method of OptsVisitor. There are many 'size' elements inside
> > qapi-schema.json.
> >
> > I don't see any code using visit_type_size() directly, but I see two users
> > of type 'size' on qapi-schema.json: NetdevTapOptions and NetdevDumpOptions.
> >
> > I didn't know that we already had a visitor method using the suffixed-int
> > parsing code. Should we change the visit_type_size() code to be to use use
> > the new generic ->type_suffixed_int method and kill ->type_size?
>
> If there isn't strong opposition to do it in incremental way,
> I'd prefer for these patches go in first.
>
> And then later fix users of visit_type_size() to use type_suffixed_int() or
> maybe have a new type_suffixed_uint() so that size could be represented as
> uint64_t instead of int64_t as it's now. That would require to
> rewrite strtosz_* and its callers a bit.
I think that seems reasonable. Between the 2 that should give us a nice
way to support [KMGx] suffixes for options in a backward-compatible way.
Shame that users don't have a well-established way to specify base 2 vs.
10, but so long as we document our choice for each option that should
cover most cases well enough.
Re: [Qemu-devel] [PATCH 1/2] qapi: add visitor for parsing int[KMGT] input string
On Mon, Dec 10, 2012 at 05:01:38PM +0100, Igor Mammedov wrote:
> On Fri, 07 Dec 2012 19:57:35 +0100
> Andreas Färber wrote:
>
> > Am 06.12.2012 22:12, schrieb Igor Mammedov:
> > > Caller of visit_type_unit_suffixed_int() will have to specify
> > > value of 'K' suffix via unit argument.
> > > For Kbytes it's 1024, for Khz it's 1000.
> > >
> > > Signed-off-by: Igor Mammedov
> > > ---
> > > v2:
> > > - convert type_freq to type_unit_suffixed_int.
> > > - provide qapi_dealloc_type_unit_suffixed_int() impl.
> > > ---
> > > qapi/qapi-dealloc-visitor.c | 7 +++
> > > qapi/qapi-visit-core.c | 13 +
> > > qapi/qapi-visit-core.h | 8
> > > qapi/string-input-visitor.c | 22 ++
> > > 4 files changed, 50 insertions(+)
> > >
> > > diff --git a/qapi/qapi-dealloc-visitor.c b/qapi/qapi-dealloc-visitor.c
> > > index 75214e7..57e662c 100644
> > > --- a/qapi/qapi-dealloc-visitor.c
> > > +++ b/qapi/qapi-dealloc-visitor.c
> > > @@ -143,6 +143,12 @@ static void qapi_dealloc_type_enum(Visitor *v, int
> > > *obj, const char *strings[], {
> > > }
> > >
> > > +static void qapi_dealloc_type_unit_suffixed_int(Visitor *v, int64_t *obj,
> > > +const char *name,
> > > +const int unit, Error
> > > **errp) +{
> > > +}
> > > +
> > > Visitor *qapi_dealloc_get_visitor(QapiDeallocVisitor *v)
> > > {
> > > return &v->visitor;
> > > @@ -170,6 +176,7 @@ QapiDeallocVisitor *qapi_dealloc_visitor_new(void)
> > > v->visitor.type_str = qapi_dealloc_type_str;
> > > v->visitor.type_number = qapi_dealloc_type_number;
> > > v->visitor.type_size = qapi_dealloc_type_size;
> > > +v->visitor.type_unit_suffixed_int =
> > > qapi_dealloc_type_unit_suffixed_int;
> > > QTAILQ_INIT(&v->stack);
> > >
> > > diff --git a/qapi/qapi-visit-core.c b/qapi/qapi-visit-core.c
> > > index 7a82b63..dcbc1a9 100644
> > > --- a/qapi/qapi-visit-core.c
> > > +++ b/qapi/qapi-visit-core.c
> > > @@ -311,3 +311,16 @@ void input_type_enum(Visitor *v, int *obj, const
> > > char *strings[], g_free(enum_str);
> > > *obj = value;
> > > }
> > > +
> > > +void visit_type_unit_suffixed_int(Visitor *v, int64_t *obj, const char
> > > *name,
> > > + const int unit, Error **errp)
> > > +{
> > > +if (!error_is_set(errp)) {
> >
> > if (error_is_set(errp)) {
> Thanks, I'll fix it.
>
> > > +return;
> > > +}
> > > +if (v->type_unit_suffixed_int) {
> > > +v->type_unit_suffixed_int(v, obj, name, unit, errp);
> > > +} else {
> > > +visit_type_int64(v, obj, name, errp);
> > > +}
> > > +}
> > > diff --git a/qapi/qapi-visit-core.h b/qapi/qapi-visit-core.h
> > > index 60aceda..04e690a 100644
> > > --- a/qapi/qapi-visit-core.h
> > > +++ b/qapi/qapi-visit-core.h
> > > @@ -62,6 +62,12 @@ struct Visitor
> > > void (*type_int64)(Visitor *v, int64_t *obj, const char *name, Error
> > > **errp); /* visit_type_size() falls back to (*type_uint64)() if type_size
> > > is unset */ void (*type_size)(Visitor *v, uint64_t *obj, const char
> > > *name, Error **errp);
> > > +/*
> > > + * visit_unit_suffixed_int() falls back to (*type_int64)()
> > > + * if type_unit_suffixed_int is unset
> > > +*/
> >
> > Indentation is one off.
> ditto
>
> >
> > > +void (*type_unit_suffixed_int)(Visitor *v, int64_t *obj, const char
> > > *name,
> > > + const int unit, Error **errp);
> >
> > Are we expecting differently suffixed ints? Otherwise we could
> > optionally shorten to type_suffixed_int (but that probably still doesn't
> > fit within one comment line ;)).
> Not with current implementation. I'll shorten it as you've suggested.
>
> >
> > > };
> > >
> > > void visit_start_handle(Visitor *v, void **obj, const char *kind,
> > > @@ -91,5 +97,7 @@ void visit_type_size(Visitor *v, uint64_t *obj, const
> > > char *name, Error **errp); void visit_type_bool(Visitor *v, bool *obj,
> > > const char *name, Error **errp); void visit_type_str(Visitor *v, char
> > > **obj, const char *name, Error **errp); void visit_type_number(Visitor
> > > *v, double *obj, const char *name, Error **errp); +void
> > > visit_type_unit_suffixed_int(Visitor *v, int64_t *obj, const char *name,
> > > + const int unit, Error **errp);
> > >
> > > #endif
> > > diff --git a/qapi/string-input-visitor.c b/qapi/string-input-visitor.c
> > > index 497eb9a..d2bd154 100644
> > > --- a/qapi/string-input-visitor.c
> > > +++ b/qapi/string-input-visitor.c
> > > @@ -110,6 +110,27 @@ static void parse_start_optional(Visitor *v, bool
> > > *present, *present = true;
> > > }
> > >
> > > +static void parse_type_unit_suffixed_int(Visitor *v, int64_t *obj,
> > > + const char *name, const int
> > > unit,
> > > + Error **errp)
> >
Re: [Qemu-devel] [PATCH 0/3] re-enable balloon stats
On Tue, Dec 11, 2012 at 03:14:02PM +, Dietmar Maurer wrote: > > > > I'm not sure I like this for two reasons. First, there will be cases > > > > where the user doesn't want this to be enabled. Second, we'll be > > > > forcing an interval on users. > > > > > > So when should we set the stats-polling-interval? I first thought I > > > simply set that at VM start, but that triggers an error: > > > > > > "guest doesn't support balloon stats" > > > > > > because the balloon driver is not loaded. > > > > Yes, it's required to have the balloon driver loaded. The stats are > > reported by > > it. > > That does not really answers my question. So you think the management > framework should start the VM, and then wait until the balloon driver is > loaded? That sound very clumsy to me. > > I mean, I just want to set the polling interval. Why does that need the > balloon drive loaded > inside the guest (polling is done by the host)? I agree. Should lack of a balloon module disable the timer completely, or just silently fail? Management can always reference stats-last-update to check that stats are being reported properly. -1 would mean driver was never loaded, longer update intervals might mean guest was rebooted/suspended/etc, but in both cases it makes sense that the timer still try to fetch stats, and the determination of what stats are valid/invalid be left to management. Alternatively, we can hook into the feature negotiation path and defer the actual timer start until the required feature is negotiated, and disable it (temporarilly or permanently based on guest behavior) somewhere in the reset path). I'm not sure this will keep it from running during hibernate/sleep...maybe hook into runstate changes?
Re: [Qemu-devel] [PATCH 0/3] re-enable balloon stats
On Tue, Dec 11, 2012 at 04:28:40PM -0200, Luiz Capitulino wrote: > On Tue, 11 Dec 2012 11:38:08 -0600 > mdroth wrote: > > > On Tue, Dec 11, 2012 at 03:14:02PM +, Dietmar Maurer wrote: > > > > > > I'm not sure I like this for two reasons. First, there will be cases > > > > > > where the user doesn't want this to be enabled. Second, we'll be > > > > > > forcing an interval on users. > > > > > > > > > > So when should we set the stats-polling-interval? I first thought I > > > > > simply set that at VM start, but that triggers an error: > > > > > > > > > > "guest doesn't support balloon stats" > > > > > > > > > > because the balloon driver is not loaded. > > > > > > > > Yes, it's required to have the balloon driver loaded. The stats are > > > > reported by > > > > it. > > > > > > That does not really answers my question. So you think the management > > > framework should start the VM, and then wait until the balloon driver is > > > loaded? That sound very clumsy to me. > > > > > > I mean, I just want to set the polling interval. Why does that need the > > > balloon drive loaded > > > inside the guest (polling is done by the host)? > > > > I agree. Should lack of a balloon module disable the timer > > completely, or just silently fail? Management can always reference > > stats-last-update to check that stats are being reported properly. -1 would > > mean driver was never loaded, longer update intervals might mean guest > > was rebooted/suspended/etc, but in both cases it makes sense that the > > timer still try to fetch stats, and the determination of what stats are > > valid/invalid be left to management. > > I could move the check for the stats feature bit from the function that > enables polling to the timer callback. This would solve Dietmar's use-case, > but it would silently fail if the feature is never negotiated (as you said > above). > > > Alternatively, we can hook into the feature negotiation path and > > defer the actual timer start until the required feature is negotiated, > > That's also possible, and helps not wasting resources. > > > and disable it (temporarilly or permanently based on guest behavior) > > somewhere in the reset path). I'm not sure this will keep it from > > running during hibernate/sleep...maybe hook into runstate changes? > > Then it starts to get somewhat complicated. > Agreed...that's only if we want to go to extremes to avoid running the timer unecessarily, but probably not worth it. I like your idea of just having the timer callback fail silently if the feature isn't enabled. And personally, I think just having a stale value for stats-last-update is sufficient for error checking, but we can squirrel away an error in a stats-last-error field if we want something more descriptive for clients, and just wipe it out on the next successful update. Not sure whether we should clear the stats upon error or not, I think it might actually be useful to leave them. If the guest crashed because we ballooned away too much memory or something for instance we'd be able to see the last value we logged.
[Qemu-devel] [ANNOUNCE] QEMU 1.2.2 Stable released
Hi everyone, I am pleased to announce that the QEMU v1.2.2 stable release is now available at: http://wiki.qemu.org/download/qemu-1.2.2.tar.bz2 The official stable-1.2 repository has also been updated to v1.2.2: http://git.qemu.org/?p=qemu-stable-1.2.git;a=summary This release includes 43 build/bug fixes. This is the last planned stable release for the QEMU 1.2 series, as I'll be moving on to supporting the QEMU 1.3 series. Any parties interested in maintaining the 1.2 series further however are free to contact myself or Anthony (on CC) to volunteer. Thanks! CHANGE LOG: update VERSION for v1.2.2 (Michael Roth) e1000: Discard packets that are too long if !SBP and !LPE (Michael Contreras) stream: fix ratelimit_set_speed (Dietmar Maurer) usb: fail usbdevice_create() when there is no USB bus (Stefan Hajnoczi) qxl: reload memslots after migration, when qxl is in UNDEFINED mode (Yonit Halperin) virtio-scsi: Fix subtle (guest) endian bug (David Gibson) virtio-scsi: Fix some endian bugs with virtio-scsi (David Gibson) iscsi: do not assume device is zero initialized (Peter Lieven) iscsi: fix deadlock during login (Peter Lieven) iscsi: fix segfault in url parsing (Peter Lieven) qapi: fix qapi_dealloc_type_size parameter type (Bruce Rogers) qapi: handle visitor->type_size() in QapiDeallocVisitor (Stefan Hajnoczi) qom: fix refcount of non-heap-allocated objects (Paolo Bonzini) PPC: Fix missing TRACE exception (Julio Guerra) hmp: do not crash on invalid SCSI hotplug (Paolo Bonzini) qom: dynamic_cast of NULL is always NULL (Paolo Bonzini) block: Fix regression for MinGW (assertion caused by short string) (Stefan Weil) tci: Fix type of tci_read_label (Richard Henderson) qcow2: Fix refcount table size calculation (Kevin Wolf) configure: avoid compiler warning in pipe2 detection (Bruce Rogers) target-openrisc: remove conflicting definitions from cpu.h (Aurelien Jarno) tcg/arm: fix cross-endian qemu_st16 (Aurelien Jarno) tcg/arm: fix TLB access in qemu-ld/st ops (Aurelien Jarno) target-mips: fix wrong microMIPS opcode encoding (陳韋任 (Wei-Ren Chen)) mips/malta: fix CBUS UART interrupt pin (Aurelien Jarno) nbd: fixes to read-only handling (Paolo Bonzini) m68k: Return semihosting errno values correctly (Meador Inge) tools: initialize main loop before block layer (Paolo Bonzini) xhci: fix usb name in caps (Gerd Hoffmann) target-sparc64: disable VGA cirrus (Aurelien Jarno) PPC: Bamboo: Fix memory size DT property (Alexander Graf) s390x: fix -initrd in virtio machine (Alexander Graf) memory: fix rendering of a region obscured by another (Avi Kivity) e1000: drop check_rxov, always treat RX ring with RDH == RDT as empty (Dmitry Fleytman) target-i386: Allow tsc-frequency to be larger then 2.147G (Don Slutz) hw: Fix return value check for bdrv_read, bdrv_write (Stefan Weil) rtc: fix overflow in mktimegm (Paolo Bonzini) qxl: always update displaysurface on resize (Gerd Hoffmann) hw/qxl: qxl_dirty_surfaces: use uintptr_t (Alon Levy) uhci: Raise interrupt when requested even for non active tds (Hans de Goede) vnc: fix "info vnc" with "-vnc ..., reverse=on" (Paolo Bonzini) ui/vnc: Only report/use TIGHT_PNG encoding if enabled. (Joel Martin) fix CONFIG_QEMU_HELPERDIR generation again (Michael Tokarev) configure: Fix CONFIG_QEMU_HELPERDIR generation (Jan Kiszka)
Re: [Qemu-devel] [RFC] Time resync by qemu-ga
On Fri, Dec 28, 2012 at 12:29:15AM +0800, Lei Li wrote:
> Hi guys,
>
> I am working on the time drift issue as background info here.
>
> http://mid.gmane.org/87pq5r5otp@codemonkey.ws
>
> As Anthony proposed, one part of the solutions is that we want
> to add a qemu-ga command to resync the guest clock by reading the
> wallclock time when the tick overflow the limit to the number
> of missed ticks.
>
> When I implement on the qemu-ga side, I got some problems and
> I'd like to send this RFC out and ask for suggestions and comments
> to make sure I am in the right direction before I send out the code.
>
> The proposed interface for qemu-ga command as link here:
>
> http://wiki.qemu.org/Features/GuestAgent/UsefulCommands
>
> This maybe easier to implement, but I think this interface may
> have problem, like the delta issue. So I am trying to implement
> it like below.
>
> The interface looks like:
>
> { 'command': 'guest-resync-time' }
>
> The way it works:
>
> When calling this command in the host, qemu-ga would try to
> get the wallclock time in the host by handling guest->host
> commands, and then write this time to the guest hardware.
>
> But looks like we don't have the way to get host information
> now, since qemu-ga just handle host->guest commands (Correct
> me if I am wrong). To solve this, we should add support to
> get host information by handling guest->host commands by qemu-ga
> first.
>
> Please correct me if I am wrong. And suggestions and comments
> are very appreciated, specifically the way to handle quest->host
> commands by qemu-ga since I am still struggling with this.
I don't think the guest actually needs to talk to the host here, it's
okay to punt some of the work to the host/management. Since we just
need to be able to set the guest time in cases where NTP is disabled, or
the guest is too far behind for the NTP clients to adjust the time (Windows
has a configurable threshold for instance), all we really need to
know to do this is the host time in UTC (which is presumably correct), and
the guest offset from that (timezone basically: UTC, UTC+4, etc.).
So to set the time we'd just need a guest-set-time interface as proposed in
the wiki, and to know what to set it to we would probably need something
like:
{'execute':'guest-get-time'}
{'seconds': 1343862720, 'nanoseconds': 12300, 'utc-offset': -6}
the seconds/nanoseconds in this case is mostly for completeness, the
utc-offset is all we really need to translate the host time into the
appropriate guest time for the set-time command.
>
> Thanks!
>
> --
> Lei
>
Re: [Qemu-devel] Using QEMU guest agent to run programs from guest path
On Mon, Dec 31, 2012 at 06:14:59PM -0200, Erlon Cruz wrote:
> Hi,
>
>
> I needed to run an external program in a guest machine. Once this must be
> triggered by the host, I first thought in qemu-ga.
> Is that possible? In QEMU help page and in the code I couldn't find such
> capability.
> So Im thinking In to implement a new GA QMP command that can run generic
> programs in the guest. It would be receive/return something like this:
>
> {"execute":"execvp",
> "arguments":{"command":"/bin/ls","cmdargs":"-la","timeout":20}}
> {"return": {"status": "0", "stdout": "aGVsbG8gd29ybGQhCg==", "stderr": ""}}
>
> Any thoughts/ideas about this?
I sent an RFC for this a while back:
http://lists.gnu.org/archive/html/qemu-devel/2011-12/msg00722.html
At the time the interface seemed a bit tedious, but AFAIK it's the only
kind of approach that'll work for longer-running commands with lots of
output, so I might just clean it up and re-spin the series.
>
> Kind Regards,
> Erlon
Re: [Qemu-devel] [PATCH] qga: add missing commas in json docs
On Wed, Jan 02, 2013 at 09:15:11AM -0700, Eric Blake wrote:
> * qga/qapi-schema.json: Use valid JSON.
>
> Signed-off-by: Eric Blake
Thanks, applied to qga branch
> ---
> qga/qapi-schema.json | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/qga/qapi-schema.json b/qga/qapi-schema.json
> index ed0eb69..d91d903 100644
> --- a/qga/qapi-schema.json
> +++ b/qga/qapi-schema.json
> @@ -31,7 +31,7 @@
> #
> # Since: 1.1
> # ##
> -{ 'command': 'guest-sync-delimited'
> +{ 'command': 'guest-sync-delimited',
>'data':{ 'id': 'int' },
>'returns': 'int' }
>
> @@ -69,7 +69,7 @@
> #
> # Since: 0.15.0
> ##
> -{ 'command': 'guest-sync'
> +{ 'command': 'guest-sync',
>'data':{ 'id': 'int' },
>'returns': 'int' }
>
> --
> 1.8.0.2
>
Re: [Qemu-devel] Using QEMU guest agent to run programs from guest path]
On Thu, Jan 03, 2013 at 11:06:02AM -0200, Erlon Cruz wrote:
> On Wed, Jan 2, 2013 at 9:04 PM, mdroth wrote:
>
> > On Mon, Dec 31, 2012 at 06:14:59PM -0200, Erlon Cruz wrote:
> > > Hi,
> > >
> > >
> > > I needed to run an external program in a guest machine. Once this must be
> > > triggered by the host, I first thought in qemu-ga.
> > > Is that possible? In QEMU help page and in the code I couldn't find such
> > > capability.
> > > So Im thinking In to implement a new GA QMP command that can run generic
> > > programs in the guest. It would be receive/return something like this:
> > >
> > > {"execute":"execvp",
> > > "arguments":{"command":"/bin/ls","cmdargs":"-la","timeout":20}}
> > > {"return": {"status": "0", "stdout": "aGVsbG8gd29ybGQhCg==", "stderr":
> > ""}}
> > >
> > > Any thoughts/ideas about this?
> >
> > I sent an RFC for this a while back:
> >
> > http://lists.gnu.org/archive/html/qemu-devel/2011-12/msg00722.html
> >
> > At the time the interface seemed a bit tedious, but AFAIK it's the only
> > kind of approach that'll work for longer-running commands with lots of
> > output, so I might just clean it up and re-spin the series.
> >
> >
> Why you say tedious? The interface seems to have a very wide usage for
The parameter passing for guest commands was kludgy (list of json
objects rather than a list of parameter strings), but I think we can handle
that now with the "gen: no" option to the code parser indicating we'll
handle it manually.
I'm not sure about the guest-file-open-pipe stuff either. It seems
clumsy, but I can't think of a better approach.
I'll look at it and shoot to get in by 1.4, but feature freeze is only a
couple weeks away so it may have to wait till 1.5.
> several scenarios and fits perfectly for what we are trying to do. Why it
> didn't go upstream? I think it would be nice to roll that up again.
>
> Erlon
>
> >
> > > Kind Regards,
> > > Erlon
> >
[Qemu-devel] [ANNOUNCE] QEMU 1.3.1 Stable released
Hi everyone, I am pleased to announce that the QEMU v1.3.1 stable release is now available at: http://wiki.qemu.org/download/qemu-1.3.1.tar.bz2 The official stable-1.3 repository has also been updated to v1.3.1: http://git.qemu.org/?p=qemu-stable-1.3.git;a=summary This release includes 29 build/bug fixes, including a number of critical fixes for OpenBSD guests. Please note this is the last planned released for the 1.3 series as we move on to maintaining 1.4, after which we'll be moving to a more regular schedule of 2 planned stable releases for each major release. Thanks! 04024de: update VERSION for v1.3.1 (Michael Roth) 1bd4397: qxl: Fix SPICE_RING_PROD_ITEM(), SPICE_RING_CONS_ITEM() sanity check (Markus Armbruster) e766724: Fix compile errors when enabling Xen debug logging. (Sander Eikelenboom) df50a7e: xen: fix trivial PCI passthrough MSI-X bug (Stefano Stabellini) 90c96d3: xen_disk: fix memory leak (Roger Pau Monne) 4ee2879: tcg/target-arm: Add missing parens to assertions (Peter Maydell) 563068a: win32-aio: Fix memory leak (Kevin Wolf) cdb4834: win32-aio: Fix vectored reads (Kevin Wolf) 9d173df: aio: Fix return value of aio_poll() (Kevin Wolf) 204dd38: raw-posix: fix bdrv_aio_ioctl (Paolo Bonzini) 86bab45: vfio-pci: Loosen sanity checks to allow future features (Alex Williamson) 006c747: pci-assign: Enable MSIX on device to match guest (Alex Williamson) f042cca: vfio-pci: Make host MSI-X enable track guest (Alex Williamson) 1205b80: target-xtensa: fix search_pc for the last TB opcode (Max Filippov) ff0c079: buffered_file: do not send more than s->bytes_xfer bytes per tick (Paolo Bonzini) d745511: migration: fix migration_bitmap leak (Paolo Bonzini) 5afd0ec: e1000: Discard oversized packets based on SBP|LPE (Michael Contreras) c4cd5b0: qxl+vnc: register a vm state change handler for dummy spice_server (Uri Lublin) 7ca2496: qxl: save qemu_create_displaysurface_from result (Gerd Hoffmann) bfae937: target-xtensa: fix ITLB/DTLB page protection flags (Max Filippov) b68c48f: pixman: fix vnc tight png/jpeg support (Gerd Hoffmann) 36fd817: Update seabios to a810e4e72a0d42c7bc04eda57382f8e019add901 (Gerd Hoffmann) 0bc5f4a: seabios: update to e8a76b0f225bba5ba9d63ab227e0a37b3beb1059 (Gerd Hoffmann) 37e1428: vfio-pci: Don't use kvm_irqchip_in_kernel (Alex Williamson) 518799a: target-mips: Fix incorrect shift for SHILO and SHILOV (Petar Jovanovic) 16c5fe4: target-mips: Fix incorrect code and test for INSV (Petar Jovanovic) f1a2195: migration: Fix madvise breakage if host and guest have different page sizes (David Gibson) 3b4fc1f: Fix off-by-1 error in RAM migration code (David Gibson) d67d95f: Disable semaphores fallback code for OpenBSD (Brad Smith) 0a7ad69: Fix semaphores fallback code (Brad Smith)
Re: [Qemu-devel] [PATCH for-1.4] Revert "e1000: no need auto-negotiation if link was down"
On Fri, Feb 01, 2013 at 03:20:59PM +0800, Amos Kong wrote:
> On Thu, Jan 31, 2013 at 05:43:51PM -0600, Michael Roth wrote:
> > This reverts commit 84dd2120247a7d25ff1bb337de21c0e76816ad2d.
> >
> > I'm not sure what issue the original commit was meant to fix, or if
> > the logic is actually wrong, but it causes e1000 to stop working
> > after a guest issues a reset.
>
> Hi Michael,
>
> What's your test scenario?
Nothing special, I just started a guest with
-net nic,model=e1000 -net user
or
-net nic,model=e1000 -net tap
and networking stopped working (could not lease an IP, no outbound
traffic) after I rebooted.
>
> I tried this test with current qemu code, link status is not reseted
> to 'up' after step 3. Is it the problem you said?
I think it's related, but I'm not so much concerned with the qmp-visible
link status changing as I am with the guest.
> This problem also exists with current virtio (existed in the past) /
> rtl8139 (introduced in 83f58e570f21c3e7227e7fbef1fc0e18b5ed7ea9)
>
> 1) boot a guest with e1000 nic
> 2) set link down in monitor
>(hmp) set_link e1000.0 down
> 3) reset guest by 'system_reset' in monitor
>(hmp) system_reset
>
>
> My original patch is used to restore the link status after guest
> reboot(execute 'reboot' insider guest system).
> The link status should always be up after virtual 'hardware' reset
> (execute 'system_reset' in monitor).
You sure you don't have that backwards? It seems to me that your
original patch was meant to *prevent* the link status from changing
after a system reset, which makes sense from the perspective of a
qmp-issued "set_link down" meaning "unplug the cable".
>
> Thanks, Amos
>
> > >From what I can tell a guest with an e1000 nic has no way of changing
> > the link status, as far as it's NetClient peer is concerned, except
> > in the auto-negotiation path, so with this patch in place there's no
> > recovery after a reset, since the link goes down and stays that way.
> >
> > Revert this patch now to fix the bigger problem, and handle any
> > lingering issues with a follow-up.
> >
> > Reproduced/tested with qemu-jeos and Ubuntu 12.10.
> >
> > Signed-off-by: Michael Roth
> > ---
> > hw/e1000.c |5 -
> > 1 file changed, 5 deletions(-)
> >
> > diff --git a/hw/e1000.c b/hw/e1000.c
> > index ef06ca1..563a58f 100644
> > --- a/hw/e1000.c
> > +++ b/hw/e1000.c
> > @@ -166,11 +166,6 @@ static void
> > set_phy_ctrl(E1000State *s, int index, uint16_t val)
> > {
> > if ((val & MII_CR_AUTO_NEG_EN) && (val & MII_CR_RESTART_AUTO_NEG)) {
> > -/* no need auto-negotiation if link was down */
> > -if (s->nic->nc.link_down) {
> > -s->phy_reg[PHY_STATUS] |= MII_SR_AUTONEG_COMPLETE;
> > -return;
> > -}
> > s->nic->nc.link_down = true;
> > e1000_link_down(s);
> > s->phy_reg[PHY_STATUS] &= ~MII_SR_AUTONEG_COMPLETE;
> > --
> > 1.7.9.5
>
Re: [Qemu-devel] [PATCH for-1.4] Revert "e1000: no need auto-negotiation if link was down"
On Fri, Feb 01, 2013 at 03:53:22PM +0800, Amos Kong wrote: > On Fri, Feb 01, 2013 at 03:20:59PM +0800, Amos Kong wrote: > > On Thu, Jan 31, 2013 at 05:43:51PM -0600, Michael Roth wrote: > > > This reverts commit 84dd2120247a7d25ff1bb337de21c0e76816ad2d. > > > > > > I'm not sure what issue the original commit was meant to fix, or if > > > the logic is actually wrong, but it causes e1000 to stop working > > > after a guest issues a reset. > > > > Hi Michael, > > > > What's your test scenario? > > > > I tried this test with current qemu code, link status is not reseted > > to 'up' after step 3. Is it the problem you said? > > This problem also exists with current virtio (existed in the past) / > > rtl8139 (introduced in 83f58e570f21c3e7227e7fbef1fc0e18b5ed7ea9) > > > > 1) boot a guest with e1000 nic > > 2) set link down in monitor > >(hmp) set_link e1000.0 down > > 3) reset guest by 'system_reset' in monitor > >(hmp) system_reset > > > > > > My original patch is used to restore the link status after guest > > reboot(execute 'reboot' insider guest system). > > > The link status should always be up after virtual 'hardware' reset > > (execute 'system_reset' in monitor). > > Is it expected? > > When we reset the virtual system, do we need to reset the status > of simulation of network cable? I don't so. If we "unplug the cable", it should stay unplugged when we reset the machine. When we reset the machine, we should set the NIC's link status in accordance with it's NetClient peer. So if that's what your patch was attempting to enforce I think we're in agreement there. I think the problem with your original patch is that it doesn't check the link status of the nic's NetClient peer, but instead checks it's own internal NetClient state, so we end up trying to enforce "leave the cable unplugged" even when "set_link down" was never issued. And if that's the case I think you can re-spin your original patch for 1.4 accordingly. But we definitely need to revert the current one ASAP since it breaks all QEMU guests that are started using the default nic. > I think it's deciced by that if we think simulation of network > cable is a part of the virtual machine. > > > commit 436e5e53c97d8fb469306b18a0c31dc60f5e546c > Author: aliguori > Date: Thu Jan 8 19:44:06 2009 + > > Add 'set_link' monitor command (Mark McLoughlin) > > Add a monitor command to setting a given network device's link status > to 'up' or 'down'. > > Allows simulation of network cable disconnect. > > Signed-off-by: Mark McLoughlin > Signed-off-by: Anthony Liguori >
Re: [Qemu-devel] [PATCH 1/2] qga: add guest-get-time command
On Wed, Jan 30, 2013 at 03:37:25PM +0800, Lei Li wrote:
> On 01/29/2013 04:24 AM, Anthony Liguori wrote:
> >Eric Blake writes:
> >
> >>On 01/27/2013 11:14 AM, Lei Li wrote:
> >>>Signed-off-by: Lei Li
> >>>---
> >>> include/qapi/qmp/qerror.h |3 +++
> >>> qga/commands-posix.c | 30 ++
> >>> qga/qapi-schema.json | 38 ++
> >>> 3 files changed, 71 insertions(+), 0 deletions(-)
> >>>
> >>>diff --git a/include/qapi/qmp/qerror.h b/include/qapi/qmp/qerror.h
> >>>index 6c0a18d..0baf1a4 100644
> >>>--- a/include/qapi/qmp/qerror.h
> >>>+++ b/include/qapi/qmp/qerror.h
> >>>@@ -129,6 +129,9 @@ void assert_no_error(Error *err);
> >>> #define QERR_FEATURE_DISABLED \
> >>> ERROR_CLASS_GENERIC_ERROR, "The feature '%s' is not enabled"
> >>>+#define QERR_GET_TIME_FAILED \
> >>>+ERROR_CLASS_GENERIC_ERROR, "Failed to get time"
> >>>+
> >>These days, you shouldn't be defining a new QERR wrapper. Instead,...[1]
> >>
> >>>+static TimeInfo *get_host_time(Error **errp)
> >>This name is unusual...[2]
> >>
> >>>+{
> >>>+int ret;
> >>>+qemu_timeval tq;
> >>>+TimeInfo *time_info;
> >>>+
> >>>+ret = qemu_gettimeofday(&tq);
> >>>+if (ret < 0) {
> >>>+error_set_errno(errp, errno, QERR_GET_TIME_FAILED);
> >>[1]...use the right idiom here:
> >>
> >>error_setg_errno(errp, errno, "Failed to get time");
> >>
> >>>+return NULL;
> >>>+}
> >>>+
> >>>+time_info = g_malloc0(sizeof(TimeInfo));
> >>>+time_info->seconds = tq.tv_sec;
> >>>+time_info->microseconds = tq.tv_usec;
> >>Is microseconds the right unit to expose through JSON, or are we going
> >>to wish we had exposed nanoseconds down the road (you can still use
> >>qemu_gettimeofday() and scale tv_usec by 1000 before assigning to
> >>time_info->nanoseconds, if we desire the extra granularity).
> >>
> >>You aren't setting time_info->utc_offset. Is that intentional?
> >Moreover, there's no need to specify microseconds and seconds. A 64-bit
> >value for nanoseconds is sufficient. A signed value can represent
> >1678 -> 2262. If anyone is using qemu-ga in 2262, they'll have to
> >introduce a new command for their quantum emulators :-)
> >
> >Setting time independent of date is a bit silly because time cannot be
> >interpreted correctly without a date.
> >
> >A single value of nanoseconds since the epoch (interpreted as UTC/GMT
> >time) is really the best strategy. The host shouldn't be involved in
> >guest time zones. In a Cloud environment, it's pretty normal to have
> >different guests using different time zones.
> >
> >Regards,
> >
> >Anthony Liguori
> >
> >>>+
> >>>+return time_info;
> >>>+}
> >>>+
> >>>+struct TimeInfo *qmp_guest_get_time(Error **errp)
> >>>+{
> >>>+TimeInfo *time_info = get_host_time(errp);
> >>[2]...given that it is only ever called from qmp_guest_get_time.
> >>
> >>>+
> >>>+if (!time_info) {
> >>>+return NULL;
> >>>+}
> >>These three lines can be deleted,
> >>
> >>>+
> >>>+return time_info;
> >>since this line will do the same thing if you let NULL time_info get
> >>this far.
> >>
> >>>+++ b/qga/qapi-schema.json
> >>>@@ -83,6 +83,44 @@
> >>> { 'command': 'guest-ping' }
> >>> ##
> >>>+# @TimeInfo
> >>>+#
> >>>+# Time Information. It is relative to the Epoch of 1970-01-01.
> >>>+#
> >>>+# @seconds: "seconds" time unit.
> >>>+#
> >>>+# @microseconds: "microseconds" time unit.
> >>>+#
> >>>+# @utc-offset: Information about utc offset. Represented as:
> >>>+# ±[] based at a minimum on minutes, with
> >>s/based at a minimum on//
> >>
> >>This still doesn't state whether two hours east of UTC is represented as
> >>120 or as 0200.
> >>
> It should be 120.
> Yeah, I should make it clear.
>
> I am thinking if this 'utc_offset' can be made as a string, represented
> like: ±[hh]:[mm], +08:45 (8 hours and 45 minutes) for example.
I'd prefer we stick with an integer value representing minutes.
QMP/qemu-ga are programmatic interfaces where human-readable string
representations are actually harder to work with for this type of thing.
>
> >>>+# negative values are west and positive values
> >>>+# are east of UTC. The bounds of @utc-offset is
> >>>+# at most 24 hours away from UTC.
> >>>+#
> >>>+# Since: 1.4
> >>>+##
> >>>+{ 'type': 'TimeInfo',
> >>>+ 'data': { 'seconds': 'int', 'microseconds': 'int',
> >>>+'utc-offset': 'int' } }
> >>>+
> >>>+##
> >>>+# @guest-get-time:
> >>>+#
> >>>+# Get the information about host time in UTC and the
> >>s/host/guest/
> >>
> >>>+# UTC offset.
> >>>+#
> >>>+# This command tries to get the host time which is
> >>>+# presumably correct, since we need to be able to
> >>>+# resynchronize guest clock to host's in guest.
> >>This sentence doesn't make sense. Better would be:
> >>
> >>Get the guest's notion of the current time.
> >>
> >>>+#
> >>>+# Returns: @TimeInfo on success.
> >>>+#
> >>>+# Since 1.4
>
Re: [Qemu-devel] [PATCH for-1.4 01/12] qmp: Fix design bug and read beyond buffer in memchar-write
On Wed, Feb 06, 2013 at 10:06:03AM +0100, Markus Armbruster wrote:
> Markus Armbruster writes:
>
> > Eric Blake writes:
> >
> >> On 02/05/2013 09:22 AM, Markus Armbruster wrote:
> >>> Command memchar-write takes data and size parameter. Begs the
> >>> question what happens when data doesn't match size.
> >>>
> >>> With format base64, qmp_memchar_write() copies the full data argument,
> >>> regardless of size argument.
> >>>
> >>> With format utf8, qmp_memchar_write() copies size bytes from data,
> >>> happily reading beyond data. Copies crap from the heap or even
> >>> crashes.
> >>>
> >>> Drop the size parameter, and always copy the full data argument.
> >>>
> >>> Signed-off-by: Markus Armbruster
> >>> ---
> >>> hmp.c| 4 +---
> >>> qapi-schema.json | 4 +---
> >>> qemu-char.c | 8 +++-
> >>> qmp-commands.hx | 4 +---
> >>> 4 files changed, 6 insertions(+), 14 deletions(-)
> >>
> >>> if (has_format && (format == DATA_FORMAT_BASE64)) {
> >>> write_data = g_base64_decode(data, &write_count);
> >>> } else {
> >>> write_data = (uint8_t *)data;
> >>> +write_count = strlen(data);
> >>> }
> >>
> >> Obviously, base64 is the only way to write an embedded NUL. But what
> >
> > Consider the JSON string "this \\u is fun". But our JSON parser
> > chokes on it, so we can ignore it until that's fixed. See my "[PATCH]
> > check-qjson: More thorough testing of UTF-8 in strings", specifically
> > the comment right at the beginning of utf8_string().
> >
> >> happens if the user requests base64 encoding, but the utf8 string that
> >> got passed in through JSON is not a valid base64-encoded string? Does
> >> g_base64_decode report an error in that case, and should you be handling
> >> the error here?
> >
> > Good question. I wish it had occured to GLib developers:
> > http://developer.gnome.org/glib/stable/glib-Base64-Encoding.html#g-base64-decode
> >
> > Seriously, I need to find out what the heck g_base64_decode() does when
> > it's fed crap. If it fails in a reliable and detectable manner, I need
> > to handle the failure. If not, I need to replace the function.
> >
> > Moreover, I should document which characters outside the base64 alphabet
> > are silently ignored, if any. All whitespace? Just newlines?
>
> As far as I can tell, it never fails, but silently ignores characters
> outside the alphabet [A-Za-z0-9+/], as well as unpadded suffixes. Oh,
> and it does something weird when padding appears in the middle.
> Craptastic.
>
> We can either document this behavior as a feature, or we replace the
> function by one that accepts only valid base64. If we do the latter, we
> better specify the language we want to accept now, but I guess we could
> choose to delay the actual checking post 1.4.
>
> There's another use of g_base64_decode() in qmp_guest_file_write().
> Which means guest agent command guest-file-write is similarly
> ill-defined. Mike, anything to be done there?
For qemu-ga I think the documentation makes it clear enough that we're
expecting b64 inputs rather than just interpreting random input as b64,
so I don't see a problem with making the checks stricter in the future.
One other concern though:
>
>
> --
> #include
> #include
> #include
>
> int
> main(int argc, char *argv[])
> {
> char **ap, *b64;
> unsigned char *buf;
> size_t sz, i;
>
> for (ap = argv + 1; *ap; ap++) {
> printf("in : %s\n", *ap);
> buf = g_base64_decode(*ap, &sz);
> printf("out:");
> for (i = 0; i < sz; i++) {
> printf(" %02x", buf[i]);
> }
> putchar('\n');
> b64 = g_base64_encode(buf, sz);
> printf("b64: %s\n", b64);
> free(buf);
> }
> }
> --
> in : 1
> out:
> b64:
> in : 1=
> out:
> b64:
> in : 1==
> out:
> b64:
> in : 1===
> out: d4
> b64: 1A==
> in : 12
> out:
> b64:
> in : 123
> out:
> b64:
> in : 1234
> out: d7 6d f8
> b64: 1234
> in : =1234
> out: 03 5d b7
> b64: A123
> in : 1=234
> out: d4 0d b7
> b64: 1A23
> in : <>?,./watch this!@#$%^&*()_+
> out: ff 06 ad 72 1b 61
> b64: /watchth
> in : /watchthis+
> out: ff 06 ad 72 1b 61
> b64: /watchth
Am I misinterpreting the output or is base64_encode() actually spitting
*out* invalid base64 strings for certain inputs? If so that seems pretty
bad for something like guest-file-read, where inputs to base64_encode()
are for all intents completely random. Addressing it in hard freeze may
not be reasonable, since qemu-ga users must already be prepared to receive
garbage from malicious/buggy agents, but I'd certainly pick up a fix for a
subsequent stable release.
Re: [Qemu-devel] [PATCH for-1.4 01/12] qmp: Fix design bug and read beyond buffer in memchar-write
On Wed, Feb 06, 2013 at 09:14:12PM +0100, Markus Armbruster wrote:
> mdroth writes:
>
> > On Wed, Feb 06, 2013 at 10:06:03AM +0100, Markus Armbruster wrote:
> >> Markus Armbruster writes:
> >>
> >> > Eric Blake writes:
> >> >
> >> >> On 02/05/2013 09:22 AM, Markus Armbruster wrote:
> >> >>> Command memchar-write takes data and size parameter. Begs the
> >> >>> question what happens when data doesn't match size.
> >> >>>
> >> >>> With format base64, qmp_memchar_write() copies the full data argument,
> >> >>> regardless of size argument.
> >> >>>
> >> >>> With format utf8, qmp_memchar_write() copies size bytes from data,
> >> >>> happily reading beyond data. Copies crap from the heap or even
> >> >>> crashes.
> >> >>>
> >> >>> Drop the size parameter, and always copy the full data argument.
> >> >>>
> >> >>> Signed-off-by: Markus Armbruster
> >> >>> ---
> >> >>> hmp.c| 4 +---
> >> >>> qapi-schema.json | 4 +---
> >> >>> qemu-char.c | 8 +++-
> >> >>> qmp-commands.hx | 4 +---
> >> >>> 4 files changed, 6 insertions(+), 14 deletions(-)
> >> >>
> >> >>> if (has_format && (format == DATA_FORMAT_BASE64)) {
> >> >>> write_data = g_base64_decode(data, &write_count);
> >> >>> } else {
> >> >>> write_data = (uint8_t *)data;
> >> >>> +write_count = strlen(data);
> >> >>> }
> >> >>
> >> >> Obviously, base64 is the only way to write an embedded NUL. But what
> >> >
> >> > Consider the JSON string "this \\u is fun". But our JSON parser
> >> > chokes on it, so we can ignore it until that's fixed. See my "[PATCH]
> >> > check-qjson: More thorough testing of UTF-8 in strings", specifically
> >> > the comment right at the beginning of utf8_string().
> >> >
> >> >> happens if the user requests base64 encoding, but the utf8 string that
> >> >> got passed in through JSON is not a valid base64-encoded string? Does
> >> >> g_base64_decode report an error in that case, and should you be handling
> >> >> the error here?
> >> >
> >> > Good question. I wish it had occured to GLib developers:
> >> > http://developer.gnome.org/glib/stable/glib-Base64-Encoding.html#g-base64-decode
> >> >
> >> > Seriously, I need to find out what the heck g_base64_decode() does when
> >> > it's fed crap. If it fails in a reliable and detectable manner, I need
> >> > to handle the failure. If not, I need to replace the function.
> >> >
> >> > Moreover, I should document which characters outside the base64 alphabet
> >> > are silently ignored, if any. All whitespace? Just newlines?
> >>
> >> As far as I can tell, it never fails, but silently ignores characters
> >> outside the alphabet [A-Za-z0-9+/], as well as unpadded suffixes. Oh,
> >> and it does something weird when padding appears in the middle.
> >> Craptastic.
> >>
> >> We can either document this behavior as a feature, or we replace the
> >> function by one that accepts only valid base64. If we do the latter, we
> >> better specify the language we want to accept now, but I guess we could
> >> choose to delay the actual checking post 1.4.
> >>
> >> There's another use of g_base64_decode() in qmp_guest_file_write().
> >> Which means guest agent command guest-file-write is similarly
> >> ill-defined. Mike, anything to be done there?
> >
> > For qemu-ga I think the documentation makes it clear enough that we're
> > expecting b64 inputs rather than just interpreting random input as b64,
> > so I don't see a problem with making the checks stricter in the future.
> >
> > One other concern though:
> >
> >>
> >>
> >> --
> >> #include
> >> #include
> >> #include
> >>
> >> int
> >> main(int argc, char *argv[])
> >> {
> >> char **ap, *b64;
> >> unsigned char *buf;
> >> size_t sz, i;
> >>
> >> for (ap = argv + 1; *ap; ap++) {
&
Re: [Qemu-devel] [PATCH for-1.4] qapi: Improve chardev-add documentation
On Tue, Feb 12, 2013 at 05:56:00PM +0100, Markus Armbruster wrote:
> Gerd Hoffmann writes:
>
> > Hi,
> >
> >> But why nested discriminators?
> >>
> >> regular files: type=file
> >> serial : type=port, data.type=serial
> >> parallel : type=port, data.type=parallel
> >>
> >> Simpler, and closer to existing -chardev:
> >>
> >> regular files: type=file
> >> serial : type=serial
> >> parallel : type=parallel
> >
> > Matter of taste IMHO.
> > I can live with that too.
> > Feel free to send patches.
> >
> >> I also dislike the pointless '"data" : {}' required by type pty and
> >> null, but I can't figure out how to express 'void' in the schema.
> >
> > Someone mentioned it is possible to leave out empty data altogether.
> > Didn't try whenever our marshaller actually accepts that though.
>
> Looks like it doesn't :(
>
> Empty objects work fine here:
>
> { 'type': 'ChardevDummy', 'data': { } }
>
> Generates the obvious
>
> struct ChardevDummy
> {
> };
>
> They don't work here:
>
> { 'union': 'ChardevBackend', 'data': { 'file' : 'ChardevFile',
>'hostdev': 'ChardevHostdev',
>'socket' : 'ChardevSocket',
>'pty': 'ChardevDummy',
>'null' : {} } }
>
> Generates
>
> struct ChardevBackend
> {
> ChardevBackendKind kind;
> union {
> void *data;
> ChardevFile * file;
> ChardevHostdev * hostdev;
> ChardevSocket * socket;
> ChardevDummy * pty;
> void null;
> };
> };
>
> which is kind of cute, but the compiler doesn't like it.
>
> Anthony, Mike, is this a bug?
Not exactly, but it's a relic that doesn't seem to be needed anymore.
The code that does this is in scripts/qapi.py:
def c_type(name):
...
elif name == None or len(name) == 0:
return 'void'
...
else:
return '%s *' % name
The 'name' param being the value/type of a particular param/key in a
QAPI dictionary that defines a schema-defined type.
The reason '{}' maps to 'void' is so that in qapi-commands.py, where we generate
stuff like the function signatures for qmp commands, we'd map something like:
{ 'command': 'my_cmd',
'data': { 'param1': 'Foo' },
'returns': {} }
to:
void my_cmd(Foo *param1);
At some point in development we rightly decided that:
{ 'command': 'my_cmd',
'data': { 'param1': 'Foo' } }
was more efficient, which triggers the 'name == None' case and has the same
effect.
So, as long as we stay consistent about defining commands in this fashion, we
can map 'param_name': {} to something like 'struct {}', and use it in place of
schema-defined dummy types.
Though, as I mentioned on IRC, I think just defining a:
{ 'type': 'Dummy', 'data' {} }
Somewhere in the schema and re-using that might actually be cleaner and have
the same affect.
Re: [Qemu-devel] [RFC PATCH 03/10] qemu-ga: Add an configure option to specify path to Windows VSS SDK
On Thu, Feb 14, 2013 at 03:10:39PM +0900, Tomoki Sekiyama wrote:
> To enable VSS support in qemu-ga for Windows, header files included in
> VSS SDK is required.
> The VSS support is enabled when the option like below:
> ./configure --with-vss-sdk="/pass/to/VSS SDK"
>
> VSS SDK is available from:
> http://www.microsoft.com/en-us/download/details.aspx?id=23490
>
> To cross-compilie using mingw32 for Linux, you need to setup the SDK on
> Windows environments to extract headers. You can also use wine to run the
> setup of SDK on Linux etc.
>
> Signed-off-by: Tomoki Sekiyama
> ---
> .gitignore |1 +
> Makefile |1 +
> configure | 41 +
> 3 files changed, 43 insertions(+)
>
> diff --git a/.gitignore b/.gitignore
> index 53fe9c3..3f450e8 100644
> --- a/.gitignore
> +++ b/.gitignore
> @@ -77,6 +77,7 @@ fsdev/virtfs-proxy-helper.pod
> *.la
> *.pc
> .libs
> +.sdk
> *.swp
> *.orig
> .pc
> diff --git a/Makefile b/Makefile
> index 0d9099a..fab664f 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -252,6 +252,7 @@ distclean: clean
> for d in $(TARGET_DIRS); do \
> rm -rf $$d || exit 1 ; \
> done
> + rm -Rf .sdk
> if test -f pixman/config.log; then make -C pixman distclean; fi
>
> KEYMAPS=da en-gb et fr fr-ch is lt modifiers no pt-br sv \
> diff --git a/configure b/configure
> index e279263..da49c52 100755
> --- a/configure
> +++ b/configure
> @@ -220,6 +220,8 @@ usb_redir=""
> opengl=""
> zlib="yes"
> guest_agent="yes"
> +guest_agent_with_vss="no"
> +vss_win32_sdk=""
> want_tools="yes"
> libiscsi=""
> coroutine=""
> @@ -884,6 +886,8 @@ for opt do
>;;
>--disable-guest-agent) guest_agent="no"
>;;
> + --with-vss-sdk=*) vss_win32_sdk="$optarg"
> + ;;
>--enable-tools) want_tools="yes"
>;;
>--disable-tools) want_tools="no"
> @@ -1142,6 +1146,7 @@ echo " --disable-usb-redir disable usb network
> redirection support"
> echo " --enable-usb-redir enable usb network redirection support"
> echo " --disable-guest-agentdisable building of the QEMU Guest Agent"
> echo " --enable-guest-agent enable building of the QEMU Guest Agent"
> +echo " --with-vss-sdk=SDK-path enable Windows VSS support in QEMU Guest
> Agent"
> echo " --disable-seccompdisable seccomp support"
> echo " --enable-seccomp enables seccomp support"
> echo " --with-coroutine=BACKEND coroutine backend. Supported options:"
> @@ -2897,6 +2902,38 @@ if test "$usb_redir" != "no" ; then
> fi
>
> ##
> +# check if we have VSS SDK headers for win
> +
> +if test "$mingw32" = "yes" -a "$guest_agent" = "yes" ; then
> + case "$vss_win32_sdk" in
> +"") vss_win32_include="" ;;
> +*\ *) # The SDK is installed in "Program Files" by default, but we cannot
> + # handle path with spaces. So we copy the headers into ".sdk/sdk".
> + vss_win32_include="-I$source_path/.sdk/vss"
> + symlink "$vss_win32_sdk/inc" "$source_path/.sdk/vss/inc"
> + ;;
> + *)vss_win32_include="-I$vss_win32_sdk"
> + esac
> + cat > $TMPC << EOF
> +#define __MIDL_user_allocate_free_DEFINED__
> +#include
> +int main(void) { return VSS_CTX_BACKUP; }
> +EOF
Hi Tomoki,
Did you happen to run into this issue compiling the test program?
[mdroth@vm qemu-build]$ ls ~/w/vsssdk/inc/win2003/
vdslun.hvsbackup.hvscoordint.idl vsmgmt.idl vsprov.idl
vss.idlvsswprv.idl
vdslun.idl vscoordint.h vsmgmt.hvsprov.hvss.h
vsswprv.h vswriter.h
[mdroth@vm qemu-build]$ cat test.c
#define __MIDL_user_allocate_free_DEFINED__
#include
int main(void) { return VSS_CTX_BACKUP; }
[mdroth@vm qemu-build]$ gcc -I ~/w/vsssdk/ -o test test.c
In file included from test.c:2:0:
/home/mdroth/w/vsssdk/inc/win2003/vss.h:25:17: fatal error: rpc.h: No
such file or directory
compilation terminated.
I can't seem to locate any mingw or microsoft package that provides that.
It's also not present anywhere on the Wine filesystem I installed the
VSS SDK to.
> + if compile_prog "$vss_win32_include" "" ; then
> +guest_agent_with_vss="yes"
> +QEMU_CFLAGS="$QEMU_CFLAGS $vss_win32_include"
> +libs_qga=&quo
Re: [Qemu-devel] [RFC PATCH 03/10] qemu-ga: Add an configure option to specify path to Windows VSS SDK
On Fri, Feb 15, 2013 at 12:55:49PM +0900, Tomoki Sekiyama wrote:
> On 2013/02/15 9:47, mdroth wrote:
> [...]
> >
> > Hi Tomoki,
>
> Hi,
>
> > Did you happen to run into this issue compiling the test program?
> >
> > [mdroth@vm qemu-build]$ ls ~/w/vsssdk/inc/win2003/
> > vdslun.hvsbackup.hvscoordint.idl vsmgmt.idl vsprov.idl
> > vss.idlvsswprv.idl
> > vdslun.idl vscoordint.h vsmgmt.hvsprov.hvss.h
> > vsswprv.h vswriter.h
> > [mdroth@vm qemu-build]$ cat test.c
> > #define __MIDL_user_allocate_free_DEFINED__
> > #include
> > int main(void) { return VSS_CTX_BACKUP; }
> > [mdroth@vm qemu-build]$ gcc -I ~/w/vsssdk/ -o test test.c
> > In file included from test.c:2:0:
> > /home/mdroth/w/vsssdk/inc/win2003/vss.h:25:17: fatal error: rpc.h: No
> > such file or directory
> > compilation terminated.
> >
> > I can't seem to locate any mingw or microsoft package that provides that.
> > It's also not present anywhere on the Wine filesystem I installed the
> > VSS SDK to.
> >
>
> Hmm, I haven't seen this yet.
> I am testing this on Fedora 18 x86_64 host, and it provides package
> "mingw32-headers.noarch" or "mingw64-headers.noarch" which contains
> the rpc.h
> (filepath is /usr/x86_64-w64-mingw32/sys-root/mingw/include/rpc.h )
Thanks. I actually did have that header, but I was doing something silly
(forgot to use cross-compiler for the test program). The configure issue
I was hitting was also an error on my part (options with "~" weren't
resolving to my home dir). I did end up missing ntverp.h however in
vss-win32.h using Fedora 15. I made some progress trying the build
without it however. Can you confirm all the headers in vss-win32.h are
needed? I went ahead and put together an FC18 environment, but would be
nice if we can get this building for older environments (if possible).
>
> --
> Tomoki Sekiyama
> Linux Technology Center
> Hitachi, Ltd., Yokohama Research Laboratory
>
Re: [Qemu-devel] [RFC PATCH 04/10] qemu-ga: Add Windows VSS provider to quiesce applications on fsfreeze
On Thu, Feb 14, 2013 at 03:10:42PM +0900, Tomoki Sekiyama wrote: > Implements a basic stub of software VSS provider. Currently, this modules > only provides a relay function of events between qemu-guest-agent and > Windows VSS when VSS finished filesystem freeze and when qemu snapshot > is done. > > In the future, this module could be extended to support the other VSS > functions, such as query for snapshot volumes and recovery. > > Signed-off-by: Tomoki Sekiyama Looks like this was only tested for in-tree builds. I ran into issues build out of tree and haven't quite gotten them all worked out. I've posted what I've gotten so far below, but with those changes applied I'm still running into this one on FC18: [mdroth@vm5 qemu-build]$ rm -rf * && ~/w/qemu3.git/configure --enable-guest-agent --cross-prefix=i686-w64-mingw32- --with-vss-sdk=/home/mdroth/w/vsssdk && make qemu-ga.exe ... [mdroth@vm5 qemu-build]$ make V=1 qemu-ga.exe make BUILD_DIR=/home/mdroth/qemu-build -C /home/mdroth/w/qemu3.git/qga/./vss-win32-provider V="1" all make[1]: Entering directory `/home/mdroth/dev/kvm/qemu3.git/qga/vss-win32-provider' make[1]: Nothing to be done for `all'. make[1]: Leaving directory `/home/mdroth/dev/kvm/qemu3.git/qga/vss-win32-provider' i686-w64-mingw32-gcc -m32 -D__USE_MINGW_ANSI_STDIO=1 -DWIN32_LEAN_AND_MEAN -DWINVER=0x501 -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -Wstrict-prototypes -Wredundant-decls -Wall -Wundef -Wwrite-strings -Wmissing-prototypes -fno-strict-aliasing -fstack-protector-all -Wendif-labels -Wmissing-include-dirs -Wempty-body -Wnested-externs -Wformat-security -Wformat-y2k -Winit-self -Wignored-qualifiers -Wold-style-declaration -Wold-style-definition -Wtype-limits -I/usr/i686-w64-mingw32/sys-root/mingw/include -I/usr/i686-w64-mingw32/sys-root/mingw/include/p11-kit-1 -I/home/mdroth/w/vsssdk -I/usr/i686-w64-mingw32/sys-root/mingw/include/pixman-1 -I. -I. -mms-bitfields -I/usr/i686-w64-mingw32/sys-root/mingw/include/glib-2.0 -I/usr/i686-w64-mingw32/sys-root/mingw/lib/glib-2.0/include -DHAS_VSS_SDK -I qga/qapi-generated -O2 -D_FORTIFY_SOURCE=2 -g -Wl,--nxcompat -Wl,--no-seh -Wl,--dynamicbase -Wl,--warn-common -m32 -g -o qemu-ga.exe qapi-types.o qapi-visit.o qga/channel-win32.o qga/commands-win32.o qga/commands.o qga/guest-agent-command-state.o qga/main.o qga/qapi-generated/qga-qapi-types.o qga/qapi-generated/qga-qapi-visit.o qga/qapi-generated/qga-qmp-marshal.o qga/service-win32.o qga/vss-win32-requester.o libqemuutil.a libqemustub.a qga/vss-win32-provider/qga-provider.dll version.o -lole32 -loleaut32 -lshlwapi -luuid -lstdc++ -Wl,--enable-stdcall-fixup -L/usr/i686-w64-mingw32/sys-root/mingw/lib -lgthread-2.0 -lglib-2.0 -lintl -lws2_32 -lwinmm -lpowrprof i686-w64-mingw32-gcc: error: qga/vss-win32-requester.o: No such file or directory i686-w64-mingw32-gcc: error: qga/vss-win32-provider/qga-provider.dll: No such file or directory make: *** [qemu-ga.exe] Error 1 [mdroth@vm5 qemu-build]$ LINK qemu-ga.exe i686-w64-mingw32-gcc: error: qga/vss-win32-requester.o: No such file or directory i686-w64-mingw32-gcc: error: qga/vss-win32-provider/qga-provider.dll: No such file or directory make: *** [qemu-ga.exe] Error 1 [mdroth@vm5 qemu-build]$ provider.cpp/install.cpp/qga-provider.dll are actually generated ok, they're just getting put into $(SRC_PATH)/qga/vss-win32-provider instead of the build directory due to the recursive Makefile invocation in qga/Makefile.obj, while gcc is expecting them in $(BUILD_DIR). I think this rule should get moved up to the top-level Makefile where the rest of the qemu-ga magic lives. Leaving the Makefile there for stuff like building the .tlb file is fine though. The one I'm not understanding is vss-win32-requestor.cpp...it's not getting built even though everything else in $(qga-obj-y) is. Some strange with the matching against the %.o: %.cpp recipe I'm guessing. Here's what I have so far: @@ -15,6 +15,6 @@ $(obj)/vss-win32-requester.o: QEMU_CXXFLAGS += -Wno-unknown-pragmas $(QGALIB_EXTDIR)/qga-provider.tlb: $(QGALIB_EXTDIR)/qga-provider.dll $(QGALIB_EXTDIR)/qga-provider.dll: $(qga-lib-src) $(obj)/vss-win32.h - $(call quiet-command,$(MAKE) $(SUBDIR_MAKEFLAGS) -C qga/$(QGALIB_EXTDIR) V="$(V)" all,) + $(call quiet-command,$(MAKE) $(SUBDIR_MAKEFLAGS) -C $(SRC_PATH)/qga/$(QGALIB_EXTDIR) V="$(V)" all,) endif diff --git a/qga/vss-win32-provider.h b/qga/vss-win32-provider.h index e312977..9173484 100644 --- a/qga/vss-win32-provider.h +++ b/qga/vss-win32-provider.h @@ -14,6 +14,7 @@ #define VSS_WIN32_PROVIDER_H #include +#include STDAPI VSSCheckOSVersion(void); diff --git a/qga/vss-win32-provider/Makefile b/qga/vss-win32-provider/Makefile index 1f213f2..6a6bf8f 100644 --- a/qga/vss-win32-provider/Makefile +++ b/qga/vss-win32-provider/Makefile @@ -1,5 +1,5 @@ --include ../../config-h
Re: [Qemu-devel] [PATCH] qga: cast to int for DWORD type
On Mon, Jan 28, 2013 at 12:49:09PM +0800, Lei Li wrote:
> This patch fixes a compiler warning when cross-build:
>
> qga/service-win32.c: In function 'printf_win_error':
> qga/service-win32.c:32:5: warning: format '%d' expects argument of type 'int',
> but argument 3 has type 'DWORD' [-Wformat]
>
> Signed-off-by: Lei Li
Thanks, applied to qga branch.
> ---
> qga/service-win32.c |2 +-
> 1 files changed, 1 insertions(+), 1 deletions(-)
>
> diff --git a/qga/service-win32.c b/qga/service-win32.c
> index 0905456..843398a 100644
> --- a/qga/service-win32.c
> +++ b/qga/service-win32.c
> @@ -29,7 +29,7 @@ static int printf_win_error(const char *text)
> MAKELANGID(LANG_NEUTRAL, SUBLANG_DEFAULT),
> (char *)&message, 0,
> NULL);
> -n = printf("%s. (Error: %d) %s", text, err, message);
> +n = printf("%s. (Error: %d) %s", text, (int)err, message);
> LocalFree(message);
>
> return n;
> --
> 1.7.7.6
>
Re: [Qemu-devel] [PATCH] qemu-ga: fix confusing GAChannelMethod comparison
On Tue, Feb 19, 2013 at 03:12:34PM +0100, Stefan Hajnoczi wrote:
> In commit 7868e26e5930f49ca942311885776b938dcf3b77
> ("qemu-ga: add initial win32 support") support was added for qemu-ga on
> Windows using virtio-serial. Other channel methods (ISA serial and UNIX
> domain socket) are not supported on Windows.
>
> Signed-off-by: Stefan Hajnoczi
Interesting... I think the original code was actually a bug that just
miraculously worked out thanks to GA_CHANNEL_VIRTIO_SERIAL being 0.
Thanks, applied to qga branch.
> ---
> qga/channel-win32.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/qga/channel-win32.c b/qga/channel-win32.c
> index 16bf44a..7ed98d7 100644
> --- a/qga/channel-win32.c
> +++ b/qga/channel-win32.c
> @@ -287,7 +287,7 @@ GIOStatus ga_channel_write_all(GAChannel *c, const char
> *buf, size_t size)
> static gboolean ga_channel_open(GAChannel *c, GAChannelMethod method,
> const gchar *path)
> {
> -if (!method == GA_CHANNEL_VIRTIO_SERIAL) {
> +if (method != GA_CHANNEL_VIRTIO_SERIAL) {
> g_critical("unsupported communication method");
> return false;
> }
> --
> 1.8.1.2
>
>
Re: [Qemu-devel] [PATCH] move qemu-ga from bin to libexec dir, use $HELPERS
On Sat, Feb 16, 2013 at 10:28:31PM +0400, Michael Tokarev wrote:
> This patch does 3 things:
>
> 1. Renames HELPERS-y Makefile variable to HELPERS
> 2. Moves its definition from Makefile to configure
> 3. Moves qemu-ga binary from TOOLS to HELPERS.
>
> The effects are:
>
> 1. qemu-ga binary is now installed into libexecdir, not bindir.
> This is the main effect/motivation of this patch, -- this binary
> has no business being in a public binary directory, it is a system
> helper which must be run by a system startup script or some event
> daemon.
I agree it's not a public binary, but I think there's a pretty close
analogue between qemu-ga and things like sshd, tftpd, etc, which
generally live in /usr/sbin. qemu-ga is a "top-level" daemon, there's
really nothing else that relies on it as a helper, so I think something
like ADMIN_TOOLS that gets installed in sbin might make more sense.
>
> 2. Another helper, qemu-bridge-helper, which is already installed
> in libexecdir, is built only when we're building one of softmmu
> targets on linux (initially it was just linux-specific, but not
> softmmu-specific).
I think that seems reasonable (along with the corresponding bit in the
patch).
qemu-ga however should get moved out of the $softmmu = "yes" block
and instead be tied to --enable-tools, or something similar like
--enable-guest-tools, or --enable-admin-tools or whatever. it does in
some sense require a softmmu target to ever run, but the build for the
guest environment should be thought of as a seperate build (that we
might just happen to do at the same time as the build for the host
package in some circumstances to save time/cycles), and in that sense
there's no dependency on a softmmu target being configured.
>
> Signed-off-by: Michael Tokarev
> ---
> Makefile | 10 --
> configure |7 ++-
> 2 files changed, 10 insertions(+), 7 deletions(-)
>
> diff --git a/Makefile b/Makefile
> index 0d9099a..ba0cd98 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -53,8 +53,6 @@ $(call set-vpath, $(SRC_PATH))
>
> LIBS+=-lz $(LIBS_TOOLS)
>
> -HELPERS-$(CONFIG_LINUX) = qemu-bridge-helper$(EXESUF)
> -
> ifdef BUILD_DOCS
> DOCS=qemu-doc.html qemu-tech.html qemu.1 qemu-img.1 qemu-nbd.8
> QMP/qmp-commands.txt
> ifdef CONFIG_VIRTFS
> @@ -115,7 +113,7 @@ ifeq ($(CONFIG_SMARTCARD_NSS),y)
> include $(SRC_PATH)/libcacard/Makefile
> endif
>
> -all: $(DOCS) $(TOOLS) $(HELPERS-y) recurse-all
> +all: $(DOCS) $(TOOLS) $(HELPERS) recurse-all
>
> config-host.h: config-host.h-timestamp
> config-host.h-timestamp: config-host.mak
> @@ -215,7 +213,7 @@ clean:
> rm -f qemu-options.def
> find . -name '*.[oda]' -type f -exec rm -f {} +
> find . -name '*.l[oa]' -type f -exec rm -f {} +
> - rm -f $(TOOLS) $(HELPERS-y) qemu-ga TAGS cscope.* *.pod *~ */*~
> + rm -f $(TOOLS) $(HELPERS) qemu-ga TAGS cscope.* *.pod *~ */*~
> rm -Rf .libs
> rm -f qemu-img-cmds.h
> @# May not be present in GENERATED_HEADERS
> @@ -305,9 +303,9 @@ install: all $(if $(BUILD_DOCS),install-doc)
> install-sysconfig install-datadir
> ifneq ($(TOOLS),)
> $(INSTALL_PROG) $(STRIP_OPT) $(TOOLS) "$(DESTDIR)$(bindir)"
> endif
> -ifneq ($(HELPERS-y),)
> +ifneq ($(HELPERS),)
> $(INSTALL_DIR) "$(DESTDIR)$(libexecdir)"
> - $(INSTALL_PROG) $(STRIP_OPT) $(HELPERS-y) "$(DESTDIR)$(libexecdir)"
> + $(INSTALL_PROG) $(STRIP_OPT) $(HELPERS) "$(DESTDIR)$(libexecdir)"
> endif
> ifneq ($(BLOBS),)
> set -e; for x in $(BLOBS); do \
> diff --git a/configure b/configure
> index 8789324..304c648 100755
> --- a/configure
> +++ b/configure
> @@ -3204,6 +3204,7 @@ qemu_confdir=$sysconfdir$confsuffix
> qemu_datadir=$datadir$confsuffix
>
> tools=""
> +helpers=""
> if test "$want_tools" = "yes" ; then
>tools="qemu-img\$(EXESUF) qemu-io\$(EXESUF) $tools"
>if [ "$linux" = "yes" -o "$bsd" = "yes" -o "$solaris" = "yes" ] ; then
> @@ -3225,9 +3226,12 @@ if test "$softmmu" = yes ; then
>fi
>if [ "$linux" = "yes" -o "$bsd" = "yes" -o "$solaris" = "yes" ] ; then
> if [ "$guest_agent" = "yes" ]; then
> - tools="qemu-ga\$(EXESUF) $tools"
> + helpers="qemu-ga\$(EXESUF) $helpers"
> fi
>fi
> + if [ "$linux" = "yes" ] ; then
> + helpers="qemu-bridge-helper\$(EXESUF) $helpers"
> + fi
> fi
>
> # Mac OS X ships with a broken assembler
> @@ -3744,6 +3748,7 @@ if test "$trace_default" = "yes"; then
> fi
>
> echo "TOOLS=$tools" >> $config_host_mak
> +echo "HELPERS=$helpers" >> $config_host_mak
> echo "ROMS=$roms" >> $config_host_mak
> echo "MAKE=$make" >> $config_host_mak
> echo "INSTALL=$install" >> $config_host_mak
> --
> 1.7.10.4
>
Re: [Qemu-devel] [PATCH] move qemu-ga from bin to libexec dir, use $HELPERS
On Thu, Feb 21, 2013 at 03:39:44PM +0100, Paolo Bonzini wrote: > Il 21/02/2013 15:30, Michael Tokarev ha scritto: > > 21.02.2013 17:35, Paolo Bonzini wrote: > >> Il 16/02/2013 19:28, Michael Tokarev ha scritto: > >>> This patch does 3 things: > >>> > >>> 1. Renames HELPERS-y Makefile variable to HELPERS > >>> 2. Moves its definition from Makefile to configure > >> > >> I prefer to have more decisions in Makefile than configure, but this > >> wouldn't block the patch. What we have now is a mess anyway. > > > > What's the difference between checking for host OS type being > > in makefile or configure? > > That you list all the build products in the Makefile, where they should be. > > > Note that in this very case current condition is a bit wrong: > > it should not only depend on linux but also about presence of > > softmmu targets (which this patch fixes too). > > > >>> 3. Moves qemu-ga binary from TOOLS to HELPERS. > >>> > >>> The effects are: > >>> > >>> 1. qemu-ga binary is now installed into libexecdir, not bindir. > >>> This is the main effect/motivation of this patch, -- this binary > >>> has no business being in a public binary directory, it is a system > >>> helper which must be run by a system startup script or some event > >>> daemon. > >> > >> There is one difference, and an important one: qemu-ga does appear in > >> system-wide configuration files, while qemu-bridge-helper does not. In > >> this sense, qemu-ga is not a helper executable. > > > > Well, it definitely is not a user-callable binary. > > Since we do not ship udev rules, we are really shipping it for the user > to call it. How it does that, we don't care. > > > In that sence it is a > > "system helper" (as opposed to "qemu helper" for qemu-bridge-helper). Ie, > > either sbin or libexec, but not bin. There's no sbindir handling > > currently, -- > > neither in makefile nor in configure, only "TOOLS" and "HELPERS" variables. > > sbindir would be more correct than libexecdir. > > The latest fad for udev is to put helpers in $prefix/lib/udev (not > $libdir, because there's no 32/64-bit differentiation). > > Perhaps the best solution is to add --with-qemu-ga-dir=... and default > it to $bindir. Then distros that ship udev rules can move it to > /usr/lib/udev, distros that ship an initscript can move it to /usr/sbin > or wherever they prefer. Seem reasonable to me. If qemu-ga isn't a TOOL or HELPER, it probably doesn't make sense to think up a new group to configure it until we have other examples to consider. > > >> I have no idea how virtfs-proxy-helper would work, but I suspect that a > >> better design would have QEMU spawning it, just like qemu-bridge-helper. > > > > QEMU can't spawn it, it is spawned in *guest* by a startup script or some > > event daemon (such as systemd or udev). > > I'm talking about virtfs-proxy-helper. > > Paolo > >
Re: [Qemu-devel] [PATCH 3/9] event poll: make epoll work for normal fd
On Thu, Feb 21, 2013 at 08:54:47PM +0800, Liu Ping Fan wrote:
> From: Liu Ping Fan
>
> When event poll can work with normal fd, we can port them
> onto the event loop.
>
> Signed-off-by: Liu Ping Fan
> ---
> hw/dataplane/event-poll.c | 36
> hw/dataplane/event-poll.h |8
> 2 files changed, 44 insertions(+), 0 deletions(-)
>
> diff --git a/hw/dataplane/event-poll.c b/hw/dataplane/event-poll.c
> index 2b55c6e..b7dea53 100644
> --- a/hw/dataplane/event-poll.c
> +++ b/hw/dataplane/event-poll.c
> @@ -32,6 +32,42 @@ void event_poll_add(EventPoll *poll, EventHandler *handler,
> }
> }
>
> +void event_poll_add_fd(EventPoll *poll, int fd, uint32_t type,
> +EventHandler *handler)
> +{
> +struct epoll_event event = {
> +.events = type,
> +.data.ptr = handler,
> +};
> +
> +if (epoll_ctl(poll->epoll_fd, EPOLL_CTL_ADD, fd, &event) != 0) {
> +fprintf(stderr, "failed to add event fd handler to epoll: %m\n");
> +exit(1);
> +}
> +
> +}
An alternative to adding these interfaces would be initializing an
EventNotifier using event_notifier_init_fd() and then passing that in to
event_poll_add()
I think doing it this way would be applicable on top of Paolo's
patches to switch dataplane to using AioContext as well.
> +void event_poll_del_fd(EventPoll *poll, int fd)
> +{
> +if (epoll_ctl(poll->epoll_fd, EPOLL_CTL_DEL, fd, NULL) != 0) {
> +fprintf(stderr, "failed to del event handler to epoll: %m\n");
> +exit(1);
> +}
> +}
> +
> +
> +void event_poll_modify_fd(EventPoll *poll, int fd, uint32_t type,
> +EventHandler *handler)
> +{
> +struct epoll_event event = {
> +.events = type,
> +.data.ptr = handler,
> +};
> +if (epoll_ctl(poll->epoll_fd, EPOLL_CTL_MOD, fd, &event) != 0) {
> +fprintf(stderr, "failed to modify event handler to epoll: %m\n");
> +exit(1);
> +}
> +}
> +
> /* Event callback for stopping event_poll() */
> static void handle_stop(EventHandler *handler)
> {
> diff --git a/hw/dataplane/event-poll.h b/hw/dataplane/event-poll.h
> index 3e8d3ec..606138c 100644
> --- a/hw/dataplane/event-poll.h
> +++ b/hw/dataplane/event-poll.h
> @@ -22,6 +22,8 @@ typedef void EventCallback(EventHandler *handler);
> struct EventHandler {
> EventNotifier *notifier;/* eventfd */
> EventCallback *callback;/* callback function */
> +void *opaque;
> +int fd; /* normal fd*/
> };
>
> typedef struct {
> @@ -32,6 +34,12 @@ typedef struct {
>
> void event_poll_add(EventPoll *poll, EventHandler *handler,
> EventNotifier *notifier, EventCallback *callback);
> +void event_poll_add_fd(EventPoll *poll, int fd, uint32_t type,
> +EventHandler *handler);
> +void event_poll_del_fd(EventPoll *poll, int fd);
> +void event_poll_modify_fd(EventPoll *poll, int fd, uint32_t type,
> +EventHandler *handler);
> +
> void event_poll_init(EventPoll *poll);
> void event_poll_cleanup(EventPoll *poll);
> void event_poll(EventPoll *poll);
> --
> 1.7.4.4
>
>
Re: [Qemu-devel] [PATCH 5/9] event poll: enable event poll handle more than one event each time
On Thu, Feb 21, 2013 at 08:54:49PM +0800, Liu Ping Fan wrote:
> From: Liu Ping Fan
>
> Set handled event count at initialized time for the dedicated
> thread.
>
> Signed-off-by: Liu Ping Fan
> ---
> hw/dataplane/event-poll.c | 31 ++-
> hw/dataplane/event-poll.h |4 +++-
> hw/dataplane/virtio-blk.c |2 +-
> 3 files changed, 18 insertions(+), 19 deletions(-)
>
> diff --git a/hw/dataplane/event-poll.c b/hw/dataplane/event-poll.c
> index e19f3f3..3009787 100644
> --- a/hw/dataplane/event-poll.c
> +++ b/hw/dataplane/event-poll.c
> @@ -25,6 +25,7 @@ void event_poll_add(EventPoll *poll, EventHandler *handler,
> };
> handler->notifier = notifier;
> handler->callback = callback;
> +
> if (epoll_ctl(poll->epoll_fd, EPOLL_CTL_ADD,
>event_notifier_get_fd(notifier), &event) != 0) {
> fprintf(stderr, "failed to add event handler to epoll: %m\n");
> @@ -74,7 +75,7 @@ static void handle_stop(EventHandler *handler, uint32_t
> events)
> /* Do nothing */
> }
>
> -void event_poll_init(EventPoll *poll)
> +void event_poll_init(EventPoll *poll, int num)
> {
> /* Create epoll file descriptor */
> poll->epoll_fd = epoll_create1(EPOLL_CLOEXEC);
> @@ -83,6 +84,8 @@ void event_poll_init(EventPoll *poll)
> exit(1);
> }
>
> +poll->num = num + 1;
> +poll->events = (struct epoll_event *)g_malloc(poll->num * sizeof(struct
> epoll_event));
> /* Set up stop notifier */
> if (event_notifier_init(&poll->stop_notifier, 0) < 0) {
> fprintf(stderr, "failed to init stop notifier\n");
> @@ -103,27 +106,21 @@ void event_poll_cleanup(EventPoll *poll)
> void event_poll(EventPoll *poll)
> {
> EventHandler *handler;
> -struct epoll_event event;
> -int nevents;
> +struct epoll_event *events;
> +int nevents, i;
>
> -/* Wait for the next event. Only do one event per call to keep the
> - * function simple, this could be changed later. */
> +events = poll->events;
> do {
> -nevents = epoll_wait(poll->epoll_fd, &event, 1, -1);
> +nevents = epoll_wait(poll->epoll_fd, events, poll->num, -1);
> } while (nevents < 0 && errno == EINTR);
> -if (unlikely(nevents != 1)) {
> -fprintf(stderr, "epoll_wait failed: %m\n");
> -exit(1); /* should never happen */
> -}
>
> -/* Find out which event handler has become active */
> -handler = event.data.ptr;
> +for (i = 0; i < nevents; i++) {
> +/* Find out which event handler has become active */
> +handler = events[i].data.ptr;
>
> -/* Clear the eventfd */
> -event_notifier_test_and_clear(handler->notifier);
> -
> -/* Handle the event */
> -handler->callback(handler, event.events);
> +/* Handle the event */
> +handler->callback(handler, events[i].events);
> +}
I see where it's needed for vnet dataplane, but removing the test and
clear might cause problems for vblock dataplane. Perhaps we could add
a flag during notifier registration to control the behavior? Not sure
if the AioContext stuff has this already.
> }
>
> /* Stop event_poll()
> diff --git a/hw/dataplane/event-poll.h b/hw/dataplane/event-poll.h
> index ff9712b..f08884b 100644
> --- a/hw/dataplane/event-poll.h
> +++ b/hw/dataplane/event-poll.h
> @@ -30,6 +30,8 @@ typedef struct {
> int epoll_fd; /* epoll(2) file descriptor */
> EventNotifier stop_notifier;/* stop poll notifier */
> EventHandler stop_handler; /* stop poll handler */
> +int num;
> +struct epoll_event *events;
> } EventPoll;
>
> void event_poll_add(EventPoll *poll, EventHandler *handler,
> @@ -40,7 +42,7 @@ void event_poll_del_fd(EventPoll *poll, int fd);
> void event_poll_modify_fd(EventPoll *poll, int fd, uint32_t type,
> EventHandler *handler);
>
> -void event_poll_init(EventPoll *poll);
> +void event_poll_init(EventPoll *poll, int num);
> void event_poll_cleanup(EventPoll *poll);
> void event_poll(EventPoll *poll);
> void event_poll_notify(EventPoll *poll);
> diff --git a/hw/dataplane/virtio-blk.c b/hw/dataplane/virtio-blk.c
> index b60211a..c6a43d8 100644
> --- a/hw/dataplane/virtio-blk.c
> +++ b/hw/dataplane/virtio-blk.c
> @@ -395,7 +395,7 @@ void virtio_blk_data_plane_start(VirtIOBlockDataPlane *s)
> return;
> }
>
> -event_poll_init(&s->event_poll);
> +event_poll_init(&s->event_poll, 2);
>
> /* Set up guest notifier (irq) */
> if (s->vdev->binding->set_guest_notifiers(s->vdev->binding_opaque,
> --
> 1.7.4.4
>
>
Re: [Qemu-devel] [PATCH 6/9] virtio net: introduce dataplane for virtio net
On Thu, Feb 21, 2013 at 08:54:50PM +0800, Liu Ping Fan wrote:
> From: Liu Ping Fan
>
> This is a emulation to virtio-blk dataplane, which push the data
> handling out of biglock. And it is a try to implement this process
> in userspace, while vhost-net in kernel.
>
> Signed-off-by: Liu Ping Fan
> ---
> hw/dataplane/virtio-net.c | 422
> +
> hw/dataplane/virtio-net.h | 26 +++
> hw/virtio-net.c | 56 +-
> hw/virtio-net.h | 61 +++
> 4 files changed, 517 insertions(+), 48 deletions(-)
> create mode 100644 hw/dataplane/virtio-net.c
> create mode 100644 hw/dataplane/virtio-net.h
>
> diff --git a/hw/dataplane/virtio-net.c b/hw/dataplane/virtio-net.c
> new file mode 100644
> index 000..9a1795d
> --- /dev/null
> +++ b/hw/dataplane/virtio-net.c
> @@ -0,0 +1,422 @@
> +/* Copyright IBM, Corp. 2013
> + *
> + * Based on vhost-net and virtio-blk dataplane code
> + *
> + * This work is licensed under the terms of the GNU GPL, version 2.
> + */
> +#include "hw/virtio.h"
> +#include "qemu/iov.h"
> +#include "vring.h"
> +#include
> +#include "net/net.h"
> +#include "net/checksum.h"
> +#include "net/tap.h"
> +#include "virtio-net.h"
> +#include "qemu/error-report.h"
> +
> +typedef struct VirtIONetDataPlane {
> +int async_tx_head;
> +Vring *rx_vring;
> +Vring *tx_vring;
> +EventHandler *rx_handler;
> +EventHandler *tx_handler;
> +bool stop;
> +} VirtIONetDataPlane;
> +
> +WorkThread virt_net_thread;
> +
> +#define VRING_MAX 128
> +
> +static int32_t virtnet_tx(VirtIONet *n, VirtQueue *vq);
> +
> +static void virtnet_tx_complete(struct NetClientState *nc, ssize_t sz)
> +{
> +int ret;
> +VirtIONet *n = DO_UPCAST(NICState, nc, nc)->opaque;
> +
> +vring_push(n->dp->tx_vring, n->dp->async_tx_head, 0);
> +ret = virtnet_tx(n, n->tx_vq);
> +if (ret != -EBUSY) {
> +vring_enable_notification(&n->vdev, n->dp->tx_vring);
> +}
> +}
> +
> +static int virtnet_tx(VirtIONet *n, VirtQueue *vq)
> +{
> +struct iovec out_iov[VRING_MAX], sg[VRING_MAX];
> +struct iovec *snd, *end = &out_iov[VRING_MAX];
> +int head;
> +unsigned int out_num, in_num, sg_num;
> +int ret;
> +int num_packets = 0;
> +
> +if (!(n->vdev.status & VIRTIO_CONFIG_S_DRIVER_OK)) {
> +return num_packets;
> +}
> +
> +assert(n->vdev.vm_running);
> +
> +if (n->async_tx.elem.out_num) {
> +return num_packets;
> +}
> +
> +while (true) {
> +head = vring_pop(&n->vdev, n->dp->tx_vring, out_iov, end, &out_num,
> +&in_num);
> +if (head < 0) {
> +break;
> +}
> +snd = out_iov;
> +assert(n->host_hdr_len <= n->guest_hdr_len);
> +if (n->host_hdr_len != n->guest_hdr_len) {
> +sg_num = iov_copy(sg, ARRAY_SIZE(sg),
> + out_iov, out_num,
> + 0, n->host_hdr_len);
> +sg_num += iov_copy(sg + sg_num, ARRAY_SIZE(sg) - sg_num,
> + out_iov, out_num,
> + n->guest_hdr_len, -1);
> +out_num = sg_num;
> +snd = sg;
> +}
> +
> +ret = qemu_sendv_packet_async(&n->nic->nc, snd, out_num,
> +virtnet_tx_complete);
> +if (ret == 0) {
> +n->dp->async_tx_head = head;
> +return -EBUSY;
> +}
> +vring_push(n->dp->tx_vring, head, 0);
> +if (num_packets++ > n->tx_burst) {
> +break;
> +}
I'm not sure why we'd break here: if we're sending out lots of packets
should we keep notifications disabled and continue sending them till
we'd block? Is it to avoid starving the rx side?
> +}
> +
> +return num_packets;
> +}
> +
> +static void virtnet_handle_tx(VirtIODevice *vdev, VirtQueue *vq)
> +{
> +int32 ret;
> +VirtIONet *n = (VirtIONet *)vdev;
> +
> +/* This happens when device was stopped but VCPU wasn't. */
> +if (!n->vdev.vm_running) {
> +return;
> +}
> +vring_disable_notification(vdev, n->dp->tx_vring);
> +ret = virtnet_tx(n, vq);
> +if (ret != -EBUSY) {
> +vring_enable_notification(vdev, n->dp->tx_vring);
> +}
> +}
> +
> +
> +static int virtio_net_can_receive(NetClientState *nc)
> +{
> +VirtIONet *n = DO_UPCAST(NICState, nc, nc)->opaque;
> +if (!n->vdev.vm_running) {
> +return 0;
> +}
> +if (!(n->vdev.status & VIRTIO_CONFIG_S_DRIVER_OK)) {
> +return 0;
> +}
> +
> +return 1;
> +}
> +
> +/* peek but not use */
> +static int rx_mergeable_buf_sz(VirtIONet *n)
> +{
> +uint16_t start, idx, head;
> +int total = 0;
> +Vring *vring = n->dp->rx_vring;
> +struct vring_desc *dsc;
> +struct vring_desc *base;
> +
> +for (start = vring->last_avail_idx; start != vring->vr.avail->idx;
> +start++) {
> +head = start%vring->vr
Re: [Qemu-devel] [PATCH 0/9] introduce virtio net dataplane
On Thu, Feb 21, 2013 at 06:05:44PM +0100, Paolo Bonzini wrote: > Il 21/02/2013 17:33, Stefan Hajnoczi ha scritto: > > Interesting patch series. I want to share my thoughts on the status of > > dataplane and what I'm currently working on. There will probably be > > some overlap that we can coordinate. > > > > First, I want to eventually delete hw/dataplane/ :). That is because > > dataplane duplicates an event loop, thread-friendly guest RAM access, > > and virtio. QEMU already has all this functionality except it's not > > thread-friendly. > > > > Because hw/dataplane/ will eventually go away we should avoid adding new > > code where possible. > > 100% agree. In particular hw/dataplane/event-poll.c should be the first > to go away, but AioContext provides the functionality that Ping Fan > needs. But hw/dataplane/vring.c will probably be here for a longer Has there been any discussion around introducing something similar to AioContexts for fd handlers? This would avoid the dataplane-specific hooks needed for NetClients in this series.
Re: [Qemu-devel] [PATCH] dataplane: remove EventPoll in favor of AioContext
On Thu, Feb 21, 2013 at 05:29:55PM +0100, Paolo Bonzini wrote:
> During the review of the dataplane code, the EventPoll API morphed itself
> (not concidentially) into something very very similar to an AioContext.
> Thus, it is trivial to convert virtio-blk-dataplane to use AioContext,
> and a first baby step towards letting dataplane talk directly to the
> QEMU block layer.
>
> The only interesting note is the value-copy of EventNotifiers. At least
> in my opinion this is part of the EventNotifier API and is even portable
> to Windows. Of course, in this case you should not close the notifier's
> underlying file descriptors or handle with event_notifier_cleanup.
>
> Signed-off-by: Paolo Bonzini
> ---
> hw/dataplane/Makefile.objs | 2 +-
> hw/dataplane/event-poll.c | 100
> -
> hw/dataplane/event-poll.h | 40 --
> hw/dataplane/virtio-blk.c | 41 ++-
> 4 files changed, 22 insertions(+), 161 deletions(-)
> delete mode 100644 hw/dataplane/event-poll.c
> delete mode 100644 hw/dataplane/event-poll.h
>
> diff --git a/hw/dataplane/Makefile.objs b/hw/dataplane/Makefile.objs
> index 3e47d05..70c 100644
> --- a/hw/dataplane/Makefile.objs
> +++ b/hw/dataplane/Makefile.objs
> @@ -1 +1 @@
> -obj-$(CONFIG_VIRTIO_BLK_DATA_PLANE) += hostmem.o vring.o event-poll.o ioq.o
> virtio-blk.o
> +obj-$(CONFIG_VIRTIO_BLK_DATA_PLANE) += hostmem.o vring.o ioq.o virtio-blk.o
> diff --git a/hw/dataplane/event-poll.c b/hw/dataplane/event-poll.c
> deleted file mode 100644
> index 2b55c6e..000
> --- a/hw/dataplane/event-poll.c
> +++ /dev/null
> @@ -1,100 +0,0 @@
> -/*
> - * Event loop with file descriptor polling
> - *
> - * Copyright 2012 IBM, Corp.
> - * Copyright 2012 Red Hat, Inc. and/or its affiliates
> - *
> - * Authors:
> - * Stefan Hajnoczi
> - *
> - * This work is licensed under the terms of the GNU GPL, version 2 or later.
> - * See the COPYING file in the top-level directory.
> - *
> - */
> -
> -#include
> -#include "hw/dataplane/event-poll.h"
> -
> -/* Add an event notifier and its callback for polling */
> -void event_poll_add(EventPoll *poll, EventHandler *handler,
> -EventNotifier *notifier, EventCallback *callback)
> -{
> -struct epoll_event event = {
> -.events = EPOLLIN,
> -.data.ptr = handler,
> -};
> -handler->notifier = notifier;
> -handler->callback = callback;
> -if (epoll_ctl(poll->epoll_fd, EPOLL_CTL_ADD,
> - event_notifier_get_fd(notifier), &event) != 0) {
> -fprintf(stderr, "failed to add event handler to epoll: %m\n");
> -exit(1);
> -}
> -}
> -
> -/* Event callback for stopping event_poll() */
> -static void handle_stop(EventHandler *handler)
> -{
> -/* Do nothing */
> -}
> -
> -void event_poll_init(EventPoll *poll)
> -{
> -/* Create epoll file descriptor */
> -poll->epoll_fd = epoll_create1(EPOLL_CLOEXEC);
> -if (poll->epoll_fd < 0) {
> -fprintf(stderr, "epoll_create1 failed: %m\n");
> -exit(1);
> -}
> -
> -/* Set up stop notifier */
> -if (event_notifier_init(&poll->stop_notifier, 0) < 0) {
> -fprintf(stderr, "failed to init stop notifier\n");
> -exit(1);
> -}
> -event_poll_add(poll, &poll->stop_handler,
> - &poll->stop_notifier, handle_stop);
> -}
> -
> -void event_poll_cleanup(EventPoll *poll)
> -{
> -event_notifier_cleanup(&poll->stop_notifier);
> -close(poll->epoll_fd);
> -poll->epoll_fd = -1;
> -}
> -
> -/* Block until the next event and invoke its callback */
> -void event_poll(EventPoll *poll)
> -{
> -EventHandler *handler;
> -struct epoll_event event;
> -int nevents;
> -
> -/* Wait for the next event. Only do one event per call to keep the
> - * function simple, this could be changed later. */
> -do {
> -nevents = epoll_wait(poll->epoll_fd, &event, 1, -1);
> -} while (nevents < 0 && errno == EINTR);
> -if (unlikely(nevents != 1)) {
> -fprintf(stderr, "epoll_wait failed: %m\n");
> -exit(1); /* should never happen */
> -}
> -
> -/* Find out which event handler has become active */
> -handler = event.data.ptr;
> -
> -/* Clear the eventfd */
> -event_notifier_test_and_clear(handler->notifier);
Wouldn't we need to move this into the handle_io/handle_notify to maintain the
old behavior?
Re: [Qemu-devel] [PATCH] slirp: fixed potential use-after-free of a socket
On Fri, Feb 15, 2013 at 12:00:13PM +0100, Vitaly Chipounov wrote:
> A socket may still have references to it in various queues
> at the time it is freed, causing memory corruptions.
>
> Signed-off-by: Vitaly Chipounov
> ---
> slirp/socket.c | 29 +
> 1 file changed, 29 insertions(+)
>
> diff --git a/slirp/socket.c b/slirp/socket.c
> index 77b0c98..8a7adc8 100644
> --- a/slirp/socket.c
> +++ b/slirp/socket.c
> @@ -55,6 +55,33 @@ socreate(Slirp *slirp)
>return(so);
> }
>
> +/**
> + * It may happen that a socket is still referenced in various
> + * mbufs at the time it is freed. Clear all references to the
> + * socket here.
> + */
> +static void soremovefromqueues(struct socket *so)
> +{
> +if (!so->so_queued) {
> +return;
> +}
> +
> +Slirp *slirp = so->slirp;
> +struct mbuf *ifm;
Declarations should come at the beginning of a block/function (see: ...
er, I could've sworn it was in HACKING or CODING_STYLE but maybe not.
That's the standard in any case)
> +
> +for (ifm = slirp->if_fastq.ifq_next; ifm != &slirp->if_fastq; ifm =
> ifm->ifq_next) {
> +if (ifm->ifq_so == so) {
> +ifm->ifq_so = NULL;
> +}
> +}
> +
> +for (ifm = slirp->if_batchq.ifq_next; ifm != &slirp->if_batchq; ifm =
> ifm->ifq_next) {
> +if (ifm->ifq_so == so) {
> +ifm->ifq_so = NULL;
> +}
> +}
Is the intention just to mark them null or actually remove? Not sure
which, but either the logic should change or the function name should
(soinvalidate() or something along that line if the former).
> +}
> +
> /*
> * remque and free a socket, clobber cache
> */
> @@ -79,6 +106,8 @@ sofree(struct socket *so)
>if(so->so_next && so->so_prev)
> remque(so); /* crashes if so is not in a queue */
>
> + soremovefromqueues(so);
> +
>free(so);
> }
>
> --
> 1.7.10
>
>
Re: [Qemu-devel] [PATCH] slirp: fixed potential use-after-free of a socket
On Thu, Feb 21, 2013 at 03:47:25PM -0600, mdroth wrote:
> On Fri, Feb 15, 2013 at 12:00:13PM +0100, Vitaly Chipounov wrote:
> > A socket may still have references to it in various queues
> > at the time it is freed, causing memory corruptions.
> >
> > Signed-off-by: Vitaly Chipounov
Meant to cc qemu-stable
> > ---
> > slirp/socket.c | 29 +
> > 1 file changed, 29 insertions(+)
> >
> > diff --git a/slirp/socket.c b/slirp/socket.c
> > index 77b0c98..8a7adc8 100644
> > --- a/slirp/socket.c
> > +++ b/slirp/socket.c
> > @@ -55,6 +55,33 @@ socreate(Slirp *slirp)
> >return(so);
> > }
> >
> > +/**
> > + * It may happen that a socket is still referenced in various
> > + * mbufs at the time it is freed. Clear all references to the
> > + * socket here.
> > + */
> > +static void soremovefromqueues(struct socket *so)
> > +{
> > +if (!so->so_queued) {
> > +return;
> > +}
> > +
> > +Slirp *slirp = so->slirp;
> > +struct mbuf *ifm;
>
> Declarations should come at the beginning of a block/function (see: ...
> er, I could've sworn it was in HACKING or CODING_STYLE but maybe not.
> That's the standard in any case)
>
> > +
> > +for (ifm = slirp->if_fastq.ifq_next; ifm != &slirp->if_fastq; ifm =
> > ifm->ifq_next) {
> > +if (ifm->ifq_so == so) {
> > +ifm->ifq_so = NULL;
> > +}
> > +}
> > +
> > +for (ifm = slirp->if_batchq.ifq_next; ifm != &slirp->if_batchq; ifm =
> > ifm->ifq_next) {
> > +if (ifm->ifq_so == so) {
> > +ifm->ifq_so = NULL;
> > +}
> > +}
>
> Is the intention just to mark them null or actually remove? Not sure
> which, but either the logic should change or the function name should
> (soinvalidate() or something along that line if the former).
>
> > +}
> > +
> > /*
> > * remque and free a socket, clobber cache
> > */
> > @@ -79,6 +106,8 @@ sofree(struct socket *so)
> >if(so->so_next && so->so_prev)
> > remque(so); /* crashes if so is not in a queue */
> >
> > + soremovefromqueues(so);
> > +
> >free(so);
> > }
> >
> > --
> > 1.7.10
> >
> >
Re: [Qemu-devel] [PATCH] fix wrong output with 'info chardev' for tcp socket.
On Fri, Feb 22, 2013 at 12:29:44AM +0400, Michael Tokarev wrote: > 22.02.2013 00:20, Serge E. Hallyn wrote: > > The snprintf format isn't taking into account the new 'left' and > > 'right' variables (for ipv6 []) when placing the ':', which should > > go immediately before the port. > > This fixes actual isse (also found by Serge), where `info chardev' > prints `tcp:127.0.0.1,server,nowait' for a monitor running on port > . > > This is definitely a stable material (CCed). > > Reviewed-by: Michael Tokarev Reviewed-by: Michael Roth > > Thanks! > > /mjt > > > Signed-off-by: Serge Hallyn > > --- > > qemu-char.c | 2 +- > > 1 file changed, 1 insertion(+), 1 deletion(-) > > > > diff --git a/qemu-char.c b/qemu-char.c > > index e4b0f53..3e152e1 100644 > > --- a/qemu-char.c > > +++ b/qemu-char.c > > @@ -2482,7 +2482,7 @@ static CharDriverState *qemu_chr_open_socket_fd(int > > fd, bool do_nodelay, > > s->do_nodelay = do_nodelay; > > getnameinfo((struct sockaddr *) &ss, ss_len, host, sizeof(host), > > serv, sizeof(serv), NI_NUMERICHOST | NI_NUMERICSERV); > > -snprintf(chr->filename, 256, "%s:%s:%s%s%s%s", > > +snprintf(chr->filename, 256, "%s:%s%s%s:%s%s", > > is_telnet ? "telnet" : "tcp", > > left, host, right, serv, > > is_listen ? ",server" : ""); > > > >
Re: [Qemu-devel] BUG: RTC issue when Windows guest is idle
On Thu, Feb 21, 2013 at 06:16:10PM +, Matthew Anderson wrote: > If this isn't the correct list just let me know, > > I've run into a bug whereby a Windows guest (tested on Server 2008R2 and > 2012) no longer receives RTC ticks when it has been idle for a random amount > of time. HPET is disabled and the guest is running Hyper-V relaxed timers > (same situation without hv_relaxed). The guest clock stands still and the > qemu process uses very little CPU (<0.5%, normally it's >5% when the guest is > idle) . Eventually the guest stops responding to network requests but if you > open the guest console via VNC and move the mouse around it comes back to > life and QEMU replays the lost RTC ticks and the guest recovers. I've also > been able to make it recover by querying the clock over the network via the > net time command, you can see the clock stand still for 30 seconds then it > replays the ticks and catches up. > > I've tried to reproduce the issue but it seems fairly illusive, the only way > I've been able to reproduce it is by letting the VM's idle and waiting. > Sometimes it's hours and sometimes minutes. Can anyone suggest a way to > narrow the issue down? > > Qemu command line is- > /usr/bin/kvm -name SQL01 -S -M pc-0.14 -cpu qemu64,hv_relaxed -enable-kvm -m > 2048 -smp 2,sockets=2,cores=1,threads=1 -uuid > 5f54333b-c250-aa72-c979-39d156814b85 -no-user-config -nodefaults -chardev > socket,id=charmonitor,path=/var/lib/libvirt/qemu/iHost-SQL01.monitor,server,nowait > -mon chardev=charmonitor,id=monitor,mode=control -rtc base=localtime > -no-hpet -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 > -drive > file=/mnt/gluster1-norep/iHost/SQL01.qed,if=none,id=drive-virtio-disk0,format=qed,cache=writeback > -device > virtio-blk-pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0 > -drive > file=/mnt/gluster1-norep/iHost/SQL01-Data.qed,if=none,id=drive-virtio-disk2,format=qed,cache=writeback > -device > virtio-blk-pci,scsi=off,bus=pci.0,addr=0x6,drive=drive-virtio-disk2,id=virtio-disk2 > -drive if=none,id=drive-ide0-1-0,readonly=on,format=raw -device > ide-cd,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0,bootindex=1 -netdev > tap,fd=29,id=hostnet0,vhost=on,vhostfd=39 -device > virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:2c:8d:23,bus=pci.0,addr=0x3 > -chardev pty,id=charserial0 -device > isa-serial,chardev=charserial0,id=serial0 -device usb-tablet,id=input0 -vnc > 127.0.0.1:22 -vga cirrus -device > virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x4 > > Environment is - > Mainline 3.7.5 and 3.8.0 > Qemu 1.2.2, 1.3.1 and 1.4.0 Were all of these with -M pc-0.14? Only thing that stands out to me is kernel_irqchip being disabled in your case. -M 1.1 and higher will enable it by default. Worth a shot. > Scientific Linux 6.3 > KSM enabled, transparent hugepages disabled. > Dual Xeon 5650 > 192GB > > Thanks all >
Re: [Qemu-devel] [PATCH 0/9] introduce virtio net dataplane
On Thu, Feb 21, 2013 at 11:12:01PM +0100, Paolo Bonzini wrote: > Il 21/02/2013 22:07, mdroth ha scritto: > >> > > >> > 100% agree. In particular hw/dataplane/event-poll.c should be the first > >> > to go away, but AioContext provides the functionality that Ping Fan > >> > needs. But hw/dataplane/vring.c will probably be here for a longer > > Has there been any discussion around introducing something similar to > > AioContexts for fd handlers? This would avoid the dataplane-specific hooks > > needed for NetClients in this series. > > AioContext can include file descriptors on POSIX systems (used for NBD > and other network backends), see aio_set_fd_handler. Sorry, was using "fd handlers" too generally. I mean specifically for the qemu_set_fd_handler interfaces, where we currently rely on a single list of IOHandlerRecords for registration and a single loop to drive them. Would be nice if we could drive subsets of those via mini main loops, similar to the way dataplane threads would with a particular AioContext via aio_poll (or perhaps the exact same way) Currently, Ping Fan's patches basically do this already by accessing a global reference to the vnet worker thread and attaching events/handlers to it's event loop via a new set of registration functions (PATCH 7). I think generalizing this by basing qemu_set_fd_handler() around AioContext, or something similar, would help to extend support to other NetClient implementations without requiring dataplane-specific hooks throughout. > > Paolo >
Re: [Qemu-devel] GTK UI is now the default
On Fri, Feb 22, 2013 at 04:48:47PM +0100, Paolo Bonzini wrote: > Il 22/02/2013 16:17, Anthony Liguori ha scritto: > >> > FWIW I'd be in favor of making grab-on-hover the default. > > It really breaks window manager key bindings. If you alt-tab between > > windows and you happen to end up with the window under your mouse, > > alt-tab stops working. > > > > I think as the default behavior, you want the least amount of > > surprises. To me alt-tab not working is a pretty big surprise. > > Alt-tab moving you away from the guest is just as much a surprise. :) True, but if you're interacting with a guest to the extent that you're alt-tabbing around, it's not an unreasonable expectation that the user initiate a screen-grab aforetime. A shiny "grab/ungrab screen" button might also help to set expectations in advance. Keeping your mouse away from the window isn't quite as easy/predictable. You might do it by accident while switching through workspaces for instance, or by closing out a different app if you're using a tiling window manager. It's not a huge deal either way if it's configurable and persistant, but IMO there's more rationale for disabling grab-on-hover by default. > > Paolo >
Re: [Qemu-devel] 'info help'
On Fri, Feb 22, 2013 at 05:10:29PM +, Serge E. Hallyn wrote: > Hi, > > up to and including 1.3.0, monitor.c:do_info(), if it got no arg or an > unknown arg, would do help_cmd(mon, "info"); That behavior is gone in > 1.4.0, so that 'info', 'info help', and 'info whatever' just say unknown > argument. Was that intended, or would re-introducing the old behavior > be acceptable? Seems intended: 84c44613f9ad8d13e0d2dbee767051527072dc12 introduced the change as a result of refactoring, and the change in behavior was documented (and the original behavior was never really 'official': 'help info' was always the documented help text). > > -serge >
Re: [Qemu-devel] [Bug 1131757] [NEW] QEMU 1.4.0 fails to boot sparc64 linux image
On Fri, Feb 22, 2013 at 02:33:43PM -, Kirill Tkhai wrote: > Public bug reported: > > Hi! > > I tried to boot sparc64 linux image > (http://packages.debian.org/sid/sparc64/linux- > image-2.6-sparc64-smp/download) with qemu and received the error. > > host:~$qemu-system-sparc64 -nographic -kernel vmlinuz-3.2.0-4-sparc64-smp > OpenBIOS for Sparc64 > Configuration device id QEMU version tion device id QEMUkernel addr n device > id QEMUkernel cmdline > CPUs: cmdline > x SUNW,UltraSPARC-IIi > UUID: 00UltraSPARC-IIi > Welcome to OpenBIOS v1.0 built on Aug 19 2012 13:06 > Type 'help' for detailed information > [sparc64] Kernel already loaded > Unhandled Exception 0x0020 > PC = 0x00404000 NPC = 0x00404004 > Stopping execution > > Also, I tried to follow instruction from Artyom Tarasenko blog > (http://tyom.blogspot.ru/2012/05/booting-linuxsparc64-on-todays- > openbios.html), but it's still impossible to boot linux. I don't see any change in behavior from 1.3.0 (haven't looked back further). It appears to be attempting to boot garbage after the initial load. From the blog you mentioned you can prevent this with the -prom-env 'auto-boot?=false' option, which seems to work for me on both 1.3.0 and 1.4.0: mdroth@loki:~/w/qemu-build2$ sparc64-softmmu/qemu-system-sparc64 -kernel ~/w/qemu-test-images/sparc/sparc-test/vmlinux-2.6.11+tcx -prom-env 'auto-boot?=false' -L pc-bios/ -nographic OpenBIOS for Sparc64 Configuration device id QEMU version 1 machine id 0 kernel addr 404000 size b3d3d5 kernel cmdline CPUs: 1 x SUNW,UltraSPARC-IIi UUID: ---- Welcome to OpenBIOS v1.0 built on Aug 19 2012 13:06 Type 'help' for detailed information 0 > Can you test with a full boot using an image/initrd? I'd point you to the one at http://wiki.qemu.org/Testing but I've never managed to get it to work: sparc64-softmmu/qemu-system-sparc64 -kernel ~/w/qemu-test-images/sparc/sparc-test/vmlinux-2.6.11+tcx -initrd ~/w/qemu-test-images/sparc/sparc-test/linux.img -append "root=/dev/ram console=ttyS0 video=tcxfb:off" -L pc-bios/ -nographic -m 16M rom: requested regions overlap (rom /home/mdroth/w/qemu-test-images/sparc/sparc-test/linux.img. free=0x01fff01b5fa0, addr=0x01fff000) rom loading failed sparc-softmmu/qemu-system-sparc -kernel ~/w/qemu-test-images/sparc/sparc-test/vmlinux-2.6.11+tcx -hda ~/w/qemu-test-images/sparc/sparc-test/linux.img -append "root=/dev/ram console=ttyS0 video=tcxfb:off" -L pc-bios/ -nographic -m 16M Unhandled Exception 0x0029 PC = 0xffd07470 NPC = 0xffd07474 Stopping execution Would be great if you had another you could add to the wiki. > > Regards, > Kirill > > ** Affects: qemu > Importance: Undecided > Status: New > > > ** Tags: linux sparc64 > > -- > You received this bug notification because you are a member of qemu- > devel-ml, which is subscribed to QEMU. > https://bugs.launchpad.net/bugs/1131757 > > Title: > QEMU 1.4.0 fails to boot sparc64 linux image > > Status in QEMU: > New > > Bug description: > Hi! > > I tried to boot sparc64 linux image > (http://packages.debian.org/sid/sparc64/linux- > image-2.6-sparc64-smp/download) with qemu and received the error. > > host:~$qemu-system-sparc64 -nographic -kernel vmlinuz-3.2.0-4-sparc64-smp > OpenBIOS for Sparc64 > Configuration device id QEMU version tion device id QEMUkernel addr n > device id QEMUkernel cmdline > CPUs: cmdline >x SUNW,UltraSPARC-IIi > UUID: 00UltraSPARC-IIi > Welcome to OpenBIOS v1.0 built on Aug 19 2012 13:06 > Type 'help' for detailed information > [sparc64] Kernel already loaded > Unhandled Exception 0x0020 > PC = 0x00404000 NPC = 0x00404004 > Stopping execution > > Also, I tried to follow instruction from Artyom Tarasenko blog > (http://tyom.blogspot.ru/2012/05/booting-linuxsparc64-on-todays- > openbios.html), but it's still impossible to boot linux. > > Regards, > Kirill > > To manage notifications about this bug go to: > https://bugs.launchpad.net/qemu/+bug/1131757/+subscriptions >
Re: [Qemu-devel] [PATCH 0/9] introduce virtio net dataplane
On Fri, Feb 22, 2013 at 09:53:11AM +0100, Paolo Bonzini wrote:
> Il 22/02/2013 00:38, mdroth ha scritto:
> > On Thu, Feb 21, 2013 at 11:12:01PM +0100, Paolo Bonzini wrote:
> >> Il 21/02/2013 22:07, mdroth ha scritto:
> >>>>>
> >>>>> 100% agree. In particular hw/dataplane/event-poll.c should be the first
> >>>>> to go away, but AioContext provides the functionality that Ping Fan
> >>>>> needs. But hw/dataplane/vring.c will probably be here for a longer
> >>> Has there been any discussion around introducing something similar to
> >>> AioContexts for fd handlers? This would avoid the dataplane-specific hooks
> >>> needed for NetClients in this series.
> >>
> >> AioContext can include file descriptors on POSIX systems (used for NBD
> >> and other network backends), see aio_set_fd_handler.
> >
> > Sorry, was using "fd handlers" too generally. I mean specifically for
> > the qemu_set_fd_handler interfaces, where we currently rely on a single list
> > of IOHandlerRecords for registration and a single loop to drive them. Would
> > be nice if we could drive subsets of those via mini main loops, similar to
> > the
> > way dataplane threads would with a particular AioContext via aio_poll (or
> > perhaps
> > the exact same way)
>
> Yes, that's what I meant actually. You can already do it for POSIX,
> unfortunately Windows poses extra complication because sockets are not
> handles.
>
> Moving more of the os_host_main_loop_wait to AioContext would be
> possible (timers are on the todo list, in fact), but we should only do
> it as need arises.
Were you planning on hanging another GSource off of AioContext to handle
timers?
I wonder if adding an interface to AioContext that allows you to just
register an arbitrary GSource might be the right approach.
We could consolidate qemu_set_fd_handler()/qemu_aio_set_fd_handler() on
POSIX by teaching the current GSource about fd_read_poll functions, and on
Windows qemu_set_fd_handler() would tie into a winsock-specific GSource
that we register with an AioContext. Might be able to do similar with
GSources for slirp and the qemu_add_wait_object() stuff.
Or maybe that's the plan already... glib-style main loops, but not
intrinsically tethered to GMainContext so we can implement our own
locking/priority/loop constructs if need be... seems to be what's been
discussed elsewhere, just trying to get an idea of what that might look
like.
> possible (timers are on the todo list, in fact), but we should only do
> it as need arises.
Yup, don't mean to get ahead of things, my main interest is just in how
we might deal with the interaction between NetClients and virtio-net
dataplane threads without introducing ad-hoc, dataplane-specific
mechanisms. If there was a general way for Nic to tell it's NetClient
peer "hey, i have my own thread/main loop, here's my {Aio,*}Context, register
your handlers there instead" I think this series might look a lot more
realistic as a default, or at least make merging it less risky. But the
right way to do that seems to tie into the discussion around making
other aio sources more GMainContext/AioContext-ish.
>
> Paolo
>
> > Currently, Ping Fan's patches basically do this already by accessing a
> > global reference to the vnet worker thread and attaching events/handlers to
> > it's event loop via a new set of registration functions (PATCH 7).
> >
> > I think generalizing this by basing qemu_set_fd_handler() around
> > AioContext, or something similar, would help to extend support to other
> > NetClient implementations without requiring dataplane-specific hooks
> > throughout.
>
Re: [Qemu-devel] [PATCH 1/9] chardev: add support for qapi-based chardev initialization
On Mon, Feb 25, 2013 at 10:03:33AM +0100, Gerd Hoffmann wrote:
> This patch add support for a new way to initialize chardev devices.
> Instead of calling a initialization function with a QemuOpts we will
> now create a (qapi) ChardevBackend, optionally call a function to
> fill ChardevBackend from QemuOpts, then go create the chardev using
> the new qapi code path which is also used by chardev-add.
>
> Signed-off-by: Gerd Hoffmann
> ---
> qemu-char.c | 30 ++
> 1 file changed, 30 insertions(+)
>
> diff --git a/qemu-char.c b/qemu-char.c
> index 160decc..cf6b98b 100644
> --- a/qemu-char.c
> +++ b/qemu-char.c
> @@ -2974,7 +2974,11 @@ static CharDriverState *qemu_chr_open_pp(QemuOpts
> *opts)
>
> static const struct {
> const char *name;
> +/* old, pre qapi */
> CharDriverState *(*open)(QemuOpts *opts);
> +/* new, qapi-based */
> +const int kind;
> +void (*parse)(QemuOpts *opts, ChardevBackend *backend, Error **errp);
> } backend_table[] = {
> { .name = "null", .open = qemu_chr_open_null },
> { .name = "socket",.open = qemu_chr_open_socket },
> @@ -3040,6 +3044,32 @@ CharDriverState *qemu_chr_new_from_opts(QemuOpts *opts,
> goto err;
> }
>
> +if (!backend_table[i].open) {
> +/* using new, qapi init */
> +ChardevBackend *backend = g_new0(ChardevBackend, 1);
> +ChardevReturn *ret = NULL;
> +const char *id = qemu_opts_id(opts);
> +
> +chr = NULL;
> +backend->kind = backend_table[i].kind;
> +if (backend_table[i].parse) {
> +backend_table[i].parse(opts, backend, errp);
> +if (error_is_set(errp)) {
> +goto qapi_out;
> +}
> +}
Have you tried using visit_type_ChardevBackend() with an OptsVisitor to
handle the option parsing? It's how -netdev options are parsed now, so
it should "just work" in theory.
Might be worth looking at now as opposed to a follow-up since it'll
avoid the need to introduce .parse functions to the backend table as you
go.
> +ret = qmp_chardev_add(qemu_opts_id(opts), backend, errp);
> +if (error_is_set(errp)) {
> +goto qapi_out;
> +}
> +chr = qemu_chr_find(id);
> +
> +qapi_out:
> +qapi_free_ChardevBackend(backend);
> +qapi_free_ChardevReturn(ret);
> +return chr;
> +}
> +
> chr = backend_table[i].open(opts);
> if (!chr) {
> error_setg(errp, "chardev: opening backend \"%s\" failed",
> --
> 1.7.9.7
>
>
Re: [Qemu-devel] [PATCH 1/3] two new file wrappers
On Wed, Feb 27, 2013 at 08:46:19AM -0500, Stefan Berger wrote:
> On 02/26/2013 09:19 PM, Anthony Liguori wrote:
> >jsch...@linux.vnet.ibm.com writes:
> >
> >I think we've moved away from using qemu-file for anything other than
> >migration.
>
> Our goal was to use the abstraction of a QEMUFile for writing into
> memory buffers and later on to also provide a wrapper for writing
> into block devices. This was all to be used for the NVRAM
> implementation necessary for storing TPM persistent state.
One approach is to just keep all of this inside BER*Visitor, and do the
distinction between memory/file via, for example,
ber_output_visitor_new_file()/ber_output_visitor_new[_buffer]()
Would be kinda cool to move that into a {BE,LE}{File,Buffer}Visitor
so you could use the visitor interfaces to handle the endian-conversions
automagically, and then use that in BER*Visitor internally, but I'm not
not sure how well that would work in practice.
Although, I did try to do something like that in the past to use it as a shim
between the qemu_{put|get}* interfaces and QEMUFile in a misguided attempt to
introduce a BER protocol for migration, but something similar might still
be useful in your case:
http://lists.gnu.org/archive/html/qemu-devel/2011-09/msg02466.html
There's currently no visitor interface for arbitrary-length byte buffers
though. The aforementioned patches relied on users creating their
own loops around visit_type_int*() and friends. There's a patch floating
around to add them but I don't see a need for it until we have a need to
add support for specifying arrays in schema-defined QAPI types.
Just another approach to consider though, I wouldn't see a problem with all
this being done inside BER*Visitor for now.
>
>Stefan
>
>
Re: [Qemu-devel] [PATCH 2/3] asn1 ber visitors
On Wed, Feb 27, 2013 at 02:03:45PM -0600, Joel Schopp wrote: > Thanks for the feedback. > > On 02/27/2013 02:28 AM, Andreas Färber wrote: > >Am 27.02.2013 00:03, schrieb jsch...@linux.vnet.ibm.com: > >>These patches implement asn1 ber visitors for encoding and decoding data. > > > >Would be good to not be lazy and spell them correctly in at least one of > >the two lines of the commit where they're being introduced. > > I will change "asn1 ber" to "ASN.1 BER" > > >I would suggest to place your headers into existing include/qapi/ for > >instance and the remaining source files into qapi/ or a subdirectory > >thereof rather than a new top-level directory. > > > >I also note that you're not changing MAINTAINERS, so the new files don't > >have a maintainer assigned that gets CC'ed on changes. > > I'll update MAINTAINERS and move the headers into include. > > As for the location of the source files I think where the patch puts > them now makes most sense, I don't think qapi makes sense. But the > location isn't important to me so I'll put them wherever. Anybody > else have an opinion on the best location? qapi/ along with all the other visitor implementations is probably the right place for now. If in the future we end up with dozens of implementations pulling into ridiculous amounts of dependencies we might want to reconsider things, but for now it's safest to assume that any serialization format for can be selected at runtime for a Visitor user, so they should all be made available via qapi-obj-y. It's also possible this might fix whatever build issue you're running into for free :)
Re: [Qemu-devel] [PATCH 1/3] two new file wrappers
On Wed, Feb 27, 2013 at 03:56:27PM -0600, mdroth wrote:
> On Wed, Feb 27, 2013 at 08:46:19AM -0500, Stefan Berger wrote:
> > On 02/26/2013 09:19 PM, Anthony Liguori wrote:
> > >jsch...@linux.vnet.ibm.com writes:
> > >
> > >I think we've moved away from using qemu-file for anything other than
> > >migration.
> >
> > Our goal was to use the abstraction of a QEMUFile for writing into
> > memory buffers and later on to also provide a wrapper for writing
> > into block devices. This was all to be used for the NVRAM
> > implementation necessary for storing TPM persistent state.
>
> One approach is to just keep all of this inside BER*Visitor, and do the
> distinction between memory/file via, for example,
> ber_output_visitor_new_file()/ber_output_visitor_new[_buffer]()
>
> Would be kinda cool to move that into a {BE,LE}{File,Buffer}Visitor
> so you could use the visitor interfaces to handle the endian-conversions
> automagically, and then use that in BER*Visitor internally, but I'm not
> not sure how well that would work in practice.
>
> Although, I did try to do something like that in the past to use it as a shim
> between the qemu_{put|get}* interfaces and QEMUFile in a misguided attempt to
> introduce a BER protocol for migration, but something similar might still
> be useful in your case:
>
> http://lists.gnu.org/archive/html/qemu-devel/2011-09/msg02466.html
>
> There's currently no visitor interface for arbitrary-length byte buffers
> though. The aforementioned patches relied on users creating their
> own loops around visit_type_int*() and friends. There's a patch floating
> around to add them but I don't see a need for it until we have a need to
> add support for specifying arrays in schema-defined QAPI types.
... okay, now i see the need :) see my response in patch 2.
>
> Just another approach to consider though, I wouldn't see a problem with all
> this being done inside BER*Visitor for now.
>
> >
> >Stefan
> >
> >
Re: [Qemu-devel] [PATCH 2/3] asn1 ber visitors
On Wed, Feb 27, 2013 at 06:24:45PM -0500, Stefan Berger wrote:
> On 02/27/2013 05:57 PM, mdroth wrote:
> >On Tue, Feb 26, 2013 at 05:03:56PM -0600, jsch...@linux.vnet.ibm.com wrote:
> >>These patches implement asn1 ber visitors for encoding and decoding data.
> >>References: <20130226230354.982917...@linux.vnet.ibm.com>
> >>Content-Disposition: inline; filename=asn1_all.diff
> >>
> >>Signed-off-by: Stefan Berger
> >>Signed-off-by: Joel Schopp
> >>---
> >> Makefile.objs | 10
> >> ber/Makefile.objs |2
> >> ber/ber-common.c| 56 ++
> >> ber/ber-input-visitor.c | 969
> >>
> >> ber/ber-input-visitor.h | 30 +
> >> ber/ber-output-visitor.c| 563 +
> >> ber/ber-output-visitor.h| 28 +
> >Please break the input/output implementations out into separate patches,
> >and follow those up with test cases. See:
> >
> >tests/test-{string,qmp}-{input,output}-visitor.c
> >tests/test-visitor-serialization.c
>
> I saw this with the other test cases... The problem with the ASN.1
> encoding is that writing ASN.1 into a buffer doesn't make much
> sense, unless of course you were to compare it against a binary
> string that was generated through other means, which I didn't do. So
> instead I wrote test cases that write the ASN.1 stream using the
> output visitor and then immediately feed the streams into the input
> visitor and then I compare the outcome against expected outcome,
> e.g., that a stucture's fields were properly filled. I hope this
> makes more sense in the ASN.1 case.
This is actually exactly what test-visitor-serialization.c does. It
feeds various normal/corner cases of visit_type_* into your output
visitor, takes that output and feeds it back into the input visitor,
then validates it against the original input. Just need to create
a SerializeOps implementation to drive your visitor and plug it
in like the others.
Just doing that would be a good start at least, but it would be really
nice to validate the encoding against some other reference implementation.
Have you looked into libsnacc? It seems to be the most readilly
available library. I wouldn't make it a formal build requirement, but it
would be nice to be able to execute test cases that use it if it's
present.
Even failing that, I personally wouldn't even mind just generating an
encoded blob outside the tree, and then checking that it along with the
textual description and steps to generate it, then validating the
visitors against it. It's not too far off from our "hand-written" JSON
to test the QMP visitors.
I don't think we have any reasonable assurance that our implementation
is correct otherwise.
>
>Stefan
>
>
Re: [Qemu-devel] The state of testing
On Mon, Mar 04, 2013 at 05:22:34PM +0100, Gerd Hoffmann wrote:
> Hi,
>
> > How are things looking with device emulation, migration, monitor, char, etc?
>
> At the moment autotest/virt-test is pretty much the only workable thing
> for non-trivial devices because libqtest lacks infrastructure for pci
> and anything building on top of pci.
>
> usb has no in-tree tests, but has autotest coverage.
>
> chardevs have some autotest coverage, /me wrote a test for
> chardev-{add,remove} qmp commands. Still need to rebase + polish +
> submit that one though.
>
> Migration coverage is pretty bad overall I think.
I've been working on a tool to try to do build-bot-like coverage for
cross-version migration. It's basically a fork of Anthony's qemu-test
suite that uses the same qemu-jeos/initrd system and a single test
harness with pre-migration/post-migration/post-migration reboots tests
case.
The plan is to get it in-tree along with qemu-test, with a wrapper
that'll test current->current migration as part of `make check`, and set up
a test bot with older qemu builds to test cross-version compatibility and
generate a report along these lines:
http://wiki.qemu.org/Migration/Compatibility
I can throw the code up somewhere if anyone is interested, but it's
mostly for internal testing atm.
> What is the state of the idl patch series btw?
The IDL to generate serialization routines for device state is fairly
well-fleshed out, but there was a lot of flux in the code-parser to
process them in later stages of getting the initial code upstreamed, so
I wanted to hold off a bit to refactor the parser and consider some
alternatives (namely, Peter's suggestion of generating device structs
from a higher-level IDL, or leveraging other tools to generate the ASTs)
I should have some cycles to work on it more after the 1.4.1 release.
>
> cheers,
> Gerd
>
Re: [Qemu-devel] [PATCH 1/2] qga: add guest-get-time command
On Mon, Mar 04, 2013 at 05:16:29PM +0800, Lei Li wrote:
> Signed-off-by: Lei Li
Reviewed-by: Michael Roth
> ---
> qga/commands-posix.c | 16
> qga/qapi-schema.json | 13 +
> 2 files changed, 29 insertions(+)
>
> diff --git a/qga/commands-posix.c b/qga/commands-posix.c
> index 0ad73f3..6fc6003 100644
> --- a/qga/commands-posix.c
> +++ b/qga/commands-posix.c
> @@ -119,6 +119,22 @@ void qmp_guest_shutdown(bool has_mode, const char *mode,
> Error **err)
> /* succeded */
> }
>
> +int64_t qmp_guest_get_time(Error **errp)
> +{
> + int ret;
> + qemu_timeval tq;
> + int64_t time_ns;
> +
> + ret = qemu_gettimeofday(&tq);
> + if (ret < 0) {
> + error_setg_errno(errp, errno, "Failed to get time");
> + return -1;
> + }
> +
> + time_ns = tq.tv_sec * 10LL + tq.tv_usec * 1000;
> + return time_ns;
> +}
> +
> typedef struct GuestFileHandle {
> uint64_t id;
> FILE *fh;
> diff --git a/qga/qapi-schema.json b/qga/qapi-schema.json
> index d91d903..52bb091 100644
> --- a/qga/qapi-schema.json
> +++ b/qga/qapi-schema.json
> @@ -83,6 +83,19 @@
> { 'command': 'guest-ping' }
>
> ##
> +# @guest-get-time:
> +#
> +# Get the information about guest time relative to the Epoch
> +# of 1970-01-01 in UTC.
> +#
> +# Returns: Time in nanoseconds on success.
> +#
> +# Since 1.5
> +##
> +{ 'command': 'guest-get-time',
> + 'returns': 'int' }
> +
> +##
> # @GuestAgentCommandInfo:
> #
> # Information about guest agent commands.
> --
> 1.7.11.7
>
Re: [Qemu-devel] [PATCH 2/2] qga: add guest-set-time command
On Mon, Mar 04, 2013 at 05:16:30PM +0800, Lei Li wrote:
> Signed-off-by: Lei Li
> ---
> qga/commands-posix.c | 54
>
> qga/qapi-schema.json | 27 ++
> 2 files changed, 81 insertions(+)
>
> diff --git a/qga/commands-posix.c b/qga/commands-posix.c
> index 6fc6003..0515f5f 100644
> --- a/qga/commands-posix.c
> +++ b/qga/commands-posix.c
> @@ -135,6 +135,60 @@ int64_t qmp_guest_get_time(Error **errp)
> return time_ns;
> }
>
> +void qmp_guest_set_time(int64_t time_ns, Error **errp)
> +{
> +int ret;
> +int status;
> +pid_t pid;
> +Error *local_err = NULL;
> +struct timeval tv;
> +
> +/* year-2038 will overflow in case time_t is 32bit */
> +if (time_ns / 10 != (time_t)(time_ns / 10)) {
> +error_setg(errp, "Time %" PRId64 " is too large", time_ns);
> +}
> +
> +tv.tv_sec = time_ns / 10;
> +tv.tv_usec = (time_ns % 10) / 1000;
> +
> +ret = settimeofday(&tv, NULL);
> +if (ret < 0) {
> +error_setg_errno(errp, errno, "Failed to set time to guest");
> +return;
> +}
> +
> +/* Set the Hardware Clock to the current System Time. */
> +pid = fork();
> +if (pid == 0) {
> +setsid();
> +reopen_fd_to_null(0);
> +reopen_fd_to_null(1);
> +reopen_fd_to_null(2);
> +
> +execle("/sbin/hwclock", "hwclock", "-w", NULL, environ);
> +_exit(EXIT_FAILURE);
> +} else if (pid < 0) {
> +error_setg_errno(errp, errno, "failed to create child process");
> +return;
> +}
> +
> +ga_wait_child(pid, &status, &local_err);
> +if (error_is_set(&local_err)) {
> +error_propagate(errp, local_err);
> +return;
> +}
> +
> +if (!WIFEXITED(status)) {
> +error_setg(errp, "child process has terminated abnormally");
> +return;
> +}
> +
> +if (WEXITSTATUS(status)) {
> +error_setg(errp, "hwclock failed to set hardware clock to system
> time");
> +return;
> +}
> +}
> +
> typedef struct GuestFileHandle {
> uint64_t id;
> FILE *fh;
> diff --git a/qga/qapi-schema.json b/qga/qapi-schema.json
> index 52bb091..ce964e9 100644
> --- a/qga/qapi-schema.json
> +++ b/qga/qapi-schema.json
> @@ -96,6 +96,33 @@
>'returns': 'int' }
>
> ##
> +# @guest-set-time:
> +#
> +# Set guest time.
> +#
> +# When a guest is paused or migrated to a file then loaded
> +# from that file, the guest OS has no idea that there
> +# was a big gap in the time. Depending on how long the
> +# gap was, NTP might not be able to resynchronize the
> +# guest.
> +#
> +# This command tries to set guest time based on the information
> +# from host or an absolute value given by management app, and
> +# set the Hardware Clock to the current System Time. This
> +# will make it easier for a guest to resynchronize without
> +# waiting for NTP.
> +#
> +# @time: time of nanoseconds, relative to the Epoch of
"time in nanoseconds", but I can fix this up myself if there are no
other comments that need to be addressed.
Reviewed-by: Michael Roth
> +#1970-01-01 in UTC.
> +#
> +# Returns: Nothing on success.
> +#
> +# Since: 1.5
> +##
> +{ 'command': 'guest-set-time',
> + 'data': { 'time': 'int' } }
> +
> +##
> # @GuestAgentCommandInfo:
> #
> # Information about guest agent commands.
> --
> 1.7.11.7
>
Re: [Qemu-devel] [PATCH] qemu-ga: make guest-sync-delimited available during fsfreeze
On Fri, Mar 01, 2013 at 11:53:40AM -0600, Michael Roth wrote:
> We currently maintain a whitelist of commands that are safe during
> fsfreeze. During fsfreeze, we disable all commands that aren't part of
> that whitelist.
>
> guest-sync-delimited meets the criteria for being whitelisted, and is
> also required for qemu-ga clients that rely on guest-sync-delimited for
> re-syncing the channel after a timeout.
>
> Signed-off-by: Michael Roth
> Cc: qemu-sta...@nongnu.org
Thanks, applied to qga branch.
> ---
> qga/main.c |1 +
> 1 file changed, 1 insertion(+)
>
> diff --git a/qga/main.c b/qga/main.c
> index db281a5..ad75171 100644
> --- a/qga/main.c
> +++ b/qga/main.c
> @@ -85,6 +85,7 @@ static const char *ga_freeze_whitelist[] = {
> "guest-ping",
> "guest-info",
> "guest-sync",
> +"guest-sync-delimited",
> "guest-fsfreeze-status",
> "guest-fsfreeze-thaw",
> NULL
> --
> 1.7.9.5
>
Re: [Qemu-devel] [PATCH 2/3] qga: implement qmp_guest_get_vcpus() for Linux with sysfs
On Mon, Mar 04, 2013 at 11:19:56PM +0100, Laszlo Ersek wrote:
>
> Signed-off-by: Laszlo Ersek
> ---
> qga/commands-posix.c | 87
> ++
> 1 files changed, 87 insertions(+), 0 deletions(-)
>
> diff --git a/qga/commands-posix.c b/qga/commands-posix.c
> index 1ad231a..d4b6bdc 100644
> --- a/qga/commands-posix.c
> +++ b/qga/commands-posix.c
> @@ -15,6 +15,9 @@
> #include
> #include
> #include
> +#include
> +#include
> +#include
> #include "qga/guest-agent-core.h"
> #include "qga-qmp-commands.h"
> #include "qapi/qmp/qerror.h"
> @@ -1083,9 +1086,93 @@ void qmp_guest_fstrim(bool has_minimum, int64_t
> minimum, Error **err)
> }
> #endif
>
> +#if defined(__linux__)
There's a section in commands-posix.c set aside under "linux-specific
implementations" for these, and another underneath for stubs so we can
avoid having too many ifdef's.
> +#define SYSCONF_EXACT(name, err) sysconf_exact((name), #name, (err))
> +
> +static long sysconf_exact(int name, const char *name_str, Error **err)
> +{
> +long ret;
> +
> +errno = 0;
> +ret = sysconf(name);
> +if (ret == -1) {
> +if (errno == 0) {
> +error_setg(err, "sysconf(%s): value indefinite", name_str);
> +} else {
> +error_setg_errno(err, errno, "sysconf(%s)", name_str);
> +}
> +}
> +return ret;
> +}
> +
> +/*
> + * Store a VCPU structure under the link, and return the link to store into
> + * at the next time.
> + */
> +static GuestLogicalProcessorList **
> +append_vcpu(int64_t logical_id, bool online, GuestLogicalProcessorList
> **link)
> +{
> +GuestLogicalProcessor *vcpu;
> +GuestLogicalProcessorList *entry;
> +
> +vcpu = g_malloc0(sizeof *vcpu);
> +vcpu->logical_id = logical_id;
> +vcpu->online = online;
> +
> +entry = g_malloc0(sizeof *entry);
> +entry->value = vcpu;
> +
> +*link = entry;
> +return &entry->next;
> +}
> +#endif
> +
> GuestLogicalProcessorList *qmp_guest_get_vcpus(Error **errp)
> {
> +#if defined(__linux__)
> +long current;
> +GuestLogicalProcessorList **link, *head;
> +long sc_max;
> +Error *local_err = NULL;
> +
> +current = 0;
> +link = append_vcpu(current++, true, &head);
> +
> +sc_max = SYSCONF_EXACT(_SC_NPROCESSORS_CONF, &local_err);
> +while (local_err == NULL && current < sc_max) {
> +char *buf;
> +FILE *f;
> +
> +buf = g_strdup_printf("/sys/devices/system/cpu/cpu%ld/online",
> + current);
> +f = fopen(buf, "r");
> +if (f == NULL) {
> +error_setg_errno(&local_err, errno, "fopen(\"%s\", \"r\")", buf);
> +} else {
> +unsigned online;
> +
> +if (fscanf(f, "%u", &online) != 1) {
On Fedora 18 and Ubuntu 12.04 at least there doesn't seem to be per-cpu
values for online/offline/etc, but instead just a 'global' entry at
/sys/devices/system/cpu/{online,offline} that provides a range. This is
what's currently described in
linux/Documentation/ABI/testing/sysfs-devices-system-cpu as well.
Is that file also available on the distro you're testing with? Hopefully
there's a single interfaces we can rely on.
> +error_setg(&local_err, "failed to read or parse \"%s\"",
> buf);
> +} else {
> +link = append_vcpu(current++, online != 0, link);
> +}
> +
> +if (fclose(f) == EOF && local_err == NULL) {
> +error_setg_errno(&local_err, errno, "fclose(\"%s\")", buf);
> +}
> +}
> +g_free(buf);
> +}
> +
> +if (local_err == NULL) {
> +return head;
> +}
> +
> +qapi_free_GuestLogicalProcessorList(head);
> +error_propagate(errp, local_err);
> +#else
> error_set(errp, QERR_UNSUPPORTED);
> +#endif
> return NULL;
> }
>
> --
> 1.7.1
>
>
Re: [Qemu-devel] [PATCH 3/3] qga: implement qmp_guest_set_vcpus() for Linux with sysfs
On Mon, Mar 04, 2013 at 11:19:57PM +0100, Laszlo Ersek wrote:
>
> Signed-off-by: Laszlo Ersek
> ---
> qga/commands-posix.c | 51
> ++
> 1 files changed, 51 insertions(+), 0 deletions(-)
>
> diff --git a/qga/commands-posix.c b/qga/commands-posix.c
> index d4b6bdc..1848df8 100644
> --- a/qga/commands-posix.c
> +++ b/qga/commands-posix.c
> @@ -40,6 +40,7 @@ extern char **environ;
> #include
> #include
> #include
> +#include
>
> #ifdef FIFREEZE
> #define CONFIG_FSFREEZE
> @@ -1178,7 +1179,57 @@ GuestLogicalProcessorList *qmp_guest_get_vcpus(Error
> **errp)
>
> void qmp_guest_set_vcpus(GuestLogicalProcessorList *vcpus, Error **errp)
> {
> +#if defined(__linux__)
> +const GuestLogicalProcessorList *entry;
> +Error *local_err = NULL;
> +
> +for (entry = vcpus; local_err == NULL && entry != NULL;
> + entry = entry->next) {
> +const GuestLogicalProcessor *vcpu;
> +
> +vcpu = entry->value;
> +if (vcpu->logical_id == 0) {
> +if (!vcpu->online) {
> +error_setg(&local_err,
> + "unable to offline logical processor #0");
> +}
> +} else {
> +char *buf;
> +FILE *f;
> +
> +buf = g_strdup_printf("/sys/devices/system/cpu/cpu%"PRId64
> + "/online", vcpu->logical_id);
> +f = fopen(buf, "r+");
> +if (f == NULL) {
> +error_setg_errno(&local_err, errno, "fopen(\"%s\", \"r+\")",
> + buf);
> +} else {
> +unsigned online;
> +
> +if (fscanf(f, "%u", &online) != 1) {
> +error_setg(&local_err, "failed to read or parse \"%s\"",
> + buf);
> +} else if ((online != 0) != vcpu->online) {
> +errno = 0;
> +rewind(f);
> +if (errno != 0 ||
> +fprintf(f, "%u\n", (unsigned)vcpu->online) < 0) {
Similar to comments in patch 2: is this supposed to be similar to the
/sys/devices/system/cpu/{probe,release} entries currently documented?
> +error_setg_errno(&local_err, errno,
> + "rewind()/fprintf() on \"%s\"",
> buf);
> +}
> +}
> +
> +if (fclose(f) == EOF && local_err == NULL) {
> +error_setg_errno(&local_err, errno, "fclose(\"%s\")",
> buf);
> +}
> +}
> +g_free(buf);
> +}
> +}
> +error_propagate(errp, local_err);
> +#else
> error_set(errp, QERR_UNSUPPORTED);
> +#endif
> }
>
> /* register init/cleanup routines for stateful command groups */
> --
> 1.7.1
>
Re: [Qemu-devel] [PATCH 2/3] qga: implement qmp_guest_get_vcpus() for Linux with sysfs
On Tue, Mar 05, 2013 at 01:22:03PM -0700, Eric Blake wrote:
> On 03/05/2013 01:03 PM, mdroth wrote:
>
> >> +buf = g_strdup_printf("/sys/devices/system/cpu/cpu%ld/online",
> >> + current);
> >> +f = fopen(buf, "r");
> >> +if (f == NULL) {
> >> +error_setg_errno(&local_err, errno, "fopen(\"%s\", \"r\")",
> >> buf);
> >> +} else {
> >> +unsigned online;
> >> +
> >> +if (fscanf(f, "%u", &online) != 1) {
> >
> > On Fedora 18 and Ubuntu 12.04 at least there doesn't seem to be per-cpu
> > values for online/offline/etc, but instead just a 'global' entry at
> > /sys/devices/system/cpu/{online,offline} that provides a range. This is
> > what's currently described in
> > linux/Documentation/ABI/testing/sysfs-devices-system-cpu as well.
>
> Actually, there is both. Here's what I have on my 2-cpu laptop, running
> Fedora 18:
>
> # find /sys/devices/system/cpu/ -name online
> /sys/devices/system/cpu/cpu1/online
> /sys/devices/system/cpu/online
>
> Notice that there is NO /sys/devices/system/cpu/cpu0/online, because
Ahh, didn't think to check the others. This seems to be the case for me
as well. I agree on your later note about special casing this though.
> this particular laptop's chipset requires that cpu0 ALWAYS be online.
> The per-cpu online file exists only for cpus that can safely be
> offlined; if it does not exist, then you must leave that cpu on.
>
> >
> > Is that file also available on the distro you're testing with? Hopefully
> > there's a single interfaces we can rely on.
>
> Libvirt also relies on the per-cpu online files, and hasn't had any
> complaints across the distros.
>
> --
> Eric Blake eblake redhat com+1-919-301-3266
> Libvirt virtualization library http://libvirt.org
>
Re: [Qemu-devel] [PATCH 0/3] *** make netlayer re-entrant ***
On Sun, Mar 03, 2013 at 09:21:19PM +0800, Liu Ping Fan wrote: > From: Liu Ping Fan > > This series aim to make netlayer re-entrant, so netlayer can > run out of biglock safely. I think most of the locking considerations are still applicable either way, but this series seems to be written under the assumption that we'll be associating hubs/ports with separate AioContexts to facilitate driving the event handling outside of the iothread. Is this the case? >From what I gathered from the other thread, the path forward was to replace the global iohandler list that we currently use to drive NetClient events and replace it with a GSource and GMainContext, rather than relying on AioContexts. I do agree that the event handlers currently grouped under iohandler.c:io_handlers look like a nice fit for AioContexts, but other things like slirp and chardevs seem better served by a more general mechanism like GSources/GMainContexts. The chardev flow control patches seem to be doing something similar already as well. > > Liu Ping Fan (3): > net: spread hub on AioContexts > net: introduce lock to protect NetClientState's send_queue > net: make netclient re-entrant with refcnt > > hw/qdev-properties-system.c | 15 ++ > include/block/aio.h |1 + > include/net/net.h | 12 + > include/net/queue.h | 15 ++ > main-loop.c |5 ++ > net/hub.c | 81 ++-- > net/net.c | 109 ++ > net/queue.c | 19 ++-- > net/slirp.c |3 +- > 9 files changed, 239 insertions(+), 21 deletions(-) > > -- > 1.7.4.4 >
Re: [Qemu-devel] [PATCH 2/3] qga: implement qmp_guest_get_vcpus() for Linux with sysfs
On Tue, Mar 05, 2013 at 10:34:08PM +0100, Laszlo Ersek wrote:
> On 03/05/13 21:25, Eric Blake wrote:
>
> > On 03/04/2013 03:19 PM, Laszlo Ersek wrote:
> >> Signed-off-by: Laszlo Ersek
> >> ---
> >> GuestLogicalProcessorList *qmp_guest_get_vcpus(Error **errp)
> >> {
> >> +#if defined(__linux__)
> >
> >> +
> >> +buf = g_strdup_printf("/sys/devices/system/cpu/cpu%ld/online",
> >> + current);
> >> +f = fopen(buf, "r");
> >> +if (f == NULL) {
> >> +error_setg_errno(&local_err, errno, "fopen(\"%s\", \"r\")",
> >> buf);
> >
> > NACK to this portion. If the file doesn't exist, but the
> > /sys/devices/system/cpu/cpu%ld/ directory exists, then the cpu should be
> > treated as always online, and not an error. In fact, on many machines,
> > cpu0 does not have an online file precisely because it cannot be taken
> > offline, even if the rest of the cpus can.
>
> The code always reports cpu0 online (without looking at the sysfs) and
> never attempts to remove it (refuses any such requests), but see below.
>
> > It is also the case that on
> > older kernels that did not support offline cpus (such as RHEL 5), there
> > will be no per-cpu online file; but again, such kernels cannot support
> > hot unplug, so all per-cpu directories imply which cpus are online. In
> > other words, you need a sane fallback if the online file does not exist
> > but the per-cpu directory does exist.
>
> In upstream "Documentation/cpu-hotplug.txt", there's
>
> Q: Why can't i remove CPU0 on some systems?
> A: Some architectures may have some special dependency on a certain
> CPU.
>
> For e.g in IA64 platforms we have ability to sent platform
> interrupts to the OS. a.k.a Corrected Platform Error Interrupts
> (CPEI). In current ACPI specifications, we didn't have a way to
> change the target CPU. Hence if the current ACPI version doesn't
> support such re-direction, we disable that CPU by making it
> not-removable.
>
> In such cases you will also notice that the online file is missing
> under cpu0.
>
> Q: Is CPU0 removable on X86?
> A: Yes. If kernel is compiled with CONFIG_BOOTPARAM_HOTPLUG_CPU0=y,
> CPU0 is removable by default. Otherwise, CPU0 is also removable by
> kernel option cpu0_hotplug.
>
> But some features depend on CPU0. Two known dependencies are:
>
> 1. Resume from hibernate/suspend depends on CPU0. Hibernate/suspend
> will fail if CPU0 is offline and you need to online CPU0 before
> hibernate/suspend can continue.
> 2. PIC interrupts also depend on CPU0. CPU0 can't be removed if a
> PIC interrupt is detected.
>
> It's said poweroff/reboot may depend on CPU0 on some machines
> although I haven't seen any poweroff/reboot failure so far after
> CPU0 is offline on a few tested machines.
>
> Please let me know if you know or see any other dependencies of
> CPU0.
>
> If the dependencies are under your control, you can turn on CPU0
> hotplug feature either by CONFIG_BOOTPARAM_HOTPLUG_CPU0 or by kernel
> parameter cpu0_hotplug.
>
> --Fenghua Yu
>
> I got this wrong in the series. The get method always reports an online
> CPU#0 (without even looking at sysfs), plus 'set' always refuses an
> attempt to offline it (even if the guest would support it; ie. 'set' too
> omits looking at the sysfs for cpu#0). This matches the common specific
> case (exactly cpu#0 is fixed online), but it is wrong in general.
>
> I think I should introduce another boolean flag in the element structure
> (only for the get method) that would say whether you can attempt to
> offline the VCPU. Then the caller of the set method can either comply or
> ignore the hint, and in the latter case it would be smacked down
> rightfully.
I think that seems reasonable.
>
> Thanks!
> Laszlo
>
Re: [Qemu-devel] [PATCH] qemu-ga: use key-value store to avoid recycling fd handles after restart
On Fri, Mar 01, 2013 at 11:40:27AM -0600, Michael Roth wrote:
> Hosts hold on to handles provided by guest-file-open for periods that can
> span beyond the life of the qemu-ga process that issued them. Since these
> are issued starting from 0 on every restart, we run the risk of issuing
> duplicate handles after restarts/reboots.
>
> As a result, users with a stale copy of these handles may end up
> reading/writing corrupted data due to their existing handles effectively
> being re-assigned to an unexpected file or offset.
>
> We unfortunately do not issue handles as strings, but as integers, so a
> solution such as using UUIDs can't be implemented without introducing a
> new interface.
>
> As a workaround, we fix this by implementing a persistent key-value store
> that will be used to track the value of the last handle that was issued
> across restarts/reboots to avoid issuing duplicates.
>
> The store is automatically written to the same directory we currently
> set via --statedir to track fsfreeze state, and so should be applicable
> for stable releases where this flag is supported.
>
> A follow-up can use this same store for handling fsfreeze state, but
> that change is cosmetic and left out for now.
>
> Signed-off-by: Michael Roth
> Cc: qemu-sta...@nongnu.org
Thanks, applied to qga branch with a s/uint64/int64/ fix-up for
guest_file_handle_add()'s return value.
> ---
> qga/commands-posix.c | 25 +--
> qga/guest-agent-core.h |1 +
> qga/main.c | 184
>
> 3 files changed, 204 insertions(+), 6 deletions(-)
>
> diff --git a/qga/commands-posix.c b/qga/commands-posix.c
> index 7a0202e..5d12716 100644
> --- a/qga/commands-posix.c
> +++ b/qga/commands-posix.c
> @@ -129,14 +129,22 @@ static struct {
> QTAILQ_HEAD(, GuestFileHandle) filehandles;
> } guest_file_state;
>
> -static void guest_file_handle_add(FILE *fh)
> +static uint64_t guest_file_handle_add(FILE *fh, Error **errp)
> {
> GuestFileHandle *gfh;
> +uint64_t handle;
> +
> +handle = ga_get_fd_handle(ga_state, errp);
> +if (error_is_set(errp)) {
> +return 0;
> +}
>
> gfh = g_malloc0(sizeof(GuestFileHandle));
> -gfh->id = fileno(fh);
> +gfh->id = handle;
> gfh->fh = fh;
> QTAILQ_INSERT_TAIL(&guest_file_state.filehandles, gfh, next);
> +
> +return handle;
> }
>
> static GuestFileHandle *guest_file_handle_find(int64_t id, Error **err)
> @@ -158,7 +166,7 @@ int64_t qmp_guest_file_open(const char *path, bool
> has_mode, const char *mode, E
> {
> FILE *fh;
> int fd;
> -int64_t ret = -1;
> +int64_t ret = -1, handle;
>
> if (!has_mode) {
> mode = "r";
> @@ -184,9 +192,14 @@ int64_t qmp_guest_file_open(const char *path, bool
> has_mode, const char *mode, E
> return -1;
> }
>
> -guest_file_handle_add(fh);
> -slog("guest-file-open, handle: %d", fd);
> -return fd;
> +handle = guest_file_handle_add(fh, err);
> +if (error_is_set(err)) {
> +fclose(fh);
> +return -1;
> +}
> +
> +slog("guest-file-open, handle: %d", handle);
> +return handle;
> }
>
> void qmp_guest_file_close(int64_t handle, Error **err)
> diff --git a/qga/guest-agent-core.h b/qga/guest-agent-core.h
> index 3354598..624a559 100644
> --- a/qga/guest-agent-core.h
> +++ b/qga/guest-agent-core.h
> @@ -35,6 +35,7 @@ bool ga_is_frozen(GAState *s);
> void ga_set_frozen(GAState *s);
> void ga_unset_frozen(GAState *s);
> const char *ga_fsfreeze_hook(GAState *s);
> +int64_t ga_get_fd_handle(GAState *s, Error **errp);
>
> #ifndef _WIN32
> void reopen_fd_to_null(int fd);
> diff --git a/qga/main.c b/qga/main.c
> index db281a5..3635430 100644
> --- a/qga/main.c
> +++ b/qga/main.c
> @@ -15,6 +15,7 @@
> #include
> #include
> #include
> +#include
> #ifndef _WIN32
> #include
> #include
> @@ -30,6 +31,7 @@
> #include "qapi/qmp/qerror.h"
> #include "qapi/qmp/dispatch.h"
> #include "qga/channel.h"
> +#include "qemu/bswap.h"
> #ifdef _WIN32
> #include "qga/service-win32.h"
> #include
> @@ -53,6 +55,11 @@
> #endif
> #define QGA_SENTINEL_BYTE 0xFF
>
> +typedef struct GAPersistentState {
> +#define QGA_PSTATE_DEFAULT_FD_COUNTER 1000
> +int64_t fd_counter;
> +} GAPersistentState;
> +
> struct GAState {
> JSONMessageParser parser;
> GMainLoop *main_loop;
> @@ -76,6 +83,8 @@ struct GAState {
> #ifdef CONFIG_FSFREEZE
> const char *fsfreeze_hook;
> #endif
> +const gchar *pstate_filepath;
> +GAPersistentState pstate;
> };
>
> struct GAState *ga_state;
> @@ -724,6 +733,171 @@ VOID WINAPI service_main(DWORD argc, TCHAR *argv[])
> }
> #endif
>
> +static void set_persistent_state_defaults(GAPersistentState *pstate)
> +{
> +g_assert(pstate);
> +pstate->fd_counter = QGA_PSTATE_DEFAULT_FD_COUNTER;
> +}
> +
> +static void persistent_state_from_keyfile(GAPersistentState *pstate,
> +
Re: [Qemu-devel] [PATCH 0/2 v5] Time resync support by qemu-ga
On Tue, Mar 05, 2013 at 05:39:10PM +0800, Lei Li wrote:
> This patch series attempts to add time resync support
> to qemu-ga by introducing qemu-ga commands guest-get-time
> and guest-set-time.
Thanks, applied to qga branch.
>
> Right now, when a guest is paused or migrated to a file
> then loaded from that file, the guest OS has no idea that
> there was a big gap in the time. Depending on how long the
> gap was, NTP might not be able to resynchronize the guest.
> So adding new guest-agent command that is called any time
> a guest is resumed and which tells the guest to update its
> own wall clock time based on the information from the host
> will make it easier for a guest to resynchronize without
> waiting for NTP.
>
> The previous RFC send for discussion and suggestion as link
> here:
>
> http://article.gmane.org/gmane.comp.emulators.qemu/186126
>
> The interface for these commands like:
>
> { 'command': 'guest-get-time', 'returns': 'int' }
>
> { 'command': 'guest-set-time', 'data': { 'time': int } }
>
> Notes:
> For the implementition of win32-specific commands, I plan
> to send it out in another thread later.
>
> Suggestions and comments are welcome!
>
> Changes since v4:
> - Fix the missing error exit pointed by Eric.
> - Doc improvement from Eric.
>
> Changes since v3:
> - Doc improvement based on Eric's suggestions.
> - Overflow check improve from Eric.
>
> Changes since v2:
> - Get rid of utc-offset, and make it just pass single nanoseconds
> relative to the Epoch in UTC/GMT according to Anthony and
> Eric's comments.
> - Make time argument mandatory.
> - Fix the overflow check for year-2038 problem.
> - Error handel improvment from Eric.
>
> Changes since v1:
> - Squashed patches add support to get host time and add
> guest-get-time command into one.
> - Documents improvment based on the suggestions from
> Eric and Mike.
> - Change the name of 'HostTimeInfo' to 'TimeInfo'.
> - Better use-case and logic for 'guest-set-time'
> command suggested by Eric.
> - Error handel improvment from Luiz.
>
> Lei Li (2):
> qga: add guest-get-time command
> qga: add guest-set-time command
>
Re: [Qemu-devel] [PATCH v2 1/3] qga: introduce guest-get-vcpus / guest-set-vcpus with stubs
On Wed, Mar 06, 2013 at 11:48:14PM +0100, Laszlo Ersek wrote:
> On 03/06/13 23:32, Eric Blake wrote:
> > On 03/06/2013 02:59 PM, Laszlo Ersek wrote:
>
> >> +##
> >> +# @GuestLogicalProcessor:
> >> +#
> >> +# @logical-id: Arbitrary guest-specific unique identifier of the VCPU.
> >> +#
> >> +# @online: Whether the VCPU is enabled.
> >> +#
> >> +# @can-offline: Whether offlining the VCPU is possible. This member is
> >> always
> >> +# filled in by the guest agent when the structure is
> >> returned,
> >> +# and always ignored on input (hence it can be omitted
> >> then).
> >
> > Other places have used the notation '#optional' when documenting a
> > parameter that need not be present on input; although we don't have
> > anything that strictly requires/enforces that notation.
>
> I'll fix this in v3 if I'll have to respin, otherwise I'd prefer a
> followup patch.
>
> >> +# Returns: The length of the initial sublist that has been successfully
> >> +# processed. The guest agent maximizes this value. Possible
> >> cases:
> >> +#
> >> +# 0:if the @vcpus list was empty on input. Guest
> >> state
> >> +#has not been changed. Otherwise,
> >> +#
> >> +# Error:processing the first node of @vcpus failed
> >> for the
> >> +#reason returned. Guest state has not been
> >> changed.
> >> +#Otherwise,
> >> +#
> >
> >> +
> >> +int64_t qmp_guest_set_vcpus(GuestLogicalProcessorList *vcpus, Error
> >> **errp)
> >> +{
> >> +error_set(errp, QERR_UNSUPPORTED);
> >> +return -1;
> >
> > This returns an error even on an empty input @vcpus, while the docs said
> > that returning 0 takes priority. But it's so much of a corner case that
> > I don't care; always returning an error seems fine.
>
> I see what you mean. In my mind, "unsupported" beats everything else, as
> if there was a big banner on top of the schema file: "you'll get
> QERR_UNSUPPORTED from any interface that's not supported".
>
> I'd like to leave this as-is even if I have to respin; distinguishing
> between zero-length-list and "unsupported" seems awkward, plus I'd also
> like to accept an empty list without error (in the supported case).
>
That's the general understanding for the current interfaces: "unsupported"
is a higher-level error than the errors that individual commands might
document. So I think we should keep this as-is for consistency, and if
it does need to be documented better then a patch adding the big banner
at the top of the schema is probably the best approach actually.
> > Thus, although there are things you might change if you have to respin
> > the series for later review comments, I'm perfectly fine leaving this
> > as-is and you can use:
> >
> > Reviewed-by: Eric Blake
>
> Thanks much!
> Laszlo
>
Re: [Qemu-devel] [RFC PATCH 0/2] port network layer onto glib
On Wed, Mar 13, 2013 at 05:21:02PM +0100, Paolo Bonzini wrote: > Il 13/03/2013 13:34, Anthony Liguori ha scritto: > > Paolo Bonzini writes: > > > >> Il 13/03/2013 06:59, Liu Ping Fan ha scritto: > >>> These series aim to port network backend onto glib, and > >>> prepare for moving towards making network layer mutlit-thread. > >>> The brief of the whole aim and plan is documented on > >>> http://wiki.qemu.org/Features/network_reentrant > >>> > >>> In these series, attach each NetClientState with a GSource > >>> At the first, I use AioContext instead of GSource, but after discussion, > >>> I think with GSource, we can integrated with glib more closely. > >> > >> Integrating with glib by itself is pointless. What is the *benefit*? > >> > >> We have a pretty good idea of how to make multithreaded device models > >> using AioContext, since we are using it for the block layer and > >> virtio-blk dataplane. Doing the same work twice, on two different > >> frameworks, doesn't seem like a very good idea. > > > > Hrm, I had thought on previous threads there was clear agreement that we > > did not want to use AioContext outside of the block layer. > > > > I think we certainly all agree that moving to a thread aware event loop > > is a necessary step toward multi-threading. I think the only question > > is whether to use AioContext or glib. > > > > AioContext is necessary for the block layer because the block layer > > still has synchronous I/O. I think we should aim to replace all sync > > I/O in the long term with coroutine based I/O. That lets us eliminate > > AioContextes entirely which is nice as the semantics are subtle. > > > > I think that's a solid argument for glib over AioContext. The former is > > well understood, documented, and makes unit testing easier. > > I don't see anything particularly subtle in AioContext, except > qemu_bh_schedule_idle and the flush callback. The flush callback only > has a naming problem really, it is a relic of qemu_aio_flush(). > qemu_bh_schedule_idle could disappear if we converted the floppy disk > drive to AIO; patches existed for that but then the poster disappeared. > > glib's main loop has its share of subtleness (GMainLoop vs. > GMainContext, anyone?), and AioContext's code is vastly simpler than > GMainLoop's. AioContext is also documented and unit tested, with tests > for both standalone and GSource operation. Unit tests for AioContext > users are trivial to write, we have one in test-thread-pool. > > > Did you have a specific concern with using glib vs. AioContext? Is it > > about reusing code in the block layer where AioContext is required? > > In the short term yes, code duplication is a concern. We already have > two implementation of virtio. I would like the dataplane virtio code to > grow everything else that needs to be in all dataplane-style devices > (for example, things such as setting up the guest<->host notifiers), and > the hw/virtio.c API implemented on top of it (or dead altogether). > Usage of AioContext is pretty much forced by the block layer. > > However, I'm more worried by overhead. GMainLoop is great because > everybody plugs into it. It enabled the GTK+ front-end, it let us reuse > GIOChannel for chardev flow control, and one can similarly think of > integrating Avahi for example. However, I think it's mostly useful for > simple-minded non-performance-critical code. QEMU has worked great in > almost all scenarios with only one non-VCPU thread, and if we are going > to move stuff to other threads we should only do that because we want > performance and control. I'm not at all confident that GMainLoop can > provide them. But isn't there also an effort to make virtio-blk/virtio-net a model for threaded devices/subsystems in general, as opposed to "accelerators" for specific use-cases like tap-based backends? I think this is the main question, because most of the planning seems contingent on that. And it seems to me that if the answer is no, then we need to consider the fact that vhost-net seems to serve this purpose already. If the answer is yes, don't we also need to look at things like interaction between slirp and a threaded network device? Based on comments in the other thread, I thought it was agreed that slirp was a particular example for something that should be rolled into a GMainContext loop as opposed to an AioContext based one? To me this suggests that some event loops will ultimately drive GMainContext handlers in addition to AioContexts (with the latter perhaps being driven at a higher priority with PI mutexs and whatever else that entails). This is already a requirement for the QEMU main loop, so perhaps that event loop can be moved to common code to lessen the subtleties between running in a dataplane thread as opposed to the io thread. What would be nice is if the difference between the iothread's event loop and a dataplane (or QMP/VNC/etc) thread's event loop was simply the set of AioContexts/GMainC
Re: [Qemu-devel] [PATCH 1/2] qga: add windows implementation for guest-get-time
On Wed, Mar 13, 2013 at 06:10:30PM +0800, li...@linux.vnet.ibm.com wrote:
> From: Lei Li
>
> Signed-off-by: Lei Li
> ---
> qga/commands-win32.c | 32
> 1 files changed, 32 insertions(+), 0 deletions(-)
>
> diff --git a/qga/commands-win32.c b/qga/commands-win32.c
> index 7e8ecb3..0a2bb34 100644
> --- a/qga/commands-win32.c
> +++ b/qga/commands-win32.c
> @@ -22,6 +22,12 @@
> #define SHTDN_REASON_FLAG_PLANNED 0x8000
> #endif
>
> +/* multiple of 100 nanoseconds elapsed between windows baseline
> + (1/1/1601) and Unix Epoch (1/1/1970), accounting for leap years */
> +#define W32_FT_OFFSET (1000ULL * 60 * 60 * 24 * \
> + (365 * (1970 - 1601) + \
> +(1970 - 1601) / 4 - 3))
> +
> static void acquire_privilege(const char *name, Error **err)
> {
> HANDLE token;
> @@ -108,6 +114,32 @@ void qmp_guest_shutdown(bool has_mode, const char *mode,
> Error **err)
> }
> }
>
> +int64_t qmp_guest_get_time(Error **errp)
> +{
> + SYSTEMTIME *ts = g_malloc0(sizeof(SYSTEMTIME));
Don't we need to free this at some point?
> + int64_t time_ns;
> + union {
> + UINT64 ns100;
> + FILETIME tf;
> + } time;
> +
> + GetSystemTime(ts);
> + if (!ts) {
> + slog("guest-get-time failed: %d", GetLastError());
GetSystemTime() does not seem to set an error that can be retrieved by
GetLastError().
> + error_setg_errno(errp, errno, "Failed to get time");
> + return -1;
> + }
> +
> + if (!SystemTimeToFileTime(ts, &time.tf)) {
> + error_setg_errno(errp, errno, "Failed to convert system time");
> + return -1;
> + }
> +
> + time_ns = (int64_t)((time.ns100 - W32_FT_OFFSET) * 100);
I'm not sure how safe this union stuff is. The documentation suggests that in
some circumstances the low/high fields in FILETIME might be padded for
64-bit alignment and that doing this type of cast could generate an
alignment fault:
http://msdn.microsoft.com/en-us/library/windows/desktop/ms724284(v=vs.85).aspx
or it might, perhaps even worse, just silently report the wrong time.
I think we should just do the math explicitly:
(((tf.dwHighDateTime << 32) | tf.dwLowDateTime) - W32_FT_OFFSET) * 100)
(or something along that line)
> +
> + return time_ns;
> +}
> +
> int64_t qmp_guest_file_open(const char *path, bool has_mode, const char
> *mode, Error **err)
> {
> error_set(err, QERR_UNSUPPORTED);
> --
> 1.7.7.6
>
Re: [Qemu-devel] [PATCH 1/2] qga: add windows implementation for guest-get-time
On Wed, Mar 13, 2013 at 03:07:52PM -0500, mdroth wrote:
> On Wed, Mar 13, 2013 at 06:10:30PM +0800, li...@linux.vnet.ibm.com wrote:
> > From: Lei Li
> >
> > Signed-off-by: Lei Li
> > ---
> > qga/commands-win32.c | 32
> > 1 files changed, 32 insertions(+), 0 deletions(-)
> >
> > diff --git a/qga/commands-win32.c b/qga/commands-win32.c
> > index 7e8ecb3..0a2bb34 100644
> > --- a/qga/commands-win32.c
> > +++ b/qga/commands-win32.c
> > @@ -22,6 +22,12 @@
> > #define SHTDN_REASON_FLAG_PLANNED 0x8000
> > #endif
> >
> > +/* multiple of 100 nanoseconds elapsed between windows baseline
> > + (1/1/1601) and Unix Epoch (1/1/1970), accounting for leap years */
> > +#define W32_FT_OFFSET (1000ULL * 60 * 60 * 24 * \
> > + (365 * (1970 - 1601) + \
> > +(1970 - 1601) / 4 - 3))
> > +
> > static void acquire_privilege(const char *name, Error **err)
> > {
> > HANDLE token;
> > @@ -108,6 +114,32 @@ void qmp_guest_shutdown(bool has_mode, const char
> > *mode, Error **err)
> > }
> > }
> >
> > +int64_t qmp_guest_get_time(Error **errp)
> > +{
> > + SYSTEMTIME *ts = g_malloc0(sizeof(SYSTEMTIME));
>
> Don't we need to free this at some point?
Actually, do we even need malloc? Can't be just allocate SYSTEMTIME
struct on the stack and pass a pointer to GetSystemTime?
>
> > + int64_t time_ns;
> > + union {
> > + UINT64 ns100;
> > + FILETIME tf;
> > + } time;
> > +
> > + GetSystemTime(ts);
> > + if (!ts) {
> > + slog("guest-get-time failed: %d", GetLastError());
>
> GetSystemTime() does not seem to set an error that can be retrieved by
> GetLastError().
>
> > + error_setg_errno(errp, errno, "Failed to get time");
> > + return -1;
> > + }
> > +
> > + if (!SystemTimeToFileTime(ts, &time.tf)) {
> > + error_setg_errno(errp, errno, "Failed to convert system time");
> > + return -1;
> > + }
> > +
> > + time_ns = (int64_t)((time.ns100 - W32_FT_OFFSET) * 100);
>
> I'm not sure how safe this union stuff is. The documentation suggests that in
> some circumstances the low/high fields in FILETIME might be padded for
> 64-bit alignment and that doing this type of cast could generate an
> alignment fault:
>
> http://msdn.microsoft.com/en-us/library/windows/desktop/ms724284(v=vs.85).aspx
>
> or it might, perhaps even worse, just silently report the wrong time.
>
> I think we should just do the math explicitly:
>
> (((tf.dwHighDateTime << 32) | tf.dwLowDateTime) - W32_FT_OFFSET) * 100)
>
> (or something along that line)
>
> > +
> > + return time_ns;
> > +}
> > +
> > int64_t qmp_guest_file_open(const char *path, bool has_mode, const char
> > *mode, Error **err)
> > {
> > error_set(err, QERR_UNSUPPORTED);
> > --
> > 1.7.7.6
> >
Re: [Qemu-devel] [PATCH 2/2] qga: add windows implementation for guest-set-time
On Wed, Mar 13, 2013 at 06:10:31PM +0800, li...@linux.vnet.ibm.com wrote:
> From: Lei Li
>
> Signed-off-by: Lei Li
> ---
> qga/commands-win32.c | 34 ++
> 1 files changed, 34 insertions(+), 0 deletions(-)
>
> diff --git a/qga/commands-win32.c b/qga/commands-win32.c
> index 0a2bb34..a0c8d43 100644
> --- a/qga/commands-win32.c
> +++ b/qga/commands-win32.c
> @@ -140,6 +140,40 @@ int64_t qmp_guest_get_time(Error **errp)
> return time_ns;
> }
>
> +void qmp_guest_set_time(int64_t time_ns, Error **errp)
> +{
> +SYSTEMTIME ts;
> +FILETIME tf;
> +LONGLONG time;
> +
> +if (time_ns < 0 || time_ns / 100 > INT64_MAX - W32_FT_OFFSET) {
> +error_setg(errp, "Time %" PRId64 "is invalid", time_ns);
> +return;
> +}
> +
> +time = time_ns / 100 + W32_FT_OFFSET;
> +
> +tf.dwLowDateTime = (DWORD) time;
> +tf.dwHighDateTime = (DWORD) (time >> 32);
> +
> +if (!FileTimeToSystemTime(&tf, &ts)) {
> +error_setg(errp, "Failed to convert system time");
> +return;
> +}
> +
> +acquire_privilege(SE_SYSTEMTIME_NAME, errp);
> +if (error_is_set(errp)) {
> +error_setg(errp, "Failed to acquire privilege");
> +return;
> +}
> +
> +if (!SetSystemTime(&ts)) {
> +slog("guest-set-time failed: %d", GetLastError());
> +error_setg_errno(errp, errno, "Failed to set time to guest");
I think we want to pass back GetLastError(), SetSystemTime doesn't set errno
(and if it did something in the slog() callchain probably would've
clobbered it)
Looks good otherwise.
> +return;
> +}
> +}
> +
> int64_t qmp_guest_file_open(const char *path, bool has_mode, const char
> *mode, Error **err)
> {
> error_set(err, QERR_UNSUPPORTED);
> --
> 1.7.7.6
>
Re: [Qemu-devel] [PATCH 5/9] qapi_sized_buffer
On Wed, Mar 13, 2013 at 01:56:24PM -0500, Joel Schopp wrote:
> Add a sized buffer interface to qapi.
Isn't this just a special case of the visit_*_carray() interfaces? We
should avoid new interfaces if possible, since it adds to feature
disparities between visitor implementations.
>
> Cc: Michael Tsirkin
> Signed-off-by: Stefan Berger
> Signed-off-by: Joel Schopp
> ---
> include/qapi/visitor-impl.h |2 ++
> include/qapi/visitor.h |2 ++
> qapi/qapi-visit-core.c |8
> 3 files changed, 12 insertions(+)
>
> diff --git a/include/qapi/visitor-impl.h b/include/qapi/visitor-impl.h
> index 9d87f2d..dc0e25c 100644
> --- a/include/qapi/visitor-impl.h
> +++ b/include/qapi/visitor-impl.h
> @@ -38,6 +38,8 @@ struct Visitor
> size_t elem_count, size_t elem_size, Error **errp);
> void (*next_carray)(Visitor *v, Error **errp);
> void (*end_carray)(Visitor *v, Error **errp);
> +void (*type_sized_buffer)(Visitor *v, uint8_t **obj, size_t size,
> + const char *name, Error **errp);
>
> /* May be NULL */
> void (*start_optional)(Visitor *v, bool *present, const char *name,
> diff --git a/include/qapi/visitor.h b/include/qapi/visitor.h
> index 74bddef..7c7bb98 100644
> --- a/include/qapi/visitor.h
> +++ b/include/qapi/visitor.h
> @@ -55,5 +55,7 @@ void visit_start_carray(Visitor *v, void **obj, const char
> *name,
> size_t elem_count, size_t elem_size, Error **errp);
> void visit_next_carray(Visitor *v, Error **errp);
> void visit_end_carray(Visitor *v, Error **errp);
> +void visit_type_sized_buffer(Visitor *v, uint8_t **obj, size_t len,
> + const char *name, Error **errp);
>
> #endif
> diff --git a/qapi/qapi-visit-core.c b/qapi/qapi-visit-core.c
> index d9982f8..4b36a54 100644
> --- a/qapi/qapi-visit-core.c
> +++ b/qapi/qapi-visit-core.c
> @@ -338,3 +338,11 @@ void visit_end_carray(Visitor *v, Error **errp)
> v->end_carray(v, errp);
> }
> }
> +
> +void visit_type_sized_buffer(Visitor *v, uint8_t **obj, size_t len,
> + const char *name, Error **errp)
> +{
> +if (!error_is_set(errp)) {
> +v->type_sized_buffer(v, obj, len, name, errp);
> +}
> +}
> --
> 1.7.10.4
>
>
Re: [Qemu-devel] [PATCH 4/9] qemu_qsb.diff
On Wed, Mar 13, 2013 at 01:56:23PM -0500, Joel Schopp wrote:
> This patch adds support functions for operating on in memory sized file
> buffers.
There's been some past refactorings to remove non-migration users of
QEMUFile, and AFAIK that's still the case today. QEMUFile satisfies
funky requirements like rate-limiting, buffering, etc that were specific
to migration.
IIUC all we want here is an abstraction on top of write()/memcpy(),
and access to qemu_{put|get}_be* utility functions.
Have you considered rolling those abstractions in the visitor
implementations as opposed to extending QEMUFile, and using
be*_to_cpus/cpus_to_be* helpers directly instead (like block/qcow2.c
does, for example)?
>
> Signed-off-by: Stefan Berger
> Signed-off-by: Joel Schopp
> ---
> include/migration/qemu-file.h | 12 +++
> include/qemu-common.h | 15
> util/qemu-file.c | 184
> +
> 3 files changed, 211 insertions(+)
>
> diff --git a/include/migration/qemu-file.h b/include/migration/qemu-file.h
> index 07d8362..2bc77b1 100644
> --- a/include/migration/qemu-file.h
> +++ b/include/migration/qemu-file.h
> @@ -24,6 +24,8 @@
> #ifndef QEMU_FILE_H
> #define QEMU_FILE_H 1
>
> +#include
> +
> /* This function writes a chunk of data to a file at the given position.
> * The pos argument can be ignored if the file is only being used for
> * streaming. The handler should try to write all of the data it can.
> @@ -58,6 +60,14 @@ typedef struct QEMUFileOps {
> QEMUFileGetFD *get_fd;
> } QEMUFileOps;
>
> +struct QEMUSizedBuffer {
> +unsigned char *buffer;
> +uint64_t size;
> +uint64_t used;
> +};
> +
> +typedef struct QEMUSizedBuffer QEMUSizedBuffer;
> +
> QEMUFile *qemu_fopen_ops(void *opaque, const QEMUFileOps *ops);
> QEMUFile *qemu_fopen(const char *filename, const char *mode);
> QEMUFile *qemu_fdopen(int fd, const char *mode);
> @@ -71,6 +81,8 @@ void qemu_put_byte(QEMUFile *f, int v);
> int qemu_read_bytes(QEMUFile *f, uint8_t *buf, int size);
> int qemu_peek_bytes(QEMUFile *f, uint8_t *buf, int size, size_t offset);
> int qemu_write_bytes(QEMUFile *f, const uint8_t *buf, int size);
> +QEMUFile *qemu_bufopen(const char *mode, QEMUSizedBuffer *input);
> +const QEMUSizedBuffer *qemu_buf_get(QEMUFile *f);
>
> static inline void qemu_put_ubyte(QEMUFile *f, unsigned int v)
> {
> diff --git a/include/qemu-common.h b/include/qemu-common.h
> index 5e13708..de1cdc0 100644
> --- a/include/qemu-common.h
> +++ b/include/qemu-common.h
> @@ -442,4 +442,19 @@ int64_t pow2floor(int64_t value);
> int uleb128_encode_small(uint8_t *out, uint32_t n);
> int uleb128_decode_small(const uint8_t *in, uint32_t *n);
>
> +/* QEMU Sized Buffer */
> +#include "include/migration/qemu-file.h"
> +QEMUSizedBuffer *qsb_create(const uint8_t *buffer, uint64_t len);
> +QEMUSizedBuffer *qsb_clone(const QEMUSizedBuffer *);
> +void qsb_free(QEMUSizedBuffer *);
> +uint64_t qsb_set_length(QEMUSizedBuffer *qsb, uint64_t length);
> +uint64_t qsb_get_length(const QEMUSizedBuffer *qsb);
> +const unsigned char *qsb_get_buffer(const QEMUSizedBuffer *, int64_t pos);
> +int qsb_write_at(QEMUSizedBuffer *qsb, const uint8_t *buf,
> + int64_t pos, int size);
> +int qsb_append_qsb(QEMUSizedBuffer *dest, const QEMUSizedBuffer *src);
> +int qsb_append(QEMUSizedBuffer *dest, const uint8_t *buffer, uint64_t len);
> +
> +
> +
> #endif
> diff --git a/util/qemu-file.c b/util/qemu-file.c
> index e698713..4442dcc 100644
> --- a/util/qemu-file.c
> +++ b/util/qemu-file.c
> @@ -710,3 +710,187 @@ int qemu_write_bytes(QEMUFile *f, const uint8_t *buf,
> int size)
>
> return size;
> }
> +
> +
> +QEMUSizedBuffer *qsb_create(const uint8_t *buffer, uint64_t len)
> +{
> +QEMUSizedBuffer *qsb;
> +uint64_t alloc_len;
> +
> +alloc_len = (len > 1024) ? len : 1024;
> +
> +qsb = g_new0(QEMUSizedBuffer, 1);
> +if (!qsb) {
> +return NULL;
> +}
> +
> +qsb->buffer = g_malloc(alloc_len);
> +if (!qsb->buffer) {
> +g_free(qsb);
> +return NULL;
> +}
> +qsb->size = alloc_len;
> +
> +if (buffer) {
> +memcpy(qsb->buffer, buffer, len);
> +qsb->used = len;
> +}
> +
> +return qsb;
> +}
> +
> +void qsb_free(QEMUSizedBuffer *qsb)
> +{
> +if (!qsb) {
> +return;
> +}
> +g_free(qsb->buffer);
> +g_free(qsb);
> +}
> +
> +uint64_t qsb_get_length(const QEMUSizedBuffer *qsb)
> +{
> +return qsb->used;
> +}
> +
> +uint64_t qsb_set_length(QEMUSizedBuffer *qsb, uint64_t new_len)
> +{
> +if (new_len <= qsb->size) {
> +qsb->used = new_len;
> +} else {
> +qsb->used = qsb->size;
> +}
> +return qsb->used;
> +}
> +
> +const unsigned char *qsb_get_buffer(const QEMUSizedBuffer *qsb, int64_t pos)
> +{
> +if (pos < qsb->used) {
> +return &qsb->buffer[pos];
> +}
> +return NULL;
> +}
> +
> +int qsb_write_at(QEMUSizedBuffer *qsb, const uint8_t
Re: [Qemu-devel] [PATCH 4/9] qemu_qsb.diff
On Wed, Mar 13, 2013 at 05:28:56PM -0400, Stefan Berger wrote:
> On 03/13/2013 05:11 PM, mdroth wrote:
> >On Wed, Mar 13, 2013 at 01:56:23PM -0500, Joel Schopp wrote:
> >>This patch adds support functions for operating on in memory sized file
> >>buffers.
> >There's been some past refactorings to remove non-migration users of
> >QEMUFile, and AFAIK that's still the case today. QEMUFile satisfies
> >funky requirements like rate-limiting, buffering, etc that were specific
> >to migration.
> >
> >IIUC all we want here is an abstraction on top of write()/memcpy(),
> >and access to qemu_{put|get}_be* utility functions.
> >
> >Have you considered rolling those abstractions in the visitor
> >implementations as opposed to extending QEMUFile, and using
> >be*_to_cpus/cpus_to_be* helpers directly instead (like block/qcow2.c
> >does, for example)?
>
> The advantage of using the QEMUFile abstractions is that now you can
> build a visitor on top of it and read from buffers, sockets, BDRV's
> (later on), plain files, and whatever else you can hide underneath
> that interface. Back in 2011 when I initially wrote this code there
Maybe a case can be made for making it a general utility library, but
I'm having a hard time thinking of any reasons that aren't specific to
migration, and I wonder if it's even necessary now that we have a
migration thread that can handle the rate-limiting/buffering
considerations.
But I'm not sure exactly why we decided to drop non-migration users, so
I think it's worth clarifying before we attempt to tether another
component to it.
Here's the thread I'm referencing:
https://lists.gnu.org/archive/html/qemu-devel/2011-09/msg02589.html
Juan, any background/input on this?
> that interface. Back in 2011 when I initially wrote this code there
> at least was talk about using ASN.1 for migration, but this is
> nearly 2 years ago and it may never be done that way, so this was
> one driving force behind using QEMUFile inside the visitor. Besides
Well, even back then goal was to abstract away direct calls to QEMUFile
and replace them with visitor calls, and then to provide legacy support
via a QEMUFile Visitor. A BER visitor could then be dropped in to
provide a BER migration protocol (how exactly this would be done was
somewhat of a ???, might've ended up layering on to of the
QEMUFile visitor anyway, but more out of pragmatism than anything else)
> that we later want to use the visitors for writing into virtual
> NVRAM, which we would build on top of a QEMUFile wrapping BDRVs. So
> there are some immediate advantages of using the common QEMUFile
> interface for reading and writing of data from different types of
> sources.
Can you describe the requirements for the BDRV wrapper a bit more?
Everything else seems reasonably doable via visitor internals but
maybe there's more to it I'm not considering.
>
>Regards,
> Stefan
>
Re: [Qemu-devel] [PATCH 4/9] qemu_qsb.diff
On Wed, Mar 13, 2013 at 05:41:33PM -0500, mdroth wrote:
> On Wed, Mar 13, 2013 at 05:28:56PM -0400, Stefan Berger wrote:
> > On 03/13/2013 05:11 PM, mdroth wrote:
> > >On Wed, Mar 13, 2013 at 01:56:23PM -0500, Joel Schopp wrote:
> > >>This patch adds support functions for operating on in memory sized file
> > >>buffers.
> > >There's been some past refactorings to remove non-migration users of
> > >QEMUFile, and AFAIK that's still the case today. QEMUFile satisfies
> > >funky requirements like rate-limiting, buffering, etc that were specific
> > >to migration.
> > >
> > >IIUC all we want here is an abstraction on top of write()/memcpy(),
> > >and access to qemu_{put|get}_be* utility functions.
> > >
> > >Have you considered rolling those abstractions in the visitor
> > >implementations as opposed to extending QEMUFile, and using
> > >be*_to_cpus/cpus_to_be* helpers directly instead (like block/qcow2.c
> > >does, for example)?
> >
> > The advantage of using the QEMUFile abstractions is that now you can
> > build a visitor on top of it and read from buffers, sockets, BDRV's
> > (later on), plain files, and whatever else you can hide underneath
> > that interface. Back in 2011 when I initially wrote this code there
>
> Maybe a case can be made for making it a general utility library, but
> I'm having a hard time thinking of any reasons that aren't specific to
> migration, and I wonder if it's even necessary now that we have a
> migration thread that can handle the rate-limiting/buffering
> considerations.
>
> But I'm not sure exactly why we decided to drop non-migration users, so
> I think it's worth clarifying before we attempt to tether another
> component to it.
>
> Here's the thread I'm referencing:
>
> https://lists.gnu.org/archive/html/qemu-devel/2011-09/msg02589.html
>
> Juan, any background/input on this?
Sorry, forgot to CC Juan :)
>
> > that interface. Back in 2011 when I initially wrote this code there
> > at least was talk about using ASN.1 for migration, but this is
> > nearly 2 years ago and it may never be done that way, so this was
> > one driving force behind using QEMUFile inside the visitor. Besides
>
> Well, even back then goal was to abstract away direct calls to QEMUFile
> and replace them with visitor calls, and then to provide legacy support
> via a QEMUFile Visitor. A BER visitor could then be dropped in to
> provide a BER migration protocol (how exactly this would be done was
> somewhat of a ???, might've ended up layering on to of the
> QEMUFile visitor anyway, but more out of pragmatism than anything else)
>
> > that we later want to use the visitors for writing into virtual
> > NVRAM, which we would build on top of a QEMUFile wrapping BDRVs. So
> > there are some immediate advantages of using the common QEMUFile
> > interface for reading and writing of data from different types of
> > sources.
>
> Can you describe the requirements for the BDRV wrapper a bit more?
> Everything else seems reasonably doable via visitor internals but
> maybe there's more to it I'm not considering.
>
> >
> >Regards,
> > Stefan
> >
Re: [Qemu-devel] [PATCH 5/9] qapi_sized_buffer
On Wed, Mar 13, 2013 at 06:00:24PM -0400, Stefan Berger wrote:
> On 03/13/2013 04:52 PM, mdroth wrote:
> >On Wed, Mar 13, 2013 at 01:56:24PM -0500, Joel Schopp wrote:
> >>Add a sized buffer interface to qapi.
> >Isn't this just a special case of the visit_*_carray() interfaces? We
> >should avoid new interfaces if possible, since it adds to feature
> >disparities between visitor implementations.
>
> Yes, it's a special case and carray seems more general.
> However, I don't understand the interface of carray. It has a
> start_carray with all parameters given, then a next_carray and an
> end_carray. Why do we need multiple calls if one call (start_carray)
> could be used to serialize all the data already?
Visitors don't have any knowledge of the data structures they're visiting
outside of what we tell them via the visit_*() API.
For output, visit_start_carray() only provides enough information to
instruct a visitor on how to calculate the offsets of each element (and
perhaps allocate some memory for it's own internal buffers etc.)
For input, it provides enough to allocate storage, and calculate offsets
to each element it's deserializing into.
As far as what to do with each element, we need to make additional calls
to instruct it.
For example, a visitor for a 16-element array of:
typedef struct ComplexType {
int32_t foo;
char *bar;
} ComplexType;
would look something like:
visit_start_carray(v, ...); // instruct visitor how to calculate offsets
for (i = 0; i < 16; i++) {
visit_type_ComplexType(v, ...) // instruct visitor how to handle elem
visit_next_carray(v, ...); // instruct visitor to move to next offset
}
visit_end_carray(v, ...); // instruct visitor to finalize array
>
> Regards,
> Stefan
>
>
>
Re: [Qemu-devel] [PATCH 5/9] qapi_sized_buffer
On Wed, Mar 13, 2013 at 09:48:11PM -0400, Stefan Berger wrote:
> On 03/13/2013 07:18 PM, mdroth wrote:
> >On Wed, Mar 13, 2013 at 06:00:24PM -0400, Stefan Berger wrote:
> >>On 03/13/2013 04:52 PM, mdroth wrote:
> >>
> >Visitors don't have any knowledge of the data structures they're visiting
> >outside of what we tell them via the visit_*() API.
> >
> >[...]
> >
> >For example, a visitor for a 16-element array of:
> >
> >typedef struct ComplexType {
> > int32_t foo;
> > char *bar;
> >} ComplexType;
> >
> >would look something like:
> >
> >visit_start_carray(v, ...); // instruct visitor how to calculate offsets
> >for (i = 0; i < 16; i++) {
> > visit_type_ComplexType(v, ...) // instruct visitor how to handle elem
> > visit_next_carray(v, ...); // instruct visitor to move to next offset
> >}
> >visit_end_carray(v, ...); // instruct visitor to finalize array
>
> Given this example above, I think we will need the sized buffer. The
> sized buffer targets binary arrays and their encoding. If I was to
> encode an 'unsigned char[n]' (e.g., n=200) using n, or n/2 or n/4
> loops like above breaking it apart in u8, u16 or u32 respectively I
> think this would 'not bed good' also considering the 2 bytes for tag
> and length being added by ASN.1 for every such datatype
> (u8,u16,u32). The sized buffer allows you to for example take a
> memory page and write it out in one chunk adding a few bytes of
> ASN.1 'decoration' around the actual data.
You could do it with this interface as well actually. The Visitor will
need to maintain some internal state to differentiate what it does with
subsequent visit_type*/visit_next_carray/visit_end_carry. There's no
reason it couldn't also track the elem size so it could tag a buffer
"en masse" when visit_end_carray() gets called.
QMP*Visitor does something similar already for building up lists/arrays
before tacking them onto the parent structure.
>
>Stefan
>
Re: [Qemu-devel] [PATCH 1/2] qga: add windows implementation for guest-get-time
On Thu, Mar 14, 2013 at 03:07:50PM +0800, Lei Li wrote:
> Signed-off-by: Lei Li
> ---
> qga/commands-win32.c | 33 +
> 1 file changed, 33 insertions(+)
>
> diff --git a/qga/commands-win32.c b/qga/commands-win32.c
> index 7e8ecb3..e24fb4a 100644
> --- a/qga/commands-win32.c
> +++ b/qga/commands-win32.c
> @@ -22,6 +22,12 @@
> #define SHTDN_REASON_FLAG_PLANNED 0x8000
> #endif
>
> +/* multiple of 100 nanoseconds elapsed between windows baseline
> + (1/1/1601) and Unix Epoch (1/1/1970), accounting for leap years */
> +#define W32_FT_OFFSET (1000ULL * 60 * 60 * 24 * \
> + (365 * (1970 - 1601) + \
> +(1970 - 1601) / 4 - 3))
> +
> static void acquire_privilege(const char *name, Error **err)
> {
> HANDLE token;
> @@ -108,6 +114,33 @@ void qmp_guest_shutdown(bool has_mode, const char *mode,
> Error **err)
> }
> }
>
> +int64_t qmp_guest_get_time(Error **errp)
> +{
> +SYSTEMTIME *ts = g_malloc0(sizeof(SYSTEMTIME));
I still don't understand why we do just do:
SYSTEM ts = {0};
> +int64_t time_ns;
> +FILETIME tf;
> +
> +GetSystemTime(ts);
followed by:
GetSystemTime(&ts);
(and same for SystemTimeToFileTime() below)
This would avoid the need to add common cleanup code for all the
return paths.
> +if (!ts) {
this is gonna always be false since we initialize it at the start of
this function.
also, GetSystemTime() does seem to provide any error indication
whatsoever, which is strange. But it also doesn't seem to have any
guarantee this it will always succeed...
I think the best we could do is probably just some kind of sanity
check, like making sure ts.wYear != 0, or maybe that
1601 <= ts.wYear <= 30827
> +error_setg(errp, "Failed to get time");
> +goto out;
> +}
> +
> +if (!SystemTimeToFileTime(ts, &tf)) {
> +error_setg_errno(errp, errno, "Failed to convert system time");
> +goto out;
> +}
> +
> +time_ns = int64_t)tf.dwHighDateTime << 32) | tf.dwLowDateTime)
> +- W32_FT_OFFSET) * 100;
> +
> +return time_ns;
> +
> +out:
> +g_free(ts);
> +return -1;
> +}
> +
> int64_t qmp_guest_file_open(const char *path, bool has_mode, const char
> *mode, Error **err)
> {
> error_set(err, QERR_UNSUPPORTED);
> --
> 1.7.11.7
>
Re: [Qemu-devel] [PATCH 2/2] qga: add windows implementation for guest-set-time
On Thu, Mar 14, 2013 at 03:07:51PM +0800, Lei Li wrote:
> Signed-off-by: Lei Li
> ---
> qga/commands-win32.c | 33 +
> 1 file changed, 33 insertions(+)
>
> diff --git a/qga/commands-win32.c b/qga/commands-win32.c
> index e24fb4a..8064c3a 100644
> --- a/qga/commands-win32.c
> +++ b/qga/commands-win32.c
> @@ -141,6 +141,39 @@ out:
> return -1;
> }
>
> +void qmp_guest_set_time(int64_t time_ns, Error **errp)
> +{
> +SYSTEMTIME ts;
> +FILETIME tf;
> +LONGLONG time;
> +
> +if (time_ns < 0 || time_ns / 100 > INT64_MAX - W32_FT_OFFSET) {
> +error_setg(errp, "Time %" PRId64 "is invalid", time_ns);
> +return;
> +}
> +
> +time = time_ns / 100 + W32_FT_OFFSET;
> +
> +tf.dwLowDateTime = (DWORD) time;
> +tf.dwHighDateTime = (DWORD) (time >> 32);
> +
> +if (!FileTimeToSystemTime(&tf, &ts)) {
> +error_setg(errp, "Failed to convert system time");
> +return;
> +}
> +
> +acquire_privilege(SE_SYSTEMTIME_NAME, errp);
> +if (error_is_set(errp)) {
> +error_setg(errp, "Failed to acquire privilege");
Sorry, missed this one on the last review:
Overriding a previously set error will cause an assert. We should just
return and pass back the error from acquire_privilege.
> +return;
> +}
> +
> +if (!SetSystemTime(&ts)) {
> +error_setg(errp, "Failed to set time to guest: %d",
> (int)GetLastError());
> +return;
> +}
> +}
> +
> int64_t qmp_guest_file_open(const char *path, bool has_mode, const char
> *mode, Error **err)
> {
> error_set(err, QERR_UNSUPPORTED);
> --
> 1.7.11.7
>
Re: [Qemu-devel] [PATCH 5/9] qapi_sized_buffer
On Thu, Mar 14, 2013 at 09:39:14AM -0400, Stefan Berger wrote:
> On 03/14/2013 08:18 AM, mdroth wrote:
> >On Wed, Mar 13, 2013 at 09:48:11PM -0400, Stefan Berger wrote:
> >>On 03/13/2013 07:18 PM, mdroth wrote:
> >>>On Wed, Mar 13, 2013 at 06:00:24PM -0400, Stefan Berger wrote:
> >>>>On 03/13/2013 04:52 PM, mdroth wrote:
> >>>>
> >>>Visitors don't have any knowledge of the data structures they're visiting
> >>>outside of what we tell them via the visit_*() API.
> >>>
> >>>[...]
> >>>
> >>>For example, a visitor for a 16-element array of:
> >>>
> >>>typedef struct ComplexType {
> >>> int32_t foo;
> >>> char *bar;
> >>>} ComplexType;
> >>>
> >>>would look something like:
> >>>
> >>>visit_start_carray(v, ...); // instruct visitor how to calculate offsets
> >>>for (i = 0; i < 16; i++) {
> >>> visit_type_ComplexType(v, ...) // instruct visitor how to handle elem
> >>> visit_next_carray(v, ...); // instruct visitor to move to next offset
> >>>}
> >>>visit_end_carray(v, ...); // instruct visitor to finalize array
> >>Given this example above, I think we will need the sized buffer. The
> >>sized buffer targets binary arrays and their encoding. If I was to
> >>encode an 'unsigned char[n]' (e.g., n=200) using n, or n/2 or n/4
> >>loops like above breaking it apart in u8, u16 or u32 respectively I
> >>think this would 'not bed good' also considering the 2 bytes for tag
> >>and length being added by ASN.1 for every such datatype
> >>(u8,u16,u32). The sized buffer allows you to for example take a
> >>memory page and write it out in one chunk adding a few bytes of
> >>ASN.1 'decoration' around the actual data.
> >You could do it with this interface as well actually. The Visitor will
> >need to maintain some internal state to differentiate what it does with
> >subsequent visit_type*/visit_next_carray/visit_end_carry. There's no
> >reason it couldn't also track the elem size so it could tag a buffer
> >"en masse" when visit_end_carray() gets called.
>
> It depends on what you pass into visit_start_carray. In your case if
> you pass in ComplexType you would pass in a sizeof(ComplexType) for
> the size of each element presumably. The problem is now you have
> char *foo, a string pointer, hanging off of this structure. How
> would you handle that? Serializing ComplexType's foo and pointer
> obviously won't do it.
Why not? visit_type_ComplexType() knows how to deal with
the individual fields, including the string pointer. I'm not sure
what's at issue here.
In this case the handling for ComplexType would look something like:
visit_type_Complex:
visit_start_struct
visit_type_uin32 //foo
visit_type_str //bar
visit_end_struct
Granted, strings are easier to deal with. If char * was instead a plain
old uint8_t*, we'd need a nested call to start_carray for each element.
in this case it would look something like:
visit_type_Complex:
visit_start_struct
visit_type_uin32 //foo field
visit_start_carray //bar field
for (i = 0; i < len_of_bar; i++):
visit_type_uint8
visit_next_carray
visit_end_carray
visit_end_struct
The key is knowing the length. In open coded visitor routines we know
this, or where to get it, for routines generated from QAPI schemas
we'd a way to tell the code generators how to field the size, or state
the size in the schema directly. I had some patches to do this, but we
don't have a QAPI user that needs this yet. When we do,
visit_*_carray() should be able to handle it, so we should consolidate
around that interface since there are a lot of things to consider in
the scope of what a visitor implementation may be used for.
> would you handle that? Serializing ComplexType's foo and pointer
> obviously won't do it. You need to follow the string pointer and
> serialize that as well. So we have different use cases here when
> wanting to serialize ComplexType versus a plain array with the
> carray calls somehow having to figure it out themselves -- how ??
for a plain array we'd just replace visit_type_ComplexType() with
visit_type_uint{8,16,32,64} and change loop/elem_size params
accordingly.
>
>Stefan
>
Re: [Qemu-devel] [PATCH 5/9] qapi_sized_buffer
On Thu, Mar 14, 2013 at 10:51:49AM -0400, Stefan Berger wrote:
> On 03/14/2013 10:28 AM, mdroth wrote:
> >On Thu, Mar 14, 2013 at 09:39:14AM -0400, Stefan Berger wrote:
> >>On 03/14/2013 08:18 AM, mdroth wrote:
> >>>On Wed, Mar 13, 2013 at 09:48:11PM -0400, Stefan Berger wrote:
> >>>>On 03/13/2013 07:18 PM, mdroth wrote:
> >>>>>On Wed, Mar 13, 2013 at 06:00:24PM -0400, Stefan Berger wrote:
> >>>>>>On 03/13/2013 04:52 PM, mdroth wrote:
> >>>>>>
> >>>>>Visitors don't have any knowledge of the data structures they're visiting
> >>>>>outside of what we tell them via the visit_*() API.
> >>>>>
> >>>>>[...]
> >>>>>
> >>>>>For example, a visitor for a 16-element array of:
> >>>>>
> >>>>>typedef struct ComplexType {
> >>>>> int32_t foo;
> >>>>> char *bar;
> >>>>>} ComplexType;
> >>>>>
> >>>>>would look something like:
> >>>>>
> >>>>>visit_start_carray(v, ...); // instruct visitor how to calculate offsets
> >>>>>for (i = 0; i < 16; i++) {
> >>>>> visit_type_ComplexType(v, ...) // instruct visitor how to handle
> >>>>> elem
> >>>>> visit_next_carray(v, ...); // instruct visitor to move to next
> >>>>> offset
> >>>>>}
> >>>>>visit_end_carray(v, ...); // instruct visitor to finalize array
> >>>>Given this example above, I think we will need the sized buffer. The
> >>>>sized buffer targets binary arrays and their encoding. If I was to
> >>>>encode an 'unsigned char[n]' (e.g., n=200) using n, or n/2 or n/4
> >>>>loops like above breaking it apart in u8, u16 or u32 respectively I
> >>>>think this would 'not bed good' also considering the 2 bytes for tag
> >>>>and length being added by ASN.1 for every such datatype
> >>>>(u8,u16,u32). The sized buffer allows you to for example take a
> >>>>memory page and write it out in one chunk adding a few bytes of
> >>>>ASN.1 'decoration' around the actual data.
> >>>You could do it with this interface as well actually. The Visitor will
> >>>need to maintain some internal state to differentiate what it does with
> >>>subsequent visit_type*/visit_next_carray/visit_end_carry. There's no
> >>>reason it couldn't also track the elem size so it could tag a buffer
> >>>"en masse" when visit_end_carray() gets called.
> >>It depends on what you pass into visit_start_carray. In your case if
> >>you pass in ComplexType you would pass in a sizeof(ComplexType) for
> >>the size of each element presumably. The problem is now you have
> >>char *foo, a string pointer, hanging off of this structure. How
> >>would you handle that? Serializing ComplexType's foo and pointer
> >>obviously won't do it.
> >Why not? visit_type_ComplexType() knows how to deal with
> >the individual fields, including the string pointer. I'm not sure
> >what's at issue here.
> >
> >In this case the handling for ComplexType would look something like:
> >
> >visit_type_Complex:
> > visit_start_struct
> > visit_type_uin32 //foo
> > visit_type_str //bar
> > visit_end_struct
> >
> >Granted, strings are easier to deal with. If char * was instead a plain
> >old uint8_t*, we'd need a nested call to start_carray for each element.
> >in this case it would look something like:
> >
> >visit_type_Complex:
> > visit_start_struct
> > visit_type_uin32 //foo field
> > visit_start_carray //bar field
> > for (i = 0; i < len_of_bar; i++):
> > visit_type_uint8
> > visit_next_carray
> > visit_end_carray
>
> You really want to create a separate element for each element in
> this potentially large binary array? I guess it depends on the
> underlying data, but this has the potential of generating a lot of
> control code around each such byte... As said, for ASN.1 encoding,
> each such byte would be decorated with a tag and a length value,
> consuming 2 more bytes per byte.
I addressed this earlier. Your visitor doesn't have tag each
element: if it know it's handling an array (because we told it via
start_carray()), it can buffer them internally and tag the array en
masse when end_carray() is issued.
>
>Stefan
>
Re: [Qemu-devel] [PATCH 5/9] qapi_sized_buffer
On Thu, Mar 14, 2013 at 11:24:03AM -0400, Stefan Berger wrote:
> On 03/14/2013 11:11 AM, mdroth wrote:
> >On Thu, Mar 14, 2013 at 10:51:49AM -0400, Stefan Berger wrote:
> >>On 03/14/2013 10:28 AM, mdroth wrote:
> >>>On Thu, Mar 14, 2013 at 09:39:14AM -0400, Stefan Berger wrote:
> >>>>On 03/14/2013 08:18 AM, mdroth wrote:
> >>>>>On Wed, Mar 13, 2013 at 09:48:11PM -0400, Stefan Berger wrote:
> >>>>>>On 03/13/2013 07:18 PM, mdroth wrote:
> >>>>>>>On Wed, Mar 13, 2013 at 06:00:24PM -0400, Stefan Berger wrote:
> >>>>>>>>On 03/13/2013 04:52 PM, mdroth wrote:
> >>>>>>>>
> >>>>>>>Visitors don't have any knowledge of the data structures they're
> >>>>>>>visiting
> >>>>>>>outside of what we tell them via the visit_*() API.
> >>>>>>>
> >>>>>>>[...]
> >>>>>>>
> >>>>>>>For example, a visitor for a 16-element array of:
> >>>>>>>
> >>>>>>>typedef struct ComplexType {
> >>>>>>> int32_t foo;
> >>>>>>> char *bar;
> >>>>>>>} ComplexType;
> >>>>>>>
> >>>>>>>would look something like:
> >>>>>>>
> >>>>>>>visit_start_carray(v, ...); // instruct visitor how to calculate
> >>>>>>>offsets
> >>>>>>>for (i = 0; i < 16; i++) {
> >>>>>>> visit_type_ComplexType(v, ...) // instruct visitor how to handle
> >>>>>>> elem
> >>>>>>> visit_next_carray(v, ...); // instruct visitor to move to next
> >>>>>>> offset
> >>>>>>>}
> >>>>>>>visit_end_carray(v, ...); // instruct visitor to finalize array
> >>>>>>Given this example above, I think we will need the sized buffer. The
> >>>>>>sized buffer targets binary arrays and their encoding. If I was to
> >>>>>>encode an 'unsigned char[n]' (e.g., n=200) using n, or n/2 or n/4
> >>>>>>loops like above breaking it apart in u8, u16 or u32 respectively I
> >>>>>>think this would 'not bed good' also considering the 2 bytes for tag
> >>>>>>and length being added by ASN.1 for every such datatype
> >>>>>>(u8,u16,u32). The sized buffer allows you to for example take a
> >>>>>>memory page and write it out in one chunk adding a few bytes of
> >>>>>>ASN.1 'decoration' around the actual data.
> >>>>>You could do it with this interface as well actually. The Visitor will
> >>>>>need to maintain some internal state to differentiate what it does with
> >>>>>subsequent visit_type*/visit_next_carray/visit_end_carry. There's no
> >>>>>reason it couldn't also track the elem size so it could tag a buffer
> >>>>>"en masse" when visit_end_carray() gets called.
> >>>>It depends on what you pass into visit_start_carray. In your case if
> >>>>you pass in ComplexType you would pass in a sizeof(ComplexType) for
> >>>>the size of each element presumably. The problem is now you havechar
> >>>>*foo, a string pointer, hanging off of this structure. How
> >>>>would you handle that? Serializing ComplexType's foo and pointer
> >>>>obviously won't do it.
> >>>Why not? visit_type_ComplexType() knows how to deal with
> >>>the individual fields, including the string pointer. I'm not sure
> >>>what's at issue here.
> >>>
> >>>In this case the handling for ComplexType would look something like:
> >>>
> >>>visit_type_Complex:
> >>> visit_start_struct
> >>> visit_type_uin32 //foo
> >>> visit_type_str //bar
> >>> visit_end_struct
> >>>
> >>>Granted, strings are easier to deal with. If char * was instead a plain
> >>>old uint8_t*, we'd need a nested call to start_carray for each element.
> >>>in this case it would look something like:
> >>>
> >>>visit_type_Complex:
> >>> visit_start_struct
> >>> visit_type_uin32 //foo field
> >>> visit_start_carray //bar field
> >>> for (i =
Re: [Qemu-devel] [PATCH 1/2] qga: add windows implementation for guest-get-time
On Thu, Mar 14, 2013 at 11:05:52PM +0800, Lei Li wrote:
> Signed-off-by: Lei Li
> ---
> qga/commands-win32.c | 29 +
> 1 file changed, 29 insertions(+)
>
> diff --git a/qga/commands-win32.c b/qga/commands-win32.c
> index 7e8ecb3..b395bd5 100644
> --- a/qga/commands-win32.c
> +++ b/qga/commands-win32.c
> @@ -22,6 +22,12 @@
> #define SHTDN_REASON_FLAG_PLANNED 0x8000
> #endif
>
> +/* multiple of 100 nanoseconds elapsed between windows baseline
> + (1/1/1601) and Unix Epoch (1/1/1970), accounting for leap years */
> +#define W32_FT_OFFSET (1000ULL * 60 * 60 * 24 * \
> + (365 * (1970 - 1601) + \
> +(1970 - 1601) / 4 - 3))
> +
> static void acquire_privilege(const char *name, Error **err)
> {
> HANDLE token;
> @@ -108,6 +114,29 @@ void qmp_guest_shutdown(bool has_mode, const char *mode,
> Error **err)
> }
> }
>
> +int64_t qmp_guest_get_time(Error **errp)
> +{
> +SYSTEMTIME ts = {0};
> +int64_t time_ns;
> +FILETIME tf;
> +
> +GetSystemTime(&ts);
> +if (ts.wYear < 1601 || ts.wYear > 30827) {
> +error_setg(errp, "Failed to get time");
> +return -1;
> +}
> +
> +if (!SystemTimeToFileTime(&ts, &tf)) {
> +error_setg_errno(errp, errno, "Failed to convert system time");
I missed this is well, SystemTimeToFileTime() doesn't set errno, but if
this is the only change needed I can fix this up in my tree.
> +return -1;
> +}
> +
> +time_ns = int64_t)tf.dwHighDateTime << 32) | tf.dwLowDateTime)
> +- W32_FT_OFFSET) * 100;
> +
> +return time_ns;
> +}
> +
> int64_t qmp_guest_file_open(const char *path, bool has_mode, const char
> *mode, Error **err)
> {
> error_set(err, QERR_UNSUPPORTED);
> --
> 1.7.11.7
>
Re: [Qemu-devel] [PATCH 0/2 v6] Add Windows support for time resync by qemu-ga
On Thu, Mar 14, 2013 at 11:05:51PM +0800, Lei Li wrote: > This patch series attempts to add Windows implementation > for qemu-ga commands guest-get-time and guest-set-time. > > The previous thread about the interfaces introduced and > the POSIX-specific command implementation has already > been accepted, the reference link: > > http://article.gmane.org/gmane.comp.emulators.qemu/198472 > > Notes: > Now It was tested on Windows XP SP3 and Windows 7. > Please comment! Series looks good other than comment in patch 1. I can fix this in tree or you can send another version. Reviewed-by: Michael Roth > > Thanks. > > Changes since v5: > - Fix the error check for GetSystemTime() from Michael. > - Other fixups from Michael. > > Changes since v4: > - Error handel improvement from Michael. > - Do the math explicitly for the time convert of FILETIME > suggested by Michael. > > Changes since v3: > - Reorder the acquire_privilege to avoid a possible > leak of privileges suggested by Eric. > > Changes since v2: > - Overflow check improvement for time_ns from Eric. > > Changes since v1: > - Make the macro for the offset between windows baseline > and Unix Epoch more readable from Eric. > - Overflow check for filetime pointed by Eric. > > Lei Li (2): > qga: add windows implementation for guest-get-time > qga: add windows implementation for guest-set-time >
Re: [Qemu-devel] [PATCH 2/2] qga schema: document generic QERR_UNSUPPORTED
On Fri, Mar 15, 2013 at 07:07:51PM +0100, Laszlo Ersek wrote: > Part of the wording was shamelessly stolen from Michael Roth's email. :) series: Reviewed-by: Michael Roth > > Suggested-by: Michael Roth > Signed-off-by: Laszlo Ersek > --- > qga/qapi-schema.json | 11 +++ > 1 files changed, 11 insertions(+), 0 deletions(-) > > diff --git a/qga/qapi-schema.json b/qga/qapi-schema.json > index 2af3515..7155b7a 100644 > --- a/qga/qapi-schema.json > +++ b/qga/qapi-schema.json > @@ -2,6 +2,17 @@ > > ## > # > +# General note concerning the use of guest agent interfaces: > +# > +# "unsupported" is a higher-level error than the errors that individual > +# commands might document. The caller should always be prepared to receive > +# QERR_UNSUPPORTED, even if the given command doesn't specify it, or doesn't > +# document any failure mode at all. > +# > +## > + > +## > +# > # Echo back a unique integer value, and prepend to response a > # leading sentinel byte (0xFF) the client can check scan for. > # > -- > 1.7.1 >
[Qemu-devel] [ANNOUNCE] QEMU 1.4.1 Stable released
Hi everyone, I am pleased to announce that the QEMU v1.4.1 stable release is now available at: http://wiki.qemu.org/download/qemu-1.4.1.tar.bz2 The official stable-1.4 repository has also been updated to v1.4.1: http://git.qemu.org/?p=qemu-stable-1.4.git;a=summary This release includes 57 build/bug fixes, including an important security fix for guests using qemu-nbd images. QEMU v1.4.2 is planned for 2013-05-24. Thanks! 57105f7: update VERSION for 1.4.1 (Michael Roth) 6e88653: Add -f FMT / --format FMT arg to qemu-nbd (Daniel P. Berrange) 6d0b135: target-mips: Fix accumulator selection for MIPS16 and microMIPS (Richard Sandiford) d89f9ba: Allow clock_gettime() monotonic clock to be utilized on more OS's (Brad Smith) 46f9071: target-i386: Check for host features before filter_features_for_kvm() (Eduardo Habkost) f85e082: help: add docs for missing 'queues' option of tap (Jason Wang) da78a1b: compiler: fix warning with GCC 4.8.0 (Paolo Bonzini) 2b92aa3: block: complete all IOs before resizing a device (Peter Lieven) e4cce2d: Revert "block: complete all IOs before .bdrv_truncate" (Peter Lieven) d15b1aa: qxl: better vga init in enter_vga_mode (Gerd Hoffmann) 65fe29e: doc: Fix texinfo @table markup in qemu-options.hx (Markus Armbruster) 888e036: acpi: initialize s4_val used in s4 shutdown (Bruce Rogers) d019dd9: target-mips: fix rndrashift_short_acc and code for EXTR_ instructions (Petar Jovanovic) dac077f: target-mips: fix DSP overflow macro and affected routines (Petar Jovanovic) b09a673: target-mips: fix for sign-issue in MULQ_W helper (Petar Jovanovic) 79a4dd4: target-mips: fix for incorrect multiplication with MULQ_S.PH (Petar Jovanovic) 57e929c: usb-tablet: Don't claim wakeup capability for USB-2 version (Hans de Goede) 27c7135: chardev: clear O_NONBLOCK on SCM_RIGHTS file descriptors (Stefan Hajnoczi) 283b7de: qemu-socket: set passed fd non-blocking in socket_connect() (Stefan Hajnoczi) a1cb89f: net: ensure "socket" backend uses non-blocking fds (Stefan Hajnoczi) 68f9df5: oslib-posix: rename socket_set_nonblock() to qemu_set_nonblock() (Stefan Hajnoczi) 0135796: update seabios to 1.7.2.1 (Gerd Hoffmann) 799a34a: linux-user/syscall.c: Don't warn about unimplemented get_robust_list (Peter Maydell) 8378910: linux-user: make bogus negative iovec lengths fail EINVAL (Peter Maydell) 7a238b9: linux-user: fix futex strace of FUTEX_CLOCK_REALTIME (John Rigby) 02493ee: linux-user/syscall.c: handle FUTEX_WAIT_BITSET in do_futex (John Rigby) 7d47b24: qcow2: flush refcount cache correctly in qcow2_write_snapshots() (Stefan Hajnoczi) 02ea844: qcow2: flush refcount cache correctly in alloc_refcount_block() (Stefan Hajnoczi) 0fcf00b: page_cache: fix memory leak (Peter Lieven) 5610ef5: Fix page_cache leak in cache_resize (Orit Wasserman) 7a687ae: virtio-blk: fix unplug + virsh reboot (Christian Borntraeger) b91aee5: ide/macio: Fix macio DMA initialisation. (Mark Cave-Ayland) e09b99b: target-ppc: Fix CPU_POWERPC_MPC8547E (Andreas Färber) 611c7f2: pseries: Add cleanup hook for PAPR virtual LAN device (David Gibson) 4e4566c: configure: Require at least spice-protocol-0.12.3 (Michal Privoznik) 43e0061: qemu-bridge-helper: force usage of a very high MAC address for the bridge (Paolo Bonzini) 3c3de7c: virtio-ccw: Queue sanity check for notify hypercall. (Cornelia Huck) b0da310: tcg: Fix occasional TCG broken problem when ldst optimization enabled (Yeongkyoon Lee) d26efd2: qga/main.c: Don't use g_key_file_get/set_int64 (Peter Crosthwaite) f305d50: qemu-ga: use key-value store to avoid recycling fd handles after restart (Michael Roth) d3652a1: qcow2: make is_allocated return true for zero clusters (Paolo Bonzini) 5194350: pseries: Add compatible property to root of device tree (David Gibson) 4d1cdb9: Allow virtio-net features for legacy s390 virtio bus (Christian Borntraeger) c3b81e0: rtc-test: Fix test failures with recent glib (Cole Robinson) 99b1f39: scsi-disk: do not complete canceled UNMAP requests (Paolo Bonzini) f23ab03: scsi: do not call scsi_read_data/scsi_write_data for a canceled request (Paolo Bonzini) 0c918dd: iscsi: look for pkg-config file too (Paolo Bonzini) a8b090e: scsi-disk: handle io_canceled uniformly and correctly (Paolo Bonzini) 4a38944: qemu-ga: make guest-sync-delimited available during fsfreeze (Michael Roth) b7ff1a7: qmp: netdev_add is like -netdev, not -net, fix documentation (Markus Armbruster) d49fed4: vga: fix byteswapping. (Gerd Hoffmann) cebb8eb: help: add docs for multiqueue tap options (Jason Wang) 3b39a11: net: reduce the unnecessary memory allocation of multiqueue (Jason Wang) ec9f828: qemu-char.c: fix waiting for telnet connection message (Igor Mitsyanko) 332e934: tap: forbid creating multiqueue tap when hub is used (Jason Wang) e6b795f: block: complete all IOs before .bdrv_truncate (Peter Lieven) 51968b8: coroutine: trim down nesting level in perf_nesting test (Paolo Bonzini) 80d8b5d: target-ppc: Fix "G2leGP3" PVR (Andreas Färber)
Re: [Qemu-devel] [qapi] Cannot use list of strings
On Tue, Apr 16, 2013 at 10:49:19AM +0200, Stefan Hajnoczi wrote:
> On Mon, Apr 15, 2013 at 10:04:24PM +0200, Lluís Vilanova wrote:
> > Tried using a list of strings as an argument to a command, but the generated
> > code references the 'strList' type, which does not exist.
> >
> > Is a specialized version for "['str']" missing, or should I define my own
> > type
> > with a single field of 'str' type?
I would say just give that a shot. Stick:
typedef struct strList
{
char *value;
struct strList *next;
} strList;
in include/qapi/visitor.h and see what happens.
It's not immediately obvious to me why that wouldn't work, except maybe
that sometimes the code generators will special case on CamelCase types,
but that shouldn't be too hard to work around if it's an issue.
If that works, the same approach can probably be taken for all "native" qapi
types: str, bool, [u]int[(8|16|32|64)]
>
> akong just hit this too.
>
> I think it's a question for aliguori, luiz, or mdroth.
>
> Stefan
>
Re: [Qemu-devel] [PATCH v2] qapi: add struct strList and visit_type_strList()
On Thu, Apr 25, 2013 at 12:08:05AM +0800, Amos Kong wrote:
> Currently we can only use ['String'] to add string to a list,
> it contains some additional JSON structure.
> "multicast": [
> {
> "str": "01:80:c2:00:00:21"
> },
> {
> "str": "00:00:00:00:00:00"
> }
> ]
>
> This patch adds struct strList and visit function, we can use ['str'] in
> schema file.
>
> "multicast": [
> "01:00:5e:00:00:01",
> "33:33:ff:12:34:57"
> ]
>
> V2: remove ugly #ifndef, add forward declaration in qapi-types.h,
> and define the struct in include/qapi/visitor.h (Paolo)
>
> Signed-off-by: Amos Kong
Sorry for the delay in getting back to you, I started to respond last
week and ended up hacking something up that take a different approach.
Namely: we don't hardcode visit_type_strList(), but instead teach the
qapi code generators about native types we currently allow in schemas
('str', 'int', 'number', 'bool') and have them handle code generation
for these like they would for any user-defined type.
This approach works too, but I think this solution is more complete.
I'm still working out some bits with the unit tests but I've pushed my
WIP here for reference:
https://github.com/mdroth/qemu/commits/qapi-native-lists
Please give that a shot.
> ---
> include/qapi/visitor.h |7 +++
> qapi/qapi-visit-core.c | 23 +++
> scripts/qapi-types.py |1 +
> scripts/qapi.py|4 ++--
> 4 files changed, 33 insertions(+), 2 deletions(-)
>
> diff --git a/include/qapi/visitor.h b/include/qapi/visitor.h
> index 1fef18c..743ff92 100644
> --- a/include/qapi/visitor.h
> +++ b/include/qapi/visitor.h
> @@ -22,6 +22,11 @@ typedef struct GenericList
> struct GenericList *next;
> } GenericList;
>
> +typedef struct strList {
> +char *value;
> +struct strList *next;
> +} strList;
> +
> typedef struct Visitor Visitor;
>
> void visit_start_handle(Visitor *v, void **obj, const char *kind,
> @@ -50,6 +55,8 @@ void visit_type_int64(Visitor *v, int64_t *obj, const char
> *name, Error **errp);
> void visit_type_size(Visitor *v, uint64_t *obj, const char *name, Error
> **errp);
> void visit_type_bool(Visitor *v, bool *obj, const char *name, Error **errp);
> void visit_type_str(Visitor *v, char **obj, const char *name, Error **errp);
> +void visit_type_strList(Visitor *m, strList ** obj, const char *name,
> +Error **errp);
> void visit_type_number(Visitor *v, double *obj, const char *name, Error
> **errp);
>
> #endif
> diff --git a/qapi/qapi-visit-core.c b/qapi/qapi-visit-core.c
> index 401ee6e..dc54cc8 100644
> --- a/qapi/qapi-visit-core.c
> +++ b/qapi/qapi-visit-core.c
> @@ -257,6 +257,29 @@ void visit_type_str(Visitor *v, char **obj, const char
> *name, Error **errp)
> }
> }
>
> +void visit_type_strList(Visitor *m, strList ** obj, const char *name,
> +Error **errp)
> +{
> +GenericList *i, **prev = (GenericList **)obj;
> +Error *err = NULL;
> +
> +if (!error_is_set(errp)) {
> +visit_start_list(m, name, &err);
> +if (!err) {
> +for (; (i = visit_next_list(m, prev, &err)) != NULL; prev = &i) {
> +strList *native_i = (strList *)i;
> +visit_type_str(m, &native_i->value, NULL, &err);
> +}
> +error_propagate(errp, err);
> +err = NULL;
> +
> +/* Always call end_list if start_list succeeded. */
> +visit_end_list(m, &err);
> +}
> +error_propagate(errp, err);
> +}
> +}
> +
> void visit_type_number(Visitor *v, double *obj, const char *name, Error
> **errp)
> {
> if (!error_is_set(errp)) {
> diff --git a/scripts/qapi-types.py b/scripts/qapi-types.py
> index 9e19920..e449a14 100644
> --- a/scripts/qapi-types.py
> +++ b/scripts/qapi-types.py
> @@ -276,6 +276,7 @@ fdecl.write(mcgen('''
> #include
> #include
>
> +struct strList;
> ''',
>guard=guardname(h_file)))
>
> diff --git a/scripts/qapi.py b/scripts/qapi.py
> index afc5f32..14f9f4d 100644
> --- a/scripts/qapi.py
> +++ b/scripts/qapi.py
> @@ -166,11 +166,11 @@ def c_fun(name, protect=True):
> return c_var(name, protect).replace('.', '_')
>
> def c_list_type(name):
> -return '%sList' % name
> +return 'struct %sList' % name
>
> def type_name(name):
> if type(name) == list:
> -return c_list_type(name[0])
> +return '%sList' % name[0]
> return name
>
> enum_types = []
> --
> 1.7.1
>
Re: [Qemu-devel] [RFC 0/9] QContext: QOM class to support multiple event loops
On Mon, May 06, 2013 at 09:54:14AM +0200, Paolo Bonzini wrote: > Il 03/05/2013 18:03, Michael Roth ha scritto: > > These patches apply on top of qemu.git master, and can also be obtained > > from: > > git://github.com/mdroth/qemu.git qcontext > > > > OVERVIEW > > > > This series introduces a set of QOM classes/interfaces for event > > registration/handling: QContext and QSource, which are based closely on > > their GMainContext/GSource GLib counterparts. > > > > QContexts can be created via the command-line via -object, and can also be > > intructed (via -object params/properties) to automatically start a > > thread/event-loop to handle QSources we attach to them. > > This is an awesome idea. > > However, it seems a bit overengineered. Why do we need QSource at all? > In my opinion, we should first change dataplane to use AioContext as a > GSource, and benchmark it thoroughly. If it is fast enough, we can I think it would be great to just stick with GSources. I didn't want to rely too heavily on GLib for the RFC since there seems to be some reservations about relying too heavily on GLib for our OneTrueEventLoop interface (mainly, lack of PI mutexes in the context of real-time device threads, or other performance considerations that might pop up and cause us to rethink our use of glib). However, knowing that we *could* do something like porting to QSources and using a different QContext implementation if the need ever became evident is enough for me, and I'm happy to drop QSources until we actually need them. The GSource->QSource conversions would be mostly mechanical. > GSource, and benchmark it thoroughly. If it is fast enough, we can > "just" introduce a glib-based QContext and be done with it. Hopefully > that is the case... Sounds good to me. I'll look into that more, and talk to some of our performance folks who were involved with the virtio-blk dataplane testing. > > Paolo >
Re: [Qemu-devel] [RFC 0/9] QContext: QOM class to support multiple event loops
On Mon, May 06, 2013 at 07:25:24AM -0500, Anthony Liguori wrote: > Paolo Bonzini writes: > > > Il 03/05/2013 18:03, Michael Roth ha scritto: > >> These patches apply on top of qemu.git master, and can also be obtained > >> from: > >> git://github.com/mdroth/qemu.git qcontext > >> > >> OVERVIEW > >> > >> This series introduces a set of QOM classes/interfaces for event > >> registration/handling: QContext and QSource, which are based closely on > >> their GMainContext/GSource GLib counterparts. > >> > >> QContexts can be created via the command-line via -object, and can also be > >> intructed (via -object params/properties) to automatically start a > >> thread/event-loop to handle QSources we attach to them. > > > > This is an awesome idea. > > Ack. > > > However, it seems a bit overengineered. > > Ack. > > > Why do we need QSource at all? > > In my opinion, we should first change dataplane to use AioContext as a > > GSource, and benchmark it thoroughly. If it is fast enough, we can > > "just" introduce a glib-based QContext and be done with it. Hopefully > > that is the case... > > Why even bother with QContext then? The QContext/GlibQContext object in general, or the QContext base class? In the case of the former, I think a wrapper around GLib that we can instantiate from the command-line line and query properties like TIDs from is necessary for robust control over event loops and CPU resources. We get this essentially for free with QOM, so I think it makes sense to use it. In the case of the latter I'm not too sure. Without the QSource abstraction there isn't much reason not to use the native GLib interfaces on the underlying GSources/GMainContexts directly. In which case GlibQContext would only need to be a container of sorts with some minor additions like spawning an event thread for itself. If we ever did need to switch it out in favor of a non-GLib implementation, it should be a mostly mechanical conversion of GSource->QSource and adding some wrappers around g_main_context_prepare/check/etc. Also along that line, if we're taking the approach of not adding infrastructure/cruft until we actually have a plan to use it, it probably makes sense to make QContext a concrete class implemented via GLib, and we can move the GLib stuff to a sub-class later if we ever end up with another QContext implementation. Does this seem reasonable? > > Regards, > > Anthony Liguori > > > > > Paolo >
Re: [Qemu-devel] [RFC 0/9] QContext: QOM class to support multiple event loops
On Mon, May 06, 2013 at 11:26:06AM +0800, liu ping fan wrote: > On Sat, May 4, 2013 at 12:03 AM, Michael Roth > wrote: > > These patches apply on top of qemu.git master, and can also be obtained > > from: > > git://github.com/mdroth/qemu.git qcontext > > > > OVERVIEW > > > > This series introduces a set of QOM classes/interfaces for event > > registration/handling: QContext and QSource, which are based closely on > > their GMainContext/GSource GLib counterparts. > > > > QContexts can be created via the command-line via -object, and can also be > > intructed (via -object params/properties) to automatically start a > > thread/event-loop to handle QSources we attach to them. > > > > The reference implementation of QContext is GlibQContext, which uses > > GMainContext/GSource interfaces underneath the covers, thus we can > > also attach GSource (and as a result, AioContexts) to it. > > > > As part of this series we also port the QEMU main loop to using QContext > > and drop virtio-blk's dataplane thread in favor of a GlibQContext thread, > > which virtio-blk dataplane attaches to via it's AioContext's GSource. > > > > With these patches in place we can do virtio-blk dataplane assignment > > like so: > > > > qemu ... \ > > -object glib-qcontext,id=ctx0,threaded=yes > > -drive file=file.raw,id=drive0,aio=native,cache=none \ > > -device virtio-blk,drive=drive0,scsi=off,x-data-plane=on,context=ctx0 > > > > And also gain the ability to assign a virtio-blk dataplane to the default > > QContext driven by the QEMU main iothread: > > > > qemu ... \ > > -drive file=file.raw,id=drive0,aio=native,cache=none \ > > -device virtio-blk,drive=drive0,scsi=off,x-data-plane=on,context=main > > > > The latter likely isn't particularly desirable, and the series is in rough > > shape in general, but the goal of this RFC to demonstrate the approach and > > get some feedback on how we might handle thread assignments for things like > > virtio-blk/virtio-net dataplane, and multi-threaded device models general. > > > > Input on this would be greatly appreciated. > > > > BACKGROUND > > > > There has been an outgoing discussion on qemu-devel about what event loop > > interface to consolidate around for virtio-blk dataplane, threaded > > virtio-net, > > and offloading device workloads to seperate threads in general. > > > > Currently the main candidates seem to be GLib GSources and AioContext, with > > virtio-blk/virtio-net dataplane being the use-cases under consideration. > > > > virtio-blk: > > > > In the case of virtio-blk dataplane, we need to drive virtqueue and AIO > > events. > > Since AioContext is used extensively throughout the block layer to drive > > AIO, > > it makes sense to re-use it here as we look toward pushing dataplane > > functionality deeper into the block layer to benefit from things like image > > format support/snapshots/etc. > > > > virtio-net: > > > > In the case of Ping Fan's virtio-net dataplane patches > > (http://thread.gmane.org/gmane.comp.emulators.qemu/196111/focus=196111), we > > need to drive virtqueue and NetClient peer events (such as TAP devices, or > > possibly things like slirp in the future). Currently NetClient events rely > > on > > IOHandler interfaces such as qemu_set_fd_handler(). These interfaces are > > global > > ones that rely on a single IOHandler list serviced by QEMU's main loop. An > > effort is currently underway to port these to GSources so that can be more > > easilly attached to other event loops (as opposed to the hooks used for the > > virtio-net dataplane series). > > > > Theoretically, much of the latter (such as TAP devices) can also be done > > around > > AioContext with some minor changes, but other NetClient backends such as > > slirp > > lend themselves to more open-ended event loop interfaces like GSources. > > Other > > devices might also find themselves needing something more open-ended (a > > somewhat > > silly but present example being virtio-serial + GSource-driven chardev) > > > > QContext: > > > > Since the direction for the forseeable future will likely continue to be > > GSources for some things, AioContext for others, a way to reconcile these > > approaches would be the age-old approach of adding a layer of abstration on > > top of the 2 so that we can handle device/backend thread assignments using > > a general mechanism. B
Re: [Qemu-devel] [PATCH 2/9] qom: add object_property_add_unnamed_child
On Mon, May 06, 2013 at 09:44:13AM +0200, Paolo Bonzini wrote:
> Il 03/05/2013 18:03, Michael Roth ha scritto:
> > This interface allows us to add a child property without specifying a
> > name. Instead, a unique name is created and passed back after adding
> > the property.
> >
> > Signed-off-by: Michael Roth
> > ---
> > include/qom/object.h | 16
> > qom/object.c | 25 +
> > 2 files changed, 41 insertions(+)
> >
> > diff --git a/include/qom/object.h b/include/qom/object.h
> > index 86f1e2e..ca0fce8 100644
> > --- a/include/qom/object.h
> > +++ b/include/qom/object.h
> > @@ -1041,6 +1041,22 @@ void object_property_add_child(Object *obj, const
> > char *name,
> > Object *child, struct Error **errp);
> >
> > /**
> > + * object_property_add_unnamed_child:
> > + *
> > + * @obj: the object to add a property to
> > + * @name: the name of the property
> > + * @child: the child object
> > + * @errp: if an error occurs, a pointer to an area to store the area
> > + *
> > + * Same as object_property_add_child, but will allocate a unique name to
> > + * identify the child property.
> > + *
> > + * Returns: The name assigned to the child property, or NULL on failure.
> > + */
> > +char *object_property_add_unnamed_child(Object *obj, Object *child,
> > +struct Error **errp);
> > +
> > +/**
> > * object_property_add_link:
> > * @obj: the object to add a property to
> > * @name: the name of the property
> > diff --git a/qom/object.c b/qom/object.c
> > index c932f64..229a9a7 100644
> > --- a/qom/object.c
> > +++ b/qom/object.c
> > @@ -926,6 +926,31 @@ static void object_finalize_child_property(Object
> > *obj, const char *name,
> > object_unref(child);
> > }
> >
> > +char *object_property_add_unnamed_child(Object *obj, Object *child, Error
> > **errp)
> > +{
> > +int idx = 0;
> > +bool next_idx_found = false;
> > +char name[64];
> > +ObjectProperty *prop;
> > +
> > +while (!next_idx_found) {
> > +sprintf(name, "unnamed[%d]", idx);
> > +QTAILQ_FOREACH(prop, &obj->properties, node) {
> > +if (strcmp(name, prop->name) == 0) {
> > +idx++;
> > +break;
> > +}
> > +}
> > +if (!prop) {
> > +next_idx_found = true;
> > +}
> > +}
> > +
> > +object_property_add_child(obj, name, child, errp);
> > +
> > +return error_is_set(errp) ? NULL : g_strdup(name);
> > +}
>
> This is O(n^3) for adding N children. O(n^2) would be not-that-great
> but fine; can you take the occasion to convert the properties list to a
> hashtable?
Sure, I'll look into it.
>
> Paolo
>
> > +
> > void object_property_add_child(Object *obj, const char *name,
> > Object *child, Error **errp)
> > {
> >
>
Re: [Qemu-devel] [PATCH 1/9] qom: add qom_init_completion
On Mon, May 06, 2013 at 09:45:22AM +0200, Paolo Bonzini wrote:
> Il 03/05/2013 18:03, Michael Roth ha scritto:
> > This is similar in concept to "realize", though semantics are a
> > bit more open-ended:
> >
> > And object might in some cases need a number of properties to be
> > specified before it can be "used"/"started"/etc. This can't always
> > be done via an open-ended new() function, the main example being objects
> > that around created via the command-line by -object.
> >
> > To support these cases we allow a function, ->instance_init_completion,
> > to be registered that will be called by the -object constructor, or can
> > be called at the end of new() constructors and such.
>
> This seems a lot like a realize property that cannot be set back to false...
I seem to recall some other conditions like properties not being
modifiable after being realized? In this case though I think event loops
should be startable/stoppable via properties (post-migration, for
instance, or maybe testing/debugging) with simple qom-set commands.
Not too sure honestly, mainly I just recalled realize() being pushed
down into DeviceState for specific reasons, and didn't want to confuse
this with being the same thing (even though it does seem very similar).
I'm not sure what the best approach is here.
>
> Paolo
>
> > Signed-off-by: Michael Roth
> > ---
> > include/qom/object.h | 19 +++
> > qom/object.c | 21 +
> > vl.c |2 ++
> > 3 files changed, 42 insertions(+)
> >
> > diff --git a/include/qom/object.h b/include/qom/object.h
> > index d0f99c5..86f1e2e 100644
> > --- a/include/qom/object.h
> > +++ b/include/qom/object.h
> > @@ -394,6 +394,11 @@ struct Object
> > * @instance_init: This function is called to initialize an object. The
> > parent
> > * class will have already been initialized so the type is only
> > responsible
> > * for initializing its own members.
> > + * @instance_init_completion: This function is used mainly cases where an
> > + * object has been instantiated via the command-line, and is called once
> > all
> > + * properties specified via command-line have been set for the object.
> > This
> > + * is not called automatically, but manually via @object_init_completion
> > once
> > + * the processing of said properties is completed.
> > * @instance_finalize: This function is called during object destruction.
> > This
> > * is called before the parent @instance_finalize function has been
> > called.
> > * An object should only free the members that are unique to its type in
> > this
> > @@ -429,6 +434,7 @@ struct TypeInfo
> >
> > size_t instance_size;
> > void (*instance_init)(Object *obj);
> > +void (*instance_init_completion)(Object *obj);
> > void (*instance_finalize)(Object *obj);
> >
> > bool abstract;
> > @@ -562,6 +568,19 @@ struct InterfaceClass
> > Object *object_new(const char *typename);
> >
> > /**
> > + * object_init_completion:
> > + * @obj: The object to complete initialization of
> > + *
> > + * In cases where an object is instantiated from a command-line with a
> > number
> > + * of properties specified as parameters (generally via -object), or for
> > cases
> > + * where a new()/helper function is used to pass/set some minimal number of
> > + * properties that are required prior to completion of object
> > initialization,
> > + * this function can be called to mark when that occurs to complete object
> > + * initialization.
> > + */
> > +void object_init_completion(Object *obj);
> > +
> > +/**
> > * object_new_with_type:
> > * @type: The type of the object to instantiate.
> > *
> > diff --git a/qom/object.c b/qom/object.c
> > index 75e6aac..c932f64 100644
> > --- a/qom/object.c
> > +++ b/qom/object.c
> > @@ -50,6 +50,7 @@ struct TypeImpl
> > void *class_data;
> >
> > void (*instance_init)(Object *obj);
> > +void (*instance_init_completion)(Object *obj);
> > void (*instance_finalize)(Object *obj);
> >
> > bool abstract;
> > @@ -110,6 +111,7 @@ static TypeImpl *type_register_internal(const TypeInfo
> > *info)
> > ti->class_data = info->class_data;
> >
> > ti->instance_init = info->instance_init;
> > +ti->instance_init_completion = info->instance_init_completion;
> > ti->instance_finalize = info->instance_finalize;
> >
> > ti->abstract = info->abstract;
> > @@ -422,6 +424,25 @@ Object *object_new(const char *typename)
> > return object_new_with_type(ti);
> > }
> >
> > +
> > +static void object_init_completion_with_type(Object *obj, TypeImpl *ti)
> > +{
> > +if (type_has_parent(ti)) {
> > +object_init_completion_with_type(obj, type_get_parent(ti));
> > +}
> > +
> > +if (ti->instance_init_completion) {
> > +ti->instance_init_completion(obj);
> > +}
> > +}
> > +
> > +void object_init_completion(Object *obj)
> > +{
> > +TypeImpl *ti = type_ge
Re: [Qemu-devel] [PATCH 7/9] iohandler: associate with main event loop via a QSource
On Mon, May 06, 2013 at 09:53:12AM +0200, Paolo Bonzini wrote:
> Il 03/05/2013 18:03, Michael Roth ha scritto:
> > This introduces a GlibQContext wrapper around the main GMainContext
> > event loop, and associates iohandlers with it via a QSource (which
> > GlibQContext creates a GSource from so that it can be driven via
> > GLib. A subsequent patch will drive the GlibQContext directly)
> >
> > We also add "QContext-aware" functionality to iohandler interfaces
> > so that they can be bound to other QContext event loops, and add
> > non-global set_fd_handler() interfaces to facilitate this. This is made
> > possible by simply searching a given QContext for a QSource by the name
> > of "iohandler" so that we can attach event handlers to the associated
> > IOHandlerState.
> >
> > Signed-off-by: Michael Roth
>
> This patch is why I think that this is a bit overengineered. The main
> loop is always glib-based, there should be no need to go through the
> QSource abstraction.
>
> BTW, this is broken for Win32. The right thing to do here is to first
> convert iohandler to a GSource in such a way that it works for both
> POSIX and Win32, and then (if needed) we can later convert GSource to
> QSource.
Yup, forgot to note that Win32 was broken and on my TODO. I'll work on
that and stick to using GSources for now.
>
> Paolo
>
> > ---
> > include/qemu/main-loop.h | 31 +-
> > iohandler.c | 238
> > --
> > main-loop.c | 21 +++-
> > 3 files changed, 213 insertions(+), 77 deletions(-)
> >
> > diff --git a/include/qemu/main-loop.h b/include/qemu/main-loop.h
> > index 6f0200a..dbadf9f 100644
> > --- a/include/qemu/main-loop.h
> > +++ b/include/qemu/main-loop.h
> > @@ -26,6 +26,7 @@
> > #define QEMU_MAIN_LOOP_H 1
> >
> > #include "block/aio.h"
> > +#include "qcontext/qcontext.h"
> >
> > #define SIG_IPI SIGUSR1
> >
> > @@ -168,9 +169,24 @@ void qemu_del_wait_object(HANDLE handle,
> > WaitObjectFunc *func, void *opaque);
> >
> > /* async I/O support */
> >
> > +#define QSOURCE_IOHANDLER "iohandler"
> > +
> > typedef void IOReadHandler(void *opaque, const uint8_t *buf, int size);
> > typedef int IOCanReadHandler(void *opaque);
> >
> > +QContext *qemu_get_qcontext(void);
> > +/**
> > + * iohandler_attach: Attach a QSource to a QContext
> > + *
> > + * This enables the use of IOHandler interfaces such as
> > + * set_fd_handler() on the given QContext. IOHandler lists will be
> > + * tracked/handled/dispatched based on a named QSource that is added to
> > + * the QContext
> > + *
> > + * @ctx: A QContext to add an IOHandler QSource to
> > + */
> > +void iohandler_attach(QContext *ctx);
> > +
> > /**
> > * qemu_set_fd_handler2: Register a file descriptor with the main loop
> > *
> > @@ -217,6 +233,13 @@ int qemu_set_fd_handler2(int fd,
> > IOHandler *fd_write,
> > void *opaque);
> >
> > +int set_fd_handler2(QContext *ctx,
> > +int fd,
> > +IOCanReadHandler *fd_read_poll,
> > +IOHandler *fd_read,
> > +IOHandler *fd_write,
> > +void *opaque);
> > +
> > /**
> > * qemu_set_fd_handler: Register a file descriptor with the main loop
> > *
> > @@ -250,6 +273,12 @@ int qemu_set_fd_handler(int fd,
> > IOHandler *fd_write,
> > void *opaque);
> >
> > +int set_fd_handler(QContext *ctx,
> > + int fd,
> > + IOHandler *fd_read,
> > + IOHandler *fd_write,
> > + void *opaque);
> > +
> > #ifdef CONFIG_POSIX
> > /**
> > * qemu_add_child_watch: Register a child process for reaping.
> > @@ -302,8 +331,6 @@ void qemu_mutex_unlock_iothread(void);
> > /* internal interfaces */
> >
> > void qemu_fd_register(int fd);
> > -void qemu_iohandler_fill(GArray *pollfds);
> > -void qemu_iohandler_poll(GArray *pollfds, int rc);
> >
> > QEMUBH *qemu_bh_new(QEMUBHFunc *cb, void *opaque);
> > void qemu_bh_schedule_idle(QEMUBH *bh);
> > diff --git a/iohandler.c b/iohandler.c
> > index ae2ef8f..8625272 100644
> > --- a/iohandler.c
> > +++ b/iohandler.c
> > @@ -41,38 +41,170 @@ typedef struct IOHandlerRecord {
> > int fd;
> > int pollfds_idx;
> > bool deleted;
> > +GPollFD pfd;
> > +bool pfd_added;
> > } IOHandlerRecord;
> >
> > -static QLIST_HEAD(, IOHandlerRecord) io_handlers =
> > -QLIST_HEAD_INITIALIZER(io_handlers);
> > +typedef struct IOHandlerState {
> > +QLIST_HEAD(, IOHandlerRecord) io_handlers;
> > +} IOHandlerState;
> >
> > +static bool iohandler_prepare(QSource *qsource, int *timeout)
> > +{
> > +QSourceClass *qsourcek = QSOURCE_GET_CLASS(qsource);
> > +IOHandlerState *s = qsourcek->get_user_data(qsource);
> > +IOHandlerRecord *ioh;
> >
> > -/* XXX: fd_read_poll should be suppressed, but an API change is
>
Re: [Qemu-devel] [PATCH 9/9] dataplane: use a QContext event loop in place of custom thread
On Mon, May 06, 2013 at 09:54:03AM +0200, Paolo Bonzini wrote:
> Il 03/05/2013 18:03, Michael Roth ha scritto:
> > virtio-blk dataplane currently creates/manages it's own thread to
> > offload work to a separate event loop.
> >
> > This patch insteads allows us to specify a QContext-based event loop by
> > adding a "context" property for virtio-blk we can use like so:
> >
> > qemu ... \
> > -object glib-qcontext,id=ctx0,threaded=yes
> > -drive file=file.raw,id=drive0,aio=native,cache=none \
> > -device virtio-blk,drive=drive0,scsi=off,x-data-plane=on,context=ctx0
> >
> > virtio-blk dataplane then simply attachs/detaches it's AioContext to the
> > ctx0 event loop on start/stop.
> >
> > This also makes available the option to drive a virtio-blk dataplane via
> > the default main loop:
> >
> > qemu ... \
> > -drive file=file.raw,id=drive0,aio=native,cache=none \
> > -device virtio-blk,drive=drive0,scsi=off,x-data-plane=on,context=main
> >
> > This doesn't do much in and of itself, but helps to demonstrate how we
> > might model a general mechanism to offload device workloads to separate
> > threads.
> >
> > Signed-off-by: Michael Roth
> > ---
> > hw/block/dataplane/virtio-blk.c | 46
> > ---
> > include/hw/virtio/virtio-blk.h |7 --
> > 2 files changed, 19 insertions(+), 34 deletions(-)
> >
> > diff --git a/hw/block/dataplane/virtio-blk.c
> > b/hw/block/dataplane/virtio-blk.c
> > index 0356665..08ea10f 100644
> > --- a/hw/block/dataplane/virtio-blk.c
> > +++ b/hw/block/dataplane/virtio-blk.c
> > @@ -24,6 +24,8 @@
> > #include "virtio-blk.h"
> > #include "block/aio.h"
> > #include "hw/virtio/virtio-bus.h"
> > +#include "qcontext/qcontext.h"
> > +#include "qcontext/glib-qcontext.h"
> >
> > enum {
> > SEG_MAX = 126, /* maximum number of I/O segments */
> > @@ -60,6 +62,7 @@ struct VirtIOBlockDataPlane {
> > * use it).
> > */
> > AioContext *ctx;
> > +QContext *qctx;
> > EventNotifier io_notifier; /* Linux AIO completion */
> > EventNotifier host_notifier;/* doorbell */
> >
> > @@ -375,26 +378,6 @@ static void handle_io(EventNotifier *e)
> > }
> > }
> >
> > -static void *data_plane_thread(void *opaque)
> > -{
> > -VirtIOBlockDataPlane *s = opaque;
> > -
> > -do {
> > -aio_poll(s->ctx, true);
> > -} while (!s->stopping || s->num_reqs > 0);
> > -return NULL;
> > -}
> > -
> > -static void start_data_plane_bh(void *opaque)
> > -{
> > -VirtIOBlockDataPlane *s = opaque;
> > -
> > -qemu_bh_delete(s->start_bh);
> > -s->start_bh = NULL;
> > -qemu_thread_create(&s->thread, data_plane_thread,
> > - s, QEMU_THREAD_JOINABLE);
> > -}
> > -
> > bool virtio_blk_data_plane_create(VirtIODevice *vdev, VirtIOBlkConf *blk,
> >VirtIOBlockDataPlane **dataplane)
> > {
> > @@ -460,6 +443,7 @@ void virtio_blk_data_plane_start(VirtIOBlockDataPlane
> > *s)
> > VirtioBusClass *k = VIRTIO_BUS_GET_CLASS(qbus);
> > VirtQueue *vq;
> > int i;
> > +Error *err = NULL;
> >
> > if (s->started) {
> > return;
> > @@ -502,9 +486,16 @@ void virtio_blk_data_plane_start(VirtIOBlockDataPlane
> > *s)
> > /* Kick right away to begin processing requests already in vring */
> > event_notifier_set(virtio_queue_get_host_notifier(vq));
> >
> > -/* Spawn thread in BH so it inherits iothread cpusets */
> > -s->start_bh = qemu_bh_new(start_data_plane_bh, s);
> > -qemu_bh_schedule(s->start_bh);
> > +/* use QEMU main loop/context by default */
> > +if (!s->blk->context) {
> > +s->blk->context = g_strdup("main");
> > +}
>
> Or rather create a device-specific context by default?
Yup, definitely.
I think I did it this way to to give an idea of how a "normal" threaded device
might look (i.e. reworked or written from the start to always be capable of
being driven by a separate event loop)
But x-data-plane=on should imply a new context be used so we don't break
things, and I'll do it that way on the next pass.
>
> Paolo
>
> > +s->qctx = qcontext_find_by_name(s->blk->context, &err);
> > +if (err) {
> > +fprintf(stderr, "virtio-blk failed to start: %s",
> > error_get_pretty(err));
> > +exit(1);
> > +}
> > +aio_context_attach(s->ctx, s->qctx);
> > }
> >
> > void virtio_blk_data_plane_stop(VirtIOBlockDataPlane *s)
> > @@ -517,15 +508,6 @@ void virtio_blk_data_plane_stop(VirtIOBlockDataPlane
> > *s)
> > s->stopping = true;
> > trace_virtio_blk_data_plane_stop(s);
> >
> > -/* Stop thread or cancel pending thread creation BH */
> > -if (s->start_bh) {
> > -qemu_bh_delete(s->start_bh);
> > -s->start_bh = NULL;
> > -} else {
> > -aio_notify(s->ctx);
> > -qemu_thread_join(&s->thread);
> > -}
> > -
> > aio_set_event_notifier(s->ctx, &s->io_no
Re: [Qemu-devel] [PATCH 1/2] qga: distinguish binary modes in "guest_file_open_modes" map
On Tue, May 07, 2013 at 11:27:03AM -0600, Eric Blake wrote: > On 05/07/2013 10:56 AM, Laszlo Ersek wrote: > > In Windows guests this may make a difference. > > > > Suggested-by: Eric Blake > > Signed-off-by: Laszlo Ersek > > --- > > qga/commands-posix.c | 22 -- > > 1 files changed, 16 insertions(+), 6 deletions(-) > > > > diff --git a/qga/commands-posix.c b/qga/commands-posix.c > > index 04c6951..2eec712 100644 > > --- a/qga/commands-posix.c > > +++ b/qga/commands-posix.c > > Oh, and only NOW do I notice that this is in a file named > commands-posix.c that doesn't get compiled into the Windows build of qga > (there, we only build commands-win32.c, and THAT file always fails this > command, because no one has ported guest-file-open there yet). But I > guess there is still the argument that some weirdnix system exists that > isn't quite POSIX compliant and does have a distinct binary mode (maybe > someone plans on compiling qga for Cygwin instead of native windows, > since at least that would be able to open files when the > commands-win32.c variant doesn't?), so I still think the patch is worth > keeping. FWIW, I have some rough patches for w32 implementations of guest-file-* commands queued up that I'll be cleaning up for 1.6, so it's not so much theoretical as just a tad early :) > > -- > Eric Blake eblake redhat com+1-919-301-3266 > Libvirt virtualization library http://libvirt.org >
Re: [Qemu-devel] [PATCH] qga: set umask 0077 when daemonizing (CVE-2013-2007)
On Tue, May 07, 2013 at 09:55:06AM -0600, Eric Blake wrote:
> On 05/07/2013 05:47 AM, Anthony Liguori wrote:
> > From: Laszlo Ersek
> >
> > The qemu guest agent creates a bunch of files with insecure permissions
> > when started in daemon mode. For example:
> >
> > -rw-rw-rw- 1 root root /var/log/qemu-ga.log
> > -rw-rw-rw- 1 root root /var/run/qga.state
> > -rw-rw-rw- 1 root root /var/log/qga-fsfreeze-hook.log
> >
> > In addition, at least all files created with the "guest-file-open" QMP
> > command, and all files created with shell output redirection (or
> > otherwise) by utilities invoked by the fsfreeze hook script are affected.
> >
> > For now mask all file mode bits for "group" and "others" in
> > become_daemon().
> >
> > Temporarily, for compatibility reasons, stick with the 0666 file-mode in
> > case of files newly created by the "guest-file-open" QMP call. Do so
> > without changing the umask temporarily.
>
> This points out that:
>
> 1. the documentation for guest-file-open is insufficient to describe our
> limitations (for example, although C11 requires support for the "wx"
> flag, this patch forbids that flag)
Got a pointer? I can fix up the docs if need be, but fopen() docs that
qemu-ga points at seem to indicate 0666 will be used for new files.
That's no longer the case?
>
> 2. guest-file-open is rather puny; we may need a newer interface that
> allows the user finer-grained control over the actual mode bits set on
Yes, shame it wasn't there at the start. a guest-file-open-full or
something with support for specifying creation mode will have to do it.
> the file that they are creating (and maybe something similar to openat()
> that would let the user open/create a file relative to an existing
> handle to a directory, rather than always having to specify an absolute
> path).
>
> But I agree that _this_ patch fixes the CVE, and that any further
> changes to resolve the above two issues are not essential to include in 1.5.
>
> > +/* http://pubs.opengroup.org/onlinepubs/9699919799/functions/fopen.html */
> > +static const struct {
> > +ccpc *forms;
> > +int oflag_base;
> > +} guest_file_open_modes[] = {
> > +{ (ccpc[]){ "r", "rb", NULL }, O_RDONLY
> > },
>
> You are mapping things according to their POSIX definition (POSIX, as an
> additional requirement above and beyond C99, states that presence or
> absence of 'b' is a no-op because there is no difference between binary
> and text mode). But C99 permits a distinct binary mode, and qga is
> compiled for Windows where binary mode is indeed different.
>
> I think that you probably should split this map into:
>
> { (ccpc[]){ "r", NULL }, O_RDONLY },
> { (ccpc[]){ "rb",NULL }, O_RDONLY | O_BINARY },
>
> and so forth (assuming that O_BINARY is properly #defined to 0 on
> POSIX-y systems that don't need it), so that you don't regress the qga
> server in a Windows guest where the management client is explicitly
> passing or omitting 'b' for the intentional difference between text and
> binary files. [1]
>
> > +
> > +if ((oflag & O_CREAT) && fchmod(fd, DEFAULT_NEW_FILE_MODE) ==
> > -1) {
> > +error_setg_errno(&local_err, errno, "failed to set
> > permission "
> > + "0%03o on new file '%s' (mode: '%s')",
> > + (unsigned)DEFAULT_NEW_FILE_MODE, path,
> > mode);
>
> On this particular error path, we've left behind an empty mode
> file. Is it worth trying to unlink() the dead file?
>
> > +} else {
> > +FILE *f;
> > +
> > +f = fdopen(fd, mode);
>
> [1] I don't know if Windows is okay using fdopen() to create a FILE in
> binary mode if you didn't match O_BINARY on the fd underlying the FILE.
>
> --
> Eric Blake eblake redhat com+1-919-301-3266
> Libvirt virtualization library http://libvirt.org
>
Re: [Qemu-devel] [PATCH 6/8] qapi: add native list coverage for visitor serialization tests
On Thu, May 09, 2013 at 02:31:03PM +0200, Laszlo Ersek wrote:
> On 05/09/13 01:33, Michael Roth wrote:
>
> > +case PTYPE_NUMBER: {
> > +numberList *ptr;
> > +char *double1, *double2;
> > +if (cur_head) {
> > +ptr = cur_head;
> > +cur_head = ptr->next;
> > +} else {
> > +cur_head = ptr = pl_copy.value.numbers;
> > +}
> > +/* we serialize with %f for our reference visitors, so rather
> > than
> > + * fuzzy * floating math to test "equality", just compare the
> > + * formatted values
> > + */
>
> I think this comment block has been copied from elsewhere in this file,
> indented more deeply and re-filled. There's an asterisk in the comment
> body now.
Yes :) Will fix it on the next pass.
>
>
> > +double1 =
> > g_malloc0(calc_float_string_storage(pt->value.number));
> > +double2 = g_malloc0(calc_float_string_storage(ptr->value));
> > +g_assert_cmpstr(double1, ==, double2);
>
> Are you comparing empty strings? Space is allocated and zeroed, but I
> can't see where the values are actually formatted.
Well, that's embarassing...
>
> (Same holds for the original instance of this code, test_primitives().)
>
but at least I can blame it on the fact that I copied it from here
(we'll just ignore who the original author was :)
Will add a patch on the next pass to fix the original instance
beforehand.
Thanks for the catch!
> > +g_free(double1);
> > +g_free(double2);
> > +break;
> > +}
>
> Thanks,
> Laszlo
>
Re: [Qemu-devel] [PATCH 0/8] qapi: add support for lists of native types
On Thu, May 09, 2013 at 03:31:08PM +0200, Laszlo Ersek wrote:
> On 05/09/13 01:33, Michael Roth wrote:
> > These patches apply on top of qemu.git master, and can also be obtained
> > from:
> > git://github.com/mdroth/qemu.git qapi-native-lists
> >
> > Sending this now since a number of series have popped up in the past that
> > wanted this, and Amos has some pending patches (query-mac-tables) that rely
> > on this as well.
> >
> > These patches add support for specifying lists of native qapi types
> > (int/bool/str/number) like so:
> >
> > { 'type': 'foo',
> > 'data': { 'bar': ['int'] }}
> >
> > for a 'bar' field that is a list of type 'int',
> >
> > { 'type': 'foo2',
> > 'data': { 'bar2': ['str'] }}
> >
> > for a 'bar2' field that is a list of type 'str', and so on.
> >
> > This uses linked list types for the native C representations, just as we do
> > for complex schema-defined types. In the future we may add schema
> > annotations
> > of some sort to specify a more natural/efficient array type for the C
> > representations, but this should serve the majority of uses-cases for now.
> >
> > Makefile |6 +-
> > qapi-schema-test.json |8 ++
> > scripts/qapi-types.py | 44 ++-
> > scripts/qapi-visit.py | 36 -
> > scripts/qapi.py| 21 +++
> > tests/test-qmp-input-visitor.c | 181 +
> > tests/test-qmp-output-visitor.c| 172
> > tests/test-visitor-serialization.c | 256
> > +---
> > 8 files changed, 692 insertions(+), 32 deletions(-)
> >
> >
>
> Two notes:
> - the remark I made for 6/8 (comparing empty strings),
>
> - for 7/8 and 8/8: the format specification %3.4f is not very useful.
> "3" is the field width, "4" is the precision, and the latter means for
> %f the number of digits printed after the radix character. It's not
> useful to specify a smaller field width (which covers the entire output
> string) than precision here.
Yup, that's a mistake. The field width isn't useful at all for what I
was intending actually (to constrain the whole portion of float so that
the fractional portion wouldn't get truncated by snprintf and make the
test less likely to catch re-encoding issues). I think just using %.4f
should suffice since we have plenty of extra buffer space for the values
we're working with (max being 32 / 3) so I'll do that for the next pass.
>
> Other than these nothing pokes me in the eye.
>
> Laszlo
>
Re: [Qemu-devel] [PATCH 01/10] qapi: qapi-types.py, native list support
On Fri, May 10, 2013 at 11:04:22AM +0800, Amos Kong wrote: > On Thu, May 09, 2013 at 09:20:53PM -0500, Michael Roth wrote: > > Teach type generators about native types so they can generate the > > appropriate linked list types. > > > > Signed-off-by: Michael Roth > > --- > > scripts/qapi-types.py | 43 --- > > scripts/qapi.py | 21 + > > 2 files changed, 61 insertions(+), 3 deletions(-) > > > > diff --git a/scripts/qapi-types.py b/scripts/qapi-types.py > > index 9e19920..96cb26d 100644 > > --- a/scripts/qapi-types.py > > +++ b/scripts/qapi-types.py > > @@ -16,7 +16,18 @@ import os > > import getopt > > import errno > > > > > diff --git a/scripts/qapi.py b/scripts/qapi.py > > index afc5f32..0ac8c2b 100644 > > --- a/scripts/qapi.py > > +++ b/scripts/qapi.py > > @@ -11,6 +11,10 @@ > > > > from ordereddict import OrderedDict > > > > +builtin_types = [ > > +'str', 'int', 'number', 'bool' > > Will we add size, int8, uint32, ... in future when they are needed? Well, that was the plan, but I hadn't recalled that support for these was already been added to the code generators as part of the Netdev opts-visitor stuff, so I'll go ahead and do a re-spin with those added. > > > +] > > + > > -- > Amos. >
Re: [Qemu-devel] [PATCH 06/10] json-parser: fix handling of large whole number values
On Fri, May 10, 2013 at 02:47:18PM +0200, Laszlo Ersek wrote:
> On 05/10/13 14:22, Eric Blake wrote:
> > On 05/09/2013 08:20 PM, Michael Roth wrote:
> >> Currently our JSON parser assumes that numbers lacking a mantissa are
> >> integers and attempts to store them as QInt/int64 values. This breaks in
> >> the case where the number overflows/underflows int64 values (which is
> >> still valid JSON)
> >>
> >> Fix this by detecting such cases and using a QFloat to store the value
> >> instead.
> >>
> >> Signed-off-by: Michael Roth
> >> ---
> >> qobject/json-parser.c | 26 +++---
> >> 1 file changed, 23 insertions(+), 3 deletions(-)
> >
> > This changes the error message handed back to QMP clients, and possibly
> > exposes problems in other qemu code that receives the result of json
> > parses. Previously, for an 'int' argument, if you passed in a too-large
> > number, you got an error that the argument was too large for int. Now,
> > the number is accepted as a double; are we guaranteed that in a context
> > that expects a qint, when that code is now handed a qfloat (a case which
> > was previously impossible because qint_from_int protected it), that the
> > code will still behave correctly?
>
> I tried to consider this while reviewing... Maybe I was wrong.
>
> The pre-patch code for JSON_INTEGER:
>
> obj = QOBJECT(qint_from_int(strtoll(token_get_value(token), NULL, 10)));
>
> doesn't check for errors at all. (I assume that JSON_INTEGER is selected
> by the parser, token_get_type(), based on syntax purely.)
>
> I thought when the pre-patch version encounters an int-looking decimal
> string that's actually too big in magnitude for an int, you'd simply end
> up with LLONG_MIN or LLONG_MAX, but no error. strtoll() clamps the
> value, errno is lost, and qint_from_int() sees nothing wrong.
>
> With the patch, you end up with a float instead of an int-typed
> LLONG_MIN/LLONG_MAX, and also no error.
>
This is correct, but it does make code acting on the resulting QObject
capable of checking for this scenario and acting accordingly in whatever
manner is appropriate. I think that's appropriate, since there's no error
as far as the JSON spec goes, but further up the stack we may have stricter
requirements on what the valid ranges are for various fields.
For instance, in the case of QmpInputVisitor, these cases will
trigger an error case that was previously being bypassed when
out-of-range values were provided in place of 'int' due to the parser
silently capping the max/min values:
in qmp_input_type_int():
QObject *qobj = qmp_input_get_object(qiv, name);
if (!qobj || qobject_type(qobj) != QTYPE_QINT) {
error_set(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
"integer");
return;
}
We'd now trigger this error for such inputs (since we'd have QTYPE_QFLOAT),
which is stricter checking than before, but as intended since we map int to
int64_t and expect values in that range (though we could stand to be clearer
about this in the QAPI documentation)
> > At any rate, libvirt already checks that all numbers that fall outside
> > the range of int64_t are never passed over qmp when passing an int
> > argument (and yes, this is annoying, in that large 64-bit unsigned
> > numbers have to be passed as negative numbers, rather than exceeding
> > INT64_MAX), so libvirt should not be triggering this newly exposed code
> > path. But even if libvirt doesn't plan on triggering it, I'd still feel
> > better if your commit message documented evidence of testing what
> > happens in this case. For example, compare what
> > {"execute":"add-fd","arguments":{"fdset-id":""}}
> > does before and after this patch.
>
> That would be likely interesting to test, yes.
That would trigger the error mentioned above, as opposed to silently changing
the value to LLONG_MAX. I'll see about covering this case somewhere in
tests/test-qmp-input-visitor.
>
> Laszlo
>
Re: [Qemu-devel] [PATCH 06/10] json-parser: fix handling of large whole number values
On Fri, May 10, 2013 at 08:08:05AM -0600, Eric Blake wrote:
> On 05/10/2013 06:47 AM, Laszlo Ersek wrote:
>
> > The pre-patch code for JSON_INTEGER:
> >
> > obj = QOBJECT(qint_from_int(strtoll(token_get_value(token), NULL, 10)));
> >
> > doesn't check for errors at all. (I assume that JSON_INTEGER is selected
> > by the parser, token_get_type(), based on syntax purely.)
> >
> > I thought when the pre-patch version encounters an int-looking decimal
> > string that's actually too big in magnitude for an int, you'd simply end
> > up with LLONG_MIN or LLONG_MAX, but no error. strtoll() clamps the
> > value, errno is lost, and qint_from_int() sees nothing wrong.
>
> Oh, right. _That's_ why libvirt had to add checks that it wasn't
> passing 0x8000ULL as a positive number - because the qemu
> parser was silently clamping it to 0x7fffLL, which is not
> what libvirt wanted. So the code was NOT erroring out with an overflow
> message, but was acting on the wrong integer.
>
> >
> > With the patch, you end up with a float instead of an int-typed
> > LLONG_MIN/LLONG_MAX, and also no error.
>
> Ah, but here we have a difference - beforehand, the code was passing a
> valid (albeit wrong value) qint, so the rest of the qemu code was
> oblivious to the fact that the QMP message contained an overflow. But
> now the code is passing a qdouble, and the rest of the qemu code may be
> unprepared to handle it when expecting a qint.
Yup, new error cases can be triggered, but in the case of
QmpInputVisitor this is handled appropriately (will add a test case to
confirm), and none of our other input visitors act on QObjects, and this
ambiguity isn't present for output visitors.
We also have monitor events that call qobject_from_json() to marshall
event payloads, but these are essentially open-coded QmpInputVisitors
where the JSON values come from native C types. The only case where I
can see this triggering the change is if they did something like:
obj = qobject_from_jsonf("{'myInt': %f}", whole_valued_float);
which would be evil, and thankfully such cases don't appear to exist:
mdroth@loki:~/w/qemu.git$ ack-grep qobject_from_json | grep "%f"
tests/check-qjson.c:987:obj = qobject_from_jsonf("%f", valuef);
mdroth@loki:~/w/qemu.git$
(the 'valuef' above is not whole-valued, and the output is expected to
be a QFloat)
I'm not aware of any other cases to consider, but I might've missed
something.
>
> >
> >> At any rate, libvirt already checks that all numbers that fall outside
> >> the range of int64_t are never passed over qmp when passing an int
> >> argument (and yes, this is annoying, in that large 64-bit unsigned
> >> numbers have to be passed as negative numbers, rather than exceeding
> >> INT64_MAX), so libvirt should not be triggering this newly exposed code
> >> path. But even if libvirt doesn't plan on triggering it, I'd still feel
> >> better if your commit message documented evidence of testing what
> >> happens in this case. For example, compare what
> >> {"execute":"add-fd","arguments":{"fdset-id":""}}
> >> does before and after this patch.
> >
> > That would be likely interesting to test, yes.
>
> add-fd may not be the best candidate (it expects an fd to be passed at
> the same time, and does its own checking that it does not get a negative
> number); but I'm sure there's plenty of other candidates (add-cpu is
> another possibility that comes quickly to mind) - basically, pick a
> command that takes an explicit 'int' argument, and overflow that
> argument to see what happens when the command now has to deal with a
> qdouble.
Command params will end up getting marshalled in QObject prior to being
passed into commands:
mi = qmp_input_visitor_new_strict(QOBJECT(args));
v = qmp_input_get_visitor(mi);
visit_start_optional(v, &has_fdset_id, "fdset-id", errp);
if (has_fdset_id) {
visit_type_int(v, &fdset_id, "fdset-id", errp);
}
visit_end_optional(v, errp);
visit_start_optional(v, &has_opaque, "opaque", errp);
if (has_opaque) {
visit_type_str(v, &opaque, "opaque", errp);
}
visit_end_optional(v, errp);
qmp_input_visitor_cleanup(mi);
if (error_is_set(errp)) {
goto out;
}
retval = qmp_add_fd(has_fdset_id, fdset_id, has_opaque, opaque, errp);
so i think a check in tests-qmp-input-visitor that verifies that values that
exceed LLONG_MAX/LLONG_MIN will get added into the QObject as QFloats
and trigger a type error when being passed to visit_type_int() should
cover the cases in question.
>
> --
> Eric Blake eblake redhat com+1-919-301-3266
> Libvirt virtualization library http://libvirt.org
>
Re: [Qemu-devel] [PATCH 01/10] qapi: qapi-types.py, native list support
On Fri, May 10, 2013 at 10:07:45AM -0400, Luiz Capitulino wrote:
> On Thu, 9 May 2013 21:20:53 -0500
> Michael Roth wrote:
>
> > Teach type generators about native types so they can generate the
> > appropriate linked list types.
> >
> > Signed-off-by: Michael Roth
> > ---
> > scripts/qapi-types.py | 43 ---
> > scripts/qapi.py | 21 +
> > 2 files changed, 61 insertions(+), 3 deletions(-)
> >
> > diff --git a/scripts/qapi-types.py b/scripts/qapi-types.py
> > index 9e19920..96cb26d 100644
> > --- a/scripts/qapi-types.py
> > +++ b/scripts/qapi-types.py
> > @@ -16,7 +16,18 @@ import os
> > import getopt
> > import errno
> >
> > -def generate_fwd_struct(name, members):
> > +def generate_fwd_struct(name, members, builtin_type=False):
> > +if builtin_type:
> > +return mcgen('''
> > +typedef struct %(name)sList
> > +{
> > +%(type)s value;
> > +struct %(name)sList *next;
> > +} %(name)sList;
> > +''',
>
> Sorry for the utterly minor comment, but as you're going to respin please
> add a newline before ''' so that we get the declarations properly separated
> when generated.
>
> > + type=c_type(name),
> > + name=name)
> > +
> > return mcgen('''
> > typedef struct %(name)s %(name)s;
> >
> > @@ -164,6 +175,7 @@ void qapi_free_%(type)s(%(c_type)s obj);
> >
> > def generate_type_cleanup(name):
> > ret = mcgen('''
> > +
> > void qapi_free_%(type)s(%(c_type)s obj)
> > {
> > QapiDeallocVisitor *md;
> > @@ -184,8 +196,9 @@ void qapi_free_%(type)s(%(c_type)s obj)
> >
> >
> > try:
> > -opts, args = getopt.gnu_getopt(sys.argv[1:], "chp:o:",
> > - ["source", "header", "prefix=",
> > "output-dir="])
> > +opts, args = getopt.gnu_getopt(sys.argv[1:], "chbp:o:",
> > + ["source", "header", "builtins",
> > +"prefix=", "output-dir="])
> > except getopt.GetoptError, err:
> > print str(err)
> > sys.exit(1)
> > @@ -197,6 +210,7 @@ h_file = 'qapi-types.h'
> >
> > do_c = False
> > do_h = False
> > +do_builtins = False
> >
> > for o, a in opts:
> > if o in ("-p", "--prefix"):
> > @@ -207,6 +221,8 @@ for o, a in opts:
> > do_c = True
> > elif o in ("-h", "--header"):
> > do_h = True
> > +elif o in ("-b", "--builtins"):
> > +do_builtins = True
> >
> > if not do_c and not do_h:
> > do_c = True
> > @@ -282,6 +298,11 @@ fdecl.write(mcgen('''
> > exprs = parse_schema(sys.stdin)
> > exprs = filter(lambda expr: not expr.has_key('gen'), exprs)
> >
> > +fdecl.write(guardstart("QAPI_TYPES_BUILTIN_STRUCT_DECL"))
> > +for typename in builtin_types:
> > +fdecl.write(generate_fwd_struct(typename, None, builtin_type=True))
> > +fdecl.write(guardend("QAPI_TYPES_BUILTIN_STRUCT_DECL"))
> > +
> > for expr in exprs:
> > ret = "\n"
> > if expr.has_key('type'):
> > @@ -298,6 +319,22 @@ for expr in exprs:
> > continue
> > fdecl.write(ret)
> >
> > +# to avoid header dependency hell, we always generate declarations
> > +# for built-in types in our header files and simply guard them
> > +fdecl.write(guardstart("QAPI_TYPES_BUILTIN_CLEANUP_DECL"))
> > +for typename in builtin_types:
> > +fdecl.write(generate_type_cleanup_decl(typename + "List"))
> > +fdecl.write(guardend("QAPI_TYPES_BUILTIN_CLEANUP_DECL"))
>
> I'm not sure I got why you're doing this. Is it because you're going to
> generate them in more .h files? This is a bit ugly :(
>
The issue is that things like the types generated from
qapi-schema-test.json or qga/qapi-schema.json may end up referencing
intList/strList/visit_type_intList/etc, which we'll also have declared for
use in the main qapi-schema.json. qapi-schema.json of course can't
depend on tests or qga, so we generate the definitions for the builtin types
when running the code generators on qapi-schema.json.
For everyone else, tests/qga/etc, to be able to use/link against those
definitions we need to include the declarations by either #include'ing
qapi-types.h/qapi-visit.h etc, or by always declaring them and simply
adding a guard.
In this case I've taken the latter approach since hard-coding a
reference in the code generators to header files created by separate calls
to the code generators seemed less modular. qapi-schema-test.json should
be capable of generating self-contained code if need be (and the only
reason we don't make it self-contained is that test-cases rely on
libqemuutil to build, which actually does have a dependency on
qapi-schema.json due to qemu-sockets.
> > +
> > +# ...this doesn't work for cases where we link in multiple objects that
> > +# have the functions defined, so we use -b option to provide control
> > +# over these cases
> > +if do_builtins:
> > +fdef.write(guardstart("QAPI_TYPES_BUILTIN_CLEANUP_DEF"))
> > +for typename in buil
Re: [Qemu-devel] [PATCH 06/10] json-parser: fix handling of large whole number values
On Fri, May 10, 2013 at 11:17:17AM -0400, Luiz Capitulino wrote:
> On Thu, 9 May 2013 21:20:58 -0500
> Michael Roth wrote:
>
> > Currently our JSON parser assumes that numbers lacking a mantissa are
> > integers and attempts to store them as QInt/int64 values. This breaks in
> > the case where the number overflows/underflows int64 values (which is
> > still valid JSON)
>
> Anthony wanted to fix this by moving to another wire format :)
>
> But, how this patch related to this series?
In v1 Laszlo pointed out that the QFloat tests I added in
test-visitor-serialization were actually non-functional, and those tests were
based on pre-existing code. So I added a patch to fix the pre-existing
code as a pre-cursor to the new unit tests based on it. But fixing that code
exposed the json-parser bug, so I added that fix as a precursor to the
precursor :)
Same with mem leak fixes, etc, to ensure the tests were functional and
leak-free within this series.
>
> >
> > Fix this by detecting such cases and using a QFloat to store the value
> > instead.
> >
> > Signed-off-by: Michael Roth
> > ---
> > qobject/json-parser.c | 26 +++---
> > 1 file changed, 23 insertions(+), 3 deletions(-)
> >
> > diff --git a/qobject/json-parser.c b/qobject/json-parser.c
> > index 05279c1..4d14e71 100644
> > --- a/qobject/json-parser.c
> > +++ b/qobject/json-parser.c
> > @@ -640,9 +640,29 @@ static QObject *parse_literal(JSONParserContext *ctxt)
> > case JSON_STRING:
> > obj = QOBJECT(qstring_from_escaped_str(ctxt, token));
> > break;
> > -case JSON_INTEGER:
> > -obj = QOBJECT(qint_from_int(strtoll(token_get_value(token), NULL,
> > 10)));
> > -break;
> > +case JSON_INTEGER: {
> > +/* A possibility exists that this is a whole-valued float where the
> > + * mantissa was left out due to being 0 (.0). It's not a big deal
> > to
> > + * treat these as ints in the parser, so long as users of the
> > + * resulting QObject know to expect a QInt in place of a QFloat in
> > + * cases like these.
> > + *
> > + * However, in some cases these values will overflow/underflow a
> > + * QInt/int64 container, thus we should assume these are to be
> > handled
> > + * as QFloats/doubles rather than silently changing their values.
> > + *
> > + * strtoll() indicates these instances by setting errno to ERANGE
> > + */
> > +int64_t value;
> > +
> > +errno = 0; /* strtoll doesn't set errno on success */
> > +value = strtoll(token_get_value(token), NULL, 10);
> > +if (errno != ERANGE) {
> > +obj = QOBJECT(qint_from_int(value));
> > +break;
> > +}
> > +/* fall through to JSON_FLOAT */
> > +}
> > case JSON_FLOAT:
> > /* FIXME dependent on locale */
> > obj = QOBJECT(qfloat_from_double(strtod(token_get_value(token),
> > NULL)));
>
Re: [Qemu-devel] [PATCH 04/10] qapi: enable generation of native list code
On Fri, May 10, 2013 at 10:10:03AM -0400, Luiz Capitulino wrote: > On Thu, 9 May 2013 21:20:56 -0500 > Michael Roth wrote: > > > Also, fix a dependency issue with libqemuutil: qemu-sockets.c needs > > qapi-types.c/qapi-visit.c > > > > Signed-off-by: Michael Roth > > --- > > Makefile |6 +++--- > > 1 file changed, 3 insertions(+), 3 deletions(-) > > > > diff --git a/Makefile b/Makefile > > index 7dc0204..9695c9d 100644 > > --- a/Makefile > > +++ b/Makefile > > @@ -178,7 +178,7 @@ Makefile: $(version-obj-y) $(version-lobj-y) > > # Build libraries > > > > libqemustub.a: $(stub-obj-y) > > -libqemuutil.a: $(util-obj-y) > > +libqemuutil.a: $(util-obj-y) qapi-types.o qapi-visit.o > > Don't we want this in for 1.5? > I don't think it's causing any issues currently since it's not causing undefined reference errors upstream. users of libqemuutil that make use of qemu-sockets seem to be pulling qapi-types/qapi-visit in through other dependencies. I only noticed it because I was attempting to generate the native list code via tests/Makefile and running into redefinition conflicts with qapi-types.o/qapi-visit.o, then noticed the qemu-sockets.o issue when I attempted to remove the qapi-types/qapi-visit dependency from tests/test-visitor-serialization Now that we're generating the native list code from top-level Makefile, it actually doesn't seem to be needed by this series anymore, so maybe I'll pull it out for now. I think a better fix would be to have qapi/Makefile.obj add these to $util-obj-y directly anyway. > > > > ## > > > > @@ -215,10 +215,10 @@ $(SRC_PATH)/qga/qapi-schema.json > > $(SRC_PATH)/scripts/qapi-commands.py $(qapi-py) > > > > qapi-types.c qapi-types.h :\ > > $(SRC_PATH)/qapi-schema.json $(SRC_PATH)/scripts/qapi-types.py $(qapi-py) > > - $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-types.py > > $(gen-out-type) -o "." < $<, " GEN $@") > > + $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-types.py > > $(gen-out-type) -o "." -b < $<, " GEN $@") > > qapi-visit.c qapi-visit.h :\ > > $(SRC_PATH)/qapi-schema.json $(SRC_PATH)/scripts/qapi-visit.py $(qapi-py) > > - $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-visit.py > > $(gen-out-type) -o "." < $<, " GEN $@") > > + $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-visit.py > > $(gen-out-type) -o "." -b < $<, " GEN $@") > > qmp-commands.h qmp-marshal.c :\ > > $(SRC_PATH)/qapi-schema.json $(SRC_PATH)/scripts/qapi-commands.py > > $(qapi-py) > > $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-commands.py > > $(gen-out-type) -m -o "." < $<, " GEN $@") >
Re: [Qemu-devel] [PATCH v2 0/2] qga umask fix addenda
On Wed, May 08, 2013 at 05:31:34PM +0200, Laszlo Ersek wrote: > I should have paid more attention to portability and error path cleanup > in the CVE-2013-2007 fix. > > (We continue to assume, like the rest of qemu code, that > qemu_set_cloexec() never fails internally. This should be a reasonable > assumption when the input fd is valid.) > > Laszlo Ersek (2): > qga: distinguish binary modes in "guest_file_open_modes" map > qga: unlink just created guest-file if fchmod() or fdopen() fails on > it Thanks, applied to qga branch: https://github.com/mdroth/qemu/commits/qga > > qga/commands-posix.c | 25 +++-- > 1 files changed, 19 insertions(+), 6 deletions(-) >
