Re: [Qemu-devel] [RFC PATCH 07/10] spapr_pci: Allow PCI host bridge DMA window to be configured

[Laurent Vivier]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1



On 24/09/2015 01:54, David Gibson wrote:
> On Wed, Sep 23, 2015 at 08:55:01PM +0200, Laurent Vivier wrote:
>> 
>> 
>> On 17/09/2015 15:09, David Gibson wrote:
>>> At present the PCI host bridge (PHB) for the pseries machine
>>> type has a fixed DMA window from 0..1GB (in PCI address space)
>>> which is mapped to real memory via the PAPR paravirtualized
>>> IOMMU.
>>> 
>>> For better support of VFIO devices, we're going to want to
>>> allow for different configurations of the DMA window.
>>> 
>>> Eventually we'll want to allow the guest itself to reconfigure
>>> the window via the PAPR dynamic DMA window interface, but as a
>>> preliminary this patch allows the user to reconfigure the
>>> window with new properties on the PHB device.
>>> 
>>> Signed-off-by: David Gibson  --- 
>>> hw/ppc/spapr_pci.c  | 7 +-- 
>>> include/hw/pci-host/spapr.h | 3 +-- 2 files changed, 6
>>> insertions(+), 4 deletions(-)
>>> 
>>> diff --git a/hw/ppc/spapr_pci.c b/hw/ppc/spapr_pci.c index
>>> b088491..622c4ac 100644 --- a/hw/ppc/spapr_pci.c +++
>>> b/hw/ppc/spapr_pci.c @@ -1394,7 +1394,7 @@ static void
>>> spapr_phb_finish_realize(sPAPRPHBState *sphb, Error **errp) 
>>> sPAPRTCETable *tcet; uint32_t nb_table;
>>> 
>>> -nb_table = SPAPR_PCI_DMA32_SIZE >> SPAPR_TCE_PAGE_SHIFT; +
>>> nb_table = sphb->dma_win_size >> SPAPR_TCE_PAGE_SHIFT; tcet =
>>> spapr_tce_new_table(DEVICE(sphb), sphb->dma_liobn, 0,
>>> SPAPR_TCE_PAGE_SHIFT, nb_table, false); if (!tcet) { @@ -1404,7
>>> +1404,7 @@ static void spapr_phb_finish_realize(sPAPRPHBState
>>> *sphb, Error **errp) }
>>> 
>>> /* Register default 32bit DMA window */ -
>>> memory_region_add_subregion(&sphb->iommu_root, 0, +
>>> memory_region_add_subregion(&sphb->iommu_root,
>>> sphb->dma_win_addr, spapr_tce_get_iommu(tcet)); }
>>> 
>>> @@ -1437,6 +1437,9 @@ static Property spapr_phb_properties[] =
>>> { SPAPR_PCI_IO_WIN_SIZE), 
>>> DEFINE_PROP_BOOL("dynamic-reconfiguration", sPAPRPHBState,
>>> dr_enabled, true), +/* Default DMA window is 0..1GB */ +
>>> DEFINE_PROP_UINT64("dma_win_addr", sPAPRPHBState, dma_win_addr,
>>> 0), +DEFINE_PROP_UINT64("dma_win_size", sPAPRPHBState,
>>> dma_win_size, 0x4000), DEFINE_PROP_END_OF_LIST(), };
>> 
>> Add them too to the vmstate_spapr_pci ?
> 
> No.  At least, not just now.
> 
> This is the sort of configuration information that typically isn't 
> migrated, instead requiring the far end to be set up matching.
> Which

I think this is the aim of VMSTATE_UINT64_EQUAL() ?

> is a problem, but not one within the scope of this patch series.
> In any case just transferring the values wouldn't be enough - we'd
> need post_load handlers to actually rewire the TCE table
> accordingly.
> 
> We will need to do this once we add dynamic DMA window support,
> but again, not in scope just now.
> 
-BEGIN PGP SIGNATURE-
Version: GnuPG v2

iEYEARECAAYFAlYDn1UACgkQNKT2yavzbFM20ACfcUVIPLObcYh4y5U8CcoLVGoO
FR8AoOgigTMFu7FOh7wI8U+fGNtYv6Ji
=4opa
-END PGP SIGNATURE-

Re: [Qemu-devel] [PATCH] audio: Use g_new() & friends where that makes obvious sense

[Markus Armbruster]

Ping?  (also ui, usb, qxl)

Markus Armbruster  writes:

> g_new(T, n) is neater than g_malloc(sizeof(T) * n).  It's also safer,
> for two reasons.  One, it catches multiplication overflowing size_t.
> Two, it returns T * rather than void *, which lets the compiler catch
> more type errors.
>
> This commit only touches allocations with size arguments of the form
> sizeof(T).  Same Coccinelle semantic patch as in commit b45c03f.
>
> Signed-off-by: Markus Armbruster 
> ---
>  audio/alsaaudio.c| 2 +-
>  audio/coreaudio.c| 2 +-
>  audio/dsoundaudio.c  | 2 +-
>  audio/ossaudio.c | 2 +-
>  audio/paaudio.c  | 2 +-
>  audio/wavaudio.c | 2 +-
>  hw/audio/intel-hda.c | 2 +-
>  7 files changed, 7 insertions(+), 7 deletions(-)
>
> diff --git a/audio/alsaaudio.c b/audio/alsaaudio.c
> index 6315b2d..76813f4 100644
> --- a/audio/alsaaudio.c
> +++ b/audio/alsaaudio.c
> @@ -1125,7 +1125,7 @@ static ALSAConf glob_conf = {
>  
>  static void *alsa_audio_init (void)
>  {
> -ALSAConf *conf = g_malloc(sizeof(ALSAConf));
> +ALSAConf *conf = g_new(ALSAConf, 1);
>  *conf = glob_conf;
>  return conf;
>  }
> diff --git a/audio/coreaudio.c b/audio/coreaudio.c
> index 6dfd63e..6390ba4 100644
> --- a/audio/coreaudio.c
> +++ b/audio/coreaudio.c
> @@ -504,7 +504,7 @@ static CoreaudioConf glob_conf = {
>  
>  static void *coreaudio_audio_init (void)
>  {
> -CoreaudioConf *conf = g_malloc(sizeof(CoreaudioConf));
> +CoreaudioConf *conf = g_new(CoreaudioConf, 1);
>  *conf = glob_conf;
>  
>  atexit(coreaudio_atexit);
> diff --git a/audio/dsoundaudio.c b/audio/dsoundaudio.c
> index e9472c1..3949fa4 100644
> --- a/audio/dsoundaudio.c
> +++ b/audio/dsoundaudio.c
> @@ -786,7 +786,7 @@ static void *dsound_audio_init (void)
>  {
>  int err;
>  HRESULT hr;
> -dsound *s = g_malloc0(sizeof(dsound));
> +dsound *s = g_new0(dsound, 1);
>  
>  s->conf = glob_conf;
>  hr = CoInitialize (NULL);
> diff --git a/audio/ossaudio.c b/audio/ossaudio.c
> index 7dbe333..2614c62 100644
> --- a/audio/ossaudio.c
> +++ b/audio/ossaudio.c
> @@ -848,7 +848,7 @@ static OSSConf glob_conf = {
>  
>  static void *oss_audio_init (void)
>  {
> -OSSConf *conf = g_malloc(sizeof(OSSConf));
> +OSSConf *conf = g_new(OSSConf, 1);
>  *conf = glob_conf;
>  
>  if (access(conf->devpath_in, R_OK | W_OK) < 0 ||
> diff --git a/audio/paaudio.c b/audio/paaudio.c
> index fea6071..9c498cf 100644
> --- a/audio/paaudio.c
> +++ b/audio/paaudio.c
> @@ -814,7 +814,7 @@ static PAConf glob_conf = {
>  
>  static void *qpa_audio_init (void)
>  {
> -paaudio *g = g_malloc(sizeof(paaudio));
> +paaudio *g = g_new(paaudio, 1);
>  g->conf = glob_conf;
>  g->mainloop = NULL;
>  g->context = NULL;
> diff --git a/audio/wavaudio.c b/audio/wavaudio.c
> index c586020..f706fae 100644
> --- a/audio/wavaudio.c
> +++ b/audio/wavaudio.c
> @@ -230,7 +230,7 @@ static WAVConf glob_conf = {
>  
>  static void *wav_audio_init (void)
>  {
> -WAVConf *conf = g_malloc(sizeof(WAVConf));
> +WAVConf *conf = g_new(WAVConf, 1);
>  *conf = glob_conf;
>  return conf;
>  }
> diff --git a/hw/audio/intel-hda.c b/hw/audio/intel-hda.c
> index 433463e..6074c2e 100644
> --- a/hw/audio/intel-hda.c
> +++ b/hw/audio/intel-hda.c
> @@ -467,7 +467,7 @@ static void intel_hda_parse_bdl(IntelHDAState *d, 
> IntelHDAStream *st)
>  addr = intel_hda_addr(st->bdlp_lbase, st->bdlp_ubase);
>  st->bentries = st->lvi +1;
>  g_free(st->bpl);
> -st->bpl = g_malloc(sizeof(bpl) * st->bentries);
> +st->bpl = g_new(bpl, st->bentries);
>  for (i = 0; i < st->bentries; i++, addr += 16) {
>  pci_dma_read(&d->pci, addr, buf, 16);
>  st->bpl[i].addr  = le64_to_cpu(*(uint64_t *)buf);

Re: [Qemu-devel] [RFC PATCH 00/10] pseries: Allow VFIO devices on spapr-pci-host-bridge

[Laurent Vivier]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1



On 24/09/2015 03:02, David Gibson wrote:
> On Wed, Sep 23, 2015 at 06:46:17PM +0200, Laurent Vivier wrote:
>> 
>> 
>> On 17/09/2015 15:09, David Gibson wrote:
>>> Currently the pseries machine type uses two types of PCI Host
>>> Bridge (PHB) devices: "spapr-pci-host-bridge" the 'normal'
>>> variant intended for emulated PCI devices, and
>>> "spapr-pci-vfio-host-bridge" intended for VFIO devices.
>>> 
>>> When using VFIO with pseries, a separate
>>> spapr-pci-vfio-host-bridge device is needed for every host
>>> IOMMU group from which you're using VFIO devices.  This is
>>> quite awkward for the user and/or management tools.  It's
>>> especially awkward since the current code makes essentially no
>>> attempt to detect and warn the user if the wrong sorts of
>>> devices are connected to the wrong PHB.
>>> 
>>> It turns out that the VFIO core code is actually general enough
>>> that VFIO devices almost work on the normal
>>> spapr-pci-host-bridge device. In fact with the right
>>> combination of circumstances they *can* work right now.
>>> 
>>> spapr-pci-vfio-host-bridge does 3 additional things:
>>> 
>>> 1) It disables KVM acceleration of the guest IOMMU.  That 
>>> acceleration breaks VFIO because it means guest IOMMU updates 
>>> bypass the VFIO infrastructure which keeps the host IOMMU in 
>>> sync.
>>> 
>>> 2) It automatically configures the guest PHB's DMA window to
>>> match the capabilities of the host IOMMU, and advertises that
>>> to the guest.
>>> 
>>> 3) It provides additional handling of EEH (Enhanced Error 
>>> Handling) functions.
>>> 
>>> This patch series: * Allows VFIO devices to be used on the
>>> spapr-pci-host-bridge by auto-switching the KVM TCE
>>> acceleration
>>> 
>>> * Adds verification that the host IOMMU can handle the DMA
>>> windows used by guest PHBs
>>> 
>>> * Allows the DMA window on the guest PHB to be configured with 
>>> device properties.  This can be used to make sure it matches a 
>>> host window, but in practice the default setting will already 
>>> work with the host IOMMU on all current systems.
>>> 
>>> * Adds support to the VFIO core to allow a VFIO device to be 
>>> hotplugged onto a bus which doesn't yet have VFIO devices.
>>> This already worked for systems without a guest-visible IOMMU 
>>> (i.e. x86), this series makes it work even with a guest
>>> visible IOMMU.
>>> 
>>> * Makes a few related cleanups along the way
>>> 
>>> This series does NOT allow EEH operations on VFIO devices on
>>> the spapr-pci-host-bridge device, so the
>>> spapr-pci-vfio-host-bridge device is left in for now.  It turns
>>> out there are some serious existing problems in both the qemu
>>> EEH implementation and (worse) in the EEH/VFIO kernel
>>> interface.  Fixing those is a problem for another day. Maybe
>>> tomorrow.
>>> 
>>> 
>>> I've tested basic assignment of an xHCI to a pseries guest,
>>> both at startup and with hotplug.  I haven't (yet) tested VFIO
>>> on x86 with this series.
>>> 
>>> This series probably needs to be merged via several different
>>> trees. I'm intending to split up as necessary once it's had
>>> some review.
>>> 
>>> David Gibson (10): vfio: Remove unneeded union from
>>> VFIOContainer vfio: Generalize vfio_listener_region_add failure
>>> path vfio: Check guest IOVA ranges against host IOMMU
>>> capabilities vfio: Record host IOMMU's available IO page sizes 
>>> memory: Allow replay of IOMMU mapping notifications vfio: Allow
>>> hotplug of containers onto existing guest IOMMU mappings 
>>> spapr_pci: Allow PCI host bridge DMA window to be configured 
>>> spapr_iommu: Rename vfio_accel parameter spapr_iommu: Provide a
>>> function to switch a TCE table to allowing VFIO spapr_pci:
>>> Allow VFIO devices to work on the normal PCI host bridge
>>> 
>>> hw/ppc/spapr_iommu.c  |  25 ++- hw/ppc/spapr_pci.c
>>> |  13 +++- hw/vfio/common.c  | 152
>>> +++--- 
>>> include/exec/memory.h |  16 + 
>>> include/hw/pci-host/spapr.h   |   3 +- include/hw/ppc/spapr.h
>>> |   6 +- include/hw/vfio/vfio-common.h |  21 +++--- memory.c
>>> |  18 + target-ppc/kvm.c  |   4 +- 
>>> target-ppc/kvm_ppc.h  |   2 +- 10 files changed, 184
>>> insertions(+), 76 deletions(-)
>>> 
>> 
>> Just a comment on the form: checkpatch.pl vociferates because of
>> "DOS line endings", "trailing whitespace" and "line over 80
>> characters".
> 
> Hmm.. I see the trailing whitespace and over 80 characters
> warnings along with a few others.  Nothing about DOS line endings
> though. Wondering if that could be something that's come from your
> mailer.

Possible, but very strange: it's the first time it happens (thunderbird)
.

>> 
>>  Please, use "vi" with "set ff=unix" 
>> 
>> Stefan has a good tip to check automatically commits:
>> 
>> http://blog.vmsplice.net/2011/03/how-to-automatically-run-checkpatchp
l.html
>
>> 
> Nice, thanks.
> 
-BEGIN PGP SIGNATURE-
Ve

Re: [Qemu-devel] [PATCH] qga: Use g_new() & friends where that makes obvious sense

[Markus Armbruster]

Ping?

Markus Armbruster  writes:

> g_new(T, n) is neater than g_malloc(sizeof(T) * n).  It's also safer,
> for two reasons.  One, it catches multiplication overflowing size_t.
> Two, it returns T * rather than void *, which lets the compiler catch
> more type errors.
>
> This commit only touches allocations with size arguments of the form
> sizeof(T).  Same Coccinelle semantic patch as in commit b45c03f.
>
> Signed-off-by: Markus Armbruster 
> ---
>  qga/channel-posix.c |  2 +-
>  qga/channel-win32.c |  2 +-
>  qga/commands-posix.c| 10 +-
>  qga/commands-win32.c| 10 +-
>  qga/commands.c  |  6 +++---
>  qga/guest-agent-command-state.c |  4 ++--
>  6 files changed, 17 insertions(+), 17 deletions(-)
>
> diff --git a/qga/channel-posix.c b/qga/channel-posix.c
> index 8aad4fe..61aa3cb 100644
> --- a/qga/channel-posix.c
> +++ b/qga/channel-posix.c
> @@ -249,7 +249,7 @@ GIOStatus ga_channel_read(GAChannel *c, gchar *buf, gsize 
> size, gsize *count)
>  GAChannel *ga_channel_new(GAChannelMethod method, const gchar *path,
>GAChannelCallback cb, gpointer opaque)
>  {
> -GAChannel *c = g_malloc0(sizeof(GAChannel));
> +GAChannel *c = g_new0(GAChannel, 1);
>  c->event_cb = cb;
>  c->user_data = opaque;
>  
> diff --git a/qga/channel-win32.c b/qga/channel-win32.c
> index 04fa5e4..215b15a 100644
> --- a/qga/channel-win32.c
> +++ b/qga/channel-win32.c
> @@ -322,7 +322,7 @@ static gboolean ga_channel_open(GAChannel *c, 
> GAChannelMethod method,
>  GAChannel *ga_channel_new(GAChannelMethod method, const gchar *path,
>GAChannelCallback cb, gpointer opaque)
>  {
> -GAChannel *c = g_malloc0(sizeof(GAChannel));
> +GAChannel *c = g_new0(GAChannel, 1);
>  SECURITY_ATTRIBUTES sec_attrs;
>  
>  if (!ga_channel_open(c, method, path)) {
> diff --git a/qga/commands-posix.c b/qga/commands-posix.c
> index b03c316..a932809 100644
> --- a/qga/commands-posix.c
> +++ b/qga/commands-posix.c
> @@ -235,7 +235,7 @@ static int64_t guest_file_handle_add(FILE *fh, Error 
> **errp)
>  return -1;
>  }
>  
> -gfh = g_malloc0(sizeof(GuestFileHandle));
> +gfh = g_new0(GuestFileHandle, 1);
>  gfh->id = handle;
>  gfh->fh = fh;
>  QTAILQ_INSERT_TAIL(&guest_file_state.filehandles, gfh, next);
> @@ -488,7 +488,7 @@ struct GuestFileRead *qmp_guest_file_read(int64_t handle, 
> bool has_count,
>  slog("guest-file-read failed, handle: %" PRId64, handle);
>  } else {
>  buf[read_count] = 0;
> -read_data = g_malloc0(sizeof(GuestFileRead));
> +read_data = g_new0(GuestFileRead, 1);
>  read_data->count = read_count;
>  read_data->eof = feof(fh);
>  if (read_count) {
> @@ -533,7 +533,7 @@ GuestFileWrite *qmp_guest_file_write(int64_t handle, 
> const char *buf_b64,
>  error_setg_errno(errp, errno, "failed to write to file");
>  slog("guest-file-write failed, handle: %" PRId64, handle);
>  } else {
> -write_data = g_malloc0(sizeof(GuestFileWrite));
> +write_data = g_new0(GuestFileWrite, 1);
>  write_data->count = write_count;
>  write_data->eof = feof(fh);
>  }
> @@ -678,7 +678,7 @@ static void build_fs_mount_list_from_mtab(FsMountList 
> *mounts, Error **errp)
>  continue;
>  }
>  
> -mount = g_malloc0(sizeof(FsMount));
> +mount = g_new0(FsMount, 1);
>  mount->dirname = g_strdup(ment->mnt_dir);
>  mount->devtype = g_strdup(ment->mnt_type);
>  mount->devmajor = devmajor;
> @@ -757,7 +757,7 @@ static void build_fs_mount_list(FsMountList *mounts, 
> Error **errp)
>  }
>  }
>  
> -mount = g_malloc0(sizeof(FsMount));
> +mount = g_new0(FsMount, 1);
>  mount->dirname = g_strdup(line + dir_s);
>  mount->devtype = g_strdup(dash + type_s);
>  mount->devmajor = devmajor;
> diff --git a/qga/commands-win32.c b/qga/commands-win32.c
> index 41bdd3f..0481646 100644
> --- a/qga/commands-win32.c
> +++ b/qga/commands-win32.c
> @@ -106,7 +106,7 @@ static int64_t guest_file_handle_add(HANDLE fh, Error 
> **errp)
>  if (handle < 0) {
>  return -1;
>  }
> -gfh = g_malloc0(sizeof(GuestFileHandle));
> +gfh = g_new0(GuestFileHandle, 1);
>  gfh->id = handle;
>  gfh->fh = fh;
>  QTAILQ_INSERT_TAIL(&guest_file_state.filehandles, gfh, next);
> @@ -298,7 +298,7 @@ GuestFileRead *qmp_guest_file_read(int64_t handle, bool 
> has_count,
>  slog("guest-file-read failed, handle %" PRId64, handle);
>  } else {
>  buf[read_count] = 0;
> -read_data = g_malloc0(sizeof(GuestFileRead));
> +read_data = g_new0(GuestFileRead, 1);
>  read_data->count = (size_t)read_count;
>  read_data->eof = read_count == 0;
>  
> @@ -342,7 +342,7 @@ GuestFileWrite *qmp_guest_file_write(int64_t handle, 
> const ch

Re: [Qemu-devel] [PATCH] linux-user: Use g_new() & friends where that makes obvious sense

[Markus Armbruster]

Ping?

Markus Armbruster  writes:

> g_new(T, n) is neater than g_malloc(sizeof(T) * n).  It's also safer,
> for two reasons.  One, it catches multiplication overflowing size_t.
> Two, it returns T * rather than void *, which lets the compiler catch
> more type errors.
>
> This commit only touches allocations with size arguments of the form
> sizeof(T).  Same Coccinelle semantic patchas in commit b45c03f.
>
> Signed-off-by: Markus Armbruster 
> ---
>  linux-user/elfload.c | 2 +-
>  linux-user/main.c| 2 +-
>  linux-user/syscall.c | 2 +-
>  3 files changed, 3 insertions(+), 3 deletions(-)
>
> diff --git a/linux-user/elfload.c b/linux-user/elfload.c
> index eca0c7f..6584f0b 100644
> --- a/linux-user/elfload.c
> +++ b/linux-user/elfload.c
> @@ -2848,7 +2848,7 @@ static int fill_note_info(struct elf_note_info *info,
>  TaskState *ts = (TaskState *)cpu->opaque;
>  int i;
>  
> -info->notes = g_malloc0(NUMNOTES * sizeof (struct memelfnote));
> +info->notes = g_new0(struct memelfnote, NUMNOTES);
>  if (info->notes == NULL)
>  return (-ENOMEM);
>  info->prstatus = g_malloc0(sizeof (*info->prstatus));
> diff --git a/linux-user/main.c b/linux-user/main.c
> index 829ee64..1ff821d 100644
> --- a/linux-user/main.c
> +++ b/linux-user/main.c
> @@ -4014,7 +4014,7 @@ int main(int argc, char **argv, char **envp)
>  }
>  target_argv[target_argc] = NULL;
>  
> -ts = g_malloc0 (sizeof(TaskState));
> +ts = g_new0(TaskState, 1);
>  init_task_state(ts);
>  /* build Task State */
>  ts->info = info;
> diff --git a/linux-user/syscall.c b/linux-user/syscall.c
> index c0fabf7..60c62b8 100644
> --- a/linux-user/syscall.c
> +++ b/linux-user/syscall.c
> @@ -4560,7 +4560,7 @@ static int do_fork(CPUArchState *env, unsigned int 
> flags, abi_ulong newsp,
>  new_thread_info info;
>  pthread_attr_t attr;
>  
> -ts = g_malloc0(sizeof(TaskState));
> +ts = g_new0(TaskState, 1);
>  init_task_state(ts);
>  /* we create a new CPU instance. */
>  new_env = cpu_copy(env);

Re: [Qemu-devel] [RFC PATCH 05/10] memory: Allow replay of IOMMU mapping notifications

[Laurent Vivier]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1



On 24/09/2015 01:50, David Gibson wrote:
> On Wed, Sep 23, 2015 at 07:04:55PM +0200, Laurent Vivier wrote:
>> 
>> 
>> On 17/09/2015 15:09, David Gibson wrote:
>>> When we have guest visible IOMMUs, we allow notifiers to be
>>> registered which will be informed of all changes to IOMMU
>>> mappings.  This is used by vfio to keep the host IOMMU mappings
>>> in sync with guest IOMMU mappings.
>>> 
>>> However, unlike with a memory region listener, an iommu
>>> notifier won't be told about any mappings which already exist
>>> in the (guest) IOMMU at the time it is registered.  This can
>>> cause problems if hotplugging a VFIO device onto a guest bus
>>> which had existing guest IOMMU mappings, but didn't previously
>>> have an VFIO devices (and hence no host IOMMU mappings).
>>> 
>>> This adds a memory_region_register_iommu_notifier_replay()
>>> function to handle this case.  As well as registering the new
>>> notifier it replays existing mappings.  Because the IOMMU
>>> memory region doesn't internally remember the granularity of
>>> the guest IOMMU it has a small hack where the caller must
>>> specify a granularity at which to replay mappings.
>>> 
>>> If there are finer mappings in the guest IOMMU these will be
>>> reported in the iotlb structures passed to the notifier which
>>> it must handle (probably causing it to flag an error).  This
>>> isn't new - the VFIO iommu notifier must already handle
>>> notifications about guest IOMMU mappings too short for it to
>>> represent in the host IOMMU.
>>> 
>>> Signed-off-by: David Gibson  --- 
>>> include/exec/memory.h | 16  memory.c
>>> | 18 ++ 2 files changed, 34 insertions(+)
>>> 
>>> diff --git a/include/exec/memory.h b/include/exec/memory.h 
>>> index 5baaf48..3cf145b 100644 --- a/include/exec/memory.h +++
>>> b/include/exec/memory.h @@ -583,6 +583,22 @@ void
>>> memory_region_notify_iommu(MemoryRegion *mr, void
>>> memory_region_register_iommu_notifier(MemoryRegion *mr,
>>> Notifier *n);
>>> 
>>> /** + * memory_region_register_iommu_notifier_replay: register
>>> a notifier + * for changes to IOMMU translation entries, and
>>> replay existing IOMMU + * translations to the new notifier. +
>>> * + * @mr: the memory region to observe + * @n: the notifier to
>>> be added; the notifier receives a pointer to an + *
>>> #IOMMUTLBEntry as the opaque value; the pointer ceases to be +
>>> * valid on exit from the notifier. + * @granularity:
>>> Minimum page granularity to replay notifications for + *
>>> @is_write: Whether to treat the replay as a translate "write" +
>>> * through the iommu + */ +void
>>> memory_region_register_iommu_notifier_replay(MemoryRegion *mr,
>>> Notifier *n, +
>>> hwaddr granularity, bool is_write); + +/** *
>>> memory_region_unregister_iommu_notifier: unregister a notifier
>>> for * changes to IOMMU translation entries. * diff --git
>>> a/memory.c b/memory.c index 0d8b2d9..6b5a2f1 100644 ---
>>> a/memory.c +++ b/memory.c @@ -1403,6 +1403,24 @@ void
>>> memory_region_register_iommu_notifier(MemoryRegion *mr,
>>> Notifier *n) notifier_list_add(&mr->iommu_notify, n); }
>>> 
>>> +void memory_region_register_iommu_notifier_replay(MemoryRegion
>>> *mr, Notifier *n, +
>>> hwaddr granularity, bool is_write) +{ +hwaddr addr; +
>>> IOMMUTLBEntry iotlb; + +
>>> memory_region_register_iommu_notifier(mr, n); + +for (addr
>>> = 0; + int128_lt(int128_make64(addr), mr->size);
>> 
>> "addr < memory_region_size(mr)" should be enough.
> 
> Ah, yes, much neater, thanks.

but rethinking about that, you can have an infinite loop (with int128
or with memory_region_size()) if mr->size >= UINT64_MAX:

as hwaddr is a 64bit and a multiple of granularity which is a power of
two. the last value of addr is UINT64 + 1 - granularity, so the next
is (uint64_t)(UINT64 + 1), which is 0, so addr is never >= mr->size.

> 
>>> + addr += granularity) { + +iotlb =
>>> mr->iommu_ops->translate(mr, addr, is_write); +if
>>> (iotlb.perm != IOMMU_NONE) +n->notify(n, &iotlb); +
>>> } +} + void memory_region_unregister_iommu_notifier(Notifier
>>> *n) { notifier_remove(n);
>>> 
>> 
> 
-BEGIN PGP SIGNATURE-
Version: GnuPG v2

iEYEARECAAYFAlYDob0ACgkQNKT2yavzbFOnFACcDk+2PHhX/WfCCkTdXKH4XhWi
UYcAoOpe+C+8tzX02VlGTsCAV9ZxiEwQ
=Cnfl
-END PGP SIGNATURE-

Re: [Qemu-devel] [PATCH] linux-user: Use g_new() & friends where that makes obvious sense

[Stefan Weil]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Am 24.09.2015 um 09:06 schrieb Markus Armbruster:
> Ping?
> 
> Markus Armbruster  writes:
> 
>> g_new(T, n) is neater than g_malloc(sizeof(T) * n).  It's also 
>> safer, for two reasons.  One, it catches multiplication 
>> overflowing size_t. Two, it returns T * rather than void *,
>> which lets the compiler catch more type errors.
>> 
>> This commit only touches allocations with size arguments of the 
>> form sizeof(T).  Same Coccinelle semantic patchas in commit 
>> b45c03f.
>> 
>> Signed-off-by: Markus Armbruster  --- 
>> linux-user/elfload.c | 2 +- linux-user/main.c| 2 +- 
>> linux-user/syscall.c | 2 +- 3 files changed, 3 insertions(+), 3 
>> deletions(-)

Reviewed-by: Stefan Weil 

This patch is trivial enough for qemu-trivial.

As Eric has written, the commit message needs
a small correction: s/patchas/patch as/

Regards
Stefan

-BEGIN PGP SIGNATURE-
Version: GnuPG v1

iQIcBAEBCAAGBQJWA6OuAAoJEOCMIdVndFCt6Q8P/R6kKihtOvr17lL3Av117n1z
YPv6Tfa9jy1fd3qaeIMhXgm63CNTWTGU4o0A9/k3thqiD9kYVpqeZkd7gt1gE6uU
QR/eReEMZNr+5AnkYnCmZ8wr5upd4RWUjCvbGSUpFgb1ui+VtwZ08XzCVehQfY39
2S6EWxR/I7ETxzue+AFZak9w4QqVF7IQmw7CPkeTCD/himdCpyeGxL9dYQnyfEdW
mQeXUR2eipaqwIt1b6JkwQCU78rY2D/OLVDipZK9M4BSmJQWS2BsLboc+VC59bN9
JkVrKgSbuciqBMXDudY71qv9XO/wyM8MwnDM5GY7V9N1PoRVcg2mvSCRKq9lP82K
V0Sb6n1j9/AgM3ngnLZqkiVYkeFrnccIkiKCJvmDnDP1ClC8Cy5V1wkPzk7TJgTK
4xvuI3kfWPIzbphWzZtcCstJKYaO5ydrEGGNGko37A6s65oE+gIhyrv/KSIhix+H
n7KMOILY4wuk7UYoIgK6cvKL6QsVyhw0lR+RIR8p33ofAj20HylQng38h6spr50F
XTl3J38ApbyxNhw+EJHXXwj14YJFrZTwN8C0X49UHkSD4wqR4lH+2kdNUYIYJJHR
Zx+HkJj/8hLQq14WjtgZlNQ7noAck/yh9Rvyu54v6yfx1RAhNkDbiysQKgqkWzNM
U5ro2khoY2BrI7Bdq29H
=/Z+4
-END PGP SIGNATURE-

[Qemu-devel] [PATCH 2/5] net/dump: Rework net-dump init functions

[Thomas Huth]

Move the creation of the dump client from net_dump_init() into
net_init_dump(), so we can later use the former function for
dump via netfilter, too. Also rename net_dump_init() to
net_dump_state_init() to make it easier distinguishable from
net_init_dump().

Signed-off-by: Thomas Huth 
---
 net/dump.c | 27 +--
 1 file changed, 13 insertions(+), 14 deletions(-)

diff --git a/net/dump.c b/net/dump.c
index f8afb83..27083d4 100644
--- a/net/dump.c
+++ b/net/dump.c
@@ -118,13 +118,10 @@ static NetClientInfo net_dump_info = {
 .cleanup = dump_cleanup,
 };
 
-static int net_dump_init(NetClientState *peer, const char *device,
- const char *name, const char *filename, int len,
- Error **errp)
+static int net_dump_state_init(DumpState *s, const char *filename,
+   int len, Error **errp)
 {
 struct pcap_file_hdr hdr;
-NetClientState *nc;
-DumpState *s;
 struct tm tm;
 int fd;
 
@@ -148,13 +145,6 @@ static int net_dump_init(NetClientState *peer, const char 
*device,
 return -1;
 }
 
-nc = qemu_new_net_client(&net_dump_info, peer, device, name);
-
-snprintf(nc->info_str, sizeof(nc->info_str),
- "dump to %s (len=%d)", filename, len);
-
-s = DO_UPCAST(DumpState, nc, nc);
-
 s->fd = fd;
 s->pcap_caplen = len;
 
@@ -167,10 +157,11 @@ static int net_dump_init(NetClientState *peer, const char 
*device,
 int net_init_dump(const NetClientOptions *opts, const char *name,
   NetClientState *peer, Error **errp)
 {
-int len;
+int len, rc;
 const char *file;
 char def_file[128];
 const NetdevDumpOptions *dump;
+NetClientState *nc;
 
 assert(opts->kind == NET_CLIENT_OPTIONS_KIND_DUMP);
 dump = opts->dump;
@@ -200,5 +191,13 @@ int net_init_dump(const NetClientOptions *opts, const char 
*name,
 len = 65536;
 }
 
-return net_dump_init(peer, "dump", name, file, len, errp);
+nc = qemu_new_net_client(&net_dump_info, peer, "dump", name);
+snprintf(nc->info_str, sizeof(nc->info_str),
+ "dump to %s (len=%d)", file, len);
+
+rc = net_dump_state_init(DO_UPCAST(DumpState, nc, nc), file, len, errp);
+if (rc) {
+qemu_del_net_client(nc);
+}
+return rc;
 }
-- 
1.8.3.1

[Qemu-devel] [PATCH 5/5] net/dump: Add documentation

[Thomas Huth]

Add a short description for the filter-dump command line options.

Signed-off-by: Thomas Huth 
---
 qemu-options.hx | 8 
 1 file changed, 8 insertions(+)

diff --git a/qemu-options.hx b/qemu-options.hx
index b09f97f..347ee28 100644
--- a/qemu-options.hx
+++ b/qemu-options.hx
@@ -2011,6 +2011,7 @@ qemu -m 512 -object 
memory-backend-file,id=mem,size=512M,mem-path=/hugetlbfs,sha
 Dump network traffic on VLAN @var{n} to file @var{file} 
(@file{qemu-vlan0.pcap} by default).
 At most @var{len} bytes (64k by default) per packet are stored. The file 
format is
 libpcap, so it can be analyzed with tools such as tcpdump or Wireshark.
+Note: For devices created with '-netdev', use '-object filter-dump,...' 
instead.
 
 @item -net none
 Indicate that no network devices should be configured. It is used to
@@ -3660,6 +3661,13 @@ chain @var{all|in|out} is an option that can be applied 
to any netfilter, defaul
 
 @option{out} means this filter will receive packets sent from the netdev
 
+@item -object 
filter-dump,id=@var{id},netdev=@var{dev},file=@var{filename}][,maxlen=@var{len}]
+
+Dump the network traffic on netdev @var{dev} to the file specified by
+@var{filename}. At most @var{len} bytes (64k by default) per packet are stored.
+The file format is libpcap, so it can be analyzed with tools such as tcpdump
+or Wireshark.
+
 @end table
 
 ETEXI
-- 
1.8.3.1

[Qemu-devel] [PATCH 1/5] net/dump: Add support for receive_iov function

[Thomas Huth]

Adding a proper receive_iov function to the net dump module.
This will make it easier to support the dump filter feature for
the -netdev option in later patches.

Signed-off-by: Thomas Huth 
---
 net/dump.c | 24 +---
 1 file changed, 21 insertions(+), 3 deletions(-)

diff --git a/net/dump.c b/net/dump.c
index 02c8064..f8afb83 100644
--- a/net/dump.c
+++ b/net/dump.c
@@ -25,6 +25,7 @@
 #include "clients.h"
 #include "qemu-common.h"
 #include "qemu/error-report.h"
+#include "qemu/iov.h"
 #include "qemu/log.h"
 #include "qemu/timer.h"
 #include "hub.h"
@@ -57,12 +58,15 @@ struct pcap_sf_pkthdr {
 uint32_t len;
 };
 
-static ssize_t dump_receive(NetClientState *nc, const uint8_t *buf, size_t 
size)
+static ssize_t dump_receive_iov(NetClientState *nc, const struct iovec *iov,
+int cnt)
 {
 DumpState *s = DO_UPCAST(DumpState, nc, nc);
 struct pcap_sf_pkthdr hdr;
 int64_t ts;
 int caplen;
+size_t size = iov_size(iov, cnt);
+struct iovec dumpiov[cnt + 1];
 
 /* Early return in case of previous error. */
 if (s->fd < 0) {
@@ -76,8 +80,12 @@ static ssize_t dump_receive(NetClientState *nc, const 
uint8_t *buf, size_t size)
 hdr.ts.tv_usec = ts % 100;
 hdr.caplen = caplen;
 hdr.len = size;
-if (write(s->fd, &hdr, sizeof(hdr)) != sizeof(hdr) ||
-write(s->fd, buf, caplen) != caplen) {
+
+dumpiov[0].iov_base = &hdr;
+dumpiov[0].iov_len = sizeof(hdr);
+cnt = iov_copy(&dumpiov[1], cnt, iov, cnt, 0, caplen);
+
+if (writev(s->fd, dumpiov, cnt + 1) != sizeof(hdr) + caplen) {
 qemu_log("-net dump write error - stop dump\n");
 close(s->fd);
 s->fd = -1;
@@ -86,6 +94,15 @@ static ssize_t dump_receive(NetClientState *nc, const 
uint8_t *buf, size_t size)
 return size;
 }
 
+static ssize_t dump_receive(NetClientState *nc, const uint8_t *buf, size_t 
size)
+{
+struct iovec iov = {
+.iov_base = (void *)buf,
+.iov_len = size
+};
+return dump_receive_iov(nc, &iov, 1);
+}
+
 static void dump_cleanup(NetClientState *nc)
 {
 DumpState *s = DO_UPCAST(DumpState, nc, nc);
@@ -97,6 +114,7 @@ static NetClientInfo net_dump_info = {
 .type = NET_CLIENT_OPTIONS_KIND_DUMP,
 .size = sizeof(DumpState),
 .receive = dump_receive,
+.receive_iov = dump_receive_iov,
 .cleanup = dump_cleanup,
 };
 
-- 
1.8.3.1

[Qemu-devel] [PATCH 0/5] Network traffic dumping via netfilter

[Thomas Huth]

The "-net dump" option only works with the "-net" option. So far, it
is not possible to dump network traffic with the "-netdev" option yet.
This patch series now fixes this ugliness by enabling dumping for the
netdev devices via the new netfilter infrastructure from Yang Hongyang
(integration into that infrastructure was pretty easy, so thanks for
this great work!)
The dumping filter can be used like this for example:

 ppc64-softmmu/qemu-system-ppc64 -device virtio-net,netdev=mynet \
 -netdev user,id=mynet,tftp=/tmp/tftp,bootfile=zImage \
 -object filter-dump,id=f0,netdev=mynet,file=/tmp/dumpfile.dat

The patches have to be applied on top of the Jason's net branch
(since the netfilter patches have not been pulled into master yet).

Thomas Huth (5):
  net/dump: Add support for receive_iov function
  net/dump: Rework net-dump init functions
  net/dump: Separate the NetClientState from the DumpState
  net/dump: Provide the dumping facility as a net filter
  net/dump: Add documentation

 net/dump.c  | 228 
 qemu-options.hx |   8 ++
 vl.c|   8 +-
 3 files changed, 212 insertions(+), 32 deletions(-)

-- 
1.8.3.1

[Qemu-devel] [PATCH 3/5] net/dump: Separate the NetClientState from the DumpState

[Thomas Huth]

With the upcoming dumping-via-netfilter patch, the DumpState
should not be related to NetClientState anymore, so move the
related information to a new struct called DumpNetClient.

Signed-off-by: Thomas Huth 
---
 net/dump.c | 74 +-
 1 file changed, 49 insertions(+), 25 deletions(-)

diff --git a/net/dump.c b/net/dump.c
index 27083d4..afff78e 100644
--- a/net/dump.c
+++ b/net/dump.c
@@ -31,7 +31,6 @@
 #include "hub.h"
 
 typedef struct DumpState {
-NetClientState nc;
 int64_t start_ts;
 int fd;
 int pcap_caplen;
@@ -58,10 +57,8 @@ struct pcap_sf_pkthdr {
 uint32_t len;
 };
 
-static ssize_t dump_receive_iov(NetClientState *nc, const struct iovec *iov,
-int cnt)
+static ssize_t dump_receive_iov(DumpState *s, const struct iovec *iov, int cnt)
 {
-DumpState *s = DO_UPCAST(DumpState, nc, nc);
 struct pcap_sf_pkthdr hdr;
 int64_t ts;
 int caplen;
@@ -94,30 +91,12 @@ static ssize_t dump_receive_iov(NetClientState *nc, const 
struct iovec *iov,
 return size;
 }
 
-static ssize_t dump_receive(NetClientState *nc, const uint8_t *buf, size_t 
size)
-{
-struct iovec iov = {
-.iov_base = (void *)buf,
-.iov_len = size
-};
-return dump_receive_iov(nc, &iov, 1);
-}
-
-static void dump_cleanup(NetClientState *nc)
+static void dump_cleanup(DumpState *s)
 {
-DumpState *s = DO_UPCAST(DumpState, nc, nc);
-
 close(s->fd);
+s->fd = -1;
 }
 
-static NetClientInfo net_dump_info = {
-.type = NET_CLIENT_OPTIONS_KIND_DUMP,
-.size = sizeof(DumpState),
-.receive = dump_receive,
-.receive_iov = dump_receive_iov,
-.cleanup = dump_cleanup,
-};
-
 static int net_dump_state_init(DumpState *s, const char *filename,
int len, Error **errp)
 {
@@ -154,6 +133,49 @@ static int net_dump_state_init(DumpState *s, const char 
*filename,
 return 0;
 }
 
+/* Dumping via VLAN netclient */
+
+struct DumpNetClient {
+NetClientState nc;
+DumpState ds;
+};
+typedef struct DumpNetClient DumpNetClient;
+
+static ssize_t dumpclient_receive(NetClientState *nc, const uint8_t *buf,
+  size_t size)
+{
+DumpNetClient *dc = DO_UPCAST(DumpNetClient, nc, nc);
+struct iovec iov = {
+.iov_base = (void *)buf,
+.iov_len = size
+};
+
+return dump_receive_iov(&dc->ds, &iov, 1);
+}
+
+static ssize_t dumpclient_receive_iov(NetClientState *nc,
+  const struct iovec *iov, int cnt)
+{
+DumpNetClient *dc = DO_UPCAST(DumpNetClient, nc, nc);
+
+return dump_receive_iov(&dc->ds, iov, cnt);
+}
+
+static void dumpclient_cleanup(NetClientState *nc)
+{
+DumpNetClient *dc = DO_UPCAST(DumpNetClient, nc, nc);
+
+dump_cleanup(&dc->ds);
+}
+
+static NetClientInfo net_dump_info = {
+.type = NET_CLIENT_OPTIONS_KIND_DUMP,
+.size = sizeof(DumpNetClient),
+.receive = dumpclient_receive,
+.receive_iov = dumpclient_receive_iov,
+.cleanup = dumpclient_cleanup,
+};
+
 int net_init_dump(const NetClientOptions *opts, const char *name,
   NetClientState *peer, Error **errp)
 {
@@ -162,6 +184,7 @@ int net_init_dump(const NetClientOptions *opts, const char 
*name,
 char def_file[128];
 const NetdevDumpOptions *dump;
 NetClientState *nc;
+DumpNetClient *dnc;
 
 assert(opts->kind == NET_CLIENT_OPTIONS_KIND_DUMP);
 dump = opts->dump;
@@ -195,7 +218,8 @@ int net_init_dump(const NetClientOptions *opts, const char 
*name,
 snprintf(nc->info_str, sizeof(nc->info_str),
  "dump to %s (len=%d)", file, len);
 
-rc = net_dump_state_init(DO_UPCAST(DumpState, nc, nc), file, len, errp);
+dnc = DO_UPCAST(DumpNetClient, nc, nc);
+rc = net_dump_state_init(&dnc->ds, file, len, errp);
 if (rc) {
 qemu_del_net_client(nc);
 }
-- 
1.8.3.1

[Qemu-devel] [PATCH 4/5] net/dump: Provide the dumping facility as a net filter

[Thomas Huth]

Use the net filter infrastructure to provide the dumping
functions for netdev devices, too.

Signed-off-by: Thomas Huth 
---
 net/dump.c | 129 -
 vl.c   |   8 +++-
 2 files changed, 134 insertions(+), 3 deletions(-)

diff --git a/net/dump.c b/net/dump.c
index afff78e..5904f58 100644
--- a/net/dump.c
+++ b/net/dump.c
@@ -28,7 +28,8 @@
 #include "qemu/iov.h"
 #include "qemu/log.h"
 #include "qemu/timer.h"
-#include "hub.h"
+#include "qapi/visitor.h"
+#include "net/filter.h"
 
 typedef struct DumpState {
 int64_t start_ts;
@@ -225,3 +226,129 @@ int net_init_dump(const NetClientOptions *opts, const 
char *name,
 }
 return rc;
 }
+
+/* Dumping via filter */
+
+#define TYPE_FILTER_DUMP "filter-dump"
+
+#define FILTER_DUMP(obj) \
+OBJECT_CHECK(NetFilterDumpState, (obj), TYPE_FILTER_DUMP)
+
+struct NetFilterDumpState {
+NetFilterState nfs;
+DumpState ds;
+char *filename;
+uint32_t maxlen;
+};
+typedef struct NetFilterDumpState NetFilterDumpState;
+
+static ssize_t filter_dump_receive_iov(NetFilterState *nf, NetClientState 
*sndr,
+   unsigned flags, const struct iovec *iov,
+   int iovcnt, NetPacketSent *sent_cb)
+{
+NetFilterDumpState *nfds = FILTER_DUMP(nf);
+
+dump_receive_iov(&nfds->ds, iov, iovcnt);
+return 0;
+}
+
+static void filter_dump_cleanup(NetFilterState *nf)
+{
+NetFilterDumpState *nfds = FILTER_DUMP(nf);
+
+dump_cleanup(&nfds->ds);
+}
+
+static void filter_dump_setup(NetFilterState *nf, Error **errp)
+{
+NetFilterDumpState *nfds = FILTER_DUMP(nf);
+
+if (!nfds->filename) {
+error_setg(errp, "dump filter needs 'file' property set!");
+return;
+}
+
+net_dump_state_init(&nfds->ds, nfds->filename, nfds->maxlen, errp);
+}
+
+static void filter_dump_get_maxlen(Object *obj, Visitor *v, void *opaque,
+   const char *name, Error **errp)
+{
+NetFilterDumpState *nfds = FILTER_DUMP(obj);
+uint32_t value = nfds->maxlen;
+
+visit_type_uint32(v, &value, name, errp);
+}
+
+static void filter_dump_set_maxlen(Object *obj, Visitor *v, void *opaque,
+   const char *name, Error **errp)
+{
+NetFilterDumpState *nfds = FILTER_DUMP(obj);
+Error *local_err = NULL;
+uint32_t value;
+
+visit_type_uint32(v, &value, name, &local_err);
+if (local_err) {
+goto out;
+}
+if (value == 0) {
+error_setg(&local_err, "Property '%s.%s' doesn't take value '%u'",
+   object_get_typename(obj), name, value);
+goto out;
+}
+nfds->maxlen = value;
+
+out:
+error_propagate(errp, local_err);
+}
+
+static char *file_dump_get_filename(Object *obj, Error **errp)
+{
+NetFilterDumpState *nfds = FILTER_DUMP(obj);
+
+return g_strdup(nfds->filename);
+}
+
+static void file_dump_set_filename(Object *obj, const char *value, Error 
**errp)
+{
+   NetFilterDumpState *nfds = FILTER_DUMP(obj);
+
+g_free(nfds->filename);
+nfds->filename = g_strdup(value);
+}
+
+static void filter_dump_instance_init(Object *obj)
+{
+NetFilterDumpState *nfds = FILTER_DUMP(obj);
+
+nfds->maxlen = 65536;
+
+object_property_add(obj, "maxlen", "int", filter_dump_get_maxlen,
+filter_dump_set_maxlen, NULL, NULL, NULL);
+object_property_add_str(obj, "file", file_dump_get_filename,
+file_dump_set_filename, NULL);
+}
+
+static void filter_dump_class_init(ObjectClass *oc, void *data)
+{
+NetFilterClass *nfc = NETFILTER_CLASS(oc);
+
+nfc->setup = filter_dump_setup;
+nfc->cleanup = filter_dump_cleanup;
+nfc->receive_iov = filter_dump_receive_iov;
+}
+
+static const TypeInfo filter_dump_info = {
+.name = TYPE_FILTER_DUMP,
+.parent = TYPE_NETFILTER,
+.class_init = filter_dump_class_init,
+.instance_init = filter_dump_instance_init,
+.instance_size = sizeof(NetFilterDumpState),
+};
+
+static void filter_dump_register_types(void)
+{
+type_register_static(&filter_dump_info);
+}
+
+type_init(filter_dump_register_types);
diff --git a/vl.c b/vl.c
index 9be4d22..aa99ef3 100644
--- a/vl.c
+++ b/vl.c
@@ -2761,8 +2761,12 @@ static bool object_create_initial(const char *type)
 return false;
 }
 
-/* return false for concrete netfilters */
-if (g_str_equal(type, "filter-buffer")) {
+/*
+ * return false for concrete netfilters since
+ * they depend on netdevs already existing
+ */
+if (g_str_equal(type, "filter-buffer") ||
+g_str_equal(type, "filter-dump")) {
 return false;
 }
 
-- 
1.8.3.1

Re: [Qemu-devel] [PATCH v11 01/12] qmp: delete qemu opts when delete an object

[Markus Armbruster]

This has finally reached the front of my review queue.  I apologize for
the lng delay.

Copying Paolo for another pair of eyeballs (he wrote this code).

Yang Hongyang  writes:

> When delete an object, we need to delete the associated qemu opts,
> otherwise, we can not add another object with the same name using
> object_add.
> The case happens when we start qemu with:
> -object xxx,id=aa
> then we delete this object with:
> object_del aa
> then add the object with:
> object_add xxx,id=aa
>
> QEMU will return error with Duplicate ID error.
>
> Signed-off-by: Yang Hongyang 
> ---
>  qmp.c | 4 
>  1 file changed, 4 insertions(+)
>
> diff --git a/qmp.c b/qmp.c
> index 9623c80..4bd44c3 100644
> --- a/qmp.c
> +++ b/qmp.c
> @@ -686,6 +686,7 @@ void qmp_object_del(const char *id, Error **errp)
>  {
>  Object *container;
>  Object *obj;
> +QemuOpts *opts;
>  
>  container = object_get_objects_root();
>  obj = object_resolve_path_component(container, id);
> @@ -699,6 +700,9 @@ void qmp_object_del(const char *id, Error **errp)
>  return;
>  }
>  object_unparent(obj);
> +
> +opts = qemu_opts_find(qemu_find_opts_err("object", NULL), id);
> +qemu_opts_del(opts);

qemu_find_opts_err("object", &error_abort) please, because when it
fails, we want to die right away, not when the null pointer it returns
gets dereferenced.

Same sloppiness in netdev_del_completion() and qmp_netdev_del(), not
your patch's fault.

Elsewhere, we store the QemuOpts in the object just so we can delete it:
DeviceState, DriveInfo.  Paolo, what do you think?

>  }
>  
>  MemoryDeviceInfoList *qmp_query_memory_devices(Error **errp)

Re: [Qemu-devel] [PATCH] virtio-9p: migrate virtio subsections

[Greg Kurz]

On Mon, 21 Sep 2015 10:09:34 +0200
Greg Kurz  wrote:

> On Sat, 19 Sep 2015 15:34:02 +0530
> "Aneesh Kumar K.V"  wrote:
> 
> > Greg Kurz  writes:
> > 
> > > In a cross-endian setup, the virtio-9p device has state in 
> > > @device_endian. It
> > > must be migrated.
> > >
> > > Signed-off-by: Greg Kurz 
> > 
> > With 9p mounted, we don't support qemu migration. We have
> > migration blocker added in v9fs_attach. 
> > 
> 
> Ok I should have been more precise... If you unmount the 9p share, then
> migrate, then try to remount, you end up with a hung mount command because
> @device_endian has the wrong value...
> 

Michael ? Any input ?

> > > ---
> > >  hw/9pfs/virtio-9p-device.c |   11 +++
> > >  1 file changed, 11 insertions(+)
> > >
> > > diff --git a/hw/9pfs/virtio-9p-device.c b/hw/9pfs/virtio-9p-device.c
> > > index 93a407c45926..e3abcfaffb2a 100644
> > > --- a/hw/9pfs/virtio-9p-device.c
> > > +++ b/hw/9pfs/virtio-9p-device.c
> > > @@ -43,6 +43,16 @@ static void virtio_9p_get_config(VirtIODevice *vdev, 
> > > uint8_t *config)
> > >  g_free(cfg);
> > >  }
> > >
> > > +static void virtio_9p_save(QEMUFile *f, void *opaque)
> > > +{
> > > +virtio_save(VIRTIO_DEVICE(opaque), f);
> > > +}
> > > +
> > > +static int virtio_9p_load(QEMUFile *f, void *opaque, int version_id)
> > > +{
> > > +return virtio_load(VIRTIO_DEVICE(opaque), f, version_id);
> > > +}
> > > +
> > >  static void virtio_9p_device_realize(DeviceState *dev, Error **errp)
> > >  {
> > >  VirtIODevice *vdev = VIRTIO_DEVICE(dev);
> > > @@ -130,6 +140,7 @@ static void virtio_9p_device_realize(DeviceState 
> > > *dev, Error **errp)
> > >  }
> > >  v9fs_path_free(&path);
> > >
> > > +register_savevm(dev, "virtio-9p", -1, 1, virtio_9p_save, 
> > > virtio_9p_load, s);
> > >  return;
> > >  out:
> > >  g_free(s->ctx.fs_root);
> 
> 

Re: [Qemu-devel] No error report when using the qemu-img.exetoconvert a disk to vmdk format which is saved on a disk that has nomorespace

[Guangmu Zhu]

Hi Kevin,


I tried the patch you provide, and I haven't seen that problem yet. If the disk 
space is full, an error will be reported with the message "Invalid argument" 
and the program will stop.


Will you merge the patch to the master?


Thanks.
Guangmu Zhu


diff --git a/block/raw-win32.c b/block/raw-win32.c
index 68f2338..b562c94 100644
--- a/block/raw-win32.c
+++ b/block/raw-win32.c
@@ -119,9 +119,9 @@ static int aio_worker(void *arg)
 case QEMU_AIO_WRITE:
 count = handle_aiocb_rw(aiocb);
 if (count == aiocb->aio_nbytes) {
-count = 0;
+ret = 0;
 } else {
-count = -EINVAL;
+ret = -EINVAL;
 }
 break;
 case QEMU_AIO_FLUSH:


-


I'll try the patch and report in a week for I'm too busy these days. And if I 
could, I would like to help to maintain the Windows backend.


Sincerely.
Guangmu Zhu


-


Am 23.09.2015 um 13:30 hat Guangmu Zhu geschrieben:
> If the "BlockDriver" is "bdrv_vmdk", the function "vmdk_co_write" will be
> called instead. In function "vmdk_write_extent" I see "ret = bdrv_pwrite
> (extent->file, write_offset, write_buf, write_len);". So the "extend->file" is
> "bdrv_file", is it?

Yes, exactly. You'll go through bdrv_vmdk first, and then the nested
call goes to bdrv_file.

> -
> 
> Correct a mistake:
> So though the "count" would be "-EINVAL" if error occurred while writing some
> file, the return value will always be zero. Maybe I missed something?

I think you're right. Instead of setting count = 0/-EINVAL in
aio_worker, we should be setting ret.

Can you try the patch below and report back?

> 3. The "bs->drv->bdrv_aio_writev" is function "raw_aio_writev" in file
> "raw-win32.c" and the quemu-img uses synchronous IO always, so the function
> "paio_submit" in the same file will be called. This function submits the "aio"
> to "worker_thread" with the callback "aio_worker". There are some codes in
> "aio_worker":
> 
> ssize_t ret = 0;
> ..
> case QEMU_AIO_WRITE:
> count = handle_aiocb_rw(aiocb);
> if (count == aiocb->aio_nbytes) {
> count = 0;
> } else {
> count = -EINVAL;
> }
> break;
> ..
> return ret;

Independently of your problem, the code in aio_worker() looks a bit
fishy, because handle_aiocb_rw() can't distinguish between an error
and 0 bytes transferred.

For writes, that probably doesn't matter, but for reads, I think we
return a successful read of zeroes instead of signalling an error. This
might need another patch.

Generally, the Windows backend is not getting a lot of attention and
could use someone who checks it, cleans it up and fixes bugs.

Kevin


diff --git a/block/raw-win32.c b/block/raw-win32.c
index 68f2338..b562c94 100644
--- a/block/raw-win32.c
+++ b/block/raw-win32.c
@@ -119,9 +119,9 @@ static int aio_worker(void *arg)
 case QEMU_AIO_WRITE:
 count = handle_aiocb_rw(aiocb);
 if (count == aiocb->aio_nbytes) {
-count = 0;
+ret = 0;
 } else {
-count = -EINVAL;
+ret = -EINVAL;
 }
 break;
 case QEMU_AIO_FLUSH:

Re: [Qemu-devel] [PATCH 4/5] net/dump: Provide the dumping facility as a net filter

[Yang Hongyang]

The filter interface part looks good to me, thanks!

On 09/24/2015 03:22 PM, Thomas Huth wrote:

Use the net filter infrastructure to provide the dumping
functions for netdev devices, too.

Signed-off-by: Thomas Huth 
---
  net/dump.c | 129 -
  vl.c   |   8 +++-
  2 files changed, 134 insertions(+), 3 deletions(-)

diff --git a/net/dump.c b/net/dump.c
index afff78e..5904f58 100644
--- a/net/dump.c
+++ b/net/dump.c
@@ -28,7 +28,8 @@
  #include "qemu/iov.h"
  #include "qemu/log.h"
  #include "qemu/timer.h"
-#include "hub.h"
+#include "qapi/visitor.h"
+#include "net/filter.h"

  typedef struct DumpState {
  int64_t start_ts;
@@ -225,3 +226,129 @@ int net_init_dump(const NetClientOptions *opts, const 
char *name,
  }
  return rc;
  }
+
+/* Dumping via filter */
+
+#define TYPE_FILTER_DUMP "filter-dump"
+
+#define FILTER_DUMP(obj) \
+OBJECT_CHECK(NetFilterDumpState, (obj), TYPE_FILTER_DUMP)
+
+struct NetFilterDumpState {
+NetFilterState nfs;
+DumpState ds;
+char *filename;
+uint32_t maxlen;
+};
+typedef struct NetFilterDumpState NetFilterDumpState;
+
+static ssize_t filter_dump_receive_iov(NetFilterState *nf, NetClientState 
*sndr,
+   unsigned flags, const struct iovec *iov,
+   int iovcnt, NetPacketSent *sent_cb)
+{
+NetFilterDumpState *nfds = FILTER_DUMP(nf);
+
+dump_receive_iov(&nfds->ds, iov, iovcnt);
+return 0;
+}
+
+static void filter_dump_cleanup(NetFilterState *nf)
+{
+NetFilterDumpState *nfds = FILTER_DUMP(nf);
+
+dump_cleanup(&nfds->ds);
+}
+
+static void filter_dump_setup(NetFilterState *nf, Error **errp)
+{
+NetFilterDumpState *nfds = FILTER_DUMP(nf);
+
+if (!nfds->filename) {
+error_setg(errp, "dump filter needs 'file' property set!");
+return;
+}
+
+net_dump_state_init(&nfds->ds, nfds->filename, nfds->maxlen, errp);
+}
+
+static void filter_dump_get_maxlen(Object *obj, Visitor *v, void *opaque,
+   const char *name, Error **errp)
+{
+NetFilterDumpState *nfds = FILTER_DUMP(obj);
+uint32_t value = nfds->maxlen;
+
+visit_type_uint32(v, &value, name, errp);
+}
+
+static void filter_dump_set_maxlen(Object *obj, Visitor *v, void *opaque,
+   const char *name, Error **errp)
+{
+NetFilterDumpState *nfds = FILTER_DUMP(obj);
+Error *local_err = NULL;
+uint32_t value;
+
+visit_type_uint32(v, &value, name, &local_err);
+if (local_err) {
+goto out;
+}
+if (value == 0) {
+error_setg(&local_err, "Property '%s.%s' doesn't take value '%u'",
+   object_get_typename(obj), name, value);
+goto out;
+}
+nfds->maxlen = value;
+
+out:
+error_propagate(errp, local_err);
+}
+
+static char *file_dump_get_filename(Object *obj, Error **errp)
+{
+NetFilterDumpState *nfds = FILTER_DUMP(obj);
+
+return g_strdup(nfds->filename);
+}
+
+static void file_dump_set_filename(Object *obj, const char *value, Error 
**errp)
+{
+   NetFilterDumpState *nfds = FILTER_DUMP(obj);
+
+g_free(nfds->filename);
+nfds->filename = g_strdup(value);
+}
+
+static void filter_dump_instance_init(Object *obj)
+{
+NetFilterDumpState *nfds = FILTER_DUMP(obj);
+
+nfds->maxlen = 65536;
+
+object_property_add(obj, "maxlen", "int", filter_dump_get_maxlen,
+filter_dump_set_maxlen, NULL, NULL, NULL);
+object_property_add_str(obj, "file", file_dump_get_filename,
+file_dump_set_filename, NULL);
+}
+
+static void filter_dump_class_init(ObjectClass *oc, void *data)
+{
+NetFilterClass *nfc = NETFILTER_CLASS(oc);
+
+nfc->setup = filter_dump_setup;
+nfc->cleanup = filter_dump_cleanup;
+nfc->receive_iov = filter_dump_receive_iov;
+}
+
+static const TypeInfo filter_dump_info = {
+.name = TYPE_FILTER_DUMP,
+.parent = TYPE_NETFILTER,
+.class_init = filter_dump_class_init,
+.instance_init = filter_dump_instance_init,
+.instance_size = sizeof(NetFilterDumpState),
+};
+
+static void filter_dump_register_types(void)
+{
+type_register_static(&filter_dump_info);
+}
+
+type_init(filter_dump_register_types);
diff --git a/vl.c b/vl.c
index 9be4d22..aa99ef3 100644
--- a/vl.c
+++ b/vl.c
@@ -2761,8 +2761,12 @@ static bool object_create_initial(const char *type)
  return false;
  }

-/* return false for concrete netfilters */
-if (g_str_equal(type, "filter-buffer")) {
+/*
+ * return false for concrete netfilters since
+ * they depend on netdevs already existing
+ */
+if (g_str_equal(type, "filter-buffer") ||
+g_str_equal(type, "filter-dump")) {
  return false;
  }




--
Thanks,
Yang.

[Qemu-devel] [RFC v5 4/6] target-arm: Create new runtime helpers for excl accesses

[Alvise Rigo]

Introduce a set of new runtime helpers do handle exclusive instructions.
This helpers are used as hooks to call the respective LL/SC helpers in
softmmu_llsc_template.h from TCG code.

Suggested-by: Jani Kokkonen 
Suggested-by: Claudio Fontana 
Signed-off-by: Alvise Rigo 
---
 target-arm/helper.h| 10 ++
 target-arm/op_helper.c | 94 ++
 2 files changed, 104 insertions(+)

diff --git a/target-arm/helper.h b/target-arm/helper.h
index 827b33d..8e7a7c2 100644
--- a/target-arm/helper.h
+++ b/target-arm/helper.h
@@ -530,6 +530,16 @@ DEF_HELPER_2(dc_zva, void, env, i64)
 DEF_HELPER_FLAGS_2(neon_pmull_64_lo, TCG_CALL_NO_RWG_SE, i64, i64, i64)
 DEF_HELPER_FLAGS_2(neon_pmull_64_hi, TCG_CALL_NO_RWG_SE, i64, i64, i64)
 
+DEF_HELPER_3(ldlink_aa32_i8, i32, env, i32, i32)
+DEF_HELPER_3(ldlink_aa32_i16, i32, env, i32, i32)
+DEF_HELPER_3(ldlink_aa32_i32, i32, env, i32, i32)
+DEF_HELPER_3(ldlink_aa32_i64, i64, env, i32, i32)
+
+DEF_HELPER_4(stcond_aa32_i8, i32, env, i32, i32, i32)
+DEF_HELPER_4(stcond_aa32_i16, i32, env, i32, i32, i32)
+DEF_HELPER_4(stcond_aa32_i32, i32, env, i32, i32, i32)
+DEF_HELPER_4(stcond_aa32_i64, i32, env, i32, i64, i32)
+
 #ifdef TARGET_AARCH64
 #include "helper-a64.h"
 #endif
diff --git a/target-arm/op_helper.c b/target-arm/op_helper.c
index 663c05d..d832ba8 100644
--- a/target-arm/op_helper.c
+++ b/target-arm/op_helper.c
@@ -969,3 +969,97 @@ uint32_t HELPER(ror_cc)(CPUARMState *env, uint32_t x, 
uint32_t i)
 return ((uint32_t)x >> shift) | (x << (32 - shift));
 }
 }
+
+/* LoadLink helpers, only unsigned. */
+static void * const qemu_ldex_helpers[16] = {
+[MO_UB]   = helper_ret_ldlinkub_mmu,
+
+[MO_LEUW] = helper_le_ldlinkuw_mmu,
+[MO_LEUL] = helper_le_ldlinkul_mmu,
+[MO_LEQ]  = helper_le_ldlinkq_mmu,
+
+[MO_BEUW] = helper_be_ldlinkuw_mmu,
+[MO_BEUL] = helper_be_ldlinkul_mmu,
+[MO_BEQ]  = helper_be_ldlinkq_mmu,
+};
+
+#define LDEX_HELPER(SUFF, OPC)  \
+uint32_t HELPER(ldlink_aa32_i##SUFF)(CPUARMState *env, uint32_t addr,   \
+   uint32_t index)  \
+{   \
+CPUArchState *state = env;  \
+TCGMemOpIdx op; \
+\
+op = make_memop_idx(OPC, index);\
+\
+tcg_target_ulong (*func)(CPUArchState *env, target_ulong addr,  \
+ TCGMemOpIdx oi, uintptr_t retaddr);\
+func = qemu_ldex_helpers[OPC];  \
+\
+return (uint32_t)func(state, addr, op, GETRA());\
+}
+
+LDEX_HELPER(8, MO_UB)
+LDEX_HELPER(16, MO_TEUW)
+LDEX_HELPER(32, MO_TEUL)
+
+uint64_t HELPER(ldlink_aa32_i64)(CPUARMState *env, uint32_t addr,
+   uint32_t index)
+{
+CPUArchState *state = env;
+TCGMemOpIdx op;
+
+op = make_memop_idx(MO_TEQ, index);
+
+uint64_t (*func)(CPUArchState *env, target_ulong addr,
+ TCGMemOpIdx oi, uintptr_t retaddr);
+func = qemu_ldex_helpers[MO_TEQ];
+
+return func(state, addr, op, GETRA());
+}
+
+/* StoreConditional helpers. Use the macro below to access them. */
+static void * const qemu_stex_helpers[16] = {
+[MO_UB]   = helper_ret_stcondb_mmu,
+[MO_LEUW] = helper_le_stcondw_mmu,
+[MO_LEUL] = helper_le_stcondl_mmu,
+[MO_LEQ]  = helper_le_stcondq_mmu,
+[MO_BEUW] = helper_be_stcondw_mmu,
+[MO_BEUL] = helper_be_stcondl_mmu,
+[MO_BEQ]  = helper_be_stcondq_mmu,
+};
+
+#define STEX_HELPER(SUFF, DATA_TYPE, OPC)   \
+uint32_t HELPER(stcond_aa32_i##SUFF)(CPUARMState *env, uint32_t addr,   \
+ uint32_t val, uint32_t index)  \
+{   \
+CPUArchState *state = env;  \
+TCGMemOpIdx op; \
+\
+op = make_memop_idx(OPC, index);\
+\
+tcg_target_ulong (*func)(CPUArchState *env, target_ulong addr,  \
+DATA_TYPE val, TCGMemOpIdx oi, uintptr_t retaddr);  \
+func = qemu_stex_helpers[OPC];  \
+\
+return (uint32_t)func(state, addr, val, op, GETRA());   \
+}
+
+STEX

[Qemu-devel] [RFC v5 3/6] softmmu: Add helpers for a new slowpath

[Alvise Rigo]

The new helpers rely on the legacy ones to perform the actual read/write.

The LoadLink helper (helper_ldlink_name) prepares the way for the
following SC operation. It sets the linked address and the size of the
access.
These helper also update the TLB entry of the page involved in the
LL/SC for those vCPUs that have the bit set (dirty), so that the
following accesses made by all the vCPUs will follow the slow path.

The StoreConditional helper (helper_stcond_name) returns 1 if the
store has to fail due to a concurrent access to the same page by
another vCPU. A 'concurrent access' can be a store made by *any* vCPU
(although, some implementations allow stores made by the CPU that issued
the LoadLink).

Suggested-by: Jani Kokkonen 
Suggested-by: Claudio Fontana 
Signed-off-by: Alvise Rigo 
---
 cputlb.c|   3 ++
 softmmu_llsc_template.h | 124 
 softmmu_template.h  |  12 +
 tcg/tcg.h   |  30 
 4 files changed, 169 insertions(+)
 create mode 100644 softmmu_llsc_template.h

diff --git a/cputlb.c b/cputlb.c
index 1e25a2a..d5aae7c 100644
--- a/cputlb.c
+++ b/cputlb.c
@@ -416,6 +416,8 @@ static inline void lookup_and_reset_cpus_ll_addr(hwaddr 
addr, hwaddr size)
 
 #define MMUSUFFIX _mmu
 
+/* Generates LoadLink/StoreConditional helpers in softmmu_template.h */
+#define GEN_EXCLUSIVE_HELPERS
 #define SHIFT 0
 #include "softmmu_template.h"
 
@@ -428,6 +430,7 @@ static inline void lookup_and_reset_cpus_ll_addr(hwaddr 
addr, hwaddr size)
 #define SHIFT 3
 #include "softmmu_template.h"
 #undef MMUSUFFIX
+#undef GEN_EXCLUSIVE_HELPERS
 
 #define MMUSUFFIX _cmmu
 #undef GETPC_ADJ
diff --git a/softmmu_llsc_template.h b/softmmu_llsc_template.h
new file mode 100644
index 000..9f22834
--- /dev/null
+++ b/softmmu_llsc_template.h
@@ -0,0 +1,124 @@
+/*
+ *  Software MMU support (esclusive load/store operations)
+ *
+ * Generate helpers used by TCG for qemu_ldlink/stcond ops.
+ *
+ * Included from softmmu_template.h only.
+ *
+ * Copyright (c) 2015 Virtual Open Systems
+ *
+ * Authors:
+ *  Alvise Rigo 
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see .
+ */
+
+/* This template does not generate together the le and be version, but only one
+ * of the two depending on whether BIGENDIAN_EXCLUSIVE_HELPERS has been set.
+ * The same nomenclature as softmmu_template.h is used for the exclusive
+ * helpers.  */
+
+#ifdef BIGENDIAN_EXCLUSIVE_HELPERS
+
+#define helper_ldlink_name  glue(glue(helper_be_ldlink, USUFFIX), MMUSUFFIX)
+#define helper_stcond_name  glue(glue(helper_be_stcond, SUFFIX), MMUSUFFIX)
+#define helper_ld glue(glue(helper_be_ld, USUFFIX), MMUSUFFIX)
+#define helper_st glue(glue(helper_be_st, SUFFIX), MMUSUFFIX)
+
+#else /* LE helpers + 8bit helpers (generated only once for both LE end BE) */
+
+#if DATA_SIZE > 1
+#define helper_ldlink_name  glue(glue(helper_le_ldlink, USUFFIX), MMUSUFFIX)
+#define helper_stcond_name  glue(glue(helper_le_stcond, SUFFIX), MMUSUFFIX)
+#define helper_ld glue(glue(helper_le_ld, USUFFIX), MMUSUFFIX)
+#define helper_st glue(glue(helper_le_st, SUFFIX), MMUSUFFIX)
+#else /* DATA_SIZE <= 1 */
+#define helper_ldlink_name  glue(glue(helper_ret_ldlink, USUFFIX), MMUSUFFIX)
+#define helper_stcond_name  glue(glue(helper_ret_stcond, SUFFIX), MMUSUFFIX)
+#define helper_ld glue(glue(helper_ret_ld, USUFFIX), MMUSUFFIX)
+#define helper_st glue(glue(helper_ret_st, SUFFIX), MMUSUFFIX)
+#endif
+
+#endif
+
+WORD_TYPE helper_ldlink_name(CPUArchState *env, target_ulong addr,
+TCGMemOpIdx oi, uintptr_t retaddr)
+{
+WORD_TYPE ret;
+int index;
+CPUState *cpu;
+hwaddr hw_addr;
+unsigned mmu_idx = get_mmuidx(oi);
+
+/* Use the proper load helper from cpu_ldst.h */
+ret = helper_ld(env, addr, mmu_idx, retaddr);
+
+index = (addr >> TARGET_PAGE_BITS) & (CPU_TLB_SIZE - 1);
+
+/* hw_addr = hwaddr of the page (i.e. section->mr->ram_addr + xlat)
+ * plus the offset (i.e. addr & ~TARGET_PAGE_MASK) */
+hw_addr = (env->iotlb[mmu_idx][index].addr & TARGET_PAGE_MASK) + addr;
+
+cpu_physical_memory_clear_excl_dirty(hw_addr, ENV_GET_CPU(env)->cpu_index);
+/* If all the vCPUs have the EXCL bit set for this page there is no need
+ * to request any flush. */
+if (cpu_physical_memory_excl_is_dirty(hw_addr, smp_cpus)) {
+   

[Qemu-devel] [RFC v5 0/6] Slow-path for atomic instruction translation

[Alvise Rigo]

This is the fifth iteration of the patch series which applies to the
upstream branch of QEMU (v2.4.0).

Changes versus previous versions are at the bottom of this cover letter.

The code is also available at following repository:
https://git.virtualopensystems.com/dev/qemu-mt.git
branch:
slowpath-for-atomic-v5-no-mttcg
(branch slowpath-for-atomic-v5-mttcg for the version based on mttcg)

This patch series provides an infrastructure for atomic instruction
implementation in QEMU, thus offering a 'legacy' solution for
translating guest atomic instructions. Moreover, it can be considered as
a first step toward a multi-thread TCG.

The underlying idea is to provide new TCG helpers (sort of softmmu
helpers) that guarantee atomicity to some memory accesses or in general
a way to define memory transactions.

More specifically, the new softmmu helpers behave as LoadLink and
StoreConditional instructions, and are called from TCG code by means of
target specific helpers. This work includes the implementation for all
the ARM atomic instructions, see target-arm/op_helper.c.

The implementation heavily uses the software TLB together with a new
bitmap that has been added to the ram_list structure which flags, on a
per-CPU basis, all the memory pages that are in the middle of a LoadLink
(LL), StoreConditional (SC) operation.  Since all these pages can be
accessed directly through the fast-path and alter a vCPU's linked value,
the new bitmap has been coupled with a new TLB flag for the TLB virtual
address which forces the slow-path execution for all the accesses to a
page containing a linked address.

The new slow-path is implemented such that:
- the LL behaves as a normal load slow-path, except for clearing the
  dirty flag in the bitmap.  The cputlb.c code while generating a TLB
  entry, checks if there is at least one vCPU that has the bit cleared
  in the exclusive bitmap, it that case the TLB entry will have the EXCL
  flag set, thus forcing the slow-path.  In order to ensure that all the
  vCPUs will follow the slow-path for that page, we flush the TLB cache
  of all the other vCPUs.

  The LL will also set the linked address and size of the access in a
  vCPU's private variable. After the corresponding SC, this address will
  be set to a reset value.

- the SC can fail returning 1, or succeed, returning 0.  It has to come
  always after a LL and has to access the same address 'linked' by the
  previous LL, otherwise it will fail. If in the time window delimited
  by a legit pair of LL/SC operations another write access happens to
  the linked address, the SC will fail.

In theory, the provided implementation of TCG LoadLink/StoreConditional
can be used to properly handle atomic instructions on any architecture.

The code has been tested with bare-metal test cases and by booting Linux.

* Performance considerations
The new slow-path adds some overhead to the translation of the ARM
atomic instructions, since their emulation doesn't happen anymore only
in the guest (by mean of pure TCG generated code), but requires the
execution of two helpers functions. Despite this, the additional time
required to boot an ARM Linux kernel on an i7 clocked at 2.5GHz is
negligible.
Instead, on a LL/SC bound test scenario - like:
https://git.virtualopensystems.com/dev/tcg_baremetal_tests.git - this
solution requires 30% (1 million iterations) and 70% (10 millions
iterations) of additional time for the test to complete.

Changes from v4:
- Reworked the exclusive bitmap to be of fixed size (8 bits per address)
- The slow-path is now TCG backend independent, no need to touch
  tcg/* anymore as suggested by Aurelien Jarno.

Changes from v3:
- based on upstream QEMU
- addressed comments from Alex Bennée
- the slow path can be enabled by the user with:
  ./configure --enable-tcg-ldst-excl only if the backend supports it
- all the ARM ldex/stex instructions make now use of the slow path
- added aarch64 TCG backend support
- part of the code has been rewritten

Changes from v2:
- the bitmap accessors are now atomic
- a rendezvous between vCPUs and a simple callback support before executing
  a TB have been added to handle the TLB flush support
- the softmmu_template and softmmu_llsc_template have been adapted to work
  on real multi-threading

Changes from v1:
- The ram bitmap is not reversed anymore, 1 = dirty, 0 = exclusive
- The way how the offset to access the bitmap is calculated has
  been improved and fixed
- A page to be set as dirty requires a vCPU to target the protected address
  and not just an address in the page
- Addressed comments from Richard Henderson to improve the logic in
  softmmu_template.h and to simplify the methods generation through
  softmmu_llsc_template.h
- Added initial implementation of qemu_{ldlink,stcond}_i32 for tcg/i386

This work has been sponsored by Huawei Technologies Duesseldorf GmbH.

Alvise Rigo (6):
  exec.c: Add new exclusive bitmap to ram_list
  softmmu: Add new TLB_EXCL flag
  softmmu: Add helpers 

[Qemu-devel] [RFC v5 6/6] target-arm: translate: Use ld/st excl for atomic insns

[Alvise Rigo]

Use the new LL/SC runtime helpers to handle the ARM atomic
instructions in softmmu_llsc_template.h.

In general, the helper generator
gen_helper_{ldlink,stcond}_aa32_i{8,16,32,64}() calls the function
helper_{le,be}_{ldlink,stcond}{ub,uw,ulq}_mmu() implemented in
softmmu_llsc_template.h.

Suggested-by: Jani Kokkonen 
Suggested-by: Claudio Fontana 
Signed-off-by: Alvise Rigo 
---
 target-arm/translate.c | 121 +++--
 1 file changed, 117 insertions(+), 4 deletions(-)

diff --git a/target-arm/translate.c b/target-arm/translate.c
index 69ac18c..fa85455 100644
--- a/target-arm/translate.c
+++ b/target-arm/translate.c
@@ -65,8 +65,12 @@ TCGv_ptr cpu_env;
 static TCGv_i64 cpu_V0, cpu_V1, cpu_M0;
 static TCGv_i32 cpu_R[16];
 static TCGv_i32 cpu_CF, cpu_NF, cpu_VF, cpu_ZF;
+#ifndef CONFIG_TCG_USE_LDST_EXCL
 static TCGv_i64 cpu_exclusive_addr;
 static TCGv_i64 cpu_exclusive_val;
+#else
+static TCGv_i32 cpu_ll_sc_context;
+#endif
 #ifdef CONFIG_USER_ONLY
 static TCGv_i64 cpu_exclusive_test;
 static TCGv_i32 cpu_exclusive_info;
@@ -99,10 +103,15 @@ void arm_translate_init(void)
 cpu_VF = tcg_global_mem_new_i32(TCG_AREG0, offsetof(CPUARMState, VF), 
"VF");
 cpu_ZF = tcg_global_mem_new_i32(TCG_AREG0, offsetof(CPUARMState, ZF), 
"ZF");
 
+#ifdef CONFIG_TCG_USE_LDST_EXCL
+cpu_ll_sc_context = tcg_global_mem_new_i32(TCG_AREG0,
+offsetof(CPUARMState, ll_sc_context), "ll_sc_context");
+#else
 cpu_exclusive_addr = tcg_global_mem_new_i64(TCG_AREG0,
 offsetof(CPUARMState, exclusive_addr), "exclusive_addr");
 cpu_exclusive_val = tcg_global_mem_new_i64(TCG_AREG0,
 offsetof(CPUARMState, exclusive_val), "exclusive_val");
+#endif
 #ifdef CONFIG_USER_ONLY
 cpu_exclusive_test = tcg_global_mem_new_i64(TCG_AREG0,
 offsetof(CPUARMState, exclusive_test), "exclusive_test");
@@ -7382,15 +7391,62 @@ static void gen_logicq_cc(TCGv_i32 lo, TCGv_i32 hi)
 tcg_gen_or_i32(cpu_ZF, lo, hi);
 }
 
-/* Load/Store exclusive instructions are implemented by remembering
+/* If the softmmu is enabled, the translation of Load/Store exclusive
+ * instructions will rely on the gen_helper_{ldlink,stcond} helpers,
+ * offloading most of the work to the softmmu_llsc_template.h functions.
+
+   Otherwise, these instructions are implemented by remembering
the value/address loaded, and seeing if these are the same
when the store is performed. This should be sufficient to implement
the architecturally mandated semantics, and avoids having to monitor
regular stores.
 
-   In system emulation mode only one CPU will be running at once, so
-   this sequence is effectively atomic.  In user emulation mode we
-   throw an exception and handle the atomic operation elsewhere.  */
+   In user emulation mode we throw an exception and handle the atomic
+   operation elsewhere.  */
+#ifdef CONFIG_TCG_USE_LDST_EXCL
+static void gen_load_exclusive(DisasContext *s, int rt, int rt2,
+   TCGv_i32 addr, int size)
+ {
+TCGv_i32 tmp = tcg_temp_new_i32();
+TCGv_i32 mem_idx = tcg_temp_new_i32();
+
+tcg_gen_movi_i32(mem_idx, get_mem_index(s));
+
+if (size != 3) {
+switch (size) {
+case 0:
+gen_helper_ldlink_aa32_i8(tmp, cpu_env, addr, mem_idx);
+break;
+case 1:
+gen_helper_ldlink_aa32_i16(tmp, cpu_env, addr, mem_idx);
+break;
+case 2:
+gen_helper_ldlink_aa32_i32(tmp, cpu_env, addr, mem_idx);
+break;
+default:
+abort();
+}
+
+store_reg(s, rt, tmp);
+} else {
+TCGv_i64 tmp64 = tcg_temp_new_i64();
+TCGv_i32 tmph = tcg_temp_new_i32();
+
+gen_helper_ldlink_aa32_i64(tmp64, cpu_env, addr, mem_idx);
+tcg_gen_extr_i64_i32(tmp, tmph, tmp64);
+
+store_reg(s, rt, tmp);
+store_reg(s, rt2, tmph);
+
+tcg_temp_free_i64(tmp64);
+}
+
+tcg_temp_free_i32(mem_idx);
+
+/* From now on we are in LL/SC context. */
+tcg_gen_movi_i32(cpu_ll_sc_context, 1);
+}
+#else
 static void gen_load_exclusive(DisasContext *s, int rt, int rt2,
TCGv_i32 addr, int size)
 {
@@ -7429,10 +7485,14 @@ static void gen_load_exclusive(DisasContext *s, int rt, 
int rt2,
 store_reg(s, rt, tmp);
 tcg_gen_extu_i32_i64(cpu_exclusive_addr, addr);
 }
+#endif
 
 static void gen_clrex(DisasContext *s)
 {
+#ifdef CONFIG_TCG_USE_LDST_EXCL
+#else
 tcg_gen_movi_i64(cpu_exclusive_addr, -1);
+#endif
 }
 
 #ifdef CONFIG_USER_ONLY
@@ -7444,6 +7504,59 @@ static void gen_store_exclusive(DisasContext *s, int rd, 
int rt, int rt2,
  size | (rd << 4) | (rt << 8) | (rt2 << 12));
 gen_exception_internal_insn(s, 4, EXCP_STREX);
 }
+#elif defined CONFIG_TCG_USE_LDST_EXCL
+static void gen_store_exclusive(DisasContext *s, int rd, int rt, int rt2,
+TCGv_i32 addr, int size)
+{
+TCGv_i32 tmp, mem_

[Qemu-devel] [RFC v5 1/6] exec.c: Add new exclusive bitmap to ram_list

[Alvise Rigo]

The purpose of this new bitmap is to flag the memory pages that are in
the middle of LL/SC operations (after a LL, before a SC) on a per-vCPU
basis.
For all these pages, the corresponding TLB entries will be generated
in such a way to force the slow-path if at least one vCPU has the bit
not set.
When the system starts, the whole memory is dirty (all the bitmap is
set). A page, after being marked as exclusively-clean, will be
restored as dirty after the SC.

For each page we keep 8 bits to be shared among all the vCPUs available
in the system. In general, the to the vCPU n correspond the bit n % 8.

Suggested-by: Jani Kokkonen 
Suggested-by: Claudio Fontana 
Signed-off-by: Alvise Rigo 
---
 exec.c  |  8 --
 include/exec/memory.h   |  3 +-
 include/exec/ram_addr.h | 75 +
 3 files changed, 83 insertions(+), 3 deletions(-)

diff --git a/exec.c b/exec.c
index 0a4a0c5..cbe559f 100644
--- a/exec.c
+++ b/exec.c
@@ -1496,11 +1496,15 @@ static ram_addr_t ram_block_add(RAMBlock *new_block, 
Error **errp)
 int i;
 
 /* ram_list.dirty_memory[] is protected by the iothread lock.  */
-for (i = 0; i < DIRTY_MEMORY_NUM; i++) {
+for (i = 0; i < DIRTY_MEMORY_EXCLUSIVE; i++) {
 ram_list.dirty_memory[i] =
 bitmap_zero_extend(ram_list.dirty_memory[i],
old_ram_size, new_ram_size);
-   }
+}
+ram_list.dirty_memory[DIRTY_MEMORY_EXCLUSIVE] = bitmap_zero_extend(
+ram_list.dirty_memory[DIRTY_MEMORY_EXCLUSIVE],
+old_ram_size * EXCL_BITMAP_CELL_SZ,
+new_ram_size * EXCL_BITMAP_CELL_SZ);
 }
 cpu_physical_memory_set_dirty_range(new_block->offset,
 new_block->used_length,
diff --git a/include/exec/memory.h b/include/exec/memory.h
index 94d20ea..b71cb98 100644
--- a/include/exec/memory.h
+++ b/include/exec/memory.h
@@ -19,7 +19,8 @@
 #define DIRTY_MEMORY_VGA   0
 #define DIRTY_MEMORY_CODE  1
 #define DIRTY_MEMORY_MIGRATION 2
-#define DIRTY_MEMORY_NUM   3/* num of dirty bits */
+#define DIRTY_MEMORY_EXCLUSIVE 3
+#define DIRTY_MEMORY_NUM   4/* num of dirty bits */
 
 #include 
 #include 
diff --git a/include/exec/ram_addr.h b/include/exec/ram_addr.h
index c113f21..0016a4b 100644
--- a/include/exec/ram_addr.h
+++ b/include/exec/ram_addr.h
@@ -21,6 +21,7 @@
 
 #ifndef CONFIG_USER_ONLY
 #include "hw/xen/xen.h"
+#include "sysemu/sysemu.h"
 
 ram_addr_t qemu_ram_alloc_from_file(ram_addr_t size, MemoryRegion *mr,
 bool share, const char *mem_path,
@@ -44,6 +45,13 @@ int qemu_ram_resize(ram_addr_t base, ram_addr_t newsize, 
Error **errp);
 #define DIRTY_CLIENTS_ALL ((1 << DIRTY_MEMORY_NUM) - 1)
 #define DIRTY_CLIENTS_NOCODE  (DIRTY_CLIENTS_ALL & ~(1 << DIRTY_MEMORY_CODE))
 
+/* Exclusive bitmap support. */
+#define EXCL_BITMAP_CELL_SZ 8
+#define EXCL_BITMAP_GET_BIT_OFFSET(addr) \
+(EXCL_BITMAP_CELL_SZ * (addr >> TARGET_PAGE_BITS))
+#define EXCL_BITMAP_GET_BYTE_OFFSET(addr) (addr >> TARGET_PAGE_BITS)
+#define EXCL_IDX(cpu) (cpu % EXCL_BITMAP_CELL_SZ)
+
 static inline bool cpu_physical_memory_get_dirty(ram_addr_t start,
  ram_addr_t length,
  unsigned client)
@@ -135,6 +143,11 @@ static inline void 
cpu_physical_memory_set_dirty_range(ram_addr_t start,
 if (unlikely(mask & (1 << DIRTY_MEMORY_CODE))) {
 bitmap_set_atomic(d[DIRTY_MEMORY_CODE], page, end - page);
 }
+if (unlikely(mask & (1 << DIRTY_MEMORY_EXCLUSIVE))) {
+bitmap_set_atomic(d[DIRTY_MEMORY_EXCLUSIVE],
+page * EXCL_BITMAP_CELL_SZ,
+(end - page) * EXCL_BITMAP_CELL_SZ);
+}
 xen_modified_memory(start, length);
 }
 
@@ -249,5 +262,67 @@ uint64_t cpu_physical_memory_sync_dirty_bitmap(unsigned 
long *dest,
 return num_dirty;
 }
 
+/* One cell for each page. The n-th bit of a cell describes all the i-th vCPUs
+ * such that (i % EXCL_BITMAP_CELL_SZ) == n.
+ * A bit set to zero ensures that all the vCPUs described by the bit have the
+ * EXCL_BIT set for the page. */
+static inline void cpu_physical_memory_set_excl_dirty(ram_addr_t addr,
+  uint32_t cpu)
+{
+set_bit_atomic(EXCL_BITMAP_GET_BIT_OFFSET(addr) + EXCL_IDX(cpu),
+ram_list.dirty_memory[DIRTY_MEMORY_EXCLUSIVE]);
+}
+
+static inline int cpu_physical_memory_excl_atleast_one_clean(ram_addr_t addr)
+{
+uint8_t *bitmap;
+
+bitmap = (uint8_t *)(ram_list.dirty_memory[DIRTY_MEMORY_EXCLUSIVE]);
+
+/* This is safe even if smp_cpus < 8 since the unused bits are always 1. */
+return bitmap[EXCL_BITMAP_GET_BYTE_OFFSET(addr)] != UCHAR_MAX;
+}
+
+/* Return true if the @cpu has the bit set for the page of @addr.
+ * If @cpu == smp_cpus return 

Re: [Qemu-devel] [PATCH v7 09/14] block: Add block job transactions

[Fam Zheng]

On Tue, 09/22 15:09, John Snow wrote:
> 
> 
> On 09/21/2015 10:46 PM, Fam Zheng wrote:
> > Sometimes block jobs must execute as a transaction group.  Finishing
> > jobs wait until all other jobs are ready to complete successfully.
> > Failure or cancellation of one job cancels the other jobs in the group.
> > 
> > Signed-off-by: Stefan Hajnoczi 
> > [Rewrite the implementation which is now contained in block_job_completed.
> > --Fam]
> > Signed-off-by: Fam Zheng 
> > Reviewed-by: Max Reitz 
> > ---
> >  blockjob.c   | 135 
> > ++-
> >  include/block/block.h|   1 +
> >  include/block/blockjob.h |  38 +
> >  3 files changed, 172 insertions(+), 2 deletions(-)
> > 
> > diff --git a/blockjob.c b/blockjob.c
> > index 36c18e0..91e8d3c 100644
> > --- a/blockjob.c
> > +++ b/blockjob.c
> > @@ -36,6 +36,19 @@
> >  #include "qemu/timer.h"
> >  #include "qapi-event.h"
> >  
> > +/* Transactional group of block jobs */
> > +struct BlockJobTxn {
> > +
> > +/* Is this txn being cancelled? */
> > +bool aborting;
> > +
> > +/* List of jobs */
> > +QLIST_HEAD(, BlockJob) jobs;
> > +
> > +/* Reference count */
> > +int refcnt;
> > +};
> > +
> >  void *block_job_create(const BlockJobDriver *driver, BlockDriverState *bs,
> > int64_t speed, BlockCompletionFunc *cb,
> > void *opaque, Error **errp)
> > @@ -90,6 +103,86 @@ void block_job_unref(BlockJob *job)
> >  }
> >  }
> >  
> > +static void block_job_completed_single(BlockJob *job)
> > +{
> > +if (!job->ret) {
> > +if (job->driver->commit) {
> > +job->driver->commit(job);
> > +}
> > +} else {
> > +if (job->driver->abort) {
> > +job->driver->abort(job);
> > +}
> > +}
> > +job->cb(job->opaque, job->ret);
> > +if (job->txn) {
> > +block_job_txn_unref(job->txn);
> > +}
> > +block_job_unref(job);
> > +}
> > +
> > +static void block_job_completed_txn_abort(BlockJob *job)
> > +{
> > +AioContext *ctx;
> > +BlockJobTxn *txn = job->txn;
> > +BlockJob *other_job, *next;
> > +
> > +if (txn->aborting) {
> > +/*
> > + * We are cancelled by another job, which will handle everything.
> > + */
> > +return;
> > +}
> > +txn->aborting = true;
> > +/* We are the first failed job. Cancel other jobs. */
> > +QLIST_FOREACH(other_job, &txn->jobs, txn_list) {
> > +ctx = bdrv_get_aio_context(other_job->bs);
> > +aio_context_acquire(ctx);
> > +}
> > +QLIST_FOREACH(other_job, &txn->jobs, txn_list) {
> > +if (other_job == job || other_job->completed) {
> > +/* Other jobs are "effectively" cancelled by us, set the 
> > status for
> > + * them; this job, however, may or may not be cancelled, 
> > depending
> > + * on the caller, so leave it. */
> > +if (other_job != job) {
> > +other_job->cancelled = true;
> > +}
> > +continue;
> > +}
> > +block_job_cancel_sync(other_job);
> > +assert(other_job->completed);
> > +}
> > +QLIST_FOREACH_SAFE(other_job, &txn->jobs, txn_list, next) {
> > +ctx = bdrv_get_aio_context(other_job->bs);
> > +block_job_completed_single(other_job);
> > +aio_context_release(ctx);
> > +}
> > +}
> > +
> > +static void block_job_completed_txn_success(BlockJob *job)
> > +{
> > +AioContext *ctx;
> > +BlockJobTxn *txn = job->txn;
> > +BlockJob *other_job, *next;
> > +/*
> > + * Successful completion, see if there are other running jobs in this
> > + * txn.
> > + */
> > +QLIST_FOREACH(other_job, &txn->jobs, txn_list) {
> > +if (!other_job->completed) {
> > +return;
> > +}
> > +}
> > +/* We are the last completed job, commit the transaction. */
> > +QLIST_FOREACH_SAFE(other_job, &txn->jobs, txn_list, next) {
> > +ctx = bdrv_get_aio_context(other_job->bs);
> > +aio_context_acquire(ctx);
> > +assert(other_job->ret == 0);
> 
> Sorry for being a dense noggin about this, but is it documented anywhere
> (through an assertion or otherwise) that we will never return a
> positive, non-zero return code for a block job?
> 
> I just don't want to get into a situation where, in the future, someone
> decides to do so and then mysteriously something breaks a version or two
> later.

No, I don't think it's documented. And yes we should document it.

Anyway if someone decides to do so he will hit the assertion here immediately,
instead of a version or two later.

Fam

[Qemu-devel] [RFC v5 2/6] softmmu: Add new TLB_EXCL flag

[Alvise Rigo]

Add a new TLB flag to force all the accesses made to a page to follow
the slow-path.

In the case we remove a TLB entry marked as EXCL, we unset the
corresponding exclusive bit in the bitmap.

Suggested-by: Jani Kokkonen 
Suggested-by: Claudio Fontana 
Signed-off-by: Alvise Rigo 
---
 cputlb.c|  40 -
 include/exec/cpu-all.h  |   8 
 include/exec/cpu-defs.h |  12 ++
 softmmu_template.h  | 112 ++--
 4 files changed, 149 insertions(+), 23 deletions(-)

diff --git a/cputlb.c b/cputlb.c
index a506086..1e25a2a 100644
--- a/cputlb.c
+++ b/cputlb.c
@@ -299,6 +299,14 @@ void tlb_set_page_with_attrs(CPUState *cpu, target_ulong 
vaddr,
 env->tlb_v_table[mmu_idx][vidx] = *te;
 env->iotlb_v[mmu_idx][vidx] = env->iotlb[mmu_idx][index];
 
+if (unlikely(!(te->addr_write & TLB_MMIO) && (te->addr_write & TLB_EXCL))) 
{
+/* We are removing an exclusive entry, set the page to dirty. This
+ * is not be necessary if the vCPU has performed both SC and LL. */
+hwaddr hw_addr = (env->iotlb[mmu_idx][index].addr & TARGET_PAGE_MASK) +
+  (te->addr_write & TARGET_PAGE_MASK);
+cpu_physical_memory_set_excl_dirty(hw_addr, cpu->cpu_index);
+}
+
 /* refill the tlb */
 env->iotlb[mmu_idx][index].addr = iotlb - vaddr;
 env->iotlb[mmu_idx][index].attrs = attrs;
@@ -324,7 +332,15 @@ void tlb_set_page_with_attrs(CPUState *cpu, target_ulong 
vaddr,
+ xlat)) {
 te->addr_write = address | TLB_NOTDIRTY;
 } else {
-te->addr_write = address;
+if (!(address & TLB_MMIO) &&
+
cpu_physical_memory_excl_atleast_one_clean(section->mr->ram_addr
+   + xlat)) {
+/* There is at least one vCPU that has flagged the address as
+ * exclusive. */
+te->addr_write = address | TLB_EXCL;
+} else {
+te->addr_write = address;
+}
 }
 } else {
 te->addr_write = -1;
@@ -376,6 +392,28 @@ tb_page_addr_t get_page_addr_code(CPUArchState *env1, 
target_ulong addr)
 return qemu_ram_addr_from_host_nofail(p);
 }
 
+/* Atomic insn translation TLB support. */
+#define EXCLUSIVE_RESET_ADDR ULLONG_MAX
+/* For every vCPU compare the exclusive address and reset it in case of a
+ * match. Since only one vCPU is running at once, no lock has to be held to
+ * guard this operation. */
+static inline void lookup_and_reset_cpus_ll_addr(hwaddr addr, hwaddr size)
+{
+CPUState *cpu;
+CPUArchState *acpu;
+
+CPU_FOREACH(cpu) {
+acpu = (CPUArchState *)cpu->env_ptr;
+
+if (acpu->excl_protected_range.begin != EXCLUSIVE_RESET_ADDR &&
+ranges_overlap(acpu->excl_protected_range.begin,
+acpu->excl_protected_range.end - acpu->excl_protected_range.begin,
+addr, size)) {
+acpu->excl_protected_range.begin = EXCLUSIVE_RESET_ADDR;
+}
+}
+}
+
 #define MMUSUFFIX _mmu
 
 #define SHIFT 0
diff --git a/include/exec/cpu-all.h b/include/exec/cpu-all.h
index ea6a9a6..ad6afcb 100644
--- a/include/exec/cpu-all.h
+++ b/include/exec/cpu-all.h
@@ -320,6 +320,14 @@ extern RAMList ram_list;
 #define TLB_NOTDIRTY(1 << 4)
 /* Set if TLB entry is an IO callback.  */
 #define TLB_MMIO(1 << 5)
+/* Set if TLB entry references a page that requires exclusive access.  */
+#define TLB_EXCL(1 << 6)
+
+/* Do not allow a TARGET_PAGE_MASK which covers one or more bits defined
+ * above. */
+#if TLB_EXCL >= TARGET_PAGE_SIZE
+#error TARGET_PAGE_MASK covering the low bits of the TLB virtual address
+#endif
 
 void dump_exec_info(FILE *f, fprintf_function cpu_fprintf);
 void dump_opcount_info(FILE *f, fprintf_function cpu_fprintf);
diff --git a/include/exec/cpu-defs.h b/include/exec/cpu-defs.h
index 98b9cff..a67f295 100644
--- a/include/exec/cpu-defs.h
+++ b/include/exec/cpu-defs.h
@@ -27,6 +27,7 @@
 #include 
 #include "qemu/osdep.h"
 #include "qemu/queue.h"
+#include "qemu/range.h"
 #include "tcg-target.h"
 #ifndef CONFIG_USER_ONLY
 #include "exec/hwaddr.h"
@@ -150,5 +151,16 @@ typedef struct CPUIOTLBEntry {
 #define CPU_COMMON  \
 /* soft mmu support */  \
 CPU_COMMON_TLB  \
+\
+/* Used by the atomic insn translation backend. */  \
+int ll_sc_context;  \
+/* vCPU current exclusive addresses range.
+ * The address is set to EXCLUSIVE_RESET_ADDR if the vCPU is not.
+ * in the middle of a LL/SC. */ \
+struct Range excl_protected_range;

[Qemu-devel] [RFC v5 5/6] configure: Use slow-path for atomic only when the softmmu is enabled

[Alvise Rigo]

Use the new slow path for atomic instruction translation when the
softmmu is enabled.

Suggested-by: Jani Kokkonen 
Suggested-by: Claudio Fontana 
Signed-off-by: Alvise Rigo 
---
 configure | 11 +++
 1 file changed, 11 insertions(+)

diff --git a/configure b/configure
index cd219d8..5f72977 100755
--- a/configure
+++ b/configure
@@ -1391,6 +1391,13 @@ if test "$ARCH" = "unknown"; then
 fi
 fi
 
+# Use the slow-path for atomic instructions if the softmmu is enabled
+if test "$softmmu" = "yes"; then
+tcg_use_ldst_excl="yes"
+else
+tcg_use_ldst_excl="no"
+fi
+
 # Consult white-list to determine whether to enable werror
 # by default.  Only enable by default for git builds
 z_version=`cut -f3 -d. $source_path/VERSION`
@@ -4542,6 +4549,7 @@ echo "Install blobs $blobs"
 echo "KVM support   $kvm"
 echo "RDMA support  $rdma"
 echo "TCG interpreter   $tcg_interpreter"
+echo "use ld/st excl$tcg_use_ldst_excl"
 echo "fdt support   $fdt"
 echo "preadv support$preadv"
 echo "fdatasync $fdatasync"
@@ -4920,6 +4928,9 @@ fi
 if test "$tcg_interpreter" = "yes" ; then
   echo "CONFIG_TCG_INTERPRETER=y" >> $config_host_mak
 fi
+if test "$tcg_use_ldst_excl" = "yes" ; then
+  echo "CONFIG_TCG_USE_LDST_EXCL=y" >> $config_host_mak
+fi
 if test "$fdatasync" = "yes" ; then
   echo "CONFIG_FDATASYNC=y" >> $config_host_mak
 fi
-- 
2.5.3

Re: [Qemu-devel] [PATCH v11 01/12] qmp: delete qemu opts when delete an object

[Yang Hongyang]

On 09/24/2015 03:43 PM, Markus Armbruster wrote:

This has finally reached the front of my review queue.  I apologize for
the lng delay.

Copying Paolo for another pair of eyeballs (he wrote this code).


[...]

+
+opts = qemu_opts_find(qemu_find_opts_err("object", NULL), id);
+qemu_opts_del(opts);


qemu_find_opts_err("object", &error_abort) please, because when it
fails, we want to die right away, not when the null pointer it returns
gets dereferenced.


Thanks for the review.
Jason, do you want me to propose a fix on top of this series or simply drop
this for now because this patch is an independent bug fix and won't affect the
other filter patch series.



Same sloppiness in netdev_del_completion() and qmp_netdev_del(), not
your patch's fault.

Elsewhere, we store the QemuOpts in the object just so we can delete it:
DeviceState, DriveInfo.  Paolo, what do you think?


I don't get it. Currently, only objects created at the beginning through
QEMU command line will be stored in the QemuOpts, objects that created
with object_add won't stored in QemuOpts. Do you mean for DeviceState,
DriveInfo they store there QemuOpts explicity so that they can delete it?
Why don't we just delete it from objects directly instead?




  }

  MemoryDeviceInfoList *qmp_query_memory_devices(Error **errp)

.



--
Thanks,
Yang.

Re: [Qemu-devel] [PULL 0/7] target-arm queue

[Pavel Fedin]

 Hello!

 Thank you very much for your support and cooperation. I am back from my 
vacation and continuing my
work on live migration. Actually i already have working code, but need to 
settle down kernel API
first.

Kind regards,
Pavel Fedin
Expert Engineer
Samsung Electronics Research center Russia

Re: [Qemu-devel] [PATCH v11 02/12] init/cleanup of netfilter object

[Markus Armbruster]

Yang Hongyang  writes:

> Add a netfilter object based on QOM.
>
> A netfilter is attached to a netdev, captures all network packets
> that pass through the netdev. When we delete the netdev, we also
> delete the netfilter object attached to it, because if the netdev is
> removed, the filter which attached to it is useless.
>
> QTAILQ_ENTRY next used by netdev, filter belongs to the specific netdev is
> in this queue.

I don't get this paragraph.  Not sure it's needed.

> Also init delayed object after net_init_clients, because netfilters need
> to be initialized after net clients initialized.

A paragraph starting with "Also" in a commit message is a pretty good
sign the patch should be split :)

>
> Signed-off-by: Yang Hongyang 
> ---
> v11: no need to free nf->netdev_id, it will be auto freeed while object 
> deleted
>  remove global_list net_filters, will add back when needed
> v10: use QOM for netfilter
> v9: use flat union instead of simple union in QAPI schema
> v8: include vhost_net header
> v7: add check for vhost
> fix error propagate bug
> v6: add multiqueue support (net_filter_init1)
> v5: remove model from NetFilterState
> add a sent_cb param to receive_iov API
> ---
>  include/net/filter.h|  60 +
>  include/net/net.h   |   1 +
>  include/qemu/typedefs.h |   1 +
>  net/Makefile.objs   |   1 +
>  net/filter.c| 138 
> 
>  net/net.c   |   7 +++
>  qapi-schema.json|  18 +++
>  vl.c|  13 ++---
>  8 files changed, 233 insertions(+), 6 deletions(-)
>  create mode 100644 include/net/filter.h
>  create mode 100644 net/filter.c
>
> diff --git a/include/net/filter.h b/include/net/filter.h
> new file mode 100644
> index 000..226f2f7
> --- /dev/null
> +++ b/include/net/filter.h
> @@ -0,0 +1,60 @@
> +/*
> + * Copyright (c) 2015 FUJITSU LIMITED
> + * Author: Yang Hongyang 
> + *
> + * This work is licensed under the terms of the GNU GPL, version 2 or
> + * later.  See the COPYING file in the top-level directory.
> + */
> +
> +#ifndef QEMU_NET_FILTER_H
> +#define QEMU_NET_FILTER_H
> +
> +#include "qom/object.h"
> +#include "qemu-common.h"
> +#include "qemu/typedefs.h"
> +#include "net/queue.h"
> +
> +#define TYPE_NETFILTER "netfilter"
> +#define NETFILTER(obj) \
> +OBJECT_CHECK(NetFilterState, (obj), TYPE_NETFILTER)
> +#define NETFILTER_GET_CLASS(obj) \
> +OBJECT_GET_CLASS(NetFilterClass, (obj), TYPE_NETFILTER)
> +#define NETFILTER_CLASS(klass) \
> +OBJECT_CLASS_CHECK(NetFilterClass, (klass), TYPE_NETFILTER)
> +
> +typedef void (FilterSetup) (NetFilterState *nf, Error **errp);
> +typedef void (FilterCleanup) (NetFilterState *nf);
> +/*
> + * Return:
> + *   0: finished handling the packet, we should continue
> + *   size: filter stolen this packet, we stop pass this packet further
> + */
> +typedef ssize_t (FilterReceiveIOV)(NetFilterState *nc,
> +   NetClientState *sender,
> +   unsigned flags,
> +   const struct iovec *iov,
> +   int iovcnt,
> +   NetPacketSent *sent_cb);
> +
> +struct NetFilterClass {
> +ObjectClass parent_class;
> +
> +FilterSetup *setup;
> +FilterCleanup *cleanup;
> +FilterReceiveIOV *receive_iov;
> +};
> +typedef struct NetFilterClass NetFilterClass;

Not splitting the declaration is more concise:

typedef struct {
ObjectClass parent_class;
FilterSetup *setup;
FilterCleanup *cleanup;
FilterReceiveIOV *receive_iov;
} NetFilterClass;

Are any of the methods optional?  If yes, please add suitable comments.

> +
> +
> +struct NetFilterState {
> +/* private */
> +Object parent;
> +
> +/* protected */
> +char *netdev_id;
> +NetClientState *netdev;
> +NetFilterChain chain;
> +QTAILQ_ENTRY(NetFilterState) next;
> +};
> +
> +#endif /* QEMU_NET_FILTER_H */
> diff --git a/include/net/net.h b/include/net/net.h
> index 6a6cbef..36e5fab 100644
> --- a/include/net/net.h
> +++ b/include/net/net.h
> @@ -92,6 +92,7 @@ struct NetClientState {
>  NetClientDestructor *destructor;
>  unsigned int queue_index;
>  unsigned rxfilter_notify_enabled:1;
> +QTAILQ_HEAD(, NetFilterState) filters;
>  };
>  
>  typedef struct NICState {
> diff --git a/include/qemu/typedefs.h b/include/qemu/typedefs.h
> index f8a9dd6..2c0648f 100644
> --- a/include/qemu/typedefs.h
> +++ b/include/qemu/typedefs.h
> @@ -45,6 +45,7 @@ typedef struct Monitor Monitor;
>  typedef struct MouseTransformInfo MouseTransformInfo;
>  typedef struct MSIMessage MSIMessage;
>  typedef struct NetClientState NetClientState;
> +typedef struct NetFilterState NetFilterState;
>  typedef struct NICInfo NICInfo;
>  typedef struct PcGuestInfo PcGuestInfo;
>  typedef struct PCIBridge PCIBridge;
> diff --git a/net/Makefile.objs b/net/Makefile.ob

[Qemu-devel] [RfC PATCH 03/10] vnc: make the Buffer capacity increase in powers of two

[Gerd Hoffmann]

From: Peter Lieven 

This makes sure the number of reallocs is in O(log N).

Signed-off-by: Peter Lieven 

[ rebased to io/buffer.c ]

Signed-off-by: Gerd Hoffmann 
---
 io/buffer.c | 5 -
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/io/buffer.c b/io/buffer.c
index 68ae68d..6aa06e7 100644
--- a/io/buffer.c
+++ b/io/buffer.c
@@ -20,10 +20,13 @@
 
 #include "io/buffer.h"
 
+#define QIO_BUFFER_MIN_INIT_SIZE 4096
+
 void qio_buffer_reserve(QIOBuffer *buffer, size_t len)
 {
 if ((buffer->capacity - buffer->offset) < len) {
-buffer->capacity += (len + 1024);
+buffer->capacity = pow2ceil(buffer->offset + len);
+buffer->capacity = MAX(buffer->capacity, QIO_BUFFER_MIN_INIT_SIZE);
 buffer->buffer = g_realloc(buffer->buffer, buffer->capacity);
 }
 }
-- 
1.8.3.1

[Qemu-devel] [RfC PATCH 04/10] io: add qio_buffer_init

[Gerd Hoffmann]

Signed-off-by: Gerd Hoffmann 
---
 include/io/buffer.h | 12 
 io/buffer.c | 11 +++
 2 files changed, 23 insertions(+)

diff --git a/include/io/buffer.h b/include/io/buffer.h
index 2b1b261..cffad19 100644
--- a/include/io/buffer.h
+++ b/include/io/buffer.h
@@ -34,12 +34,24 @@ typedef struct QIOBuffer QIOBuffer;
  */
 
 struct QIOBuffer {
+char *name;
 size_t capacity;
 size_t offset;
 uint8_t *buffer;
 };
 
 /**
+ * qio_buffer_init:
+ * @buffer: the buffer object
+ * @name: buffer name
+ *
+ * Optionally attach a name to the buffer, to make it easier
+ * to identify in debug traces.
+ */
+void qio_buffer_init(QIOBuffer *buffer, const char *name, ...)
+GCC_FMT_ATTR(2, 3);
+
+/**
  * qio_buffer_reserve:
  * @buffer: the buffer object
  * @len: the minimum required free space
diff --git a/io/buffer.c b/io/buffer.c
index 6aa06e7..daa3ebf 100644
--- a/io/buffer.c
+++ b/io/buffer.c
@@ -22,6 +22,15 @@
 
 #define QIO_BUFFER_MIN_INIT_SIZE 4096
 
+void qio_buffer_init(QIOBuffer *buffer, const char *name, ...)
+{
+va_list ap;
+
+va_start(ap, name);
+buffer->name = g_strdup_vprintf(name, ap);
+va_end(ap);
+}
+
 void qio_buffer_reserve(QIOBuffer *buffer, size_t len)
 {
 if ((buffer->capacity - buffer->offset) < len) {
@@ -49,9 +58,11 @@ void qio_buffer_reset(QIOBuffer *buffer)
 void qio_buffer_free(QIOBuffer *buffer)
 {
 g_free(buffer->buffer);
+g_free(buffer->name);
 buffer->offset = 0;
 buffer->capacity = 0;
 buffer->buffer = NULL;
+buffer->name = NULL;
 }
 
 void qio_buffer_append(QIOBuffer *buffer, const void *data, size_t len)
-- 
1.8.3.1

[Qemu-devel] [RfC PATCH 06/10] io: add qio_buffer_move

[Gerd Hoffmann]

Signed-off-by: Gerd Hoffmann 
---
 include/io/buffer.h | 10 ++
 io/buffer.c | 16 
 2 files changed, 26 insertions(+)

diff --git a/include/io/buffer.h b/include/io/buffer.h
index 1dddc73..5676aff 100644
--- a/include/io/buffer.h
+++ b/include/io/buffer.h
@@ -137,4 +137,14 @@ gboolean qio_buffer_empty(QIOBuffer *buffer);
  */
 void qio_buffer_move_empty(QIOBuffer *to, QIOBuffer *from);
 
+/**
+ * qio_buffer_move:
+ * @to: destination buffer object
+ * @from: source buffer object
+ *
+ * Moves buffer, copying data (unless 'to' buffer happens to be empty).
+ * 'from' buffer is empty and zero-sized on return.
+ */
+void qio_buffer_move(QIOBuffer *to, QIOBuffer *from);
+
 #endif /* QIO_BUFFER_H__ */
diff --git a/io/buffer.c b/io/buffer.c
index 09ca321..96077d3 100644
--- a/io/buffer.c
+++ b/io/buffer.c
@@ -91,3 +91,19 @@ void qio_buffer_move_empty(QIOBuffer *to, QIOBuffer *from)
 from->capacity = 0;
 from->buffer = NULL;
 }
+
+void qio_buffer_move(QIOBuffer *to, QIOBuffer *from)
+{
+if (to->offset == 0) {
+qio_buffer_move_empty(to, from);
+return;
+}
+
+qio_buffer_reserve(to, from->offset);
+qio_buffer_append(to, from->buffer, from->offset);
+
+g_free(from->buffer);
+from->offset = 0;
+from->capacity = 0;
+from->buffer = NULL;
+}
-- 
1.8.3.1

[Qemu-devel] [RfC PATCH 08/10] name vnc buffers

[Gerd Hoffmann]

Signed-off-by: Gerd Hoffmann 
---
 ui/vnc-jobs.c |  1 +
 ui/vnc.c  | 16 
 2 files changed, 17 insertions(+)

diff --git a/ui/vnc-jobs.c b/ui/vnc-jobs.c
index 9824c34..7a234da 100644
--- a/ui/vnc-jobs.c
+++ b/ui/vnc-jobs.c
@@ -303,6 +303,7 @@ static VncJobQueue *vnc_queue_init(void)
 
 qemu_cond_init(&queue->cond);
 qemu_mutex_init(&queue->mutex);
+qio_buffer_init(&queue->buffer, "vnc-job-queue");
 QTAILQ_INIT(&queue->jobs);
 return queue;
 }
diff --git a/ui/vnc.c b/ui/vnc.c
index e4b0e3a..fc0311f 100644
--- a/ui/vnc.c
+++ b/ui/vnc.c
@@ -2978,6 +2978,22 @@ static void vnc_connect(VncDisplay *vd, int csock,
 vs->csock = csock;
 vs->vd = vd;
 
+qio_buffer_init(&vs->input,  "vnc-input/%d", csock);
+qio_buffer_init(&vs->output, "vnc-output/%d", csock);
+qio_buffer_init(&vs->ws_input,   "vnc-ws_input/%d", csock);
+qio_buffer_init(&vs->ws_output,  "vnc-ws_output/%d", csock);
+qio_buffer_init(&vs->jobs_buffer,"vnc-jobs_buffer/%d", csock);
+
+qio_buffer_init(&vs->tight.tight,"vnc-tight/%d", csock);
+qio_buffer_init(&vs->tight.zlib, "vnc-tight-zlib/%d", csock);
+qio_buffer_init(&vs->tight.gradient, "vnc-tight-gradient/%d", csock);
+qio_buffer_init(&vs->tight.jpeg, "vnc-tight-jpeg/%d", csock);
+qio_buffer_init(&vs->tight.png,  "vnc-tight-png/%d", csock);
+qio_buffer_init(&vs->zlib.zlib,  "vnc-zlib/%d", csock);
+qio_buffer_init(&vs->zrle.zrle,  "vnc-zrle/%d", csock);
+qio_buffer_init(&vs->zrle.fb,"vnc-zrle-fb/%d", csock);
+qio_buffer_init(&vs->zrle.zlib,  "vnc-zrle-zlib/%d", csock);
+
 if (skipauth) {
vs->auth = VNC_AUTH_NONE;
vs->subauth = VNC_AUTH_INVALID;
-- 
1.8.3.1

[Qemu-devel] [RfC PATCH 00/10] vnc buffer handling

[Gerd Hoffmann]

  Hi,

Here is a patch series to improve the vnc buffer handling.  It picks up
the qio_buffer patches from Daniel, adds move calls (move data from one
buffer to another) and tracing, makes vnc use the new features.  Net
effect should be that (a) vnc copies less data around and (b) buffers
don't grow forever.

It's RfC because it depends on wip patches.  My plan is to wait for
Daniels patch series to be merged (which should obsolete patches #1+#2),
then rebase and repost the series.

Patches are also available from git:
  git://git.kraxel.org/qemu rebase/ui-vnc-next

please test & review,
  Gerd

Daniel P. Berrange (1):
  io: pull Buffer code out of VNC module

Gerd Hoffmann (8):
  io/ makefile fluff
  io: add qio_buffer_init
  io: add qio_buffer_move_empty
  io: add qio_buffer_move
  io: add qio_buffer tracing
  name vnc buffers
  vnc: kill jobs queue buffer
  vnc-jobs: move buffer reset, use new buffer move

Peter Lieven (1):
  vnc: make the Buffer capacity increase in powers of two

 Makefile|   2 +
 Makefile.objs   |   5 ++
 Makefile.target |   2 +
 include/io/buffer.h | 150 
 io/Makefile.objs|   1 +
 io/buffer.c | 119 +
 trace-events|   6 +++
 ui/vnc-auth-sasl.c  |   4 +-
 ui/vnc-enc-tight.c  |  38 ++---
 ui/vnc-enc-zlib.c   |   6 +--
 ui/vnc-enc-zrle.c   |  18 +++
 ui/vnc-jobs.c   |  16 ++
 ui/vnc-ws.c |  36 ++---
 ui/vnc-ws.h |   6 +--
 ui/vnc.c|  83 ++---
 ui/vnc.h|  50 +++---
 16 files changed, 389 insertions(+), 153 deletions(-)
 create mode 100644 include/io/buffer.h
 create mode 100644 io/Makefile.objs
 create mode 100644 io/buffer.c

-- 
1.8.3.1

[Qemu-devel] [RfC PATCH 07/10] io: add qio_buffer tracing

[Gerd Hoffmann]

Signed-off-by: Gerd Hoffmann 
---
 io/buffer.c  | 10 ++
 trace-events |  6 ++
 2 files changed, 16 insertions(+)

diff --git a/io/buffer.c b/io/buffer.c
index 96077d3..05425c2 100644
--- a/io/buffer.c
+++ b/io/buffer.c
@@ -19,6 +19,7 @@
  */
 
 #include "io/buffer.h"
+#include "trace.h"
 
 #define QIO_BUFFER_MIN_INIT_SIZE 4096
 
@@ -37,6 +38,7 @@ void qio_buffer_reserve(QIOBuffer *buffer, size_t len)
 buffer->capacity = pow2ceil(buffer->offset + len);
 buffer->capacity = MAX(buffer->capacity, QIO_BUFFER_MIN_INIT_SIZE);
 buffer->buffer = g_realloc(buffer->buffer, buffer->capacity);
+trace_qio_buffer_resize(buffer->name ?: "unnamed", buffer->capacity);
 }
 }
 
@@ -57,6 +59,7 @@ void qio_buffer_reset(QIOBuffer *buffer)
 
 void qio_buffer_free(QIOBuffer *buffer)
 {
+trace_qio_buffer_free(buffer->name ?: "unnamed");
 g_free(buffer->buffer);
 g_free(buffer->name);
 buffer->offset = 0;
@@ -80,6 +83,9 @@ void qio_buffer_advance(QIOBuffer *buffer, size_t len)
 
 void qio_buffer_move_empty(QIOBuffer *to, QIOBuffer *from)
 {
+trace_qio_buffer_move_empty(to->name ?: "unnamed",
+from->offset,
+from->name ?: "unnamed");
 assert(to->offset == 0);
 
 g_free(to->buffer);
@@ -99,6 +105,10 @@ void qio_buffer_move(QIOBuffer *to, QIOBuffer *from)
 return;
 }
 
+trace_qio_buffer_move(to->name ?: "unnamed",
+  from->offset,
+  from->name ?: "unnamed");
+
 qio_buffer_reserve(to, from->offset);
 qio_buffer_append(to, from->buffer, from->offset);
 
diff --git a/trace-events b/trace-events
index 88a2f14..0f06b64 100644
--- a/trace-events
+++ b/trace-events
@@ -1376,6 +1376,12 @@ spapr_iommu_new_table(uint64_t liobn, void *tcet, void 
*table, int fd) "liobn=%"
 # hw/ppc/ppc.c
 ppc_tb_adjust(uint64_t offs1, uint64_t offs2, int64_t diff, int64_t seconds) 
"adjusted from 0x%"PRIx64" to 0x%"PRIx64", diff %"PRId64" (%"PRId64"s)"
 
+# io/buffer.c
+qio_buffer_resize(const char *buf, size_t len) "%s: len %zd"
+qio_buffer_move_empty(const char *buf, size_t len, const char *from) "%s: %zd 
bytes from %s"
+qio_buffer_move(const char *buf, size_t len, const char *from) "%s: %zd bytes 
from %s"
+qio_buffer_free(const char *buf) "%s"
+
 # util/hbitmap.c
 hbitmap_iter_skip_words(const void *hb, void *hbi, uint64_t pos, unsigned long 
cur) "hb %p hbi %p pos %"PRId64" cur 0x%lx"
 hbitmap_reset(void *hb, uint64_t start, uint64_t count, uint64_t sbit, 
uint64_t ebit) "hb %p items %"PRIu64",%"PRIu64" bits %"PRIu64"..%"PRIu64
-- 
1.8.3.1

[Qemu-devel] [RfC PATCH 01/10] io/ makefile fluff

[Gerd Hoffmann]

Signed-off-by: Gerd Hoffmann 
---
 Makefile| 2 ++
 Makefile.objs   | 5 +
 Makefile.target | 2 ++
 3 files changed, 9 insertions(+)

diff --git a/Makefile b/Makefile
index 8ec9b69..ca18209 100644
--- a/Makefile
+++ b/Makefile
@@ -157,6 +157,7 @@ dummy := $(call unnest-vars,, \
 crypto-obj-y \
 crypto-aes-obj-y \
 qom-obj-y \
+io-obj-y \
 common-obj-y \
 common-obj-m)
 
@@ -180,6 +181,7 @@ SOFTMMU_SUBDIR_RULES=$(filter %-softmmu,$(SUBDIR_RULES))
 $(SOFTMMU_SUBDIR_RULES): $(block-obj-y)
 $(SOFTMMU_SUBDIR_RULES): $(crypto-obj-y)
 $(SOFTMMU_SUBDIR_RULES): $(qom-obj-y)
+$(SOFTMMU_SUBDIR_RULES): $(io-obj-y)
 $(SOFTMMU_SUBDIR_RULES): config-all-devices.mak
 
 subdir-%:
diff --git a/Makefile.objs b/Makefile.objs
index ce87778..8c9461e 100644
--- a/Makefile.objs
+++ b/Makefile.objs
@@ -32,6 +32,11 @@ crypto-aes-obj-y = crypto/
 
 qom-obj-y = qom/
 
+###
+# io-obj-y is code used by both qemu system emulation and qemu-img
+
+io-obj-y = io/
+
 ##
 # smartcard
 
diff --git a/Makefile.target b/Makefile.target
index 962d004..34ddb7e 100644
--- a/Makefile.target
+++ b/Makefile.target
@@ -176,6 +176,7 @@ dummy := $(call unnest-vars,.., \
crypto-obj-y \
crypto-aes-obj-y \
qom-obj-y \
+   io-obj-y \
common-obj-y \
common-obj-m)
 target-obj-y := $(target-obj-y-save)
@@ -185,6 +186,7 @@ all-obj-y += $(qom-obj-y)
 all-obj-$(CONFIG_SOFTMMU) += $(block-obj-y)
 all-obj-$(CONFIG_USER_ONLY) += $(crypto-aes-obj-y)
 all-obj-$(CONFIG_SOFTMMU) += $(crypto-obj-y)
+all-obj-$(CONFIG_SOFTMMU) += $(io-obj-y)
 
 $(QEMU_PROG_BUILD): config-devices.mak
 
-- 
1.8.3.1

[Qemu-devel] [RfC PATCH 02/10] io: pull Buffer code out of VNC module

[Gerd Hoffmann]

From: "Daniel P. Berrange" 

The Buffer code in the VNC server is useful for the IO channel
code, so pull it out into a shared module, QIOBuffer.

Signed-off-by: Daniel P. Berrange 
---
 include/io/buffer.h | 118 
 io/Makefile.objs|   1 +
 io/buffer.c |  65 +
 ui/vnc-auth-sasl.c  |   4 +-
 ui/vnc-enc-tight.c  |  38 -
 ui/vnc-enc-zlib.c   |   6 +--
 ui/vnc-enc-zrle.c   |  18 
 ui/vnc-jobs.c   |  15 +++
 ui/vnc-ws.c |  36 
 ui/vnc-ws.h |   6 +--
 ui/vnc.c|  67 ++---
 ui/vnc.h|  50 --
 12 files changed, 276 insertions(+), 148 deletions(-)
 create mode 100644 include/io/buffer.h
 create mode 100644 io/Makefile.objs
 create mode 100644 io/buffer.c

diff --git a/include/io/buffer.h b/include/io/buffer.h
new file mode 100644
index 000..2b1b261
--- /dev/null
+++ b/include/io/buffer.h
@@ -0,0 +1,118 @@
+/*
+ * QEMU I/O buffers
+ *
+ * Copyright (c) 2015 Red Hat, Inc.
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see .
+ *
+ */
+
+#ifndef QIO_BUFFER_H__
+#define QIO_BUFFER_H__
+
+#include "qemu-common.h"
+
+typedef struct QIOBuffer QIOBuffer;
+
+/**
+ * QIOBuffer:
+ *
+ * The QIOBuffer object provides a simple dynamically resizing
+ * array, with separate tracking of capacity and usage. This
+ * is typically useful when buffering I/O data.
+ */
+
+struct QIOBuffer {
+size_t capacity;
+size_t offset;
+uint8_t *buffer;
+};
+
+/**
+ * qio_buffer_reserve:
+ * @buffer: the buffer object
+ * @len: the minimum required free space
+ *
+ * Ensure that the buffer has space allocated for at least
+ * @len bytes. If the current buffer is too small, it will
+ * be reallocated, possibly to a larger size than requested.
+ */
+void qio_buffer_reserve(QIOBuffer *buffer, size_t len);
+
+/**
+ * qio_buffer_reset:
+ * @buffer: the buffer object
+ *
+ * Reset the length of the stored data to zero, but do
+ * not free / reallocate the memory buffer
+ */
+void qio_buffer_reset(QIOBuffer *buffer);
+
+/**
+ * qio_buffer_free:
+ * @buffer: the buffer object
+ *
+ * Reset the length of the stored data to zero and also
+ * free the internal memory buffer
+ */
+void qio_buffer_free(QIOBuffer *buffer);
+
+/**
+ * qio_buffer_append:
+ * @buffer: the buffer object
+ * @data: the data block to append
+ * @len: the length of @data in bytes
+ *
+ * Append the contents of @data to the end of the buffer.
+ * The caller must ensure that the buffer has sufficient
+ * free space for @len bytes, typically by calling the
+ * qio_buffer_reserve() method prior to appending.
+ */
+void qio_buffer_append(QIOBuffer *buffer, const void *data, size_t len);
+
+/**
+ * qio_buffer_advance:
+ * @buffer: the buffer object
+ * @len: the number of bytes to skip
+ *
+ * Remove @len bytes of data from the head of the buffer.
+ * The internal buffer will not be reallocated, so will
+ * have at least @len bytes of free space after this
+ * call completes
+ */
+void qio_buffer_advance(QIOBuffer *buffer, size_t len);
+
+/**
+ * qio_buffer_end:
+ * @buffer: the buffer object
+ *
+ * Get a pointer to the tail end of the internal buffer
+ * The returned pointer is only valid until the next
+ * call to qio_buffer_reserve().
+ *
+ * Returns: the tail of the buffer
+ */
+uint8_t *qio_buffer_end(QIOBuffer *buffer);
+
+/**
+ * qio_buffer_empty:
+ * @buffer: the buffer object
+ *
+ * Determine if the buffer contains any current data
+ *
+ * Returns: true if the buffer holds data, false otherwise
+ */
+gboolean qio_buffer_empty(QIOBuffer *buffer);
+
+#endif /* QIO_BUFFER_H__ */
diff --git a/io/Makefile.objs b/io/Makefile.objs
new file mode 100644
index 000..3de4e47
--- /dev/null
+++ b/io/Makefile.objs
@@ -0,0 +1 @@
+io-obj-y = buffer.o
diff --git a/io/buffer.c b/io/buffer.c
new file mode 100644
index 000..68ae68d
--- /dev/null
+++ b/io/buffer.c
@@ -0,0 +1,65 @@
+/*
+ * QEMU I/O buffers
+ *
+ * Copyright (c) 2015 Red Hat, Inc.
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in

[Qemu-devel] [RfC PATCH 05/10] io: add qio_buffer_move_empty

[Gerd Hoffmann]

Signed-off-by: Gerd Hoffmann 
---
 include/io/buffer.h | 10 ++
 io/buffer.c | 14 ++
 2 files changed, 24 insertions(+)

diff --git a/include/io/buffer.h b/include/io/buffer.h
index cffad19..1dddc73 100644
--- a/include/io/buffer.h
+++ b/include/io/buffer.h
@@ -127,4 +127,14 @@ uint8_t *qio_buffer_end(QIOBuffer *buffer);
  */
 gboolean qio_buffer_empty(QIOBuffer *buffer);
 
+/**
+ * qio_buffer_move_empty:
+ * @to: destination buffer object
+ * @from: source buffer object
+ *
+ * Moves buffer, without copying data.  'to' buffer must be empty.
+ * 'from' buffer is empty and zero-sized on return.
+ */
+void qio_buffer_move_empty(QIOBuffer *to, QIOBuffer *from);
+
 #endif /* QIO_BUFFER_H__ */
diff --git a/io/buffer.c b/io/buffer.c
index daa3ebf..09ca321 100644
--- a/io/buffer.c
+++ b/io/buffer.c
@@ -77,3 +77,17 @@ void qio_buffer_advance(QIOBuffer *buffer, size_t len)
 (buffer->offset - len));
 buffer->offset -= len;
 }
+
+void qio_buffer_move_empty(QIOBuffer *to, QIOBuffer *from)
+{
+assert(to->offset == 0);
+
+g_free(to->buffer);
+to->offset = from->offset;
+to->capacity = from->capacity;
+to->buffer = from->buffer;
+
+from->offset = 0;
+from->capacity = 0;
+from->buffer = NULL;
+}
-- 
1.8.3.1

[Qemu-devel] [RfC PATCH 09/10] vnc: kill jobs queue buffer

[Gerd Hoffmann]

Signed-off-by: Gerd Hoffmann 
---
 ui/vnc-jobs.c | 6 --
 1 file changed, 6 deletions(-)

diff --git a/ui/vnc-jobs.c b/ui/vnc-jobs.c
index 7a234da..50e6b37 100644
--- a/ui/vnc-jobs.c
+++ b/ui/vnc-jobs.c
@@ -54,7 +54,6 @@ struct VncJobQueue {
 QemuCond cond;
 QemuMutex mutex;
 QemuThread thread;
-QIOBuffer buffer;
 bool exit;
 QTAILQ_HEAD(, VncJob) jobs;
 };
@@ -193,7 +192,6 @@ static void vnc_async_encoding_start(VncState *orig, 
VncState *local)
 local->zlib = orig->zlib;
 local->hextile = orig->hextile;
 local->zrle = orig->zrle;
-local->output =  queue->buffer;
 local->csock = -1; /* Don't do any network work on this thread */
 
 qio_buffer_reset(&local->output);
@@ -206,8 +204,6 @@ static void vnc_async_encoding_end(VncState *orig, VncState 
*local)
 orig->hextile = local->hextile;
 orig->zrle = local->zrle;
 orig->lossy_rect = local->lossy_rect;
-
-queue->buffer = local->output;
 }
 
 static int vnc_worker_thread_loop(VncJobQueue *queue)
@@ -303,7 +299,6 @@ static VncJobQueue *vnc_queue_init(void)
 
 qemu_cond_init(&queue->cond);
 qemu_mutex_init(&queue->mutex);
-qio_buffer_init(&queue->buffer, "vnc-job-queue");
 QTAILQ_INIT(&queue->jobs);
 return queue;
 }
@@ -312,7 +307,6 @@ static void vnc_queue_clear(VncJobQueue *q)
 {
 qemu_cond_destroy(&queue->cond);
 qemu_mutex_destroy(&queue->mutex);
-qio_buffer_free(&queue->buffer);
 g_free(q);
 queue = NULL; /* Unset global queue */
 }
-- 
1.8.3.1

Re: [Qemu-devel] [PATCH v11 02/12] init/cleanup of netfilter object

[Yang Hongyang]

On 09/24/2015 04:41 PM, Markus Armbruster wrote:

Yang Hongyang  writes:


Add a netfilter object based on QOM.

A netfilter is attached to a netdev, captures all network packets
that pass through the netdev. When we delete the netdev, we also
delete the netfilter object attached to it, because if the netdev is
removed, the filter which attached to it is useless.

QTAILQ_ENTRY next used by netdev, filter belongs to the specific netdev is
in this queue.


I don't get this paragraph.  Not sure it's needed.


Also init delayed object after net_init_clients, because netfilters need
to be initialized after net clients initialized.


A paragraph starting with "Also" in a commit message is a pretty good
sign the patch should be split :)



Signed-off-by: Yang Hongyang 
---
v11: no need to free nf->netdev_id, it will be auto freeed while object deleted
  remove global_list net_filters, will add back when needed
v10: use QOM for netfilter
v9: use flat union instead of simple union in QAPI schema
v8: include vhost_net header
v7: add check for vhost
 fix error propagate bug
v6: add multiqueue support (net_filter_init1)
v5: remove model from NetFilterState
 add a sent_cb param to receive_iov API
---
  include/net/filter.h|  60 +
  include/net/net.h   |   1 +
  include/qemu/typedefs.h |   1 +
  net/Makefile.objs   |   1 +
  net/filter.c| 138 
  net/net.c   |   7 +++
  qapi-schema.json|  18 +++
  vl.c|  13 ++---
  8 files changed, 233 insertions(+), 6 deletions(-)
  create mode 100644 include/net/filter.h
  create mode 100644 net/filter.c

diff --git a/include/net/filter.h b/include/net/filter.h
new file mode 100644
index 000..226f2f7
--- /dev/null
+++ b/include/net/filter.h
@@ -0,0 +1,60 @@
+/*
+ * Copyright (c) 2015 FUJITSU LIMITED
+ * Author: Yang Hongyang 
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or
+ * later.  See the COPYING file in the top-level directory.
+ */
+
+#ifndef QEMU_NET_FILTER_H
+#define QEMU_NET_FILTER_H
+
+#include "qom/object.h"
+#include "qemu-common.h"
+#include "qemu/typedefs.h"
+#include "net/queue.h"
+
+#define TYPE_NETFILTER "netfilter"
+#define NETFILTER(obj) \
+OBJECT_CHECK(NetFilterState, (obj), TYPE_NETFILTER)
+#define NETFILTER_GET_CLASS(obj) \
+OBJECT_GET_CLASS(NetFilterClass, (obj), TYPE_NETFILTER)
+#define NETFILTER_CLASS(klass) \
+OBJECT_CLASS_CHECK(NetFilterClass, (klass), TYPE_NETFILTER)
+
+typedef void (FilterSetup) (NetFilterState *nf, Error **errp);
+typedef void (FilterCleanup) (NetFilterState *nf);
+/*
+ * Return:
+ *   0: finished handling the packet, we should continue
+ *   size: filter stolen this packet, we stop pass this packet further
+ */
+typedef ssize_t (FilterReceiveIOV)(NetFilterState *nc,
+   NetClientState *sender,
+   unsigned flags,
+   const struct iovec *iov,
+   int iovcnt,
+   NetPacketSent *sent_cb);
+
+struct NetFilterClass {
+ObjectClass parent_class;
+
+FilterSetup *setup;
+FilterCleanup *cleanup;
+FilterReceiveIOV *receive_iov;
+};
+typedef struct NetFilterClass NetFilterClass;


Not splitting the declaration is more concise:

 typedef struct {
 ObjectClass parent_class;
 FilterSetup *setup;
 FilterCleanup *cleanup;
 FilterReceiveIOV *receive_iov;
 } NetFilterClass;

Are any of the methods optional?  If yes, please add suitable comments.


Hi Markus, I split it because the checkpatch.pl told me to do so...




+
+
+struct NetFilterState {
+/* private */
+Object parent;
+
+/* protected */
+char *netdev_id;
+NetClientState *netdev;
+NetFilterChain chain;
+QTAILQ_ENTRY(NetFilterState) next;
+};
+
+#endif /* QEMU_NET_FILTER_H */
diff --git a/include/net/net.h b/include/net/net.h
index 6a6cbef..36e5fab 100644
--- a/include/net/net.h
+++ b/include/net/net.h
@@ -92,6 +92,7 @@ struct NetClientState {
  NetClientDestructor *destructor;
  unsigned int queue_index;
  unsigned rxfilter_notify_enabled:1;
+QTAILQ_HEAD(, NetFilterState) filters;
  };

  typedef struct NICState {
diff --git a/include/qemu/typedefs.h b/include/qemu/typedefs.h
index f8a9dd6..2c0648f 100644
--- a/include/qemu/typedefs.h
+++ b/include/qemu/typedefs.h
@@ -45,6 +45,7 @@ typedef struct Monitor Monitor;
  typedef struct MouseTransformInfo MouseTransformInfo;
  typedef struct MSIMessage MSIMessage;
  typedef struct NetClientState NetClientState;
+typedef struct NetFilterState NetFilterState;
  typedef struct NICInfo NICInfo;
  typedef struct PcGuestInfo PcGuestInfo;
  typedef struct PCIBridge PCIBridge;
diff --git a/net/Makefile.objs b/net/Makefile.objs
index ec19cb3..914aec0 100644
--- a/net/Makefile.objs
+++ b/net/Makefile.objs
@@ -13,3 +13,4 

[Qemu-devel] [RfC PATCH 10/10] vnc-jobs: move buffer reset, use new buffer move

[Gerd Hoffmann]

Signed-off-by: Gerd Hoffmann 
---
 ui/vnc-jobs.c | 10 +++---
 1 file changed, 3 insertions(+), 7 deletions(-)

diff --git a/ui/vnc-jobs.c b/ui/vnc-jobs.c
index 50e6b37..dfc5139 100644
--- a/ui/vnc-jobs.c
+++ b/ui/vnc-jobs.c
@@ -165,8 +165,7 @@ void vnc_jobs_consume_buffer(VncState *vs)
 
 vnc_lock_output(vs);
 if (vs->jobs_buffer.offset) {
-vnc_write(vs, vs->jobs_buffer.buffer, vs->jobs_buffer.offset);
-qio_buffer_reset(&vs->jobs_buffer);
+qio_buffer_move(&vs->output, &vs->jobs_buffer);
 }
 flush = vs->csock != -1 && vs->abort != true;
 vnc_unlock_output(vs);
@@ -193,8 +192,6 @@ static void vnc_async_encoding_start(VncState *orig, 
VncState *local)
 local->hextile = orig->hextile;
 local->zrle = orig->zrle;
 local->csock = -1; /* Don't do any network work on this thread */
-
-qio_buffer_reset(&local->output);
 }
 
 static void vnc_async_encoding_end(VncState *orig, VncState *local)
@@ -271,14 +268,13 @@ static int vnc_worker_thread_loop(VncJobQueue *queue)
 vnc_lock_output(job->vs);
 
 if (job->vs->csock != -1) {
-qio_buffer_reserve(&job->vs->jobs_buffer, vs.output.offset);
-qio_buffer_append(&job->vs->jobs_buffer, vs.output.buffer,
-  vs.output.offset);
+qio_buffer_move(&job->vs->jobs_buffer, &vs.output);
 /* Copy persistent encoding data */
 vnc_async_encoding_end(job->vs, &vs);
 
qemu_bh_schedule(job->vs->bh);
 }  else {
+qio_buffer_reset(&vs.output);
 /* Copy persistent encoding data */
 vnc_async_encoding_end(job->vs, &vs);
 }
-- 
1.8.3.1

Re: [Qemu-devel] [PATCH v11 02/12] init/cleanup of netfilter object

[Yang Hongyang]

On 09/24/2015 04:41 PM, Markus Armbruster wrote:

Yang Hongyang  writes:


Add a netfilter object based on QOM.

A netfilter is attached to a netdev, captures all network packets
that pass through the netdev. When we delete the netdev, we also
delete the netfilter object attached to it, because if the netdev is
removed, the filter which attached to it is useless.

QTAILQ_ENTRY next used by netdev, filter belongs to the specific netdev is
in this queue.


I don't get this paragraph.  Not sure it's needed.


Also init delayed object after net_init_clients, because netfilters need
to be initialized after net clients initialized.


A paragraph starting with "Also" in a commit message is a pretty good
sign the patch should be split :)



Signed-off-by: Yang Hongyang 
---
v11: no need to free nf->netdev_id, it will be auto freeed while object deleted
  remove global_list net_filters, will add back when needed
v10: use QOM for netfilter
v9: use flat union instead of simple union in QAPI schema
v8: include vhost_net header
v7: add check for vhost
 fix error propagate bug
v6: add multiqueue support (net_filter_init1)
v5: remove model from NetFilterState
 add a sent_cb param to receive_iov API
---
  include/net/filter.h|  60 +
  include/net/net.h   |   1 +
  include/qemu/typedefs.h |   1 +
  net/Makefile.objs   |   1 +
  net/filter.c| 138 
  net/net.c   |   7 +++
  qapi-schema.json|  18 +++
  vl.c|  13 ++---
  8 files changed, 233 insertions(+), 6 deletions(-)
  create mode 100644 include/net/filter.h
  create mode 100644 net/filter.c

diff --git a/include/net/filter.h b/include/net/filter.h
new file mode 100644
index 000..226f2f7
--- /dev/null
+++ b/include/net/filter.h
@@ -0,0 +1,60 @@
+/*
+ * Copyright (c) 2015 FUJITSU LIMITED
+ * Author: Yang Hongyang 
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or
+ * later.  See the COPYING file in the top-level directory.
+ */
+
+#ifndef QEMU_NET_FILTER_H
+#define QEMU_NET_FILTER_H
+
+#include "qom/object.h"
+#include "qemu-common.h"
+#include "qemu/typedefs.h"
+#include "net/queue.h"
+
+#define TYPE_NETFILTER "netfilter"
+#define NETFILTER(obj) \
+OBJECT_CHECK(NetFilterState, (obj), TYPE_NETFILTER)
+#define NETFILTER_GET_CLASS(obj) \
+OBJECT_GET_CLASS(NetFilterClass, (obj), TYPE_NETFILTER)
+#define NETFILTER_CLASS(klass) \
+OBJECT_CLASS_CHECK(NetFilterClass, (klass), TYPE_NETFILTER)
+
+typedef void (FilterSetup) (NetFilterState *nf, Error **errp);
+typedef void (FilterCleanup) (NetFilterState *nf);
+/*
+ * Return:
+ *   0: finished handling the packet, we should continue
+ *   size: filter stolen this packet, we stop pass this packet further
+ */
+typedef ssize_t (FilterReceiveIOV)(NetFilterState *nc,
+   NetClientState *sender,
+   unsigned flags,
+   const struct iovec *iov,
+   int iovcnt,
+   NetPacketSent *sent_cb);
+
+struct NetFilterClass {
+ObjectClass parent_class;
+
+FilterSetup *setup;
+FilterCleanup *cleanup;
+FilterReceiveIOV *receive_iov;
+};
+typedef struct NetFilterClass NetFilterClass;


Not splitting the declaration is more concise:

 typedef struct {
 ObjectClass parent_class;
 FilterSetup *setup;
 FilterCleanup *cleanup;
 FilterReceiveIOV *receive_iov;
 } NetFilterClass;

Are any of the methods optional?  If yes, please add suitable comments.


+
+
+struct NetFilterState {
+/* private */
+Object parent;
+
+/* protected */
+char *netdev_id;
+NetClientState *netdev;
+NetFilterChain chain;
+QTAILQ_ENTRY(NetFilterState) next;
+};
+
+#endif /* QEMU_NET_FILTER_H */
diff --git a/include/net/net.h b/include/net/net.h
index 6a6cbef..36e5fab 100644
--- a/include/net/net.h
+++ b/include/net/net.h
@@ -92,6 +92,7 @@ struct NetClientState {
  NetClientDestructor *destructor;
  unsigned int queue_index;
  unsigned rxfilter_notify_enabled:1;
+QTAILQ_HEAD(, NetFilterState) filters;
  };

  typedef struct NICState {
diff --git a/include/qemu/typedefs.h b/include/qemu/typedefs.h
index f8a9dd6..2c0648f 100644
--- a/include/qemu/typedefs.h
+++ b/include/qemu/typedefs.h
@@ -45,6 +45,7 @@ typedef struct Monitor Monitor;
  typedef struct MouseTransformInfo MouseTransformInfo;
  typedef struct MSIMessage MSIMessage;
  typedef struct NetClientState NetClientState;
+typedef struct NetFilterState NetFilterState;
  typedef struct NICInfo NICInfo;
  typedef struct PcGuestInfo PcGuestInfo;
  typedef struct PCIBridge PCIBridge;
diff --git a/net/Makefile.objs b/net/Makefile.objs
index ec19cb3..914aec0 100644
--- a/net/Makefile.objs
+++ b/net/Makefile.objs
@@ -13,3 +13,4 @@ common-obj-$(CONFIG_HAIKU) += tap-haiku.o
  common-obj-$(CONFIG_SLIRP) 

Re: [Qemu-devel] [PATCH v3] spapr: generate DT node names

[Thomas Huth]

On 23/09/15 14:14, Laurent Vivier wrote:
> When DT node names for PCI devices are generated by SLOF,
> they are generated according to the type of the device
> (for instance, ethernet for virtio-net-pci device).
> 
> Node name for hotplugged devices is generated by QEMU.
> This patch adds the mechanic to QEMU to create the node
> name according to the device type too.
> 
> The data structure has been roughly copied from OpenBIOS/OpenHackware,
> node names from SLOF.
[...]
> diff --git a/hw/ppc/spapr_pci.c b/hw/ppc/spapr_pci.c
> index a2feb4c..c521d31 100644
> --- a/hw/ppc/spapr_pci.c
> +++ b/hw/ppc/spapr_pci.c
> @@ -38,6 +38,7 @@
>  
>  #include "hw/pci/pci_bridge.h"
>  #include "hw/pci/pci_bus.h"
> +#include "hw/pci/pci_ids.h"
>  #include "hw/ppc/spapr_drc.h"
>  #include "sysemu/device_tree.h"
>  
> @@ -944,6 +945,280 @@ static void populate_resource_props(PCIDevice *d, 
> ResourceProps *rp)
>  rp->assigned_len = assigned_idx * sizeof(ResourceFields);
>  }
>  
> +typedef struct PCIClass PCIClass;
> +typedef struct PCISubClass PCISubClass;
> +typedef struct PCIIFace PCIIFace;
> +
> +struct PCIIFace {
> +uint8_t iface;
> +const char *name;
> +};
> +
> +struct PCISubClass {
> +uint8_t subclass;
> +const char *name;
> +const PCIIFace *iface;
> +};
> +#define SUBCLASS(a) ((uint8_t)a)
> +#define IFACE(a)((uint8_t)a)
> +
> +struct PCIClass {
> +const char *name;
> +const PCISubClass *subc;
> +};
> +
> +static const PCISubClass undef_subclass[] = {
> +{ IFACE(PCI_CLASS_NOT_DEFINED_VGA), "display", NULL },
> +{ 0xFF, NULL, NULL, NULL },
> +};
> +
> +static const PCISubClass mass_subclass[] = {
> +{ SUBCLASS(PCI_CLASS_STORAGE_SCSI), "scsi", NULL },
> +{ SUBCLASS(PCI_CLASS_STORAGE_IDE), "ide", NULL },
> +{ SUBCLASS(PCI_CLASS_STORAGE_FLOPPY), "fdc", NULL },
> +{ SUBCLASS(PCI_CLASS_STORAGE_IPI), "ipi", NULL },
> +{ SUBCLASS(PCI_CLASS_STORAGE_RAID), "raid", NULL },
> +{ SUBCLASS(PCI_CLASS_STORAGE_ATA), "ata", NULL },
> +{ SUBCLASS(PCI_CLASS_STORAGE_SATA), "sata", NULL },
> +{ SUBCLASS(PCI_CLASS_STORAGE_SAS), "sas", NULL },
> +{ 0xFF, NULL, NULL },
> +};
> +
> +static const PCISubClass net_subclass[] = {
> +{ SUBCLASS(PCI_CLASS_NETWORK_ETHERNET), "ethernet", NULL },
> +{ SUBCLASS(PCI_CLASS_NETWORK_TOKEN_RING), "token-ring", NULL },
> +{ SUBCLASS(PCI_CLASS_NETWORK_FDDI), "fddi", NULL },
> +{ SUBCLASS(PCI_CLASS_NETWORK_ATM), "atm", NULL },
> +{ SUBCLASS(PCI_CLASS_NETWORK_ISDN), "isdn", NULL },
> +{ SUBCLASS(PCI_CLASS_NETWORK_WORDFIP), "worldfip", NULL },
> +{ SUBCLASS(PCI_CLASS_NETWORK_PICMG214), "picmg", NULL },
> +{ 0xFF, NULL, NULL },
> +};
> +
> +static const PCISubClass displ_subclass[] = {
> +{ SUBCLASS(PCI_CLASS_DISPLAY_VGA), "vga", NULL },
> +{ SUBCLASS(PCI_CLASS_DISPLAY_XGA), "xga", NULL },
> +{ SUBCLASS(PCI_CLASS_DISPLAY_3D), "3d-controller", NULL },
> +{ 0xFF, NULL, NULL },
> +};
> +
> +static const PCISubClass media_subclass[] = {
> +{ SUBCLASS(PCI_CLASS_MULTIMEDIA_VIDEO), "video", NULL },
> +{ SUBCLASS(PCI_CLASS_MULTIMEDIA_AUDIO), "sound", NULL },
> +{ SUBCLASS(PCI_CLASS_MULTIMEDIA_PHONE), "telephony", NULL },
> +{ 0xFF, NULL, NULL },
> +};
> +
> +static const PCISubClass mem_subclass[] = {
> +{ SUBCLASS(PCI_CLASS_MEMORY_RAM), "memory", NULL },
> +{ SUBCLASS(PCI_CLASS_MEMORY_FLASH), "flash", NULL },
> +{ 0xFF, NULL, NULL },
> +};
> +
> +

One new-line should be sufficient.

[...]
> +static const PCISubClass cpu_subclass[] = {
> +{ SUBCLASS(PCI_CLASS_PROCESSOR_386), "386", NULL },
> +{ SUBCLASS(PCI_CLASS_PROCESSOR_486), "486", NULL },
> +{ SUBCLASS(PCI_CLASS_PROCESSOR_PENTIUM), "pentium", NULL },
> +{ SUBCLASS(PCI_CLASS_PROCESSOR_ALPHA), "alpha", NULL },
> +{ SUBCLASS(PCI_CLASS_PROCESSOR_POWERPC), "powerpc", NULL },
> +{ SUBCLASS(PCI_CLASS_PROCESSOR_MIPS), "mips", NULL },
> +{ SUBCLASS(PCI_CLASS_PROCESSOR_CO), "co-processor", NULL },
> +{ 0xFF, NULL, NULL },
> +};

I really really doubt that we'll ever see such a device on the spapr PCI
bus ... could you at least omit entries like "386", "486" and "alpha" ?

[...]
> +
> +static const PCISubClass spc_subclass[] = {
> +{ SUBCLASS(PCI_CLASS_SP_DPIO), "dpio", NULL },
> +{ SUBCLASS(PCI_CLASS_SP_PERF), "counter", NULL },
> +{ SUBCLASS(PCI_CLASS_SP_SYNCH), "measurement", NULL },
> +{ SUBCLASS(PCI_CLASS_SP_MANAGEMENT), "management-card", NULL },
> +{ 0xFF, NULL, NULL },
> +};
> +
> +static const PCIClass pci_classes[] = {
> +{ "unknown-legacy-device", undef_subclass },

Maybe just "legacy-device" instead of "unknown-legacy-device" ?

> +{ "mass-storage",  mass_subclass },
> +{ "network", net_subclass },
> +{ "display", displ_subclass, },
> +{ "multimedia-device", media_subclass },
> +{ "memory-controller", mem_subclass },
> +{ "unknown-bridge", bridg_subclass },
> +{ "communication-controller", comm_subclass},
> +{ "system-perip

[Qemu-devel] [PATCH v2 03/11] ui/console: add opengl context and scanout support interfaces.

[Gerd Hoffmann]

Add callbacks for opengl context management and scanout texture
configuration to DisplayChangeListenerOps.

Signed-off-by: Gerd Hoffmann 
Reviewed-by: Marc-André Lureau 
---
 include/ui/console.h | 37 +
 ui/console.c | 67 ++--
 2 files changed, 102 insertions(+), 2 deletions(-)

diff --git a/include/ui/console.h b/include/ui/console.h
index 047a2b4..d887f91 100644
--- a/include/ui/console.h
+++ b/include/ui/console.h
@@ -157,6 +157,14 @@ void cursor_set_mono(QEMUCursor *c,
 void cursor_get_mono_image(QEMUCursor *c, int foreground, uint8_t *mask);
 void cursor_get_mono_mask(QEMUCursor *c, int transparent, uint8_t *mask);
 
+typedef void *QEMUGLContext;
+typedef struct QEMUGLParams QEMUGLParams;
+
+struct QEMUGLParams {
+int major_ver;
+int minor_ver;
+};
+
 typedef struct DisplayChangeListenerOps {
 const char *dpy_name;
 
@@ -183,6 +191,21 @@ typedef struct DisplayChangeListenerOps {
   int x, int y, int on);
 void (*dpy_cursor_define)(DisplayChangeListener *dcl,
   QEMUCursor *cursor);
+
+QEMUGLContext (*dpy_gl_ctx_create)(DisplayChangeListener *dcl,
+   QEMUGLParams *params);
+void (*dpy_gl_ctx_destroy)(DisplayChangeListener *dcl,
+   QEMUGLContext ctx);
+int (*dpy_gl_ctx_make_current)(DisplayChangeListener *dcl,
+   QEMUGLContext ctx);
+QEMUGLContext (*dpy_gl_ctx_get_current)(DisplayChangeListener *dcl);
+
+void (*dpy_gl_scanout)(DisplayChangeListener *dcl,
+   uint32_t backing_id, bool backing_y_0_top,
+   uint32_t x, uint32_t y, uint32_t w, uint32_t h);
+void (*dpy_gl_update)(DisplayChangeListener *dcl,
+  uint32_t x, uint32_t y, uint32_t w, uint32_t h);
+
 } DisplayChangeListenerOps;
 
 struct DisplayChangeListener {
@@ -244,6 +267,20 @@ bool dpy_cursor_define_supported(QemuConsole *con);
 bool dpy_gfx_check_format(QemuConsole *con,
   pixman_format_code_t format);
 
+void dpy_gl_scanout(QemuConsole *con,
+uint32_t backing_id, bool backing_y_0_top,
+uint32_t x, uint32_t y, uint32_t w, uint32_t h);
+void dpy_gl_update(QemuConsole *con,
+   uint32_t x, uint32_t y, uint32_t w, uint32_t h);
+
+QEMUGLContext dpy_gl_ctx_create(QemuConsole *con,
+QEMUGLParams *params);
+void dpy_gl_ctx_destroy(QemuConsole *con, QEMUGLContext ctx);
+int dpy_gl_ctx_make_current(QemuConsole *con, QEMUGLContext ctx);
+QEMUGLContext dpy_gl_ctx_get_current(QemuConsole *con);
+
+bool console_has_gl(QemuConsole *con);
+
 static inline int surface_stride(DisplaySurface *s)
 {
 return pixman_image_get_stride(s->image);
diff --git a/ui/console.c b/ui/console.c
index 75fc492..31f0d35 100644
--- a/ui/console.c
+++ b/ui/console.c
@@ -121,6 +121,7 @@ struct QemuConsole {
 DisplayState *ds;
 DisplaySurface *surface;
 int dcls;
+DisplayChangeListener *gl;
 
 /* Graphic console state.  */
 Object *device;
@@ -1332,6 +1333,11 @@ void qemu_free_displaysurface(DisplaySurface *surface)
 g_free(surface);
 }
 
+bool console_has_gl(QemuConsole *con)
+{
+return con->gl != NULL;
+}
+
 void register_displaychangelistener(DisplayChangeListener *dcl)
 {
 static const char nodev[] =
@@ -1339,6 +1345,17 @@ void 
register_displaychangelistener(DisplayChangeListener *dcl)
 static DisplaySurface *dummy;
 QemuConsole *con;
 
+if (dcl->ops->dpy_gl_ctx_create) {
+/* display has opengl support */
+assert(dcl->con);
+if (dcl->con->gl) {
+fprintf(stderr, "can't register two opengl displays (%s, %s)\n",
+dcl->ops->dpy_name, dcl->con->gl->ops->dpy_name);
+exit(1);
+}
+dcl->con->gl = dcl;
+}
+
 trace_displaychangelistener_register(dcl, dcl->ops->dpy_name);
 dcl->ds = get_alloc_displaystate();
 QLIST_INSERT_HEAD(&dcl->ds->listeners, dcl, next);
@@ -1417,9 +1434,13 @@ void dpy_gfx_update(QemuConsole *con, int x, int y, int 
w, int h)
 {
 DisplayState *s = con->ds;
 DisplayChangeListener *dcl;
-int width = surface_width(con->surface);
-int height = surface_height(con->surface);
+int width = w;
+int height = h;
 
+if (con->surface) {
+width = surface_width(con->surface);
+height = surface_height(con->surface);
+}
 x = MAX(x, 0);
 y = MAX(y, 0);
 x = MIN(x, width);
@@ -1619,6 +1640,48 @@ bool dpy_cursor_define_supported(QemuConsole *con)
 return false;
 }
 
+QEMUGLContext dpy_gl_ctx_create(QemuConsole *con,
+struct QEMUGLParams *qparams)
+{
+assert(con->gl);
+return con->gl->ops->dpy_gl_ctx_create(con->gl, qparams);
+}
+
+void dpy_gl_ctx_destroy(QemuConsole *con, QEMUGLCont

[Qemu-devel] [PATCH v2 02/11] sdl2: stop flickering

[Gerd Hoffmann]

Optimizing updates by copying the dirty rectangle
only do not work because of double-buffering.

Signed-off-by: Gerd Hoffmann 
Reviewed-by: Marc-André Lureau 
Reviewed-by: Max Reitz 
---
 ui/sdl2-2d.c | 13 +
 1 file changed, 13 insertions(+)

diff --git a/ui/sdl2-2d.c b/ui/sdl2-2d.c
index d0b340f..191ee3b 100644
--- a/ui/sdl2-2d.c
+++ b/ui/sdl2-2d.c
@@ -45,10 +45,23 @@ void sdl2_2d_update(DisplayChangeListener *dcl,
 return;
 }
 
+/*
+ * SDL2 seems to do some double-buffering, and trying to only
+ * update the changed areas results in only one of the two buffers
+ * being updated.  Which flickers alot.  So lets not try to be
+ * clever do a full update every time ...
+ */
+#if 0
 rect.x = x;
 rect.y = y;
 rect.w = w;
 rect.h = h;
+#else
+rect.x = 0;
+rect.y = 0;
+rect.w = surface_width(surf);
+rect.h = surface_height(surf);
+#endif
 
 SDL_UpdateTexture(scon->texture, NULL, surface_data(surf),
   surface_stride(surf));
-- 
1.8.3.1

[Qemu-devel] [PATCH v2 04/11] virtio-gpu: move iov free to virtio_gpu_cleanup_mapping_iov

[Gerd Hoffmann]

Signed-off-by: Gerd Hoffmann 
---
 hw/display/virtio-gpu.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/hw/display/virtio-gpu.c b/hw/display/virtio-gpu.c
index a67d927..73bd9b6 100644
--- a/hw/display/virtio-gpu.c
+++ b/hw/display/virtio-gpu.c
@@ -563,7 +563,6 @@ int virtio_gpu_create_mapping_iov(struct 
virtio_gpu_resource_attach_backing *ab,
   __func__, ab->resource_id, i);
 virtio_gpu_cleanup_mapping_iov(*iov, i);
 g_free(ents);
-g_free(*iov);
 *iov = NULL;
 return -1;
 }
@@ -580,12 +579,12 @@ void virtio_gpu_cleanup_mapping_iov(struct iovec *iov, 
uint32_t count)
 cpu_physical_memory_unmap(iov[i].iov_base, iov[i].iov_len, 1,
   iov[i].iov_len);
 }
+g_free(iov);
 }
 
 static void virtio_gpu_cleanup_mapping(struct virtio_gpu_simple_resource *res)
 {
 virtio_gpu_cleanup_mapping_iov(res->iov, res->iov_cnt);
-g_free(res->iov);
 res->iov = NULL;
 res->iov_cnt = 0;
 }
-- 
1.8.3.1

[Qemu-devel] [PATCH v2 00/11] add virgl rendering support

[Gerd Hoffmann]

  Hi,

This patch series adds infrastructure for opengl rendering
(context management, define scanout textures), adds support
for this to sdl2 and gtk user interfaces and adds virgl/3d
mode to virtio-gpu by hooking up the virglrenderer library.

v2 fixes a bunch of issues found in patch review, which
hopefully also fixes the issues seen in testing.  This works
for me with both gtk3/sdl2, gl on/off, rhel7 host, fedora22
guest, intel gpu hardware.

please test & review,
  Gerd

Gerd Hoffmann (11):
  shaders: initialize vertexes once
  sdl2: stop flickering
  ui/console: add opengl context and scanout support interfaces.
  virtio-gpu: move iov free to virtio_gpu_cleanup_mapping_iov
  virtio-gpu: change licence from GPLv2 to GPLv2+
  virtio-gpu: update headers for virgl/3d
  virtio-gpu: add 3d mode and virgl rendering support.
  sdl2/opengl: add opengl context and scanout support
  opengl: add egl-context.[ch] helpers
  gtk/opengl: add opengl context and scanout support (egl)
  gtk/opengl: add opengl context and scanout support (GtkGLArea)

 configure   |  40 ++
 hw/display/Makefile.objs|   6 +-
 hw/display/virtio-gpu-3d.c  | 598 
 hw/display/virtio-gpu-pci.c |   4 +-
 hw/display/virtio-gpu.c | 142 ++-
 include/hw/virtio/virtio-gpu.h  |  22 +-
 include/standard-headers/linux/virtio_gpu.h | 112 +-
 include/ui/console.h|  37 ++
 include/ui/egl-context.h|  14 +
 include/ui/gtk.h|  39 ++
 include/ui/sdl2.h   |  22 +-
 include/ui/shader.h |   4 +-
 trace-events|   8 +
 ui/Makefile.objs|   6 +
 ui/console-gl.c |   7 +-
 ui/console.c|  67 +++-
 ui/egl-context.c|  34 ++
 ui/gtk-egl.c| 131 +-
 ui/gtk-gl-area.c| 223 +++
 ui/gtk.c| 149 +--
 ui/sdl2-2d.c|  13 +
 ui/sdl2-gl.c| 135 +++
 ui/sdl2.c   |   7 +
 ui/shader.c |  31 +-
 24 files changed, 1786 insertions(+), 65 deletions(-)
 create mode 100644 hw/display/virtio-gpu-3d.c
 create mode 100644 include/ui/egl-context.h
 create mode 100644 ui/egl-context.c
 create mode 100644 ui/gtk-gl-area.c

-- 
1.8.3.1

[Qemu-devel] [PATCH v2 09/11] opengl: add egl-context.[ch] helpers

[Gerd Hoffmann]

Add helper functions to manage opengl contexts using egl.

Signed-off-by: Gerd Hoffmann 
Reviewed-by: Marc-André Lureau 
---
 include/ui/egl-context.h | 14 ++
 ui/Makefile.objs |  1 +
 ui/egl-context.c | 34 ++
 3 files changed, 49 insertions(+)
 create mode 100644 include/ui/egl-context.h
 create mode 100644 ui/egl-context.c

diff --git a/include/ui/egl-context.h b/include/ui/egl-context.h
new file mode 100644
index 000..f004ce1
--- /dev/null
+++ b/include/ui/egl-context.h
@@ -0,0 +1,14 @@
+#ifndef EGL_CONTEXT_H
+#define EGL_CONTEXT_H
+
+#include "ui/console.h"
+#include "ui/egl-helpers.h"
+
+QEMUGLContext qemu_egl_create_context(DisplayChangeListener *dcl,
+  QEMUGLParams *params);
+void qemu_egl_destroy_context(DisplayChangeListener *dcl, QEMUGLContext ctx);
+int qemu_egl_make_context_current(DisplayChangeListener *dcl,
+  QEMUGLContext ctx);
+QEMUGLContext qemu_egl_get_current_context(DisplayChangeListener *dcl);
+
+#endif /* EGL_CONTEXT_H */
diff --git a/ui/Makefile.objs b/ui/Makefile.objs
index 0034fbb..7a49026 100644
--- a/ui/Makefile.objs
+++ b/ui/Makefile.objs
@@ -31,6 +31,7 @@ ifeq ($(CONFIG_OPENGL),y)
 common-obj-y += shader.o
 common-obj-y += console-gl.o
 common-obj-y += egl-helpers.o
+common-obj-y += egl-context.o
 common-obj-$(CONFIG_GTK) += gtk-egl.o
 endif
 
diff --git a/ui/egl-context.c b/ui/egl-context.c
new file mode 100644
index 000..40102e3
--- /dev/null
+++ b/ui/egl-context.c
@@ -0,0 +1,34 @@
+#include "qemu-common.h"
+#include "ui/egl-context.h"
+
+QEMUGLContext qemu_egl_create_context(DisplayChangeListener *dcl,
+  QEMUGLParams *params)
+{
+   EGLContext ctx;
+   EGLint ctx_att[] = {
+  EGL_CONTEXT_CLIENT_VERSION, params->major_ver,
+  EGL_CONTEXT_MINOR_VERSION_KHR, params->minor_ver,
+  EGL_NONE
+   };
+
+   ctx = eglCreateContext(qemu_egl_display, qemu_egl_config,
+  eglGetCurrentContext(), ctx_att);
+   return ctx;
+}
+
+void qemu_egl_destroy_context(DisplayChangeListener *dcl, QEMUGLContext ctx)
+{
+eglDestroyContext(qemu_egl_display, ctx);
+}
+
+int qemu_egl_make_context_current(DisplayChangeListener *dcl,
+  QEMUGLContext ctx)
+{
+   return eglMakeCurrent(qemu_egl_display,
+ EGL_NO_SURFACE, EGL_NO_SURFACE, ctx);
+}
+
+QEMUGLContext qemu_egl_get_current_context(DisplayChangeListener *dcl)
+{
+return eglGetCurrentContext();
+}
-- 
1.8.3.1

[Qemu-devel] [PATCH v2 10/11] gtk/opengl: add opengl context and scanout support (egl)

[Gerd Hoffmann]

This allows virtio-gpu to render in 3d mode.
Uses egl, for gtk versions 3.14 and older.

Signed-off-by: Gerd Hoffmann 
Reviewed-by: Marc-André Lureau 
---
 include/ui/gtk.h |  16 +++
 ui/gtk-egl.c | 131 +++
 ui/gtk.c |   7 +++
 3 files changed, 146 insertions(+), 8 deletions(-)

diff --git a/include/ui/gtk.h b/include/ui/gtk.h
index ee6dffd..6d152e5 100644
--- a/include/ui/gtk.h
+++ b/include/ui/gtk.h
@@ -24,6 +24,7 @@
 
 #if defined(CONFIG_OPENGL)
 #include "ui/egl-helpers.h"
+#include "ui/egl-context.h"
 #endif
 
 /* Compatibility define to let us build on both Gtk2 and Gtk3 */
@@ -50,6 +51,11 @@ typedef struct VirtualGfxConsole {
 EGLContext ectx;
 EGLSurface esurface;
 int glupdates;
+int x, y, w, h;
+GLuint tex_id;
+GLuint fbo_id;
+bool y0_top;
+bool scanout_mode;
 #endif
 } VirtualGfxConsole;
 
@@ -94,6 +100,16 @@ void gd_egl_update(DisplayChangeListener *dcl,
 void gd_egl_refresh(DisplayChangeListener *dcl);
 void gd_egl_switch(DisplayChangeListener *dcl,
DisplaySurface *surface);
+QEMUGLContext gd_egl_create_context(DisplayChangeListener *dcl,
+QEMUGLParams *params);
+void gd_egl_scanout(DisplayChangeListener *dcl,
+uint32_t backing_id, bool backing_y_0_top,
+uint32_t x, uint32_t y,
+uint32_t w, uint32_t h);
+void gd_egl_scanout_flush(DisplayChangeListener *dcl,
+  uint32_t x, uint32_t y, uint32_t w, uint32_t h);
 void gtk_egl_init(void);
+int gd_egl_make_current(DisplayChangeListener *dcl,
+QEMUGLContext ctx);
 
 #endif /* UI_GTK_H */
diff --git a/ui/gtk-egl.c b/ui/gtk-egl.c
index 15b41f2..500c42c 100644
--- a/ui/gtk-egl.c
+++ b/ui/gtk-egl.c
@@ -21,6 +21,29 @@
 
 #include "sysemu/sysemu.h"
 
+static void gtk_egl_set_scanout_mode(VirtualConsole *vc, bool scanout)
+{
+if (vc->gfx.scanout_mode == scanout) {
+return;
+}
+
+vc->gfx.scanout_mode = scanout;
+if (!vc->gfx.scanout_mode) {
+if (vc->gfx.fbo_id) {
+glFramebufferTexture2DEXT(GL_FRAMEBUFFER_EXT,
+  GL_COLOR_ATTACHMENT0_EXT,
+  GL_TEXTURE_2D, 0, 0);
+glBindFramebuffer(GL_FRAMEBUFFER_EXT, 0);
+glDeleteFramebuffers(1, &vc->gfx.fbo_id);
+vc->gfx.fbo_id = 0;
+}
+if (vc->gfx.surface) {
+surface_gl_destroy_texture(vc->gfx.gls, vc->gfx.ds);
+surface_gl_create_texture(vc->gfx.gls, vc->gfx.ds);
+}
+}
+}
+
 /** DisplayState Callbacks (opengl version) **/
 
 void gd_egl_init(VirtualConsole *vc)
@@ -50,19 +73,26 @@ void gd_egl_draw(VirtualConsole *vc)
 GdkWindow *window;
 int ww, wh;
 
-if (!vc->gfx.gls || !vc->gfx.ds) {
+if (!vc->gfx.gls) {
 return;
 }
 
-eglMakeCurrent(qemu_egl_display, vc->gfx.esurface,
-   vc->gfx.esurface, vc->gfx.ectx);
+if (vc->gfx.scanout_mode) {
+gd_egl_scanout_flush(&vc->gfx.dcl, 0, 0, vc->gfx.w, vc->gfx.h);
+} else {
+if (!vc->gfx.ds) {
+return;
+}
+eglMakeCurrent(qemu_egl_display, vc->gfx.esurface,
+   vc->gfx.esurface, vc->gfx.ectx);
 
-window = gtk_widget_get_window(vc->gfx.drawing_area);
-gdk_drawable_get_size(window, &ww, &wh);
-surface_gl_setup_viewport(vc->gfx.gls, vc->gfx.ds, ww, wh);
-surface_gl_render_texture(vc->gfx.gls, vc->gfx.ds);
+window = gtk_widget_get_window(vc->gfx.drawing_area);
+gdk_drawable_get_size(window, &ww, &wh);
+surface_gl_setup_viewport(vc->gfx.gls, vc->gfx.ds, ww, wh);
+surface_gl_render_texture(vc->gfx.gls, vc->gfx.ds);
 
-eglSwapBuffers(qemu_egl_display, vc->gfx.esurface);
+eglSwapBuffers(qemu_egl_display, vc->gfx.esurface);
+}
 }
 
 void gd_egl_update(DisplayChangeListener *dcl,
@@ -99,6 +129,7 @@ void gd_egl_refresh(DisplayChangeListener *dcl)
 
 if (vc->gfx.glupdates) {
 vc->gfx.glupdates = 0;
+gtk_egl_set_scanout_mode(vc, false);
 gd_egl_draw(vc);
 }
 }
@@ -128,6 +159,81 @@ void gd_egl_switch(DisplayChangeListener *dcl,
 }
 }
 
+QEMUGLContext gd_egl_create_context(DisplayChangeListener *dcl,
+QEMUGLParams *params)
+{
+VirtualConsole *vc = container_of(dcl, VirtualConsole, gfx.dcl);
+
+eglMakeCurrent(qemu_egl_display, vc->gfx.esurface,
+   vc->gfx.esurface, vc->gfx.ectx);
+return qemu_egl_create_context(dcl, params);
+}
+
+void gd_egl_scanout(DisplayChangeListener *dcl,
+uint32_t backing_id, bool backing_y_0_top,
+uint32_t x, uint32_t y,
+uint32_t w, uint32_t h)
+{
+VirtualConsole *vc = container_of(dcl, VirtualConsole, gfx.dcl);
+
+vc->gfx.x = x;
+vc->gfx.y = y;
+vc->gfx.w = w;
+  

[Qemu-devel] [PATCH v2 01/11] shaders: initialize vertexes once

[Gerd Hoffmann]

Create a buffer for the vertex data and place vertexes
there at initialization time.  Then just use the buffer
for each texture blit.

Signed-off-by: Gerd Hoffmann 
Reviewed-by: Marc-André Lureau 
---
 include/ui/shader.h |  4 +++-
 ui/console-gl.c |  7 ++-
 ui/shader.c | 31 ++-
 3 files changed, 35 insertions(+), 7 deletions(-)

diff --git a/include/ui/shader.h b/include/ui/shader.h
index 8509596..f7d8618 100644
--- a/include/ui/shader.h
+++ b/include/ui/shader.h
@@ -3,7 +3,9 @@
 
 #include 
 
-void qemu_gl_run_texture_blit(GLint texture_blit_prog);
+GLuint qemu_gl_init_texture_blit(GLint texture_blit_prog);
+void qemu_gl_run_texture_blit(GLint texture_blit_prog,
+  GLint texture_blit_vao);
 
 GLuint qemu_gl_create_compile_shader(GLenum type, const GLchar *src);
 GLuint qemu_gl_create_link_program(GLuint vert, GLuint frag);
diff --git a/ui/console-gl.c b/ui/console-gl.c
index cb45cf8..baf397b 100644
--- a/ui/console-gl.c
+++ b/ui/console-gl.c
@@ -33,6 +33,7 @@
 
 struct ConsoleGLState {
 GLint texture_blit_prog;
+GLint texture_blit_vao;
 };
 
 /* -- */
@@ -47,6 +48,9 @@ ConsoleGLState *console_gl_init_context(void)
 exit(1);
 }
 
+gls->texture_blit_vao =
+qemu_gl_init_texture_blit(gls->texture_blit_prog);
+
 return gls;
 }
 
@@ -131,7 +135,8 @@ void surface_gl_render_texture(ConsoleGLState *gls,
 glClearColor(0.1f, 0.1f, 0.1f, 0.0f);
 glClear(GL_COLOR_BUFFER_BIT);
 
-qemu_gl_run_texture_blit(gls->texture_blit_prog);
+qemu_gl_run_texture_blit(gls->texture_blit_prog,
+ gls->texture_blit_vao);
 }
 
 void surface_gl_destroy_texture(ConsoleGLState *gls,
diff --git a/ui/shader.c b/ui/shader.c
index 52a4632..0588655 100644
--- a/ui/shader.c
+++ b/ui/shader.c
@@ -29,21 +29,42 @@
 
 /* -- */
 
-void qemu_gl_run_texture_blit(GLint texture_blit_prog)
+GLuint qemu_gl_init_texture_blit(GLint texture_blit_prog)
 {
-GLfloat in_position[] = {
+static const GLfloat in_position[] = {
 -1, -1,
 1,  -1,
 -1,  1,
 1,   1,
 };
 GLint l_position;
+GLuint vao, buffer;
+
+glGenVertexArrays(1, &vao);
+glBindVertexArray(vao);
+
+/* this is the VBO that holds the vertex data */
+glGenBuffers(1, &buffer);
+glBindBuffer(GL_ARRAY_BUFFER, buffer);
+glBufferData(GL_ARRAY_BUFFER, sizeof(in_position), in_position,
+ GL_STATIC_DRAW);
 
-glUseProgram(texture_blit_prog);
 l_position = glGetAttribLocation(texture_blit_prog, "in_position");
-glVertexAttribPointer(l_position, 2, GL_FLOAT, GL_FALSE, 0, in_position);
+glVertexAttribPointer(l_position, 2, GL_FLOAT, GL_FALSE, 0, 0);
 glEnableVertexAttribArray(l_position);
-glDrawArrays(GL_TRIANGLE_STRIP, l_position, 4);
+
+glBindBuffer(GL_ARRAY_BUFFER, 0);
+glBindVertexArray(0);
+
+return vao;
+}
+
+void qemu_gl_run_texture_blit(GLint texture_blit_prog,
+  GLint texture_blit_vao)
+{
+glUseProgram(texture_blit_prog);
+glBindVertexArray(texture_blit_vao);
+glDrawArrays(GL_TRIANGLE_STRIP, 0, 4);
 }
 
 /* -- */
-- 
1.8.3.1

[Qemu-devel] [PATCH v2 08/11] sdl2/opengl: add opengl context and scanout support

[Gerd Hoffmann]

This allows virtio-gpu to render in 3d mode.

Signed-off-by: Gerd Hoffmann 
---
 include/ui/sdl2.h |  22 -
 ui/sdl2-gl.c  | 135 ++
 ui/sdl2.c |   7 +++
 3 files changed, 163 insertions(+), 1 deletion(-)

diff --git a/include/ui/sdl2.h b/include/ui/sdl2.h
index 2fdad8f..b7ac38f 100644
--- a/include/ui/sdl2.h
+++ b/include/ui/sdl2.h
@@ -15,12 +15,18 @@ struct sdl2_console {
 SDL_Renderer *real_renderer;
 int idx;
 int last_vm_running; /* per console for caption reasons */
-int x, y;
+int x, y, w, h;
 int hidden;
 int opengl;
 int updates;
 SDL_GLContext winctx;
+#ifdef CONFIG_OPENGL
 ConsoleGLState *gls;
+GLuint tex_id;
+GLuint fbo_id;
+bool y0_top;
+bool scanout_mode;
+#endif
 };
 
 void sdl2_window_create(struct sdl2_console *scon);
@@ -48,4 +54,18 @@ void sdl2_gl_switch(DisplayChangeListener *dcl,
 void sdl2_gl_refresh(DisplayChangeListener *dcl);
 void sdl2_gl_redraw(struct sdl2_console *scon);
 
+QEMUGLContext sdl2_gl_create_context(DisplayChangeListener *dcl,
+ QEMUGLParams *params);
+void sdl2_gl_destroy_context(DisplayChangeListener *dcl, QEMUGLContext ctx);
+int sdl2_gl_make_context_current(DisplayChangeListener *dcl,
+ QEMUGLContext ctx);
+QEMUGLContext sdl2_gl_get_current_context(DisplayChangeListener *dcl);
+
+void sdl2_gl_scanout(DisplayChangeListener *dcl,
+ uint32_t backing_id, bool backing_y_0_top,
+ uint32_t x, uint32_t y,
+ uint32_t w, uint32_t h);
+void sdl2_gl_scanout_flush(DisplayChangeListener *dcl,
+   uint32_t x, uint32_t y, uint32_t w, uint32_t h);
+
 #endif /* SDL2_H */
diff --git a/ui/sdl2-gl.c b/ui/sdl2-gl.c
index b604c06..2bb3d06 100644
--- a/ui/sdl2-gl.c
+++ b/ui/sdl2-gl.c
@@ -31,11 +31,37 @@
 #include "ui/sdl2.h"
 #include "sysemu/sysemu.h"
 
+#include 
+
+static void sdl2_set_scanout_mode(struct sdl2_console *scon, bool scanout)
+{
+if (scon->scanout_mode == scanout) {
+return;
+}
+
+scon->scanout_mode = scanout;
+if (!scon->scanout_mode) {
+if (scon->fbo_id) {
+glFramebufferTexture2DEXT(GL_FRAMEBUFFER_EXT,
+  GL_COLOR_ATTACHMENT0_EXT,
+  GL_TEXTURE_2D, 0, 0);
+glDeleteFramebuffers(1, &scon->fbo_id);
+glBindFramebuffer(GL_FRAMEBUFFER_EXT, 0);
+scon->fbo_id = 0;
+}
+if (scon->surface) {
+surface_gl_destroy_texture(scon->gls, scon->surface);
+surface_gl_create_texture(scon->gls, scon->surface);
+}
+}
+}
+
 static void sdl2_gl_render_surface(struct sdl2_console *scon)
 {
 int ww, wh;
 
 SDL_GL_MakeCurrent(scon->real_window, scon->winctx);
+sdl2_set_scanout_mode(scon, false);
 
 SDL_GetWindowSize(scon->real_window, &ww, &wh);
 surface_gl_setup_viewport(scon->gls, scon->surface, ww, wh);
@@ -110,3 +136,112 @@ void sdl2_gl_redraw(struct sdl2_console *scon)
 sdl2_gl_render_surface(scon);
 }
 }
+
+QEMUGLContext sdl2_gl_create_context(DisplayChangeListener *dcl,
+ QEMUGLParams *params)
+{
+struct sdl2_console *scon = container_of(dcl, struct sdl2_console, dcl);
+SDL_GLContext ctx;
+
+assert(scon->opengl);
+
+SDL_GL_MakeCurrent(scon->real_window, scon->winctx);
+
+SDL_GL_SetAttribute(SDL_GL_SHARE_WITH_CURRENT_CONTEXT, 1);
+SDL_GL_SetAttribute(SDL_GL_CONTEXT_PROFILE_MASK,
+SDL_GL_CONTEXT_PROFILE_CORE);
+SDL_GL_SetAttribute(SDL_GL_CONTEXT_MAJOR_VERSION, params->major_ver);
+SDL_GL_SetAttribute(SDL_GL_CONTEXT_MINOR_VERSION, params->minor_ver);
+
+ctx = SDL_GL_CreateContext(scon->real_window);
+return (QEMUGLContext)ctx;
+}
+
+void sdl2_gl_destroy_context(DisplayChangeListener *dcl, QEMUGLContext ctx)
+{
+SDL_GLContext sdlctx = (SDL_GLContext)ctx;
+
+SDL_GL_DeleteContext(sdlctx);
+}
+
+int sdl2_gl_make_context_current(DisplayChangeListener *dcl,
+ QEMUGLContext ctx)
+{
+struct sdl2_console *scon = container_of(dcl, struct sdl2_console, dcl);
+SDL_GLContext sdlctx = (SDL_GLContext)ctx;
+
+assert(scon->opengl);
+
+return SDL_GL_MakeCurrent(scon->real_window, sdlctx);
+}
+
+QEMUGLContext sdl2_gl_get_current_context(DisplayChangeListener *dcl)
+{
+SDL_GLContext sdlctx;
+
+sdlctx = SDL_GL_GetCurrentContext();
+return (QEMUGLContext)sdlctx;
+}
+
+void sdl2_gl_scanout(DisplayChangeListener *dcl,
+ uint32_t backing_id, bool backing_y_0_top,
+ uint32_t x, uint32_t y,
+ uint32_t w, uint32_t h)
+{
+struct sdl2_console *scon = container_of(dcl, struct sdl2_console, dcl);
+
+assert(scon->opengl);
+scon->x = x;
+scon->y = y;
+scon->w = w;
+scon->h = h

[Qemu-devel] [PATCH v2 05/11] virtio-gpu: change licence from GPLv2 to GPLv2+

[Gerd Hoffmann]

---
 hw/display/virtio-gpu-pci.c | 4 ++--
 hw/display/virtio-gpu.c | 2 +-
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/hw/display/virtio-gpu-pci.c b/hw/display/virtio-gpu-pci.c
index 5bc62cf..eef137f 100644
--- a/hw/display/virtio-gpu-pci.c
+++ b/hw/display/virtio-gpu-pci.c
@@ -6,8 +6,8 @@
  * Authors:
  *  Dave Airlie
  *
- * This work is licensed under the terms of the GNU GPL, version 2.  See
- * the COPYING file in the top-level directory.
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
  *
  */
 #include "hw/pci/pci.h"
diff --git a/hw/display/virtio-gpu.c b/hw/display/virtio-gpu.c
index 73bd9b6..8c35af3 100644
--- a/hw/display/virtio-gpu.c
+++ b/hw/display/virtio-gpu.c
@@ -7,7 +7,7 @@
  * Dave Airlie 
  * Gerd Hoffmann 
  *
- * This work is licensed under the terms of the GNU GPL, version 2.
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
  * See the COPYING file in the top-level directory.
  */
 
-- 
1.8.3.1

[Qemu-devel] [PATCH v2 06/11] virtio-gpu: update headers for virgl/3d

[Gerd Hoffmann]

Sync with linux kernel headers with virgl/3d patches applied.

Signed-off-by: Gerd Hoffmann 
Reviewed-by: Marc-André Lureau 
---
 include/standard-headers/linux/virtio_gpu.h | 112 +++-
 1 file changed, 111 insertions(+), 1 deletion(-)

diff --git a/include/standard-headers/linux/virtio_gpu.h 
b/include/standard-headers/linux/virtio_gpu.h
index 72ef815..76e5e52 100644
--- a/include/standard-headers/linux/virtio_gpu.h
+++ b/include/standard-headers/linux/virtio_gpu.h
@@ -40,6 +40,8 @@
 
 #include "standard-headers/linux/types.h"
 
+#define VIRTIO_GPU_FEATURE_VIRGL 0
+
 enum virtio_gpu_ctrl_type {
VIRTIO_GPU_UNDEFINED = 0,
 
@@ -52,6 +54,18 @@ enum virtio_gpu_ctrl_type {
VIRTIO_GPU_CMD_TRANSFER_TO_HOST_2D,
VIRTIO_GPU_CMD_RESOURCE_ATTACH_BACKING,
VIRTIO_GPU_CMD_RESOURCE_DETACH_BACKING,
+   VIRTIO_GPU_CMD_GET_CAPSET_INFO,
+   VIRTIO_GPU_CMD_GET_CAPSET,
+
+   /* 3d commands */
+   VIRTIO_GPU_CMD_CTX_CREATE = 0x0200,
+   VIRTIO_GPU_CMD_CTX_DESTROY,
+   VIRTIO_GPU_CMD_CTX_ATTACH_RESOURCE,
+   VIRTIO_GPU_CMD_CTX_DETACH_RESOURCE,
+   VIRTIO_GPU_CMD_RESOURCE_CREATE_3D,
+   VIRTIO_GPU_CMD_TRANSFER_TO_HOST_3D,
+   VIRTIO_GPU_CMD_TRANSFER_FROM_HOST_3D,
+   VIRTIO_GPU_CMD_SUBMIT_3D,
 
/* cursor commands */
VIRTIO_GPU_CMD_UPDATE_CURSOR = 0x0300,
@@ -60,6 +74,8 @@ enum virtio_gpu_ctrl_type {
/* success responses */
VIRTIO_GPU_RESP_OK_NODATA = 0x1100,
VIRTIO_GPU_RESP_OK_DISPLAY_INFO,
+   VIRTIO_GPU_RESP_OK_CAPSET_INFO,
+   VIRTIO_GPU_RESP_OK_CAPSET,
 
/* error responses */
VIRTIO_GPU_RESP_ERR_UNSPEC = 0x1200,
@@ -180,13 +196,107 @@ struct virtio_gpu_resp_display_info {
} pmodes[VIRTIO_GPU_MAX_SCANOUTS];
 };
 
+/* data passed in the control vq, 3d related */
+
+struct virtio_gpu_box {
+   uint32_t x, y, z;
+   uint32_t w, h, d;
+};
+
+/* VIRTIO_GPU_CMD_TRANSFER_TO_HOST_3D, VIRTIO_GPU_CMD_TRANSFER_FROM_HOST_3D */
+struct virtio_gpu_transfer_host_3d {
+   struct virtio_gpu_ctrl_hdr hdr;
+   struct virtio_gpu_box box;
+   uint64_t offset;
+   uint32_t resource_id;
+   uint32_t level;
+   uint32_t stride;
+   uint32_t layer_stride;
+};
+
+/* VIRTIO_GPU_CMD_RESOURCE_CREATE_3D */
+#define VIRTIO_GPU_RESOURCE_FLAG_Y_0_TOP (1 << 0)
+struct virtio_gpu_resource_create_3d {
+   struct virtio_gpu_ctrl_hdr hdr;
+   uint32_t resource_id;
+   uint32_t target;
+   uint32_t format;
+   uint32_t bind;
+   uint32_t width;
+   uint32_t height;
+   uint32_t depth;
+   uint32_t array_size;
+   uint32_t last_level;
+   uint32_t nr_samples;
+   uint32_t flags;
+   uint32_t padding;
+};
+
+/* VIRTIO_GPU_CMD_CTX_CREATE */
+struct virtio_gpu_ctx_create {
+   struct virtio_gpu_ctrl_hdr hdr;
+   uint32_t nlen;
+   uint32_t padding;
+   char debug_name[64];
+};
+
+/* VIRTIO_GPU_CMD_CTX_DESTROY */
+struct virtio_gpu_ctx_destroy {
+   struct virtio_gpu_ctrl_hdr hdr;
+};
+
+/* VIRTIO_GPU_CMD_CTX_ATTACH_RESOURCE, VIRTIO_GPU_CMD_CTX_DETACH_RESOURCE */
+struct virtio_gpu_ctx_resource {
+   struct virtio_gpu_ctrl_hdr hdr;
+   uint32_t resource_id;
+   uint32_t padding;
+};
+
+/* VIRTIO_GPU_CMD_SUBMIT_3D */
+struct virtio_gpu_cmd_submit {
+   struct virtio_gpu_ctrl_hdr hdr;
+   uint32_t size;
+   uint32_t padding;
+};
+
+#define VIRTIO_GPU_CAPSET_VIRGL 1
+
+/* VIRTIO_GPU_CMD_GET_CAPSET_INFO */
+struct virtio_gpu_get_capset_info {
+   struct virtio_gpu_ctrl_hdr hdr;
+   uint32_t capset_index;
+   uint32_t padding;
+};
+
+/* VIRTIO_GPU_RESP_OK_CAPSET_INFO */
+struct virtio_gpu_resp_capset_info {
+   struct virtio_gpu_ctrl_hdr hdr;
+   uint32_t capset_id;
+   uint32_t capset_max_version;
+   uint32_t capset_max_size;
+   uint32_t padding;
+};
+
+/* VIRTIO_GPU_CMD_GET_CAPSET */
+struct virtio_gpu_get_capset {
+   struct virtio_gpu_ctrl_hdr hdr;
+   uint32_t capset_id;
+   uint32_t capset_version;
+};
+
+/* VIRTIO_GPU_RESP_OK_CAPSET */
+struct virtio_gpu_resp_capset {
+   struct virtio_gpu_ctrl_hdr hdr;
+   uint8_t capset_data[];
+};
+
 #define VIRTIO_GPU_EVENT_DISPLAY (1 << 0)
 
 struct virtio_gpu_config {
uint32_t events_read;
uint32_t events_clear;
uint32_t num_scanouts;
-   uint32_t reserved;
+   uint32_t num_capsets;
 };
 
 /* simple formats for fbcon/X use */
-- 
1.8.3.1

[Qemu-devel] [PATCH v2 07/11] virtio-gpu: add 3d mode and virgl rendering support.

[Gerd Hoffmann]

Add virglrenderer library detection.  Add 3d mode to virtio-gpu,
wire up virglrenderer library.  When in 3d mode render using the
new context management and texture scanout callbacks.

Signed-off-by: Gerd Hoffmann 
---
 configure  |  32 +++
 hw/display/Makefile.objs   |   6 +-
 hw/display/virtio-gpu-3d.c | 598 +
 hw/display/virtio-gpu.c| 137 +-
 include/hw/virtio/virtio-gpu.h |  22 +-
 trace-events   |   8 +
 6 files changed, 792 insertions(+), 11 deletions(-)
 create mode 100644 hw/display/virtio-gpu-3d.c

diff --git a/configure b/configure
index 52f5b79..dc114bb 100755
--- a/configure
+++ b/configure
@@ -331,6 +331,7 @@ gtkabi=""
 gnutls=""
 gnutls_hash=""
 vte=""
+virglrenderer=""
 tpm="yes"
 libssh2=""
 vhdx=""
@@ -1122,6 +1123,10 @@ for opt do
   ;;
   --enable-vte) vte="yes"
   ;;
+  --disable-virglrenderer) virglrenderer="no"
+  ;;
+  --enable-virglrenderer) virglrenderer="yes"
+  ;;
   --disable-tpm) tpm="no"
   ;;
   --enable-tpm) tpm="yes"
@@ -3950,6 +3955,27 @@ EOF
 fi
 
 ##
+# virgl renderer probe
+
+if test "$virglrenderer" != "no" ; then
+  cat > $TMPC << EOF
+#include 
+int main(void) { virgl_renderer_poll(); return 0; }
+EOF
+  virgl_cflags=$($pkg_config --cflags virglrenderer 2>/dev/null)
+  virgl_libs=$($pkg_config --libs virglrenderer 2>/dev/null)
+  if $pkg_config virglrenderer >/dev/null 2>&1 && \
+ compile_prog "$virgl_cflags" "$virgl_libs" ; then
+virglrenderer="yes"
+  else
+if test "$virglrenderer" = "yes" ; then
+  feature_not_found "virglrenderer"
+fi
+virglrenderer="no"
+  fi
+fi
+
+##
 # check if we have fdatasync
 
 fdatasync=no
@@ -4566,6 +4592,7 @@ echo "GNUTLS nettle $gnutls_nettle 
${gnutls_nettle+($nettle_version)}"
 echo "libtasn1  $tasn1"
 echo "VTE support   $vte"
 echo "curses support$curses"
+echo "virgl support $virglrenderer"
 echo "curl support  $curl"
 echo "mingw32 support   $mingw32"
 echo "Audio drivers $audio_drv_list"
@@ -4938,6 +4965,11 @@ if test "$vte" = "yes" ; then
   echo "CONFIG_VTE=y" >> $config_host_mak
   echo "VTE_CFLAGS=$vte_cflags" >> $config_host_mak
 fi
+if test "$virglrenderer" = "yes" ; then
+  echo "CONFIG_VIRGL=y" >> $config_host_mak
+  echo "VIRGL_CFLAGS=$virgl_cflags" >> $config_host_mak
+  echo "VIRGL_LIBS=$virgl_libs" >> $config_host_mak
+fi
 if test "$xen" = "yes" ; then
   echo "CONFIG_XEN_BACKEND=y" >> $config_host_mak
   echo "CONFIG_XEN_CTRL_INTERFACE_VERSION=$xen_ctrl_version" >> 
$config_host_mak
diff --git a/hw/display/Makefile.objs b/hw/display/Makefile.objs
index dd8ea76..f0cf431 100644
--- a/hw/display/Makefile.objs
+++ b/hw/display/Makefile.objs
@@ -35,6 +35,10 @@ obj-$(CONFIG_VGA) += vga.o
 
 common-obj-$(CONFIG_QXL) += qxl.o qxl-logger.o qxl-render.o
 
-obj-$(CONFIG_VIRTIO) += virtio-gpu.o
+obj-$(CONFIG_VIRTIO) += virtio-gpu.o virtio-gpu-3d.o
 obj-$(CONFIG_VIRTIO_PCI) += virtio-gpu-pci.o
 obj-$(CONFIG_VIRTIO_VGA) += virtio-vga.o
+virtio-gpu.o-cflags := $(VIRGL_CFLAGS)
+virtio-gpu.o-libs += $(VIRGL_LIBS)
+virtio-gpu-3d.o-cflags := $(VIRGL_CFLAGS)
+virtio-gpu-3d.o-libs += $(VIRGL_LIBS)
diff --git a/hw/display/virtio-gpu-3d.c b/hw/display/virtio-gpu-3d.c
new file mode 100644
index 000..28dccfd
--- /dev/null
+++ b/hw/display/virtio-gpu-3d.c
@@ -0,0 +1,598 @@
+/*
+ * Virtio GPU Device
+ *
+ * Copyright Red Hat, Inc. 2013-2014
+ *
+ * Authors:
+ * Dave Airlie 
+ * Gerd Hoffmann 
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+
+#include "qemu-common.h"
+#include "qemu/iov.h"
+#include "trace.h"
+#include "hw/virtio/virtio.h"
+#include "hw/virtio/virtio-gpu.h"
+
+#ifdef CONFIG_VIRGL
+
+#include "virglrenderer.h"
+
+static struct virgl_renderer_callbacks virtio_gpu_3d_cbs;
+
+static void virgl_cmd_create_resource_2d(VirtIOGPU *g,
+ struct virtio_gpu_ctrl_command *cmd)
+{
+struct virtio_gpu_resource_create_2d c2d;
+struct virgl_renderer_resource_create_args args;
+
+VIRTIO_GPU_FILL_CMD(c2d);
+trace_virtio_gpu_cmd_res_create_2d(c2d.resource_id, c2d.format,
+   c2d.width, c2d.height);
+
+args.handle = c2d.resource_id;
+args.target = 2;
+args.format = c2d.format;
+args.bind = (1 << 1);
+args.width = c2d.width;
+args.height = c2d.height;
+args.depth = 1;
+args.array_size = 1;
+args.last_level = 0;
+args.nr_samples = 0;
+args.flags = VIRTIO_GPU_RESOURCE_FLAG_Y_0_TOP;
+virgl_renderer_resource_create(&args, NULL, 0);
+}
+
+static void virgl_cmd_create_resource_3d(VirtIOGPU *g,
+ struct virtio_gpu_ctrl_command *cmd)
+{
+struct virtio_gpu_resource_create_3d c3d;
+struct virgl_renderer_resource_create_args args;
+
+VI

[Qemu-devel] [PATCH v2 11/11] gtk/opengl: add opengl context and scanout support (GtkGLArea)

[Gerd Hoffmann]

This allows virtio-gpu to render in 3d mode.
Uses native opengl support which is present
in gtk versions 3.16 and newer.

Signed-off-by: Gerd Hoffmann 
---
 configure|   8 ++
 include/ui/gtk.h |  23 ++
 ui/Makefile.objs |   5 ++
 ui/gtk-gl-area.c | 223 +++
 ui/gtk.c | 142 +++
 5 files changed, 371 insertions(+), 30 deletions(-)
 create mode 100644 ui/gtk-gl-area.c

diff --git a/configure b/configure
index dc114bb..21cb9a0 100755
--- a/configure
+++ b/configure
@@ -328,6 +328,7 @@ glusterfs_zerofill="no"
 archipelago="no"
 gtk=""
 gtkabi=""
+gtk_gl="no"
 gnutls=""
 gnutls_hash=""
 vte=""
@@ -3205,6 +3206,9 @@ if test "$opengl" != "no" ; then
 opengl_cflags="$($pkg_config --cflags $opengl_pkgs) $x11_cflags"
 opengl_libs="$($pkg_config --libs $opengl_pkgs) $x11_libs"
 opengl=yes
+if test "$gtk" = "yes" && $pkg_config --exists "$gtkpackage >= 3.16"; then
+gtk_gl="yes"
+fi
   else
 if test "$opengl" = "yes" ; then
   feature_not_found "opengl" "Please install opengl (mesa) devel pkgs: 
$opengl_pkgs"
@@ -4585,6 +4589,7 @@ fi
 echo "pixman$pixman"
 echo "SDL support   $sdl"
 echo "GTK support   $gtk"
+echo "GTK GL support$gtk_gl"
 echo "GNUTLS support$gnutls"
 echo "GNUTLS hash   $gnutls_hash"
 echo "GNUTLS gcrypt $gnutls_gcrypt"
@@ -4944,6 +4949,9 @@ if test "$gtk" = "yes" ; then
   echo "CONFIG_GTK=y" >> $config_host_mak
   echo "CONFIG_GTKABI=$gtkabi" >> $config_host_mak
   echo "GTK_CFLAGS=$gtk_cflags" >> $config_host_mak
+  if test "$gtk_gl" = "yes" ; then
+echo "CONFIG_GTK_GL=y" >> $config_host_mak
+  fi
 fi
 if test "$gnutls" = "yes" ; then
   echo "CONFIG_GNUTLS=y" >> $config_host_mak
diff --git a/include/ui/gtk.h b/include/ui/gtk.h
index 6d152e5..dbd6871 100644
--- a/include/ui/gtk.h
+++ b/include/ui/gtk.h
@@ -112,4 +112,27 @@ void gtk_egl_init(void);
 int gd_egl_make_current(DisplayChangeListener *dcl,
 QEMUGLContext ctx);
 
+/* ui/gtk-gl-area.c */
+void gd_gl_area_init(VirtualConsole *vc);
+void gd_gl_area_draw(VirtualConsole *vc);
+void gd_gl_area_update(DisplayChangeListener *dcl,
+   int x, int y, int w, int h);
+void gd_gl_area_refresh(DisplayChangeListener *dcl);
+void gd_gl_area_switch(DisplayChangeListener *dcl,
+   DisplaySurface *surface);
+QEMUGLContext gd_gl_area_create_context(DisplayChangeListener *dcl,
+QEMUGLParams *params);
+void gd_gl_area_destroy_context(DisplayChangeListener *dcl,
+QEMUGLContext ctx);
+void gd_gl_area_scanout(DisplayChangeListener *dcl,
+uint32_t backing_id, bool backing_y_0_top,
+uint32_t x, uint32_t y,
+uint32_t w, uint32_t h);
+void gd_gl_area_scanout_flush(DisplayChangeListener *dcl,
+  uint32_t x, uint32_t y, uint32_t w, uint32_t h);
+void gtk_gl_area_init(void);
+QEMUGLContext gd_gl_area_get_current_context(DisplayChangeListener *dcl);
+int gd_gl_area_make_current(DisplayChangeListener *dcl,
+QEMUGLContext ctx);
+
 #endif /* UI_GTK_H */
diff --git a/ui/Makefile.objs b/ui/Makefile.objs
index 7a49026..728393c 100644
--- a/ui/Makefile.objs
+++ b/ui/Makefile.objs
@@ -32,11 +32,16 @@ common-obj-y += shader.o
 common-obj-y += console-gl.o
 common-obj-y += egl-helpers.o
 common-obj-y += egl-context.o
+ifeq ($(CONFIG_GTK_GL),y)
+common-obj-$(CONFIG_GTK) += gtk-gl-area.o
+else
 common-obj-$(CONFIG_GTK) += gtk-egl.o
 endif
+endif
 
 gtk.o-cflags := $(GTK_CFLAGS) $(VTE_CFLAGS)
 gtk-egl.o-cflags := $(GTK_CFLAGS) $(VTE_CFLAGS) $(OPENGL_CFLAGS)
+gtk-gl-area.o-cflags := $(GTK_CFLAGS) $(VTE_CFLAGS) $(OPENGL_CFLAGS)
 shader.o-cflags += $(OPENGL_CFLAGS)
 console-gl.o-cflags += $(OPENGL_CFLAGS)
 egl-helpers.o-cflags += $(OPENGL_CFLAGS)
diff --git a/ui/gtk-gl-area.c b/ui/gtk-gl-area.c
new file mode 100644
index 000..dec3edb
--- /dev/null
+++ b/ui/gtk-gl-area.c
@@ -0,0 +1,223 @@
+/*
+ * GTK UI -- glarea opengl code.
+ *
+ * Requires 3.16+ (GtkGLArea widget).
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+
+#include "qemu-common.h"
+
+#include "trace.h"
+
+#include "ui/console.h"
+#include "ui/gtk.h"
+#include "ui/egl-helpers.h"
+
+#include "sysemu/sysemu.h"
+
+static void gtk_gl_area_set_scanout_mode(VirtualConsole *vc, bool scanout)
+{
+if (vc->gfx.scanout_mode == scanout) {
+return;
+}
+
+vc->gfx.scanout_mode = scanout;
+if (!vc->gfx.scanout_mode) {
+if (vc->gfx.fbo_id) {
+glFramebufferTexture2DEXT(GL_FRAMEBUFFER_EXT,
+  GL_COLOR_ATTACHMENT0_EXT,
+  GL_TEXTURE_2D, 0, 0);
+glBindFramebuffer(GL_FRAMEBUFFER_EXT, 0);
+

Re: [Qemu-devel] [PATCH v11 09/12] netfilter: add a netbuffer filter

[Markus Armbruster]

Yang Hongyang  writes:

> This filter is to buffer/release packets, this feature can be used
> when using MicroCheckpointing, or other Remus like VM FT solutions, you

What's "Remus"?

> can also use it to simulate the network delay.
> It has an interval option, if supplied, this filter will release
> packets by interval.

Suggest "will delay packets by that time interval."

Is interval really optional?

>
> Usage:
>  -netdev tap,id=bn0
>  -object filter-buffer,id=f0,netdev=bn0,chain=in,interval=1000
>
> NOTE:
>  the scale of interval is microsecond.

Perhaps "interval is in microseconds".

>
> Signed-off-by: Yang Hongyang 
> ---
> v11: add a fixme comment from Jason
> v10: use NetQueue flush api to flush packets
>  sent_cb can not be called when we already return size
> v9: adjustment due to the qapi change
> v7: use QTAILQ_FOREACH_SAFE() when flush packets
> v6: move the interval check earlier and some comment adjust
> v5: remove dummy sent_cb
> change interval type from int64 to uint32
> check interval!=0 when initialise
> rename FILTERBUFFERState to FilterBufferState
> v4: remove bh
> pass the packet to next filter instead of receiver
> v3: check packet's sender and sender->peer when flush it
> ---
>  net/Makefile.objs   |   1 +
>  net/filter-buffer.c | 170 
> 
>  qemu-options.hx |  18 ++
>  vl.c|   7 ++-
>  4 files changed, 195 insertions(+), 1 deletion(-)
>  create mode 100644 net/filter-buffer.c
>
> diff --git a/net/Makefile.objs b/net/Makefile.objs
> index 914aec0..5fa2f97 100644
> --- a/net/Makefile.objs
> +++ b/net/Makefile.objs
> @@ -14,3 +14,4 @@ common-obj-$(CONFIG_SLIRP) += slirp.o
>  common-obj-$(CONFIG_VDE) += vde.o
>  common-obj-$(CONFIG_NETMAP) += netmap.o
>  common-obj-y += filter.o
> +common-obj-y += filter-buffer.o
> diff --git a/net/filter-buffer.c b/net/filter-buffer.c
> new file mode 100644
> index 000..ef94e91
> --- /dev/null
> +++ b/net/filter-buffer.c
> @@ -0,0 +1,170 @@
> +/*
> + * Copyright (c) 2015 FUJITSU LIMITED
> + * Author: Yang Hongyang 
> + *
> + * This work is licensed under the terms of the GNU GPL, version 2 or
> + * later.  See the COPYING file in the top-level directory.
> + */
> +
> +#include "net/filter.h"
> +#include "net/queue.h"
> +#include "qemu-common.h"
> +#include "qemu/timer.h"
> +#include "qemu/iov.h"
> +#include "qapi/qmp/qerror.h"
> +#include "qapi-visit.h"
> +#include "qom/object.h"
> +
> +#define TYPE_FILTER_BUFFER "filter-buffer"
> +
> +#define FILTER_BUFFER(obj) \
> +OBJECT_CHECK(FilterBufferState, (obj), TYPE_FILTER_BUFFER)
> +
> +struct FilterBufferState {
> +NetFilterState parent_obj;
> +
> +NetQueue *incoming_queue;
> +uint32_t interval;
> +QEMUTimer release_timer;
> +};
> +typedef struct FilterBufferState FilterBufferState;

Again, not splitting the declaration is more concise.

> +
> +static void filter_buffer_flush(NetFilterState *nf)
> +{
> +FilterBufferState *s = FILTER_BUFFER(nf);
> +
> +if (!qemu_net_queue_flush(s->incoming_queue)) {
> +/* Unable to empty the queue, purge remaining packets */
> +qemu_net_queue_purge(s->incoming_queue, nf->netdev);
> +}
> +}

This either flushes or purges incoming_queue, where "purge" means
dropping packets.  Correct?

> +
> +static void filter_buffer_release_timer(void *opaque)
> +{
> +NetFilterState *nf = opaque;
> +FilterBufferState *s = FILTER_BUFFER(nf);

Style nit: blank line between declarations and statements, please.

> +filter_buffer_flush(nf);

Is purging correct here?

> +timer_mod(&s->release_timer,
> +  qemu_clock_get_us(QEMU_CLOCK_VIRTUAL) + s->interval);

Timer rearmed to fire again in s->interval microseconds.

> +}
> +
> +/* filter APIs */
> +static ssize_t filter_buffer_receive_iov(NetFilterState *nf,
> + NetClientState *sender,
> + unsigned flags,
> + const struct iovec *iov,
> + int iovcnt,
> + NetPacketSent *sent_cb)
> +{
> +FilterBufferState *s = FILTER_BUFFER(nf);
> +
> +/*
> + * we return size when buffer a packet, the sender will take it as
> + * a already sent packet, so sent_cb should not be called later

Humor me: when a comment has multiple sentences, start each one with a
capital letter, and end it with punctuation.

> + * FIXME: even if guest can't receive packet for some reasons. Filter
> + * can still accept packet until its internal queue is full.
> + */

I'm not sure I understand the comment.

> +qemu_net_queue_append_iov(s->incoming_queue, sender, flags,
> +  iov, iovcnt, NULL);
> +return iov_size(iov, iovcnt);
> +}
> +
> +static void filter_buffer_cleanup(NetFilterState *nf)
> +{
> +FilterBufferState *s = FILTER_BUFFER(nf);
> +
> +  

Re: [Qemu-devel] No error report when using the qemu-img.exetoconvert a disk to vmdk format which is saved on a disk that has nomorespace

[Kevin Wolf]

Am 24.09.2015 um 10:01 hat Guangmu Zhu geschrieben:
> Hi Kevin,
> 
> I tried the patch you provide, and I haven't seen that problem yet. If the 
> disk
> space is full, an error will be reported with the message "Invalid argument"
> and the program will stop.
> 
> Will you merge the patch to the master?

I'll post it in a separate email thread as a proper patch now in order
to give people a chance to review it. After that, I'll include it in a
pull request for master, but it might still be a few days until then.

Kevin

> diff --git a/block/raw-win32.c b/block/raw-win32.c
> index 68f2338..b562c94 100644
> --- a/block/raw-win32.c
> +++ b/block/raw-win32.c
> @@ -119,9 +119,9 @@ static int aio_worker(void *arg)
>  case QEMU_AIO_WRITE:
>  count = handle_aiocb_rw(aiocb);
>  if (count == aiocb->aio_nbytes) {
> -count = 0;
> +ret = 0;
>  } else {
> -count = -EINVAL;
> +ret = -EINVAL;
>  }
>  break;
>  case QEMU_AIO_FLUSH:
> 
> -
> 
> I'll try the patch and report in a week for I'm too busy these days. And if I
> could, I would like to help to maintain the Windows backend.
> 
> Sincerely.
> Guangmu Zhu
> 
> -
> 
> Am 23.09.2015 um 13:30 hat Guangmu Zhu geschrieben:
> > If the "BlockDriver" is "bdrv_vmdk", the function "vmdk_co_write" will be
> > called instead. In function "vmdk_write_extent" I see "ret = bdrv_pwrite
> > (extent->file, write_offset, write_buf, write_len);". So the "extend->file"
> is
> > "bdrv_file", is it?
> 
> Yes, exactly. You'll go through bdrv_vmdk first, and then the nested
> call goes to bdrv_file.
> 
> > -
> >
> > Correct a mistake:
> > So though the "count" would be "-EINVAL" if error occurred while writing 
> > some
> > file, the return value will always be zero. Maybe I missed something?
> 
> I think you're right. Instead of setting count = 0/-EINVAL in
> aio_worker, we should be setting ret.
> 
> Can you try the patch below and report back?
> 
> > 3. The "bs->drv->bdrv_aio_writev" is function "raw_aio_writev" in file
> > "raw-win32.c" and the quemu-img uses synchronous IO always, so the function
> > "paio_submit" in the same file will be called. This function submits the
> "aio"
> > to "worker_thread" with the callback "aio_worker". There are some codes in
> > "aio_worker":
> >
> > ssize_t ret = 0;
> > ..
> > case QEMU_AIO_WRITE:
> > count = handle_aiocb_rw(aiocb);
> > if (count == aiocb->aio_nbytes) {
> > count = 0;
> > } else {
> > count = -EINVAL;
> > }
> > break;
> > ..
> > return ret;
> 
> Independently of your problem, the code in aio_worker() looks a bit
> fishy, because handle_aiocb_rw() can't distinguish between an error
> and 0 bytes transferred.
> 
> For writes, that probably doesn't matter, but for reads, I think we
> return a successful read of zeroes instead of signalling an error. This
> might need another patch.
> 
> Generally, the Windows backend is not getting a lot of attention and
> could use someone who checks it, cleans it up and fixes bugs.
> 
> Kevin
> 
> 
> diff --git a/block/raw-win32.c b/block/raw-win32.c
> index 68f2338..b562c94 100644
> --- a/block/raw-win32.c
> +++ b/block/raw-win32.c
> @@ -119,9 +119,9 @@ static int aio_worker(void *arg)
>  case QEMU_AIO_WRITE:
>  count = handle_aiocb_rw(aiocb);
>  if (count == aiocb->aio_nbytes) {
> -count = 0;
> +ret = 0;
>  } else {
> -count = -EINVAL;
> +ret = -EINVAL;
>  }
>  break;
>  case QEMU_AIO_FLUSH:

[Qemu-devel] [PATCH] raw-win32: Fix write request error handling

[Kevin Wolf]

aio_worker() wrote the return code to the wrong variable.

Signed-off-by: Kevin Wolf 
Tested-by: Guangmu Zhu 
---
 block/raw-win32.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/block/raw-win32.c b/block/raw-win32.c
index 68f2338..b562c94 100644
--- a/block/raw-win32.c
+++ b/block/raw-win32.c
@@ -119,9 +119,9 @@ static int aio_worker(void *arg)
 case QEMU_AIO_WRITE:
 count = handle_aiocb_rw(aiocb);
 if (count == aiocb->aio_nbytes) {
-count = 0;
+ret = 0;
 } else {
-count = -EINVAL;
+ret = -EINVAL;
 }
 break;
 case QEMU_AIO_FLUSH:
-- 
1.8.3.1

Re: [Qemu-devel] [PATCH] libseccomp: add cacheflush to whitelist

[Eduardo Otubo]

On Wed, Jul 01, 2015 at 09=12=33AM -0400, Andrew Jones wrote:
> cacheflush is an arm-specific syscall that qemu built for arm
> uses. Add it to the whitelist.
> 
> Signed-off-by: Andrew Jones 
> 
> ---
> 
> I'm not sure about the priority selection. Maybe cacheflush gets
> used frequently enough that it deserves a higher one?

The frequency is measured using strace and comparing the frequency they
appear among other syscalls. Can you run this analysis and double check
if the lowest priority is still accurate?

Thanks for the patch.

> 
> This patch isn't really necessary yet due to ae6e8ef11e6c: "Revert
> seccomp tests that allow it to be used on non-x86 architectures",
> which we can't revert until libseccomp has released a fix for
> arm-specific syscall symbol naming, but when linking to a patched
> libseccomp and reverting ae6e8ef11e6c, then this patch allows
> guests to boot with '-sandbox on'.
> 
> Signed-off-by: Andrew Jones 
> ---
>  qemu-seccomp.c | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/qemu-seccomp.c b/qemu-seccomp.c
> index f9de0d3390feb..33644a4e3c3d3 100644
> --- a/qemu-seccomp.c
> +++ b/qemu-seccomp.c
> @@ -237,7 +237,8 @@ static const struct QemuSeccompSyscall 
> seccomp_whitelist[] = {
>  { SCMP_SYS(fadvise64), 240 },
>  { SCMP_SYS(inotify_init1), 240 },
>  { SCMP_SYS(inotify_add_watch), 240 },
> -{ SCMP_SYS(mbind), 240 }
> +{ SCMP_SYS(mbind), 240 },
> +{ SCMP_SYS(cacheflush), 240 },
>  };
>  
>  int seccomp_start(void)
> -- 
> 2.1.0
> 

-- 
Eduardo Otubo
ProfitBricks GmbH


signature.asc
Description: Digital signature

Re: [Qemu-devel] [PATCH v3] spapr: generate DT node names

[Michael S. Tsirkin]

On Wed, Sep 23, 2015 at 02:14:03PM +0200, Laurent Vivier wrote:
> When DT node names for PCI devices are generated by SLOF,
> they are generated according to the type of the device
> (for instance, ethernet for virtio-net-pci device).
> 
> Node name for hotplugged devices is generated by QEMU.
> This patch adds the mechanic to QEMU to create the node
> name according to the device type too.
> 
> The data structure has been roughly copied from OpenBIOS/OpenHackware,
> node names from SLOF.
> 
> Example:
> 
> Hotplugging some PCI cards with QEMU monitor:
> 
> device_add virtio-tablet-pci
> device_add virtio-serial-pci
> device_add virtio-mouse-pci
> device_add virtio-scsi-pci
> device_add virtio-gpu-pci
> device_add ne2k_pci
> device_add nec-usb-xhci
> device_add intel-hda
> 
> What we can see in linux device tree:
> 
> for dir in /proc/device-tree/pci@8002000/*@*/; do
> echo $dir
> cat $dir/name
> echo
> done
> 
> WITHOUT this patch:
> 
> /proc/device-tree/pci@8002000/pci@0/
> pci
> /proc/device-tree/pci@8002000/pci@1/
> pci
> /proc/device-tree/pci@8002000/pci@2/
> pci
> /proc/device-tree/pci@8002000/pci@3/
> pci
> /proc/device-tree/pci@8002000/pci@4/
> pci
> /proc/device-tree/pci@8002000/pci@5/
> pci
> /proc/device-tree/pci@8002000/pci@6/
> pci
> /proc/device-tree/pci@8002000/pci@7/
> pci
> 
> WITH this patch:
> 
> /proc/device-tree/pci@8002000/communication-controller@1/
> communication-controller
> /proc/device-tree/pci@8002000/display@4/
> display
> /proc/device-tree/pci@8002000/ethernet@5/
> ethernet
> /proc/device-tree/pci@8002000/input-controller@0/
> input-controller
> /proc/device-tree/pci@8002000/mouse@2/
> mouse
> /proc/device-tree/pci@8002000/multimedia-device@7/
> multimedia-device
> /proc/device-tree/pci@8002000/scsi@3/
> scsi
> /proc/device-tree/pci@8002000/usb-xhci@6/
> usb-xhci
> 
> Signed-off-by: Laurent Vivier 

I'm not familiar enough with spapr to judge whether
this makes sense.

The pci_ids bits looks ok to me
Reviewed-by: Michael S. Tsirkin 
just for this part.

> ---
> v3: use values from pci_ids.h, update pci_ids.h values
> keep only details for USB (xhci, ohci, ...) and PIC (IO-APIC, IO-XAPIC)
> v2: Use CamelCase name, remove misc-* name,
> remove _OTHER entries to fallback to class name (as SLOF does).
> Fix typo (IPMI-bltr).
> 
>  hw/ppc/spapr_pci.c   | 296 
> ---
>  include/hw/pci/pci_ids.h | 115 --
>  2 files changed, 388 insertions(+), 23 deletions(-)
> 
> diff --git a/hw/ppc/spapr_pci.c b/hw/ppc/spapr_pci.c
> index a2feb4c..c521d31 100644
> --- a/hw/ppc/spapr_pci.c
> +++ b/hw/ppc/spapr_pci.c
> @@ -38,6 +38,7 @@
>  
>  #include "hw/pci/pci_bridge.h"
>  #include "hw/pci/pci_bus.h"
> +#include "hw/pci/pci_ids.h"
>  #include "hw/ppc/spapr_drc.h"
>  #include "sysemu/device_tree.h"
>  
> @@ -944,6 +945,280 @@ static void populate_resource_props(PCIDevice *d, 
> ResourceProps *rp)
>  rp->assigned_len = assigned_idx * sizeof(ResourceFields);
>  }
>  
> +typedef struct PCIClass PCIClass;
> +typedef struct PCISubClass PCISubClass;
> +typedef struct PCIIFace PCIIFace;
> +
> +struct PCIIFace {
> +uint8_t iface;
> +const char *name;
> +};
> +
> +struct PCISubClass {
> +uint8_t subclass;
> +const char *name;
> +const PCIIFace *iface;
> +};
> +#define SUBCLASS(a) ((uint8_t)a)
> +#define IFACE(a)((uint8_t)a)
> +
> +struct PCIClass {
> +const char *name;
> +const PCISubClass *subc;
> +};
> +
> +static const PCISubClass undef_subclass[] = {
> +{ IFACE(PCI_CLASS_NOT_DEFINED_VGA), "display", NULL },
> +{ 0xFF, NULL, NULL, NULL },
> +};
> +
> +static const PCISubClass mass_subclass[] = {
> +{ SUBCLASS(PCI_CLASS_STORAGE_SCSI), "scsi", NULL },
> +{ SUBCLASS(PCI_CLASS_STORAGE_IDE), "ide", NULL },
> +{ SUBCLASS(PCI_CLASS_STORAGE_FLOPPY), "fdc", NULL },
> +{ SUBCLASS(PCI_CLASS_STORAGE_IPI), "ipi", NULL },
> +{ SUBCLASS(PCI_CLASS_STORAGE_RAID), "raid", NULL },
> +{ SUBCLASS(PCI_CLASS_STORAGE_ATA), "ata", NULL },
> +{ SUBCLASS(PCI_CLASS_STORAGE_SATA), "sata", NULL },
> +{ SUBCLASS(PCI_CLASS_STORAGE_SAS), "sas", NULL },
> +{ 0xFF, NULL, NULL },
> +};
> +
> +static const PCISubClass net_subclass[] = {
> +{ SUBCLASS(PCI_CLASS_NETWORK_ETHERNET), "ethernet", NULL },
> +{ SUBCLASS(PCI_CLASS_NETWORK_TOKEN_RING), "token-ring", NULL },
> +{ SUBCLASS(PCI_CLASS_NETWORK_FDDI), "fddi", NULL },
> +{ SUBCLASS(PCI_CLASS_NETWORK_ATM), "atm", NULL },
> +{ SUBCLASS(PCI_CLASS_NETWORK_ISDN), "isdn", NULL },
> +{ SUBCLASS(PCI_CLASS_NETWORK_WORDFIP), "worldfip", NULL },
> +{ SUBCLASS(PCI_CLASS_NETWORK_PICMG214), "picmg", NULL },
> +{ 0xFF, NULL, NULL },
> +};
> +
> +static const PCISubClass displ_subclass[] = {
> +{ SUBCLASS(PCI_CLASS_DISPLAY_VGA), "vga", NULL },
> +{ SUBCLASS(PCI_CLASS_D

Re: [Qemu-devel] [Qemu-block] [PATCH] tests: Fix test 049 fallout from improved HMP error messages

[Alberto Garcia]

On Wed 23 Sep 2015 01:15:52 AM CEST, Eric Blake  wrote:
> Commit 50b7b000 improved HMP error messages, but forgot to update
> qemu-iotests to match.
>
> Reported-by: Kevin Wolf 
> Signed-off-by: Eric Blake 

Reviewed-by: Alberto Garcia 

Berto

Re: [Qemu-devel] [PATCH v3] spapr: generate DT node names

[Laurent Vivier]

On 24/09/2015 11:34, Michael S. Tsirkin wrote:
> On Wed, Sep 23, 2015 at 02:14:03PM +0200, Laurent Vivier wrote:
>> When DT node names for PCI devices are generated by SLOF,
>> they are generated according to the type of the device
>> (for instance, ethernet for virtio-net-pci device).
>>
>> Node name for hotplugged devices is generated by QEMU.
>> This patch adds the mechanic to QEMU to create the node
>> name according to the device type too.
>>
>> The data structure has been roughly copied from OpenBIOS/OpenHackware,
>> node names from SLOF.
>>
>> Example:
>>
>> Hotplugging some PCI cards with QEMU monitor:
>>
>> device_add virtio-tablet-pci
>> device_add virtio-serial-pci
>> device_add virtio-mouse-pci
>> device_add virtio-scsi-pci
>> device_add virtio-gpu-pci
>> device_add ne2k_pci
>> device_add nec-usb-xhci
>> device_add intel-hda
>>
>> What we can see in linux device tree:
>>
>> for dir in /proc/device-tree/pci@8002000/*@*/; do
>> echo $dir
>> cat $dir/name
>> echo
>> done
>>
>> WITHOUT this patch:
>>
>> /proc/device-tree/pci@8002000/pci@0/
>> pci
>> /proc/device-tree/pci@8002000/pci@1/
>> pci
>> /proc/device-tree/pci@8002000/pci@2/
>> pci
>> /proc/device-tree/pci@8002000/pci@3/
>> pci
>> /proc/device-tree/pci@8002000/pci@4/
>> pci
>> /proc/device-tree/pci@8002000/pci@5/
>> pci
>> /proc/device-tree/pci@8002000/pci@6/
>> pci
>> /proc/device-tree/pci@8002000/pci@7/
>> pci
>>
>> WITH this patch:
>>
>> /proc/device-tree/pci@8002000/communication-controller@1/
>> communication-controller
>> /proc/device-tree/pci@8002000/display@4/
>> display
>> /proc/device-tree/pci@8002000/ethernet@5/
>> ethernet
>> /proc/device-tree/pci@8002000/input-controller@0/
>> input-controller
>> /proc/device-tree/pci@8002000/mouse@2/
>> mouse
>> /proc/device-tree/pci@8002000/multimedia-device@7/
>> multimedia-device
>> /proc/device-tree/pci@8002000/scsi@3/
>> scsi
>> /proc/device-tree/pci@8002000/usb-xhci@6/
>> usb-xhci
>>
>> Signed-off-by: Laurent Vivier 
> 
> I'm not familiar enough with spapr to judge whether
> this makes sense.
> 
> The pci_ids bits looks ok to me
> Reviewed-by: Michael S. Tsirkin 
> just for this part.

Thank you Michael.

As suggested by Thomas, I'm going to split the patch in two parts, one
for the PCI ids and another for spapr part.

I will repost the PCI part with your Reviewed-by.

> 
>> ---
>> v3: use values from pci_ids.h, update pci_ids.h values
>> keep only details for USB (xhci, ohci, ...) and PIC (IO-APIC, IO-XAPIC)
>> v2: Use CamelCase name, remove misc-* name,
>> remove _OTHER entries to fallback to class name (as SLOF does).
>> Fix typo (IPMI-bltr).
>>
>>  hw/ppc/spapr_pci.c   | 296 
>> ---
>>  include/hw/pci/pci_ids.h | 115 --
>>  2 files changed, 388 insertions(+), 23 deletions(-)
>>
>> diff --git a/hw/ppc/spapr_pci.c b/hw/ppc/spapr_pci.c
>> index a2feb4c..c521d31 100644
>> --- a/hw/ppc/spapr_pci.c
>> +++ b/hw/ppc/spapr_pci.c
>> @@ -38,6 +38,7 @@
>>  
>>  #include "hw/pci/pci_bridge.h"
>>  #include "hw/pci/pci_bus.h"
>> +#include "hw/pci/pci_ids.h"
>>  #include "hw/ppc/spapr_drc.h"
>>  #include "sysemu/device_tree.h"
>>  
>> @@ -944,6 +945,280 @@ static void populate_resource_props(PCIDevice *d, 
>> ResourceProps *rp)
>>  rp->assigned_len = assigned_idx * sizeof(ResourceFields);
>>  }
>>  
>> +typedef struct PCIClass PCIClass;
>> +typedef struct PCISubClass PCISubClass;
>> +typedef struct PCIIFace PCIIFace;
>> +
>> +struct PCIIFace {
>> +uint8_t iface;
>> +const char *name;
>> +};
>> +
>> +struct PCISubClass {
>> +uint8_t subclass;
>> +const char *name;
>> +const PCIIFace *iface;
>> +};
>> +#define SUBCLASS(a) ((uint8_t)a)
>> +#define IFACE(a)((uint8_t)a)
>> +
>> +struct PCIClass {
>> +const char *name;
>> +const PCISubClass *subc;
>> +};
>> +
>> +static const PCISubClass undef_subclass[] = {
>> +{ IFACE(PCI_CLASS_NOT_DEFINED_VGA), "display", NULL },
>> +{ 0xFF, NULL, NULL, NULL },
>> +};
>> +
>> +static const PCISubClass mass_subclass[] = {
>> +{ SUBCLASS(PCI_CLASS_STORAGE_SCSI), "scsi", NULL },
>> +{ SUBCLASS(PCI_CLASS_STORAGE_IDE), "ide", NULL },
>> +{ SUBCLASS(PCI_CLASS_STORAGE_FLOPPY), "fdc", NULL },
>> +{ SUBCLASS(PCI_CLASS_STORAGE_IPI), "ipi", NULL },
>> +{ SUBCLASS(PCI_CLASS_STORAGE_RAID), "raid", NULL },
>> +{ SUBCLASS(PCI_CLASS_STORAGE_ATA), "ata", NULL },
>> +{ SUBCLASS(PCI_CLASS_STORAGE_SATA), "sata", NULL },
>> +{ SUBCLASS(PCI_CLASS_STORAGE_SAS), "sas", NULL },
>> +{ 0xFF, NULL, NULL },
>> +};
>> +
>> +static const PCISubClass net_subclass[] = {
>> +{ SUBCLASS(PCI_CLASS_NETWORK_ETHERNET), "ethernet", NULL },
>> +{ SUBCLASS(PCI_CLASS_NETWORK_TOKEN_RING), "token-ring", NULL },
>> +{ SUBCLASS(PCI_CLASS_NETWORK_FDDI), "fddi", NULL },
>> +{ SUBCLASS(PCI_CLASS_NETWORK

Re: [Qemu-devel] [PATCH v11 01/12] qmp: delete qemu opts when delete an object

[Markus Armbruster]

Yang Hongyang  writes:

> On 09/24/2015 03:43 PM, Markus Armbruster wrote:
>> This has finally reached the front of my review queue.  I apologize for
>> the lng delay.
>>
>> Copying Paolo for another pair of eyeballs (he wrote this code).
>>
> [...]
>>> +
>>> +opts = qemu_opts_find(qemu_find_opts_err("object", NULL), id);
>>> +qemu_opts_del(opts);
>>
>> qemu_find_opts_err("object", &error_abort) please, because when it
>> fails, we want to die right away, not when the null pointer it returns
>> gets dereferenced.
>
> Thanks for the review.
> Jason, do you want me to propose a fix on top of this series or simply drop
> this for now because this patch is an independent bug fix and won't affect the
> other filter patch series.
>
>>
>> Same sloppiness in netdev_del_completion() and qmp_netdev_del(), not
>> your patch's fault.
>>
>> Elsewhere, we store the QemuOpts in the object just so we can delete it:
>> DeviceState, DriveInfo.  Paolo, what do you think?
>
> I don't get it. Currently, only objects created at the beginning through
> QEMU command line will be stored in the QemuOpts, objects that created
> with object_add won't stored in QemuOpts. Do you mean for DeviceState,
> DriveInfo they store there QemuOpts explicity so that they can delete it?
> Why don't we just delete it from objects directly instead?

Let me elaborate.

We have the same pattern in multiple places: some kind of object gets
configured via QemuOpts, and an object's QemuOpts need to stay around
until the object dies.

Example 1: Block device backends

DriveInfo has a member opts.

drive_new() stores the QemuOpts in dinfo->opts.

drive_info_del() destroys dinfo->opts.

Note: DriveInfo member opts is always non-null.  But not every
BlockBackend has a DriveInfo.

Example 2: Device frontends

DeviceState has a member opts.

qdev_device_add() stores the QemuOpts in dev->opts.

device_finalize() destroys dev->opts.

Note: DeviceState member opts may be null (not every device is
created by qdev_device_add()).  Fine, because qemu_opts_del(NULL) is
a no-op.

Example 3: Character device backends

CharDriverState has a member opts.

qemu_chr_new_from_opts() stores the QemuOpts in chr->opts.

qemu_chr_delete() destroys chr->opts.

Example 4: Network device backends

Two cases

A. netdev

   qmp_netdev_add() does not store the QemuOpts.

   qmp_netdev_del() still needs to destroy it.  It has to find it
   somehow.  Here's how it does it:

   opts = qemu_opts_find(qemu_find_opts_err("netdev", NULL), id);
   if (!opts) {
   error_setg(errp, "Device '%s' is not a netdev", id);
   return;
   }

   The !opts condition is a non-obvious way to test "not created
   with -netdev", see commit 645c949.  Note that the commit's claim
   that qemu_opts_del(NULL) crashes is no longer true since commit
   4782183.

B. Legacy net

   hmp_host_net_add() does not store the QemuOpts.

   hmp_host_net_remove() still needs to destroy it.  I can't see
   where that happens, and I'm not sure it does.

Example 5: Generic object

object_create() does not store the QemuOpts.

It still needs to be destroyed along with the object.  It isn't, and
your patch fixes it.

Personally, I find the technique in example 1-3 easier to understand
than the one in example 4-5.

Re: [Qemu-devel] [PATCH] pc: Set broken_reserved_end on pc-*-2.4, not 2.5

[Igor Mammedov]

On Wed, 23 Sep 2015 12:04:49 -0300
Eduardo Habkost  wrote:

> Version 1 of the pc-*-2.5 machine class series was applied to the PCI
> tree instead of v3 (which was rebased after the broken_reserved_end
> patch by Igor was included).
> 
> This patch includes the missing hunks from v3, to make sure
> broken_reserved_end is set at the right machine class.
> 
> Signed-off-by: Eduardo Habkost 

Reviewed-by: Igor Mammedov 

> ---
>  hw/i386/pc_piix.c | 4 ++--
>  hw/i386/pc_q35.c  | 4 ++--
>  2 files changed, 4 insertions(+), 4 deletions(-)
> 
> diff --git a/hw/i386/pc_piix.c b/hw/i386/pc_piix.c
> index caa4edc..3ffb05f 100644
> --- a/hw/i386/pc_piix.c
> +++ b/hw/i386/pc_piix.c
> @@ -466,9 +466,7 @@ static void pc_i440fx_machine_options(MachineClass *m)
>  
>  static void pc_i440fx_2_5_machine_options(MachineClass *m)
>  {
> -PCMachineClass *pcmc = PC_MACHINE_CLASS(m);
>  pc_i440fx_machine_options(m);
> -pcmc->broken_reserved_end = true;
>  m->alias = "pc";
>  m->is_default = 1;
>  }
> @@ -479,9 +477,11 @@ DEFINE_I440FX_MACHINE(v2_5, "pc-i440fx-2.5", NULL,
>  
>  static void pc_i440fx_2_4_machine_options(MachineClass *m)
>  {
> +PCMachineClass *pcmc = PC_MACHINE_CLASS(m);
>  pc_i440fx_2_5_machine_options(m);
>  m->alias = NULL;
>  m->is_default = 0;
> +pcmc->broken_reserved_end = true;
>  SET_MACHINE_COMPAT(m, PC_COMPAT_2_4);
>  }
>  
> diff --git a/hw/i386/pc_q35.c b/hw/i386/pc_q35.c
> index 506b6bf..1b7d3b6 100644
> --- a/hw/i386/pc_q35.c
> +++ b/hw/i386/pc_q35.c
> @@ -372,9 +372,7 @@ static void pc_q35_machine_options(MachineClass *m)
>  
>  static void pc_q35_2_5_machine_options(MachineClass *m)
>  {
> -PCMachineClass *pcmc = PC_MACHINE_CLASS(m);
>  pc_q35_machine_options(m);
> -pcmc->broken_reserved_end = true;
>  m->alias = "q35";
>  }
>  
> @@ -383,8 +381,10 @@ DEFINE_Q35_MACHINE(v2_5, "pc-q35-2.5", NULL,
>  
>  static void pc_q35_2_4_machine_options(MachineClass *m)
>  {
> +PCMachineClass *pcmc = PC_MACHINE_CLASS(m);
>  pc_q35_2_5_machine_options(m);
>  m->alias = NULL;
> +pcmc->broken_reserved_end = true;
>  SET_MACHINE_COMPAT(m, PC_COMPAT_2_4);
>  }
>  

[Qemu-devel] [PATCH v3] docs: describe the QEMU build system structure / design

[Daniel P. Berrange]

Developers who are new to QEMU, or have a background familiarity
with GNU autotools, can have trouble getting their head around the
home-grown QEMU build system. This document attempts to explain
the structure / design of the configure script and the various
Makefile pieces that live across the source tree.

Signed-off-by: Daniel P. Berrange 
---

Changed in v3:

 - More speling eror fixes
 - Rephrased more paragraphs as suggested

Changed in v2:

 - Misc speling eror fixes
 - Rephrased some paragraphs as suggested
 - Added note about config-host.h file generation & use

 docs/build-system.txt | 507 ++
 1 file changed, 507 insertions(+)
 create mode 100644 docs/build-system.txt

diff --git a/docs/build-system.txt b/docs/build-system.txt
new file mode 100644
index 000..a264698
--- /dev/null
+++ b/docs/build-system.txt
@@ -0,0 +1,507 @@
+The QEMU build system architecture
+==
+
+This document aims to help developers understand the architecture of the
+QEMU build system. As with projects using GNU autotools, the QEMU build
+system has two stages, first the developer runs the "configure" script
+to determine the local build environment characteristics, then they run
+"make" to build the project. There is about where the similarities with
+GNU autotools end, so try to forget what you know about them.
+
+
+Stage 1: configure
+==
+
+The QEMU configure script is written directly in shell, and should be
+compatible with any POSIX shell, hence it uses #!/bin/sh. An important
+implication of this is that it is important to avoid using bash-isms on
+development platforms where bash is the primary host.
+
+In contrast to autoconf scripts, QEMU's configure is expected to be
+silent while it is checking for features. It will only display output
+when an error occurs, or to show the final feature enablement summary
+on completion.
+
+Adding new checks to the configure script usually comprises the
+following tasks:
+
+ - Initialize one or more variables with the default feature state.
+
+   Ideally features should auto-detect whether they are present,
+   so try to avoid hardcoding the initial state to either enabled
+   or disabled, as that forces the user to pass a --enable-XXX
+   / --disable-XXX flag on every invocation of configure.
+
+ - Add support to the command line arg parser to handle any new
+   --enable-XXX / --disable-XXX flags required by the feature XXX.
+
+ - Add information to the help output message to report on the new
+   feature flag.
+
+ - Add code to perform the actual feature check. As noted above, try to
+   be fully dynamic in checking enablement/disablement.
+
+ - Add code to print out the feature status in the configure summary
+   upon completion.
+
+ - Add any new makefile variables to $config_host_mak on completion.
+
+
+Taking (a simplified version of) the probe for gnutls from configure,
+we have the following pieces:
+
+  # Initial variable state
+  gnutls=""
+
+  ..snip..
+
+  # Configure flag processing
+  --disable-gnutls) gnutls="no"
+  ;;
+  --enable-gnutls) gnutls="yes"
+  ;;
+
+  ..snip..
+
+  # Help output feature message
+  gnutls  GNUTLS cryptography support
+
+  ..snip..
+
+  # Test for gnutls
+  if test "$gnutls" != "no"; then
+ if ! $pkg_config --exists "gnutls"; then
+gnutls_cflags=`$pkg_config --cflags gnutls`
+gnutls_libs=`$pkg_config --libs gnutls`
+libs_softmmu="$gnutls_libs $libs_softmmu"
+libs_tools="$gnutls_libs $libs_tools"
+QEMU_CFLAGS="$QEMU_CFLAGS $gnutls_cflags"
+gnutls="yes"
+ elif test "$gnutls" = "yes"; then
+feature_not_found "gnutls" "Install gnutls devel"
+ else
+gnutls="no"
+ fi
+  fi
+
+  ..snip..
+
+  # Completion feature summary
+  echo "GNUTLS support$gnutls"
+
+  ..snip..
+
+  # Define make variables
+  if test "$gnutls" = "yes" ; then
+ echo "CONFIG_GNUTLS=y" >> $config_host_mak
+  fi
+
+
+Helper functions
+
+
+The configure script provides a variety of helper functions to assist
+developers in checking for system features:
+
+ - do_cc $ARGS...
+
+   Attempt to run the system C compiler passing it $ARGS...
+
+ - do_cxx $ARGS...
+
+   Attempt to run the system C++ compiler passing it $ARGS...
+
+ - compile_object $CFLAGS
+
+   Attempt to compile a test program with the system C compiler using
+   $CFLAGS. The test program must have been previously written to a file
+   called $TMPC.
+
+ - compile_prog $CFLAGS $LDFLAGS
+
+   Attempt to compile a test program with the system C compiler using
+   $CFLAGS and link it with the system linker using $LDFLAGS. The test
+   program must have been previously written to a file called $TMPC.
+
+ - has $COMMAND
+
+   Determine if $COMMAND exists in the current environment, either as a
+   shell builtin, or executable binary, returning 0 on success.
+
+ - path_of $COMMAND
+
+   Return the fully quali

Re: [Qemu-devel] [PATCH] tests: Fix test 049 fallout from improved HMP error messages

[Kevin Wolf]

Am 23.09.2015 um 01:15 hat Eric Blake geschrieben:
> Commit 50b7b000 improved HMP error messages, but forgot to update
> qemu-iotests to match.
> 
> Reported-by: Kevin Wolf 
> Signed-off-by: Eric Blake 

Thanks, applied to the block branch.

> I'm not sure if this should go through Markus' "error" tree,
> since that's where the break was introduced, or through a
> "block" tree since it touches qemu-iotests.

Doesn't really matter as long as someone takes it. And if both took
it, I think git should be clever enough to figure it out without merge
conflicts.

Kevin

Re: [Qemu-devel] [PATCH RFC V4 0/4] Implement GIC-500 from GICv3 family for arm64

[Pavel Fedin]

 Hello!

 Nice to see things moving on. I am waiting for PULL to be applied to master, 
and after this i'll be able to proceed with other things, including live 
migration. As part of it, i already rewrote state structure using bitops, so 
that it's free of CPU number restrictions. Would be nice if we upstream this 
too. I will post my patches as RFC as soon as master gets updated. RFC only 
because kernel API isn't there yet.

Kind regards,
Pavel Fedin
Expert Engineer
Samsung Electronics Research center Russia

Re: [Qemu-devel] [PATCH v7 22/42] migrate_start_postcopy: Command to trigger transition to postcopy

[Dr. David Alan Gilbert]

* Amit Shah (amit.s...@redhat.com) wrote:
> On (Mon) 13 Jul 2015 [20:07:52], Juan Quintela wrote:
> > "Dr. David Alan Gilbert"  wrote:
> > > * Juan Quintela (quint...@redhat.com) wrote:
> 
> > >> > +void qmp_migrate_start_postcopy(Error **errp)
> > >> > +{
> > >> > +MigrationState *s = migrate_get_current();
> > >> > +
> > >> > +if (!migrate_postcopy_ram()) {
> > >> > +error_setg(errp, "Enable postcopy with 
> > >> > migration_set_capability before"
> > >> > + " the start of migration");
> > >> > +return;
> > >> > +}
> > >> > +
> > >> > +if (s->state == MIGRATION_STATUS_NONE) {
> > >> 
> > >> I would claim that this check should be:
> > >> 
> > >> if (s->state != MIGRATION_STATUS_ACTIVE) {
> > >> ??
> > >> 
> > >> FAILED, COMPLETED, CANCELL* don't make sense, right?
> > >
> > > What I'm trying to catch here is people doing:
> > >  migrate_start_postcopy
> > >  migrate tcp::whereever
> > >
> > >   which wont work, because migrate_init reinitialises
> > > the flag that start previously set.
> > >
> > > However, I also don't want to create a race, since what you do is
> > > typically:
> > >  migrate  tcp::whereever
> > >
> > >  migrate_start_postcopy
> > >
> > > if you're unlucky, and the migration finishes just
> > > at the same time you do the migrate_start_postcopy, do you
> > > want migrate_start_postcopy to fail?  My guess was it
> > > was best for it not to fail, in this case.
> > 
> > Change the order, if it is ACTIVE: do the postcopy thing, otherwise, do
> > the clause that is protected now?  Moving to postcopy only make sense if
> > we are in active.
> 
> Yeah, I tend to agree, because in the cases where migration has failed
> or has been cancelled, we'll end up setting the postcopy bit.  Then,
> upon the next migration, this bit could get reused - resulting in the
> previous condition of setting postcopy bit before starting migration.

No, that doesn't happen;  the bit is cleared at the start of migration
so that race condition doesn't exist.

Dave

> 
> 
>   Amit
--
Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK

Re: [Qemu-devel] [PATCH v2] Add argument filters to the seccomp sandbox

[Eduardo Otubo]

On Thu, Sep 10, 2015 at 08=54=28PM -0400, namn...@safe-mail.net wrote:
> > The current intention of the seccomp filter in QEMU, is that /all/ existing
> > QEMU features continue to work unchanged. So even if a flag is used in a
> > seemingly uncommon code path, we still need to allow that in a seccomp
> > filter.
> It already doesn't work very well, e.g. with -chroot, it fails because 
> chroot()
> is not whitelisted, same with -runas because setgid() etc isn't whitelisted.
> Maybe there should be an extra option for -sandbox, like "-sandbox 
> experimental"
> which does argument filtering, which of course may break something, and the 
> old
> behavior would do plain syscall filtering without caring about arguments, 
> because
> that's so much easier to guarantee to work, even if it provides little 
> security.

Can you point out which exact use case breaks if you don't whitelist the
below mentioned system calls' flags?

> 
> We could also change the default behavior from SCMP_ACT_KILL (which kills the
> entire thing as soon as a single violation occurs) to SCMP_ACT_ERRNO(EPERM), 
> which
> will just return EPERM for a syscall with a violation. The software will be 
> much
> more capable of handling a permission denied error without crashing. Although 
> of
> course that violates the principle of fast-fail.

We thought about this in beggining of the development of seccomp on
qemu. Some feature like allow all, which would print to stderr all
illegal hits and a another argument like
-sandbox_add="syscall1,syscall2", but this would be against the concept
of the whole security schema. We don't want the user to take full
control of it, and if you're a developer, you know what to do.

> 
> > So we need to add DODUMP, DONTDUMP, UNMERGABLE and WILLNEED here. That
> > is still stricter than the previous allow-everything rule, so a net
> > win.
> And MADV_INVALID too I assume? That was one of the others I got with grep.
> 
> > > +
> > > +/* shmget */
> > > +rc = seccomp_rule_add(ctx, SCMP_ACT_ALLOW, SCMP_SYS(shmget), 2,
> > > +SCMP_A0(SCMP_CMP_EQ, IPC_PRIVATE),
> > > +SCMP_A2(SCMP_CMP_EQ, IP_CREAT|0777));
> > 
> > I'm not familiar with semantics of these seccomp rules, but is this
> > saying that the second arg must be exactly equal to IP_CREAT|0777 ?
> > If the app passes IP_CREAT|0600, would that be permitted instead ?
> > The latter is what I see gtk2 source code passing for mode.
> Argument 2 must be exactly equal to IP_CREAT|0777, yes, otherwise Qemu dies 
> with
> SIGSYS. I did check the Qemu source and saw 0777 was harcoded. Does the 0600 
> mask
> in GTK2 ever get called in Qemu? Anyway I added the MADV flags and the 0600 
> mask
> in the v2 patch.
> 
> Signed-off-by: Namsun Ch'o 
> ---
> diff --git a/qemu-seccomp.c b/qemu-seccomp.c
> index f9de0d3..a353ef9 100644
> --- a/qemu-seccomp.c
> +++ b/qemu-seccomp.c
> @@ -14,6 +14,8 @@
>   */
>  #include 
>  #include 
> +#include 
> +#include 
>  #include "sysemu/seccomp.h"
>  
>  struct QemuSeccompSyscall {
> @@ -105,7 +107,6 @@ static const struct QemuSeccompSyscall 
> seccomp_whitelist[] = {
>  { SCMP_SYS(rt_sigreturn), 245 },
>  { SCMP_SYS(sync), 245 },
>  { SCMP_SYS(pread64), 245 },
> -{ SCMP_SYS(madvise), 245 },
>  { SCMP_SYS(set_robust_list), 245 },
>  { SCMP_SYS(lseek), 245 },
>  { SCMP_SYS(pselect6), 245 },
> @@ -224,11 +225,9 @@ static const struct QemuSeccompSyscall 
> seccomp_whitelist[] = {
>  { SCMP_SYS(arch_prctl), 240 },
>  { SCMP_SYS(mkdir), 240 },
>  { SCMP_SYS(fchmod), 240 },
> -{ SCMP_SYS(shmget), 240 },
>  { SCMP_SYS(shmat), 240 },
>  { SCMP_SYS(shmdt), 240 },
>  { SCMP_SYS(timerfd_create), 240 },
> -{ SCMP_SYS(shmctl), 240 },
>  { SCMP_SYS(mlockall), 240 },
>  { SCMP_SYS(mlock), 240 },
>  { SCMP_SYS(munlock), 240 },
> @@ -264,6 +263,60 @@ int seccomp_start(void)
>  }
>  }
>  
> +/* madvise */
> +static const int madvise_flags[] = {
> +MADV_DODUMP,
> +MADV_DONTDUMP,
> +MADV_INVALID,
> +MADV_UNMERGEABLE,
> +MADV_WILLNEED,
> +MADV_DONTFORK,
> +MADV_DONTNEED,
> +MADV_HUGEPAGE,
> +MADV_MERGEABLE,
> +};
> +for (i = 0; i < ARRAY_SIZE(madvise_flags); i++) {
> +rc = seccomp_rule_add(ctx, SCMP_ACT_ALLOW, SCMP_SYS(madvise), 1,
> +SCMP_A2(SCMP_CMP_EQ, madvise_flags[i]));
> +if (rc < 0) {
> +goto seccomp_return;
> +}
> +}
> +rc = seccomp_syscall_priority(ctx, SCMP_SYS(madvise), 245);
> +if (rc < 0) {
> +goto seccomp_return;
> +}
> +
> +/* shmget */
> +rc = seccomp_rule_add(ctx, SCMP_ACT_ALLOW, SCMP_SYS(shmget), 2,
> +SCMP_A0(SCMP_CMP_EQ, IPC_PRIVATE),
> +SCMP_A2(SCMP_CMP_EQ, IP_CREAT|0777));
> +if (rc < 0) {
> +goto seccomp_return;
> +}
> +rc = seccomp_rule_add(ctx, SCMP_ACT_ALLOW, SCMP_SYS(shmget), 2,
> +SCMP_A0(SCMP_CMP_EQ, IPC_PRIVATE),
> 

Re: [Qemu-devel] [PATCH v11 01/12] qmp: delete qemu opts when delete an object

[Yang Hongyang]

On 09/24/2015 05:42 PM, Markus Armbruster wrote:

Yang Hongyang  writes:


On 09/24/2015 03:43 PM, Markus Armbruster wrote:

This has finally reached the front of my review queue.  I apologize for
the lng delay.

Copying Paolo for another pair of eyeballs (he wrote this code).


[...]

+
+opts = qemu_opts_find(qemu_find_opts_err("object", NULL), id);
+qemu_opts_del(opts);


qemu_find_opts_err("object", &error_abort) please, because when it
fails, we want to die right away, not when the null pointer it returns
gets dereferenced.


Thanks for the review.
Jason, do you want me to propose a fix on top of this series or simply drop
this for now because this patch is an independent bug fix and won't affect the
other filter patch series.



Same sloppiness in netdev_del_completion() and qmp_netdev_del(), not
your patch's fault.

Elsewhere, we store the QemuOpts in the object just so we can delete it:
DeviceState, DriveInfo.  Paolo, what do you think?


I don't get it. Currently, only objects created at the beginning through
QEMU command line will be stored in the QemuOpts, objects that created
with object_add won't stored in QemuOpts. Do you mean for DeviceState,
DriveInfo they store there QemuOpts explicity so that they can delete it?
Why don't we just delete it from objects directly instead?


Let me elaborate.


Thanks very much for the elaboration.



We have the same pattern in multiple places: some kind of object gets
configured via QemuOpts, and an object's QemuOpts need to stay around
until the object dies.

Example 1: Block device backends

 DriveInfo has a member opts.

 drive_new() stores the QemuOpts in dinfo->opts.

 drive_info_del() destroys dinfo->opts.

 Note: DriveInfo member opts is always non-null.  But not every
 BlockBackend has a DriveInfo.

Example 2: Device frontends

 DeviceState has a member opts.

 qdev_device_add() stores the QemuOpts in dev->opts.

 device_finalize() destroys dev->opts.

 Note: DeviceState member opts may be null (not every device is
 created by qdev_device_add()).  Fine, because qemu_opts_del(NULL) is
 a no-op.

Example 3: Character device backends

 CharDriverState has a member opts.

 qemu_chr_new_from_opts() stores the QemuOpts in chr->opts.

 qemu_chr_delete() destroys chr->opts.


1-3 store there ops in there own state, not in global ops group right?



Example 4: Network device backends

 Two cases

 A. netdev

qmp_netdev_add() does not store the QemuOpts.

qmp_netdev_del() still needs to destroy it.  It has to find it
somehow.  Here's how it does it:

opts = qemu_opts_find(qemu_find_opts_err("netdev", NULL), id);
if (!opts) {
error_setg(errp, "Device '%s' is not a netdev", id);
return;
}

The !opts condition is a non-obvious way to test "not created
with -netdev", see commit 645c949.  Note that the commit's claim
that qemu_opts_del(NULL) crashes is no longer true since commit
4782183.

 B. Legacy net

hmp_host_net_add() does not store the QemuOpts.


I'm afraid it does store the QemuOpts, but not in it's own state.
net/net.c:
1088 qemu_opt_set(opts, "type", device, &error_abort);
This will store the QemuOpts, or am I misunderstood it?



hmp_host_net_remove() still needs to destroy it.  I can't see
where that happens, and I'm not sure it does.

Example 5: Generic object

 object_create() does not store the QemuOpts.

 It still needs to be destroyed along with the object.  It isn't, and
 your patch fixes it.

Personally, I find the technique in example 1-3 easier to understand
than the one in example 4-5.


I agree that opts should not be used to determine not created something
while there's case when something created but Opts not stored.


.



--
Thanks,
Yang.

Re: [Qemu-devel] [PATCH v11 0/7] vhost-user multiple queue support

[Marcel Apfelbaum]

On 09/23/2015 07:19 AM, Yuanhan Liu wrote:

Hi,

Here is the updated patch set for enabling vhost-user multiple queue. And I did
proper and formal testing this time.

This patch set introduces 2 more vhost user messages: VHOST_USER_GET_QUEUE_NUM,
for querying how many queues the backend supports, and 
VHOST_USER_SET_VRING_ENABLE,
for enabling/disabling a specific virt queue.

Both of the two new messages are treated as vhost protocol extension,
and that's why Michaels's patch "vhost-user: add protocol feature
negotiation" is also included here.

Patch 1-5 are all prepare works for actually enabling multiple queue.

Patch 6 is the major patch for enabling multiple queue, which also tries
to address two major concerns from Michael: no feedback from backend if
it can't support # of requested queues, and all messages are sent N time.
It also fixes a hidden bug.

Patch 7 introduces the VHOST_USER_SET_VRING_ENABLE message, to enable
or disable a specific vring.

v11: - typo fixes pointed out by Eric

  - define a dummy vhost_net_get_max_queues() when !CONFIG_VHOST_NET.

  - Per suggested by Jason Wang, invoke qmp_set_link() directly to remove
duplicate code.


v10: - typo fixes pointed out by Eric

  - [PATCH 6]: don't treat VHOST_USER_SET/GET_[PROTOCOL]_FEATURES as
one time request, as the two feature bits need to be stored at
per-device.

v9: - Per suggested by Jason Wang, patch 5 introduces a new vhost
   backend method: vhost_backend_get_vq_index().

 - Use qemu_find_net_clients_except() at net/vhost-user.c for
   gathering all related ncs so that we could register chr dev
   event handler once. Which is also suggested by Jason Wang.


Thanks.

 --yliu

---
Changchun Ouyang (2):
   vhost-user: add multiple queue support
   vhost-user: add a new message to disable/enable a specific virt queue.

Michael S. Tsirkin (1):
   vhost-user: add protocol feature negotiation

Yuanhan Liu (4):
   vhost-user: use VHOST_USER_XXX macro for switch statement
   vhost: rename VHOST_RESET_OWNER to VHOST_RESET_DEVICE
   vhost-user: add VHOST_USER_GET_QUEUE_NUM message
   vhost: introduce vhost_backend_get_vq_index method

  docs/specs/vhost-user.txt |  77 -
  hw/net/vhost_net.c|  44 ++--
  hw/net/virtio-net.c   |   8 +++
  hw/virtio/vhost-backend.c |  10 ++-
  hw/virtio/vhost-user.c| 139 +++--
  hw/virtio/vhost.c |  20 +++---
  include/hw/virtio/vhost-backend.h |   4 ++
  include/hw/virtio/vhost.h |   2 +
  include/net/vhost_net.h   |   3 +
  linux-headers/linux/vhost.h   |   2 +-
  net/vhost-user.c  | 141 +-
  qapi-schema.json  |   6 +-
  qemu-options.hx   |   5 +-
  tests/vhost-user-test.c   |   2 +-
  14 files changed, 371 insertions(+), 92 deletions(-)



Thanks a lot for your work!

Tested-by: Marcel Apfelbaum 

Re: [Qemu-devel] [PATCH v11 01/12] qmp: delete qemu opts when delete an object

[Yang Hongyang]

On 09/24/2015 05:42 PM, Markus Armbruster wrote:

Yang Hongyang  writes:


On 09/24/2015 03:43 PM, Markus Armbruster wrote:

This has finally reached the front of my review queue.  I apologize for
the lng delay.

Copying Paolo for another pair of eyeballs (he wrote this code).


[...]

+
+opts = qemu_opts_find(qemu_find_opts_err("object", NULL), id);
+qemu_opts_del(opts);


qemu_find_opts_err("object", &error_abort) please, because when it
fails, we want to die right away, not when the null pointer it returns
gets dereferenced.


Thanks for the review.
Jason, do you want me to propose a fix on top of this series or simply drop
this for now because this patch is an independent bug fix and won't affect the
other filter patch series.



Same sloppiness in netdev_del_completion() and qmp_netdev_del(), not
your patch's fault.

Elsewhere, we store the QemuOpts in the object just so we can delete it:
DeviceState, DriveInfo.  Paolo, what do you think?


I don't get it. Currently, only objects created at the beginning through
QEMU command line will be stored in the QemuOpts, objects that created
with object_add won't stored in QemuOpts. Do you mean for DeviceState,
DriveInfo they store there QemuOpts explicity so that they can delete it?
Why don't we just delete it from objects directly instead?


Let me elaborate.

We have the same pattern in multiple places: some kind of object gets
configured via QemuOpts, and an object's QemuOpts need to stay around
until the object dies.

Example 1: Block device backends

 DriveInfo has a member opts.

 drive_new() stores the QemuOpts in dinfo->opts.

 drive_info_del() destroys dinfo->opts.

 Note: DriveInfo member opts is always non-null.  But not every
 BlockBackend has a DriveInfo.

Example 2: Device frontends

 DeviceState has a member opts.

 qdev_device_add() stores the QemuOpts in dev->opts.

 device_finalize() destroys dev->opts.

 Note: DeviceState member opts may be null (not every device is
 created by qdev_device_add()).  Fine, because qemu_opts_del(NULL) is
 a no-op.

Example 3: Character device backends

 CharDriverState has a member opts.

 qemu_chr_new_from_opts() stores the QemuOpts in chr->opts.

 qemu_chr_delete() destroys chr->opts.

Example 4: Network device backends

 Two cases

 A. netdev

qmp_netdev_add() does not store the QemuOpts.


The QemuOpts stored by qmp_netdev_add() and also hmp_netdev_add().
through this function:
net/net.c: qmp_netdev_add()
1134 opts = qemu_opts_from_qdict(opts_list, qdict, &local_err);

hmp.c: hmp_netdev_add()
1579 opts = qemu_opts_from_qdict(qemu_find_opts("netdev"), qdict, &err);



qmp_netdev_del() still needs to destroy it.  It has to find it
somehow.  Here's how it does it:

opts = qemu_opts_find(qemu_find_opts_err("netdev", NULL), id);
if (!opts) {
error_setg(errp, "Device '%s' is not a netdev", id);
return;
}

The !opts condition is a non-obvious way to test "not created
with -netdev", see commit 645c949.  Note that the commit's claim
that qemu_opts_del(NULL) crashes is no longer true since commit
4782183.

 B. Legacy net

hmp_host_net_add() does not store the QemuOpts.

hmp_host_net_remove() still needs to destroy it.  I can't see
where that happens, and I'm not sure it does.

Example 5: Generic object

 object_create() does not store the QemuOpts.

 It still needs to be destroyed along with the object.  It isn't, and
 your patch fixes it.

Personally, I find the technique in example 1-3 easier to understand
than the one in example 4-5.
.



--
Thanks,
Yang.

Re: [Qemu-devel] [PATCH] iotests: disable core dumps in test 061

[Kevin Wolf]

Am 24.09.2015 um 08:45 hat Markus Armbruster geschrieben:
> Max Reitz  writes:
> 
> > On 23.09.2015 18:11, Alberto Garcia wrote:
> >> Commit 934659c460 disabled the supression of segmentation faults in
> >> bash tests. The new output of test 061, however, assumes that a core
> >> dump will be produced if a program aborts. This is not necessarily the
> >> case because core dumps can be disabled using ulimit.
> >> 
> >> We cannot guarantee that core dumps can be enabled in all cases, so we
> >> should disable them completely and update the test output accordingly.
> >> 
> >> Signed-off-by: Alberto Garcia 
> >> ---
> >>  tests/qemu-iotests/061 | 3 +++
> >>  tests/qemu-iotests/061.out | 4 ++--
> >>  2 files changed, 5 insertions(+), 2 deletions(-)
> >
> > As noted in the commit message for
> > 3f394472c5bca59de5cab9baafdff1984b0213a3, ulimit -c 0 does not work for
> > everyone (for instance, for me it fails, probably because I'm using
> > systemd's coredumpctl). Generally speaking, it'll only prevent a core
> > dump from being created if your /proc/sys/kernel/core_pattern points to
> > a file, but it won't if it points to a program for gathering the dump.
> >
> > What we really want is to use "sigraise $(kill -l KILL)" instead of
> > "abort", because SIGKILL never creates a core dump, but will have
> > basically the same effect of crashing qemu-io.
> 
> No, we don't want that.  SIGABRT gives the user the option to have core
> dumps.  By switching to SIGKILL, you'd take away that option.

Why do you need a core dump for a test case that intentionally simulates
a crash without any actual misbehaviour causing it? Isn't it actually
annoying to get useless core dumps?

> Because modern systems have complicated the ways you can get and not get
> core dumps, user qemu-iotests is having difficulties getting one
> reliably.  Since it's just as fine with getting none reliably, and a
> reliably way to ask for that still exists (ulimit -c 0), it could do
> just that.

If you reread Max' email carefully, his very point is that 'ulimit -c 0'
is _not_ reliable.

> Inconvenience: when a test fails, you can't examine its core dump
> anymore, but have to instrument the test to create one, or splice in
> gdb, or whatever else it takes.  On the other hand, you don't have to
> delete core dumps anymore.

If we switched the intentional crash to SIGKILL, you could still get
core dumps for cases where there is actual misbehaviour without touching
the script. 'ulimit -c 0' in contrast, in addition to not being
reliable, is all or nothing.

> Possible alternative: normalize the crash message differences before
> diffing against golden output.

Extending _filter_qemu_io is another viable option to make the test
pass, yes. However, you would still need to manually delete core dumps
from the intentional crash.

Kevin

Re: [Qemu-devel] [PATCH] hw/arm/virt: smbios: inform guest of kvm

[Andrew Jones]

On Wed, Sep 23, 2015 at 08:43:39AM -0700, Peter Maydell wrote:
> On 23 September 2015 at 07:18, Andrew Jones  wrote:
> > ARM/AArch64 KVM guests don't have any way to identify
> > themselves as KVM guests (x86 guests use a CPUID leaf). Now, we
> > could discuss all sorts of reasons why guests shouldn't need to
> > know that, but then there's always some case where it'd be
> > nice... Anyway, now that we have SMBIOS tables in ARM guests,
> > it's easy for the guest to know that it's a QEMU instance. This
> > patch takes that one step further, also identifying KVM, when
> > appropriate. Again, we could debate why generally nothing
> > should care whether it's of type QEMU or QEMU/KVM, but again,
> > sometimes it's nice to know...
> 
> This doesn't seem great to me, because it's ACPI/SMBIOS
> specific. A mechanism that worked whether the guest was
> booted via APCI or DT would seem preferable to me...

SMBIOS is populated on both ACPI and devicetree boots. We already
have detection in virt-what and systemd-detect-virt for DT boots,
although it only detects QEMU (it can't determine if KVM is used).
That detection is DT-specific, and much more of a heuristic, it
checks for the presence of the fw-cfg node in the DT. Actually, I'd
like to patch those virt detection tools to try SMBIOS first (which,
with this patch, could also give KVM info), and then fall back to
trying the current DT-only, QEMU-only detection, before giving up.

Thanks,
drew

Re: [Qemu-devel] [PATCH v11 2/7] vhost-user: add protocol feature negotiation

[Marcel Apfelbaum]

On 09/23/2015 07:19 AM, Yuanhan Liu wrote:

From: "Michael S. Tsirkin" 

Support a separate bitmask for vhost-user protocol features,
and messages to get/set protocol features.

Invoke them at init.

No features are defined yet.

[ leverage vhost_user_call for request handling -- Yuanhan Liu ]

Signed-off-by: Michael S. Tsirkin 
Signed-off-by: Yuanhan Liu 
---
  docs/specs/vhost-user.txt | 37 +
  hw/net/vhost_net.c|  2 ++
  hw/virtio/vhost-user.c| 31 +++
  include/hw/virtio/vhost.h |  1 +
  4 files changed, 71 insertions(+)

diff --git a/docs/specs/vhost-user.txt b/docs/specs/vhost-user.txt
index 650bb18..70da3b1 100644
--- a/docs/specs/vhost-user.txt
+++ b/docs/specs/vhost-user.txt
@@ -113,6 +113,7 @@ message replies. Most of the requests don't require 
replies. Here is a list of
  the ones that do:

   * VHOST_GET_FEATURES
+ * VHOST_GET_PROTOCOL_FEATURES
   * VHOST_GET_VRING_BASE

  There are several messages that the master sends with file descriptors passed
@@ -127,6 +128,13 @@ in the ancillary data:
  If Master is unable to send the full message or receives a wrong reply it will
  close the connection. An optional reconnection mechanism can be implemented.

+Any protocol extensions are gated by protocol feature bits,
+which allows full backwards compatibility on both master
+and slave.
+As older slaves don't support negotiating protocol features,
+a feature bit was dedicated for this purpose:
+#define VHOST_USER_F_PROTOCOL_FEATURES 30
+
  Message types
  -

@@ -138,6 +146,8 @@ Message types
Slave payload: u64

Get from the underlying vhost implementation the features bitmask.
+  Feature bit VHOST_USER_F_PROTOCOL_FEATURES signals slave support for
+  VHOST_USER_GET_PROTOCOL_FEATURES and VHOST_USER_SET_PROTOCOL_FEATURES.

   * VHOST_USER_SET_FEATURES

@@ -146,6 +156,33 @@ Message types
Master payload: u64

Enable features in the underlying vhost implementation using a bitmask.
+  Feature bit VHOST_USER_F_PROTOCOL_FEATURES signals slave support for
+  VHOST_USER_GET_PROTOCOL_FEATURES and VHOST_USER_SET_PROTOCOL_FEATURES.
+
+ * VHOST_USER_GET_PROTOCOL_FEATURES
+
+  Id: 15
+  Equivalent ioctl: VHOST_GET_FEATURES


VHOST_USER_GET_PROTOCOL_FEATURES does not have an equivalent ioctl


+  Master payload: N/A
+  Slave payload: u64
+
+  Get the protocol feature bitmask from the underlying vhost 
implementation.
+  Only legal if feature bit VHOST_USER_F_PROTOCOL_FEATURES is present in
+  VHOST_USER_GET_FEATURES.
+  Note: slave that reported VHOST_USER_F_PROTOCOL_FEATURES must support
+  this message even before VHOST_USER_SET_FEATURES was called.
+
+ * VHOST_USER_SET_PROTOCOL_FEATURES
+
+  Id: 16
+  Ioctl: VHOST_SET_FEATURES


Same here


+  Master payload: u64
+
+  Enable protocol features in the underlying vhost implementation.
+  Only legal if feature bit VHOST_USER_F_PROTOCOL_FEATURES is present in
+  VHOST_USER_GET_FEATURES.
+  Note: slave that reported VHOST_USER_F_PROTOCOL_FEATURES must support
+  this message even before VHOST_USER_SET_FEATURES was called.

   * VHOST_USER_SET_OWNER

diff --git a/hw/net/vhost_net.c b/hw/net/vhost_net.c
index 1d76b94..9d32d76 100644
--- a/hw/net/vhost_net.c
+++ b/hw/net/vhost_net.c
@@ -152,8 +152,10 @@ struct vhost_net *vhost_net_init(VhostNetOptions *options)
  net->dev.backend_features = qemu_has_vnet_hdr(options->net_backend)
  ? 0 : (1ULL << VHOST_NET_F_VIRTIO_NET_HDR);
  net->backend = r;
+net->dev.protocol_features = 0;
  } else {
  net->dev.backend_features = 0;
+net->dev.protocol_features = 0;
  net->backend = -1;
  }


Maybe protocol_features assignment should be outside the if clause.
(assigned to 0 in both cases)


  net->nc = options->net_backend;
diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c
index 13677ac..7fe35c6 100644
--- a/hw/virtio/vhost-user.c
+++ b/hw/virtio/vhost-user.c
@@ -24,6 +24,8 @@
  #include 

  #define VHOST_MEMORY_MAX_NREGIONS8
+#define VHOST_USER_F_PROTOCOL_FEATURES 30
+#define VHOST_USER_PROTOCOL_FEATURE_MASK 0x0ULL

  typedef enum VhostUserRequest {
  VHOST_USER_NONE = 0,
@@ -41,6 +43,8 @@ typedef enum VhostUserRequest {
  VHOST_USER_SET_VRING_KICK = 12,
  VHOST_USER_SET_VRING_CALL = 13,
  VHOST_USER_SET_VRING_ERR = 14,
+VHOST_USER_GET_PROTOCOL_FEATURES = 15,
+VHOST_USER_SET_PROTOCOL_FEATURES = 16,
  VHOST_USER_MAX
  } VhostUserRequest;

@@ -206,11 +210,13 @@ static int vhost_user_call(struct vhost_dev *dev, 
unsigned long int request,

  switch (msg_request) {
  case VHOST_USER_GET_FEATURES:
+case VHOST_USER_GET_PROTOCOL_FEATURES:
  need_reply = 1;
  break;

  case VHOST_USER_SET_FEATURES:
  case VHOST_USER_SET_LOG_BASE:
+case VHOST_USER_SET_PROTOCOL_FEATURES:
  msg.u64 = *(

Re: [Qemu-devel] [PATCH v2 04/11] virtio-gpu: move iov free to virtio_gpu_cleanup_mapping_iov

[Michael S. Tsirkin]

On Thu, Sep 24, 2015 at 11:04:55AM +0200, Gerd Hoffmann wrote:
> Signed-off-by: Gerd Hoffmann 

It's easy to see this is what the patch does.  But why?  Some
explanation in the commit log about why it's done, as opposed to what is
done, would be better.

> ---
>  hw/display/virtio-gpu.c | 3 +--
>  1 file changed, 1 insertion(+), 2 deletions(-)
> 
> diff --git a/hw/display/virtio-gpu.c b/hw/display/virtio-gpu.c
> index a67d927..73bd9b6 100644
> --- a/hw/display/virtio-gpu.c
> +++ b/hw/display/virtio-gpu.c
> @@ -563,7 +563,6 @@ int virtio_gpu_create_mapping_iov(struct 
> virtio_gpu_resource_attach_backing *ab,
>__func__, ab->resource_id, i);
>  virtio_gpu_cleanup_mapping_iov(*iov, i);
>  g_free(ents);
> -g_free(*iov);
>  *iov = NULL;
>  return -1;
>  }
> @@ -580,12 +579,12 @@ void virtio_gpu_cleanup_mapping_iov(struct iovec *iov, 
> uint32_t count)
>  cpu_physical_memory_unmap(iov[i].iov_base, iov[i].iov_len, 1,
>iov[i].iov_len);
>  }
> +g_free(iov);
>  }
>  
>  static void virtio_gpu_cleanup_mapping(struct virtio_gpu_simple_resource 
> *res)
>  {
>  virtio_gpu_cleanup_mapping_iov(res->iov, res->iov_cnt);
> -g_free(res->iov);
>  res->iov = NULL;
>  res->iov_cnt = 0;
>  }
> -- 
> 1.8.3.1

Re: [Qemu-devel] [PATCH] hw/arm/virt: smbios: inform guest of kvm

[Andrew Jones]

On Wed, Sep 23, 2015 at 09:09:11AM -0700, Peter Maydell wrote:
> On 23 September 2015 at 08:43, Peter Maydell  wrote:
> > On 23 September 2015 at 07:18, Andrew Jones  wrote:
> >> ARM/AArch64 KVM guests don't have any way to identify
> >> themselves as KVM guests (x86 guests use a CPUID leaf). Now, we
> >> could discuss all sorts of reasons why guests shouldn't need to
> >> know that, but then there's always some case where it'd be
> >> nice... Anyway, now that we have SMBIOS tables in ARM guests,
> >> it's easy for the guest to know that it's a QEMU instance. This
> >> patch takes that one step further, also identifying KVM, when
> >> appropriate. Again, we could debate why generally nothing
> >> should care whether it's of type QEMU or QEMU/KVM, but again,
> >> sometimes it's nice to know...
> >
> > This doesn't seem great to me, because it's ACPI/SMBIOS
> > specific. A mechanism that worked whether the guest was
> > booted via APCI or DT would seem preferable to me...
> 
> ...and, on a more careful reread, I don't like the way the
> behaviour is changing for "this is KVM" rather than "and it's
> KVM" being an additional piece of information.

It is "and it's KVM". We leave the SMBIOS Manufacturer as "QEMU",
so QEMU is still there, and even in the "primary" field. We only
change the Product Name to be KVM only, although that could
certainly be "QEMU/KVM Virtual Machine", if preferred.

Thanks,
drew

Re: [Qemu-devel] [PATCH v11 1/7] vhost-user: use VHOST_USER_XXX macro for switch statement

[Michael S. Tsirkin]

On Thu, Sep 24, 2015 at 01:05:11PM +0300, Marcel Apfelbaum wrote:
> On 09/23/2015 07:19 AM, Yuanhan Liu wrote:
> >So that we could let vhost_user_call to handle extented requests,
> >such as VHOST_USER_GET/SET_PROTOCOL_FEATURES, instead of invoking
> >vhost_user_read/write and constructing the msg again by ourself.
> >
> >Signed-off-by: Yuanhan Liu 
> >---
> >  hw/virtio/vhost-user.c | 38 ++
> >  1 file changed, 22 insertions(+), 16 deletions(-)
> >
> >diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c
> >index e7ab829..13677ac 100644
> >--- a/hw/virtio/vhost-user.c
> >+++ b/hw/virtio/vhost-user.c
> >@@ -193,27 +193,33 @@ static int vhost_user_call(struct vhost_dev *dev, 
> >unsigned long int request,
> >
> >  assert(dev->vhost_ops->backend_type == VHOST_BACKEND_TYPE_USER);
> >
> >-msg_request = vhost_user_request_translate(request);
> >+/* only translate vhost ioctl requests */
> >+if (request > VHOST_USER_MAX) {
> >+msg_request = vhost_user_request_translate(request);
> >+} else {
> >+msg_request = request;
> >+}
> >+
> >  msg.request = msg_request;
> >  msg.flags = VHOST_USER_VERSION;
> >  msg.size = 0;
> >
> >-switch (request) {
> >-case VHOST_GET_FEATURES:
> >+switch (msg_request) {
> >+case VHOST_USER_GET_FEATURES:
> >  need_reply = 1;
> >  break;
> >
> >-case VHOST_SET_FEATURES:
> >-case VHOST_SET_LOG_BASE:
> >+case VHOST_USER_SET_FEATURES:
> >+case VHOST_USER_SET_LOG_BASE:
> >  msg.u64 = *((__u64 *) arg);
> >  msg.size = sizeof(m.u64);
> >  break;
> >
> >-case VHOST_SET_OWNER:
> >-case VHOST_RESET_OWNER:
> >+case VHOST_USER_SET_OWNER:
> >+case VHOST_USER_RESET_OWNER:
> >  break;
> >
> >-case VHOST_SET_MEM_TABLE:
> >+case VHOST_USER_SET_MEM_TABLE:
> >  for (i = 0; i < dev->mem->nregions; ++i) {
> >  struct vhost_memory_region *reg = dev->mem->regions + i;
> >  ram_addr_t ram_addr;
> >@@ -246,30 +252,30 @@ static int vhost_user_call(struct vhost_dev *dev, 
> >unsigned long int request,
> >
> >  break;
> >
> >-case VHOST_SET_LOG_FD:
> >+case VHOST_USER_SET_LOG_FD:
> >  fds[fd_num++] = *((int *) arg);
> >  break;
> >
> >-case VHOST_SET_VRING_NUM:
> >-case VHOST_SET_VRING_BASE:
> >+case VHOST_USER_SET_VRING_NUM:
> >+case VHOST_USER_SET_VRING_BASE:
> >  memcpy(&msg.state, arg, sizeof(struct vhost_vring_state));
> >  msg.size = sizeof(m.state);
> >  break;
> >
> >-case VHOST_GET_VRING_BASE:
> >+case VHOST_USER_GET_VRING_BASE:
> >  memcpy(&msg.state, arg, sizeof(struct vhost_vring_state));
> >  msg.size = sizeof(m.state);
> >  need_reply = 1;
> >  break;
> >
> >-case VHOST_SET_VRING_ADDR:
> >+case VHOST_USER_SET_VRING_ADDR:
> >  memcpy(&msg.addr, arg, sizeof(struct vhost_vring_addr));
> >  msg.size = sizeof(m.addr);
> >  break;
> >
> >-case VHOST_SET_VRING_KICK:
> >-case VHOST_SET_VRING_CALL:
> >-case VHOST_SET_VRING_ERR:
> >+case VHOST_USER_SET_VRING_KICK:
> >+case VHOST_USER_SET_VRING_CALL:
> >+case VHOST_USER_SET_VRING_ERR:
> >  file = arg;
> >  msg.u64 = file->index & VHOST_USER_VRING_IDX_MASK;
> >  msg.size = sizeof(m.u64);
> >
> 
> 
> -- 
> Reviewed-by: Marcel Apfelbaum 

The -- is the signature separator.
Please don't put text after that - I almost missed it.

Re: [Qemu-devel] [PATCH v11 3/7] vhost: rename VHOST_RESET_OWNER to VHOST_RESET_DEVICE

[Marcel Apfelbaum]

On 09/23/2015 07:19 AM, Yuanhan Liu wrote:

Quote from Michael:

 We really should rename VHOST_RESET_OWNER to VHOST_RESET_DEVICE.


I suggest to change this to a better commit message :)
You can take anything from the mail thread discussion about it.



Suggested-by: Michael S. Tsirkin 
Signed-off-by: Yuanhan Liu 


Besides that:
--
Reviewed-by: Marcel Apfelbaum 



---
  docs/specs/vhost-user.txt   | 4 ++--
  hw/net/vhost_net.c  | 2 +-
  hw/virtio/vhost-user.c  | 6 +++---
  linux-headers/linux/vhost.h | 2 +-
  tests/vhost-user-test.c | 2 +-
  5 files changed, 8 insertions(+), 8 deletions(-)

diff --git a/docs/specs/vhost-user.txt b/docs/specs/vhost-user.txt
index 70da3b1..ccbbcbb 100644
--- a/docs/specs/vhost-user.txt
+++ b/docs/specs/vhost-user.txt
@@ -194,10 +194,10 @@ Message types
as an owner of the session. This can be used on the Slave as a
"session start" flag.

- * VHOST_USER_RESET_OWNER
+ * VHOST_USER_RESET_DEVICE

Id: 4
-  Equivalent ioctl: VHOST_RESET_OWNER
+  Equivalent ioctl: VHOST_RESET_DEVICE
Master payload: N/A

Issued when a new connection is about to be closed. The Master will no
diff --git a/hw/net/vhost_net.c b/hw/net/vhost_net.c
index 9d32d76..b7d29b7 100644
--- a/hw/net/vhost_net.c
+++ b/hw/net/vhost_net.c
@@ -287,7 +287,7 @@ static void vhost_net_stop_one(struct vhost_net *net,
  } else if (net->nc->info->type == NET_CLIENT_OPTIONS_KIND_VHOST_USER) {
  for (file.index = 0; file.index < net->dev.nvqs; ++file.index) {
  const VhostOps *vhost_ops = net->dev.vhost_ops;
-int r = vhost_ops->vhost_call(&net->dev, VHOST_RESET_OWNER,
+int r = vhost_ops->vhost_call(&net->dev, VHOST_RESET_DEVICE,
NULL);
  assert(r >= 0);
  }
diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c
index 7fe35c6..9cb2f52 100644
--- a/hw/virtio/vhost-user.c
+++ b/hw/virtio/vhost-user.c
@@ -32,7 +32,7 @@ typedef enum VhostUserRequest {
  VHOST_USER_GET_FEATURES = 1,
  VHOST_USER_SET_FEATURES = 2,
  VHOST_USER_SET_OWNER = 3,
-VHOST_USER_RESET_OWNER = 4,
+VHOST_USER_RESET_DEVICE = 4,
  VHOST_USER_SET_MEM_TABLE = 5,
  VHOST_USER_SET_LOG_BASE = 6,
  VHOST_USER_SET_LOG_FD = 7,
@@ -98,7 +98,7 @@ static unsigned long int 
ioctl_to_vhost_user_request[VHOST_USER_MAX] = {
  VHOST_GET_FEATURES, /* VHOST_USER_GET_FEATURES */
  VHOST_SET_FEATURES, /* VHOST_USER_SET_FEATURES */
  VHOST_SET_OWNER,/* VHOST_USER_SET_OWNER */
-VHOST_RESET_OWNER,  /* VHOST_USER_RESET_OWNER */
+VHOST_RESET_DEVICE,  /* VHOST_USER_RESET_DEVICE */
  VHOST_SET_MEM_TABLE,/* VHOST_USER_SET_MEM_TABLE */
  VHOST_SET_LOG_BASE, /* VHOST_USER_SET_LOG_BASE */
  VHOST_SET_LOG_FD,   /* VHOST_USER_SET_LOG_FD */
@@ -222,7 +222,7 @@ static int vhost_user_call(struct vhost_dev *dev, unsigned 
long int request,
  break;

  case VHOST_USER_SET_OWNER:
-case VHOST_USER_RESET_OWNER:
+case VHOST_USER_RESET_DEVICE:
  break;

  case VHOST_USER_SET_MEM_TABLE:
diff --git a/linux-headers/linux/vhost.h b/linux-headers/linux/vhost.h
index ead86db..14a0160 100644
--- a/linux-headers/linux/vhost.h
+++ b/linux-headers/linux/vhost.h
@@ -78,7 +78,7 @@ struct vhost_memory {
  #define VHOST_SET_OWNER _IO(VHOST_VIRTIO, 0x01)
  /* Give up ownership, and reset the device to default values.
   * Allows subsequent call to VHOST_OWNER_SET to succeed. */
-#define VHOST_RESET_OWNER _IO(VHOST_VIRTIO, 0x02)
+#define VHOST_RESET_DEVICE _IO(VHOST_VIRTIO, 0x02)

  /* Set up/modify memory layout */
  #define VHOST_SET_MEM_TABLE   _IOW(VHOST_VIRTIO, 0x03, struct vhost_memory)
diff --git a/tests/vhost-user-test.c b/tests/vhost-user-test.c
index 75fedf0..e301db7 100644
--- a/tests/vhost-user-test.c
+++ b/tests/vhost-user-test.c
@@ -58,7 +58,7 @@ typedef enum VhostUserRequest {
  VHOST_USER_GET_FEATURES = 1,
  VHOST_USER_SET_FEATURES = 2,
  VHOST_USER_SET_OWNER = 3,
-VHOST_USER_RESET_OWNER = 4,
+VHOST_USER_RESET_DEVICE = 4,
  VHOST_USER_SET_MEM_TABLE = 5,
  VHOST_USER_SET_LOG_BASE = 6,
  VHOST_USER_SET_LOG_FD = 7,

Re: [Qemu-devel] [PATCH v11 1/7] vhost-user: use VHOST_USER_XXX macro for switch statement

[Marcel Apfelbaum]

On 09/23/2015 07:19 AM, Yuanhan Liu wrote:

So that we could let vhost_user_call to handle extented requests,
such as VHOST_USER_GET/SET_PROTOCOL_FEATURES, instead of invoking
vhost_user_read/write and constructing the msg again by ourself.

Signed-off-by: Yuanhan Liu 
---
  hw/virtio/vhost-user.c | 38 ++
  1 file changed, 22 insertions(+), 16 deletions(-)

diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c
index e7ab829..13677ac 100644
--- a/hw/virtio/vhost-user.c
+++ b/hw/virtio/vhost-user.c
@@ -193,27 +193,33 @@ static int vhost_user_call(struct vhost_dev *dev, 
unsigned long int request,

  assert(dev->vhost_ops->backend_type == VHOST_BACKEND_TYPE_USER);

-msg_request = vhost_user_request_translate(request);
+/* only translate vhost ioctl requests */
+if (request > VHOST_USER_MAX) {
+msg_request = vhost_user_request_translate(request);
+} else {
+msg_request = request;
+}
+
  msg.request = msg_request;
  msg.flags = VHOST_USER_VERSION;
  msg.size = 0;

-switch (request) {
-case VHOST_GET_FEATURES:
+switch (msg_request) {
+case VHOST_USER_GET_FEATURES:
  need_reply = 1;
  break;

-case VHOST_SET_FEATURES:
-case VHOST_SET_LOG_BASE:
+case VHOST_USER_SET_FEATURES:
+case VHOST_USER_SET_LOG_BASE:
  msg.u64 = *((__u64 *) arg);
  msg.size = sizeof(m.u64);
  break;

-case VHOST_SET_OWNER:
-case VHOST_RESET_OWNER:
+case VHOST_USER_SET_OWNER:
+case VHOST_USER_RESET_OWNER:
  break;

-case VHOST_SET_MEM_TABLE:
+case VHOST_USER_SET_MEM_TABLE:
  for (i = 0; i < dev->mem->nregions; ++i) {
  struct vhost_memory_region *reg = dev->mem->regions + i;
  ram_addr_t ram_addr;
@@ -246,30 +252,30 @@ static int vhost_user_call(struct vhost_dev *dev, 
unsigned long int request,

  break;

-case VHOST_SET_LOG_FD:
+case VHOST_USER_SET_LOG_FD:
  fds[fd_num++] = *((int *) arg);
  break;

-case VHOST_SET_VRING_NUM:
-case VHOST_SET_VRING_BASE:
+case VHOST_USER_SET_VRING_NUM:
+case VHOST_USER_SET_VRING_BASE:
  memcpy(&msg.state, arg, sizeof(struct vhost_vring_state));
  msg.size = sizeof(m.state);
  break;

-case VHOST_GET_VRING_BASE:
+case VHOST_USER_GET_VRING_BASE:
  memcpy(&msg.state, arg, sizeof(struct vhost_vring_state));
  msg.size = sizeof(m.state);
  need_reply = 1;
  break;

-case VHOST_SET_VRING_ADDR:
+case VHOST_USER_SET_VRING_ADDR:
  memcpy(&msg.addr, arg, sizeof(struct vhost_vring_addr));
  msg.size = sizeof(m.addr);
  break;

-case VHOST_SET_VRING_KICK:
-case VHOST_SET_VRING_CALL:
-case VHOST_SET_VRING_ERR:
+case VHOST_USER_SET_VRING_KICK:
+case VHOST_USER_SET_VRING_CALL:
+case VHOST_USER_SET_VRING_ERR:
  file = arg;
  msg.u64 = file->index & VHOST_USER_VRING_IDX_MASK;
  msg.size = sizeof(m.u64);




--
Reviewed-by: Marcel Apfelbaum 

Re: [Qemu-devel] [PATCH v11 4/7] vhost-user: add VHOST_USER_GET_QUEUE_NUM message

[Marcel Apfelbaum]

On 09/23/2015 07:19 AM, Yuanhan Liu wrote:

This is for querying how many queues the backend supports if it has mq
support(when VHOST_USER_PROTOCOL_F_MQ flag is set from the quried


/s/quried/queried

Only if you plan to send another version, we can fix it on top.


protocol features).

vhost_net_get_max_queues() is the interface to export that value, and
to tell if the backend supports # of queues user requested, which is
done in the following patch.

Signed-off-by: Yuanhan Liu 

---
v11: define a dummy vhost_net_get_max_queues when !CONFIG_VHOST_NET.
---
  docs/specs/vhost-user.txt | 11 +++
  hw/net/vhost_net.c| 12 
  hw/virtio/vhost-user.c| 15 ++-
  include/hw/virtio/vhost.h |  1 +
  include/net/vhost_net.h   |  1 +
  5 files changed, 39 insertions(+), 1 deletion(-)

diff --git a/docs/specs/vhost-user.txt b/docs/specs/vhost-user.txt
index ccbbcbb..43db9b4 100644
--- a/docs/specs/vhost-user.txt
+++ b/docs/specs/vhost-user.txt
@@ -301,3 +301,14 @@ Message types
Bits (0-7) of the payload contain the vring index. Bit 8 is the
invalid FD flag. This flag is set when there is no file descriptor
in the ancillary data.
+
+ * VHOST_USER_GET_QUEUE_NUM
+
+  Id: 17
+  Equivalent ioctl: N/A
+  Master payload: N/A
+  Slave payload: u64
+
+  Query how many queues the backend supports. This request should be
+  sent only when VHOST_USER_PROTOCOL_F_MQ is set in quried protocol
+  features by VHOST_USER_GET_PROTOCOL_FEATURES.
diff --git a/hw/net/vhost_net.c b/hw/net/vhost_net.c
index b7d29b7..f663e5a 100644
--- a/hw/net/vhost_net.c
+++ b/hw/net/vhost_net.c
@@ -122,6 +122,11 @@ void vhost_net_ack_features(struct vhost_net *net, 
uint64_t features)
  vhost_ack_features(&net->dev, vhost_net_get_feature_bits(net), features);
  }

+uint64_t vhost_net_get_max_queues(VHostNetState *net)
+{
+return net->dev.max_queues;
+}
+
  static int vhost_net_get_fd(NetClientState *backend)
  {
  switch (backend->info->type) {
@@ -144,6 +149,8 @@ struct vhost_net *vhost_net_init(VhostNetOptions *options)
  goto fail;
  }

+net->dev.max_queues = 1;
+
  if (backend_kernel) {
  r = vhost_net_get_fd(options->net_backend);
  if (r < 0) {
@@ -414,6 +421,11 @@ VHostNetState *get_vhost_net(NetClientState *nc)
  return vhost_net;
  }
  #else
+uint64_t vhost_net_get_max_queues(VHostNetState *net)
+{
+return 1;
+}
+
  struct vhost_net *vhost_net_init(VhostNetOptions *options)
  {
  error_report("vhost-net support is not compiled in");
diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c
index 9cb2f52..694fde5 100644
--- a/hw/virtio/vhost-user.c
+++ b/hw/virtio/vhost-user.c
@@ -25,7 +25,9 @@

  #define VHOST_MEMORY_MAX_NREGIONS8
  #define VHOST_USER_F_PROTOCOL_FEATURES 30
-#define VHOST_USER_PROTOCOL_FEATURE_MASK 0x0ULL
+#define VHOST_USER_PROTOCOL_FEATURE_MASK 0x1ULL
+
+#define VHOST_USER_PROTOCOL_F_MQ0

  typedef enum VhostUserRequest {
  VHOST_USER_NONE = 0,
@@ -45,6 +47,7 @@ typedef enum VhostUserRequest {
  VHOST_USER_SET_VRING_ERR = 14,
  VHOST_USER_GET_PROTOCOL_FEATURES = 15,
  VHOST_USER_SET_PROTOCOL_FEATURES = 16,
+VHOST_USER_GET_QUEUE_NUM = 17,
  VHOST_USER_MAX
  } VhostUserRequest;

@@ -211,6 +214,7 @@ static int vhost_user_call(struct vhost_dev *dev, unsigned 
long int request,
  switch (msg_request) {
  case VHOST_USER_GET_FEATURES:
  case VHOST_USER_GET_PROTOCOL_FEATURES:
+case VHOST_USER_GET_QUEUE_NUM:
  need_reply = 1;
  break;

@@ -315,6 +319,7 @@ static int vhost_user_call(struct vhost_dev *dev, unsigned 
long int request,
  switch (msg_request) {
  case VHOST_USER_GET_FEATURES:
  case VHOST_USER_GET_PROTOCOL_FEATURES:
+case VHOST_USER_GET_QUEUE_NUM:
  if (msg.size != sizeof(m.u64)) {
  error_report("Received bad msg size.");
  return -1;
@@ -366,6 +371,14 @@ static int vhost_user_init(struct vhost_dev *dev, void 
*opaque)
  if (err < 0) {
  return err;
  }
+
+/* query the max queues we support if backend supports Multiple Queue 
*/
+if (dev->protocol_features & (1ULL << VHOST_USER_PROTOCOL_F_MQ)) {
+err = vhost_user_call(dev, VHOST_USER_GET_QUEUE_NUM, 
&dev->max_queues);
+if (err < 0) {
+return err;
+}
+}
  }

  return 0;
diff --git a/include/hw/virtio/vhost.h b/include/hw/virtio/vhost.h
index 6467c73..c3758f3 100644
--- a/include/hw/virtio/vhost.h
+++ b/include/hw/virtio/vhost.h
@@ -48,6 +48,7 @@ struct vhost_dev {
  unsigned long long acked_features;
  unsigned long long backend_features;
  unsigned long long protocol_features;
+unsigned long long max_queues;
  bool started;
  bool log_enabled;
  unsigned long long log_size;
diff --git a/include/net/vhost_net.h b/include/net/vhost_net.h
index 840

Re: [Qemu-devel] RFC: virtio-peer shared memory based peer communication device

[Michael S. Tsirkin]

On Mon, Sep 21, 2015 at 02:32:10PM +0200, Jan Kiszka wrote:
> On 2015-09-21 14:13, Michael S. Tsirkin wrote:
> > On Fri, Sep 18, 2015 at 06:29:27PM +0200, Claudio Fontana wrote:
> >> Hello,
> >>
> >> this is a first RFC for virtio-peer 0.1, which is still very much a work 
> >> in progress:
> >>
> >> https://github.com/hw-claudio/virtio-peer/wiki
> >>
> >> It is also available as PDF there, but the text is reproduced here for 
> >> commenting:
> >>
> >> Peer shared memory communication device (virtio-peer)
> >>
> >> General Overview
> >>
> >> (I recommend looking at the PDF for some clarifying pictures)
> >>
> >> The Virtio Peer shared memory communication device (virtio-peer) is a
> >> virtual device which allows high performance low latency guest to
> >> guest communication. It uses a new queue extension feature tentatively
> >> called VIRTIO_F_WINDOW which indicates that descriptor tables,
> >> available and used rings and Queue Data reside in physical memory
> >> ranges called Windows, each identified with an unique identifier
> >> called WindowID.
> > 
> > So if I had to summarize the difference from regular virtio,
> > I'd say the main one is that this uses window id + offset
> > instead of the physical address.
> > 
> > 
> > My question is - why do it?
> > 
> > All windows are in memory space, are they not?
> > 
> > How about guest using full physical addresses,
> > and hypervisor sending the window physical address
> > to VM2?
> > 
> > VM2 can uses that to find both window id and offset.
> > 
> > 
> > This way at least VM1 can use regular virtio without changes.
> 
> What would be the value of having different drivers in VM1 and VM2,
> specifically if both run Linux?
> 
> Jan

It's common to have a VM act as a switch between others.

In this setup, there's value in being able to support existing guests as
endpoints, with new drivers only required for the switch.

> -- 
> Siemens AG, Corporate Technology, CT RTC ITP SES-DE
> Corporate Competence Center Embedded Linux

[Qemu-devel] [PATCH v4 2/2] spapr: generate DT node names

[Laurent Vivier]

When DT node names for PCI devices are generated by SLOF,
they are generated according to the type of the device
(for instance, ethernet for virtio-net-pci device).

Node name for hotplugged devices is generated by QEMU.
This patch adds the mechanic to QEMU to create the node
name according to the device type too.

The data structure has been roughly copied from OpenBIOS/OpenHackware,
node names from SLOF.

Example:

Hotplugging some PCI cards with QEMU monitor:

device_add virtio-tablet-pci
device_add virtio-serial-pci
device_add virtio-mouse-pci
device_add virtio-scsi-pci
device_add virtio-gpu-pci
device_add ne2k_pci
device_add nec-usb-xhci
device_add intel-hda

What we can see in linux device tree:

for dir in /proc/device-tree/pci@8002000/*@*/; do
echo $dir
cat $dir/name
echo
done

WITHOUT this patch:

/proc/device-tree/pci@8002000/pci@0/
pci
/proc/device-tree/pci@8002000/pci@1/
pci
/proc/device-tree/pci@8002000/pci@2/
pci
/proc/device-tree/pci@8002000/pci@3/
pci
/proc/device-tree/pci@8002000/pci@4/
pci
/proc/device-tree/pci@8002000/pci@5/
pci
/proc/device-tree/pci@8002000/pci@6/
pci
/proc/device-tree/pci@8002000/pci@7/
pci

WITH this patch:

/proc/device-tree/pci@8002000/communication-controller@1/
communication-controller
/proc/device-tree/pci@8002000/display@4/
display
/proc/device-tree/pci@8002000/ethernet@5/
ethernet
/proc/device-tree/pci@8002000/input-controller@0/
input-controller
/proc/device-tree/pci@8002000/mouse@2/
mouse
/proc/device-tree/pci@8002000/multimedia-device@7/
multimedia-device
/proc/device-tree/pci@8002000/scsi@3/
scsi
/proc/device-tree/pci@8002000/usb-xhci@6/
usb-xhci

Signed-off-by: Laurent Vivier 
Reviewed-by: Thomas Huth 
---
 hw/ppc/spapr_pci.c | 292 ++---
 1 file changed, 278 insertions(+), 14 deletions(-)

diff --git a/hw/ppc/spapr_pci.c b/hw/ppc/spapr_pci.c
index a2feb4c..63eb28c 100644
--- a/hw/ppc/spapr_pci.c
+++ b/hw/ppc/spapr_pci.c
@@ -38,6 +38,7 @@
 
 #include "hw/pci/pci_bridge.h"
 #include "hw/pci/pci_bus.h"
+#include "hw/pci/pci_ids.h"
 #include "hw/ppc/spapr_drc.h"
 #include "sysemu/device_tree.h"
 
@@ -944,6 +945,276 @@ static void populate_resource_props(PCIDevice *d, 
ResourceProps *rp)
 rp->assigned_len = assigned_idx * sizeof(ResourceFields);
 }
 
+typedef struct PCIClass PCIClass;
+typedef struct PCISubClass PCISubClass;
+typedef struct PCIIFace PCIIFace;
+
+struct PCIIFace {
+uint8_t iface;
+const char *name;
+};
+
+struct PCISubClass {
+uint8_t subclass;
+const char *name;
+const PCIIFace *iface;
+};
+#define SUBCLASS(a) ((uint8_t)a)
+#define IFACE(a)((uint8_t)a)
+
+struct PCIClass {
+const char *name;
+const PCISubClass *subc;
+};
+
+static const PCISubClass undef_subclass[] = {
+{ IFACE(PCI_CLASS_NOT_DEFINED_VGA), "display", NULL },
+{ 0xFF, NULL, NULL, NULL },
+};
+
+static const PCISubClass mass_subclass[] = {
+{ SUBCLASS(PCI_CLASS_STORAGE_SCSI), "scsi", NULL },
+{ SUBCLASS(PCI_CLASS_STORAGE_IDE), "ide", NULL },
+{ SUBCLASS(PCI_CLASS_STORAGE_FLOPPY), "fdc", NULL },
+{ SUBCLASS(PCI_CLASS_STORAGE_IPI), "ipi", NULL },
+{ SUBCLASS(PCI_CLASS_STORAGE_RAID), "raid", NULL },
+{ SUBCLASS(PCI_CLASS_STORAGE_ATA), "ata", NULL },
+{ SUBCLASS(PCI_CLASS_STORAGE_SATA), "sata", NULL },
+{ SUBCLASS(PCI_CLASS_STORAGE_SAS), "sas", NULL },
+{ 0xFF, NULL, NULL },
+};
+
+static const PCISubClass net_subclass[] = {
+{ SUBCLASS(PCI_CLASS_NETWORK_ETHERNET), "ethernet", NULL },
+{ SUBCLASS(PCI_CLASS_NETWORK_TOKEN_RING), "token-ring", NULL },
+{ SUBCLASS(PCI_CLASS_NETWORK_FDDI), "fddi", NULL },
+{ SUBCLASS(PCI_CLASS_NETWORK_ATM), "atm", NULL },
+{ SUBCLASS(PCI_CLASS_NETWORK_ISDN), "isdn", NULL },
+{ SUBCLASS(PCI_CLASS_NETWORK_WORLDFIP), "worldfip", NULL },
+{ SUBCLASS(PCI_CLASS_NETWORK_PICMG214), "picmg", NULL },
+{ 0xFF, NULL, NULL },
+};
+
+static const PCISubClass displ_subclass[] = {
+{ SUBCLASS(PCI_CLASS_DISPLAY_VGA), "vga", NULL },
+{ SUBCLASS(PCI_CLASS_DISPLAY_XGA), "xga", NULL },
+{ SUBCLASS(PCI_CLASS_DISPLAY_3D), "3d-controller", NULL },
+{ 0xFF, NULL, NULL },
+};
+
+static const PCISubClass media_subclass[] = {
+{ SUBCLASS(PCI_CLASS_MULTIMEDIA_VIDEO), "video", NULL },
+{ SUBCLASS(PCI_CLASS_MULTIMEDIA_AUDIO), "sound", NULL },
+{ SUBCLASS(PCI_CLASS_MULTIMEDIA_PHONE), "telephony", NULL },
+{ 0xFF, NULL, NULL },
+};
+
+static const PCISubClass mem_subclass[] = {
+{ SUBCLASS(PCI_CLASS_MEMORY_RAM), "memory", NULL },
+{ SUBCLASS(PCI_CLASS_MEMORY_FLASH), "flash", NULL },
+{ 0xFF, NULL, NULL },
+};
+
+static const PCISubClass bridg_subclass[] = {
+{ SUBCLASS(PCI_CLASS_BRIDGE_HOST), "host", NULL },
+{ SUBCLASS(PCI_CLASS_BRIDGE_ISA), "isa", NULL },
+{ SUBCLASS(PCI_CLASS_BRIDGE_EISA), "eisa", NULL },
+{ SUBCLASS(PCI_CLA

[Qemu-devel] [PATCH v4 0/2] spapr: generate DT node names

[Laurent Vivier]

When DT node names for PCI devices are generated by SLOF,
they are generated according to the type of the device
(for instance, ethernet for virtio-net-pci device).

Node name for hotplugged devices is generated by QEMU.
This series adds the mechanic to QEMU to create the node
name according to the device type too.

v4: move pci_ids.h to a separate patch, fix PCI_CLASS_NETWORK_WORDFIP
remove  duplicate NL, remove 386, 486 and alpha subclasses
rename "unknown-legacy-device", correctly check array size
add Thomas and Michael "Reviewed-by".

v3: use values from pci_ids.h, update pci_ids.h values
keep only details for USB (xhci, ohci, ...) and PIC (IO-APIC, IO-XAPIC)

v2: Use CamelCase name, remove misc-* name,
remove _OTHER entries to fallback to class name (as SLOF does).
Fix typo (IPMI-bltr).

Laurent Vivier (2):
  PCI: add missing classes in pci_ids.h to build device tree
  spapr: generate DT node names

 hw/ppc/spapr_pci.c   | 292 ---
 include/hw/pci/pci_ids.h | 112 --
 2 files changed, 381 insertions(+), 23 deletions(-)

-- 
2.4.3

[Qemu-devel] [PATCH v4 1/2] PCI: add missing classes in pci_ids.h to build device tree

[Laurent Vivier]

To allow QEMU to add PCI entries in device tree,
we must have a more exhaustive list of PCI class IDs.

This patch synchronizes as much as possible with
pci_ids.h and add some missing IDs from SLOF.

Signed-off-by: Laurent Vivier 
Reviewed-by: Michael S. Tsirkin 
---
 include/hw/pci/pci_ids.h | 112 +++
 1 file changed, 103 insertions(+), 9 deletions(-)

diff --git a/include/hw/pci/pci_ids.h b/include/hw/pci/pci_ids.h
index d98e6c9..e27dc39 100644
--- a/include/hw/pci/pci_ids.h
+++ b/include/hw/pci/pci_ids.h
@@ -12,41 +12,84 @@
 
 /* Device classes and subclasses */
 
-#define PCI_BASE_CLASS_STORAGE   0x01
-#define PCI_BASE_CLASS_NETWORK   0x02
+#define PCI_CLASS_NOT_DEFINED0x
+#define PCI_CLASS_NOT_DEFINED_VGA0x0001
 
+#define PCI_BASE_CLASS_STORAGE   0x01
 #define PCI_CLASS_STORAGE_SCSI   0x0100
 #define PCI_CLASS_STORAGE_IDE0x0101
+#define PCI_CLASS_STORAGE_FLOPPY 0x0102
+#define PCI_CLASS_STORAGE_IPI0x0103
 #define PCI_CLASS_STORAGE_RAID   0x0104
+#define PCI_CLASS_STORAGE_ATA0x0105
 #define PCI_CLASS_STORAGE_SATA   0x0106
+#define PCI_CLASS_STORAGE_SAS0x0107
 #define PCI_CLASS_STORAGE_EXPRESS0x0108
 #define PCI_CLASS_STORAGE_OTHER  0x0180
 
+#define PCI_BASE_CLASS_NETWORK   0x02
 #define PCI_CLASS_NETWORK_ETHERNET   0x0200
+#define PCI_CLASS_NETWORK_TOKEN_RING 0x0201
+#define PCI_CLASS_NETWORK_FDDI   0x0202
+#define PCI_CLASS_NETWORK_ATM0x0203
+#define PCI_CLASS_NETWORK_ISDN   0x0204
+#define PCI_CLASS_NETWORK_WORLDFIP   0x0205
+#define PCI_CLASS_NETWORK_PICMG214   0x0206
 #define PCI_CLASS_NETWORK_OTHER  0x0280
 
+#define PCI_BASE_CLASS_DISPLAY   0x03
 #define PCI_CLASS_DISPLAY_VGA0x0300
+#define PCI_CLASS_DISPLAY_XGA0x0301
+#define PCI_CLASS_DISPLAY_3D 0x0302
 #define PCI_CLASS_DISPLAY_OTHER  0x0380
 
+#define PCI_BASE_CLASS_MULTIMEDIA0x04
+#define PCI_CLASS_MULTIMEDIA_VIDEO   0x0400
 #define PCI_CLASS_MULTIMEDIA_AUDIO   0x0401
+#define PCI_CLASS_MULTIMEDIA_PHONE   0x0402
+#define PCI_CLASS_MULTIMEDIA_OTHER   0x0480
 
+#define PCI_BASE_CLASS_MEMORY0x05
 #define PCI_CLASS_MEMORY_RAM 0x0500
+#define PCI_CLASS_MEMORY_FLASH   0x0501
+#define PCI_CLASS_MEMORY_OTHER   0x0580
 
-#define PCI_CLASS_SYSTEM_SDHCI   0x0805
-#define PCI_CLASS_SYSTEM_OTHER   0x0880
-
-#define PCI_CLASS_SERIAL_USB 0x0c03
-#define PCI_CLASS_SERIAL_SMBUS   0x0c05
-
+#define PCI_BASE_CLASS_BRIDGE0x06
 #define PCI_CLASS_BRIDGE_HOST0x0600
 #define PCI_CLASS_BRIDGE_ISA 0x0601
+#define PCI_CLASS_BRIDGE_EISA0x0602
+#define PCI_CLASS_BRIDGE_MC  0x0603
 #define PCI_CLASS_BRIDGE_PCI 0x0604
 #define PCI_CLASS_BRIDGE_PCI_INF_SUB 0x01
+#define PCI_CLASS_BRIDGE_PCMCIA  0x0605
+#define PCI_CLASS_BRIDGE_NUBUS   0x0606
+#define PCI_CLASS_BRIDGE_CARDBUS 0x0607
+#define PCI_CLASS_BRIDGE_RACEWAY 0x0608
+#define PCI_CLASS_BRIDGE_PCI_SEMITP  0x0609
+#define PCI_CLASS_BRIDGE_IB_PCI  0x060a
 #define PCI_CLASS_BRIDGE_OTHER   0x0680
 
+#define PCI_BASE_CLASS_COMMUNICATION 0x07
 #define PCI_CLASS_COMMUNICATION_SERIAL   0x0700
+#define PCI_CLASS_COMMUNICATION_PARALLEL 0x0701
+#define PCI_CLASS_COMMUNICATION_MULTISERIAL 0x0702
+#define PCI_CLASS_COMMUNICATION_MODEM0x0703
+#define PCI_CLASS_COMMUNICATION_GPIB 0x0704
+#define PCI_CLASS_COMMUNICATION_SC   0x0705
 #define PCI_CLASS_COMMUNICATION_OTHER0x0780
 
+#define PCI_BASE_CLASS_SYSTEM0x08
+#define PCI_CLASS_SYSTEM_PIC 0x0800
+#define PCI_CLASS_SYSTEM_PIC_IOAPIC  0x080010
+#define PCI_CLASS_SYSTEM_PIC_IOXAPIC 0x080020
+#define PCI_CLASS_SYSTEM_DMA 0x0801
+#define PCI_CLASS_SYSTEM_TIMER   0x0802
+#define PCI_CLASS_SYSTEM_RTC 0x0803
+#define PCI_CLASS_SYSTEM_PCI_HOTPLUG 0x0804
+#define PCI_CLASS_SYSTEM_SDHCI   0x0805
+#define PCI_CLASS_SYSTEM_OTHER   0x0880
+
+#define PCI_BASE_CLASS_INPUT 0x09
 #define PCI_CLASS_INPUT_KEYBOARD 0x0900
 #define PCI_CLASS_INPUT_PEN  0x0901
 #define PCI_CLASS_INPUT_MOUSE0x0902
@@ -54,8 +97,59 @@
 #define PCI_CLASS_INPUT_GAMEPORT 0x0904
 #define PCI_CLASS_INPUT_OTHER0x0980
 
-#define PCI_CLASS_PROCESSOR_CO   0x0b40
+#define PCI_BASE_CLASS_DOCKING   0x0a
+#define PCI_CLASS_DOCKING_GENERIC0x0a00
+#define PCI_CLASS_DOCKING_OTHER  0x0a80
+
+#define PCI_BASE_CLASS_PROCESSOR 0x0b
+#define PCI_CLASS_PROCESSOR_PENTIUM  0x0b02
 #define PCI_CLASS_PROCESSOR_POWERPC  0x0b20
+#define PCI_CLASS_PROCESSOR_MIPS 0x0b30
+#define PCI_CLASS_PROCESSOR_CO   0x0

Re: [Qemu-devel] [PATCH v11 1/7] vhost-user: use VHOST_USER_XXX macro for switch statement

[Marcel Apfelbaum]

On 09/24/2015 01:18 PM, Michael S. Tsirkin wrote:

On Thu, Sep 24, 2015 at 01:05:11PM +0300, Marcel Apfelbaum wrote:

On 09/23/2015 07:19 AM, Yuanhan Liu wrote:

So that we could let vhost_user_call to handle extented requests,
such as VHOST_USER_GET/SET_PROTOCOL_FEATURES, instead of invoking
vhost_user_read/write and constructing the msg again by ourself.

Signed-off-by: Yuanhan Liu 
---
  hw/virtio/vhost-user.c | 38 ++
  1 file changed, 22 insertions(+), 16 deletions(-)

diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c
index e7ab829..13677ac 100644
--- a/hw/virtio/vhost-user.c
+++ b/hw/virtio/vhost-user.c
@@ -193,27 +193,33 @@ static int vhost_user_call(struct vhost_dev *dev, 
unsigned long int request,

  assert(dev->vhost_ops->backend_type == VHOST_BACKEND_TYPE_USER);

-msg_request = vhost_user_request_translate(request);
+/* only translate vhost ioctl requests */
+if (request > VHOST_USER_MAX) {
+msg_request = vhost_user_request_translate(request);
+} else {
+msg_request = request;
+}
+
  msg.request = msg_request;
  msg.flags = VHOST_USER_VERSION;
  msg.size = 0;

-switch (request) {
-case VHOST_GET_FEATURES:
+switch (msg_request) {
+case VHOST_USER_GET_FEATURES:
  need_reply = 1;
  break;

-case VHOST_SET_FEATURES:
-case VHOST_SET_LOG_BASE:
+case VHOST_USER_SET_FEATURES:
+case VHOST_USER_SET_LOG_BASE:
  msg.u64 = *((__u64 *) arg);
  msg.size = sizeof(m.u64);
  break;

-case VHOST_SET_OWNER:
-case VHOST_RESET_OWNER:
+case VHOST_USER_SET_OWNER:
+case VHOST_USER_RESET_OWNER:
  break;

-case VHOST_SET_MEM_TABLE:
+case VHOST_USER_SET_MEM_TABLE:
  for (i = 0; i < dev->mem->nregions; ++i) {
  struct vhost_memory_region *reg = dev->mem->regions + i;
  ram_addr_t ram_addr;
@@ -246,30 +252,30 @@ static int vhost_user_call(struct vhost_dev *dev, 
unsigned long int request,

  break;

-case VHOST_SET_LOG_FD:
+case VHOST_USER_SET_LOG_FD:
  fds[fd_num++] = *((int *) arg);
  break;

-case VHOST_SET_VRING_NUM:
-case VHOST_SET_VRING_BASE:
+case VHOST_USER_SET_VRING_NUM:
+case VHOST_USER_SET_VRING_BASE:
  memcpy(&msg.state, arg, sizeof(struct vhost_vring_state));
  msg.size = sizeof(m.state);
  break;

-case VHOST_GET_VRING_BASE:
+case VHOST_USER_GET_VRING_BASE:
  memcpy(&msg.state, arg, sizeof(struct vhost_vring_state));
  msg.size = sizeof(m.state);
  need_reply = 1;
  break;

-case VHOST_SET_VRING_ADDR:
+case VHOST_USER_SET_VRING_ADDR:
  memcpy(&msg.addr, arg, sizeof(struct vhost_vring_addr));
  msg.size = sizeof(m.addr);
  break;

-case VHOST_SET_VRING_KICK:
-case VHOST_SET_VRING_CALL:
-case VHOST_SET_VRING_ERR:
+case VHOST_USER_SET_VRING_KICK:
+case VHOST_USER_SET_VRING_CALL:
+case VHOST_USER_SET_VRING_ERR:
  file = arg;
  msg.u64 = file->index & VHOST_USER_VRING_IDX_MASK;
  msg.size = sizeof(m.u64);




--
Reviewed-by: Marcel Apfelbaum 


The -- is the signature separator.
Please don't put text after that - I almost missed it.



Ooops, I'll be more careful next time (I did it already for this series...)

Thanks,
Marcel

Re: [Qemu-devel] [PATCH v7 41/42] Disable mlock around incoming postcopy

[Dr. David Alan Gilbert]

* Juan Quintela (quint...@redhat.com) wrote:
> "Dr. David Alan Gilbert (git)"  wrote:
> > From: "Dr. David Alan Gilbert" 
> >
> > Userfault doesn't work with mlock; mlock is designed to nail down pages
> > so they don't move, userfault is designed to tell you when they're not
> > there.
> >
> > munlock the pages we userfault protect before postcopy.
> > mlock everything again at the end if mlock is enabled.
> >
> > Signed-off-by: Dr. David Alan Gilbert 
> > Reviewed-by: David Gibson 
> > ---
> >  include/sysemu/sysemu.h  |  1 +
> >  migration/postcopy-ram.c | 24 
> >  2 files changed, 25 insertions(+)
> >
> > diff --git a/include/sysemu/sysemu.h b/include/sysemu/sysemu.h
> > index 1af2ea0..c1f3da4 100644
> > --- a/include/sysemu/sysemu.h
> > +++ b/include/sysemu/sysemu.h
> > @@ -171,6 +171,7 @@ extern int boot_menu;
> >  extern bool boot_strict;
> >  extern uint8_t *boot_splash_filedata;
> >  extern size_t boot_splash_filedata_size;
> > +extern bool enable_mlock;
> >  extern uint8_t qemu_extra_params_fw[2];
> >  extern QEMUClockType rtc_clock;
> >  extern const char *mem_path;
> > diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c
> > index 7eb1fb9..be7e5f2 100644
> > --- a/migration/postcopy-ram.c
> > +++ b/migration/postcopy-ram.c
> > @@ -85,6 +85,11 @@ static bool ufd_version_check(int ufd)
> >  return true;
> >  }
> >  
> > +/*
> > + * Note: This has the side effect of munlock'ing all of RAM, that's
> > + * normally fine since if the postcopy succeeds it gets turned back on at 
> > the
> > + * end.
> > + */
> >  bool postcopy_ram_supported_by_host(void)
> >  {
> >  long pagesize = getpagesize();
> > @@ -113,6 +118,15 @@ bool postcopy_ram_supported_by_host(void)
> >  }
> >  
> >  /*
> > + * userfault and mlock don't go together; we'll put it back later if
> > + * it was enabled.
> > + */
> > +if (munlockall()) {
> > +error_report("%s: munlockall: %s", __func__,  strerror(errno));
> 
> 
> why is this not proteced by enable_mlock?

Because there's no harm in doing the 'unlock', and if something else somewhere
other than the enable_mlock had enabled it then postcopy would break.

> > +return -1;
> > +}
> > +
> > +/*
> >   *  We need to check that the ops we need are supported on anon memory
> >   *  To do that we need to register a chunk and see the flags that
> >   *  are returned.
> > @@ -303,6 +317,16 @@ int 
> > postcopy_ram_incoming_cleanup(MigrationIncomingState *mis)
> >  mis->have_fault_thread = false;
> >  }
> >  
> > +if (enable_mlock) {
> > +if (os_mlock() < 0) {
> > +error_report("mlock: %s", strerror(errno));
> > +/*
> > + * It doesn't feel right to fail at this point, we have a valid
> > + * VM state.
> > + */
> 
> realtime_init() exit in case of os_mlock() fails, so current code is:
> 
> - we start qemu with mlock requset
> - we mlock memory
> - we start postcopy
> - we munlock memory
> - we mlock memory
> 
> I wmill really, really preffer having a check if memory is mlocked, and
> it that case, just abort migration altogether.

Although it does look like users want the two together.

>  Or better still, wait to
> enable mlock *until* we have finished postcopy, no?

I think that is likely to:
   a) Produce a longer downtime
Lets follow your summary points above:

- we start qemu with mlock requset
- we mlock memory
  !! This can be expensive - we might have to do some swap and the kernel
 has to make sure it has this available.  But at the end we have enough
 memory.Anyway, this isn't on any critical path; the source is still
 running.
- we start postcopy
- we munlock memory
  !! OK, that's not good, but
- we mlock memory
  !! There's a pretty good chance that most of the memory we force allocated
 during the 1st mlock is still available, so this should be faster than 
the
 1st mlock.

If we flip it so we do just the mlock at the end of postcopy, it's got a 
much
higher chance of needing to swap stuff in.


   b) the main mlock for realtime happens very early in startup in vl.c;
 and that's way before the destination knows it's about to have
 a postcopy migration incoming.

Dave
> 
  
> 
> Later, Juan.
> 
> > +}
> > +}
> > +
> >  postcopy_state_set(mis, POSTCOPY_INCOMING_END);
> >  migrate_send_rp_shut(mis, qemu_file_get_error(mis->file) != 0);
--
Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK

Re: [Qemu-devel] [PATCH] pc: Set broken_reserved_end on pc-*-2.4, not 2.5

[Michael S. Tsirkin]

On Wed, Sep 23, 2015 at 12:04:49PM -0300, Eduardo Habkost wrote:
> Version 1 of the pc-*-2.5 machine class series was applied to the PCI
> tree instead of v3 (which was rebased after the broken_reserved_end
> patch by Igor was included).
> 
> This patch includes the missing hunks from v3, to make sure
> broken_reserved_end is set at the right machine class.
> 
> Signed-off-by: Eduardo Habkost 

Thanks!
I squashed this in.
Can you pls confirm I did this correctly this time?

> ---
>  hw/i386/pc_piix.c | 4 ++--
>  hw/i386/pc_q35.c  | 4 ++--
>  2 files changed, 4 insertions(+), 4 deletions(-)
> 
> diff --git a/hw/i386/pc_piix.c b/hw/i386/pc_piix.c
> index caa4edc..3ffb05f 100644
> --- a/hw/i386/pc_piix.c
> +++ b/hw/i386/pc_piix.c
> @@ -466,9 +466,7 @@ static void pc_i440fx_machine_options(MachineClass *m)
>  
>  static void pc_i440fx_2_5_machine_options(MachineClass *m)
>  {
> -PCMachineClass *pcmc = PC_MACHINE_CLASS(m);
>  pc_i440fx_machine_options(m);
> -pcmc->broken_reserved_end = true;
>  m->alias = "pc";
>  m->is_default = 1;
>  }
> @@ -479,9 +477,11 @@ DEFINE_I440FX_MACHINE(v2_5, "pc-i440fx-2.5", NULL,
>  
>  static void pc_i440fx_2_4_machine_options(MachineClass *m)
>  {
> +PCMachineClass *pcmc = PC_MACHINE_CLASS(m);
>  pc_i440fx_2_5_machine_options(m);
>  m->alias = NULL;
>  m->is_default = 0;
> +pcmc->broken_reserved_end = true;
>  SET_MACHINE_COMPAT(m, PC_COMPAT_2_4);
>  }
>  
> diff --git a/hw/i386/pc_q35.c b/hw/i386/pc_q35.c
> index 506b6bf..1b7d3b6 100644
> --- a/hw/i386/pc_q35.c
> +++ b/hw/i386/pc_q35.c
> @@ -372,9 +372,7 @@ static void pc_q35_machine_options(MachineClass *m)
>  
>  static void pc_q35_2_5_machine_options(MachineClass *m)
>  {
> -PCMachineClass *pcmc = PC_MACHINE_CLASS(m);
>  pc_q35_machine_options(m);
> -pcmc->broken_reserved_end = true;
>  m->alias = "q35";
>  }
>  
> @@ -383,8 +381,10 @@ DEFINE_Q35_MACHINE(v2_5, "pc-q35-2.5", NULL,
>  
>  static void pc_q35_2_4_machine_options(MachineClass *m)
>  {
> +PCMachineClass *pcmc = PC_MACHINE_CLASS(m);
>  pc_q35_2_5_machine_options(m);
>  m->alias = NULL;
> +pcmc->broken_reserved_end = true;
>  SET_MACHINE_COMPAT(m, PC_COMPAT_2_4);
>  }
>  
> -- 
> 2.1.0

Re: [Qemu-devel] [PATCH v4 1/2] PCI: add missing classes in pci_ids.h to build device tree

[Thomas Huth]

On 24/09/15 12:27, Laurent Vivier wrote:
> To allow QEMU to add PCI entries in device tree,
> we must have a more exhaustive list of PCI class IDs.
> 
> This patch synchronizes as much as possible with
> pci_ids.h and add some missing IDs from SLOF.
> 
> Signed-off-by: Laurent Vivier 
> Reviewed-by: Michael S. Tsirkin 
> ---
>  include/hw/pci/pci_ids.h | 112 
> +++
>  1 file changed, 103 insertions(+), 9 deletions(-)

Reviewed-by: Thomas Huth 

[Qemu-devel] [RFC PATCH v1 0/2] spapr: Abort when HTAB size requirement can't be met

[Bharata B Rao]

HTAB size is a factor of maximum memory size that is specified by maxmem=
command line option. In cases where there is shortage of host memory, host
will not be able to allocate contiguous memory for guest HTAB and will
instead allocate a smaller HTAB. This usually is not a problem but when
user starts hotplugging memory to the guest, we can run out of HTAB entries
and hence memory hotplug fails. This failure should have been handled
gracefully by the guest kernel, but currently it leads to guest kernel OOPS.
This will eventually get fixed when the handling of memory hotplug is
completely moved to kernel for PowerKVM.

Prevent such kernel failure by refusing to boot the guest when requested
HTAB size can't be allocated. However HTAB allocation happens in the
reset path from where it is too late to abort. Hence this patchset
moves the HTAB allocation to machine init and aborts if HTAB size
requirement isn't met.

This patchset applies against David Gibson's spapr-next. With this
patchset, simple boot, reboot and migration tests pass with HV KVM guest.
For PR KVM guest, boot and reboot tests were done since migration already
appears broken for PR KVM.

Changes in v1
-
- Correctly handle HTAB for PR KVM guests that use QEMU allocated HTAB.
- Verbose error message when aborting.
- Check for returned shift value in reset path too and abort when
  there is mismatch in htab shift values.
- Move the marking of htab_fd as stale from spapr_alloc_htab() to
  spapr_reset_htab() where it originally belonged.

v0: http://lists.nongnu.org/archive/html/qemu-devel/2015-09/msg05491.html

Bharata B Rao (2):
  spapr: Allocate HTAB from machine init
  spapr: Abort when HTAB of requested size isn't allocated

 hw/ppc/spapr.c | 42 +++---
 1 file changed, 35 insertions(+), 7 deletions(-)

-- 
2.1.0

Re: [Qemu-devel] [PATCH] qga: Use g_new() & friends where that makes obvious sense

[Michael Roth]

Quoting Markus Armbruster (2015-09-24 02:02:41)
> Ping?

Thanks, applied to qga tree:

  https://github.com/mdroth/qemu/commits/qga

> 
> Markus Armbruster  writes:
> 
> > g_new(T, n) is neater than g_malloc(sizeof(T) * n).  It's also safer,
> > for two reasons.  One, it catches multiplication overflowing size_t.
> > Two, it returns T * rather than void *, which lets the compiler catch
> > more type errors.
> >
> > This commit only touches allocations with size arguments of the form
> > sizeof(T).  Same Coccinelle semantic patch as in commit b45c03f.
> >
> > Signed-off-by: Markus Armbruster 
> > ---
> >  qga/channel-posix.c |  2 +-
> >  qga/channel-win32.c |  2 +-
> >  qga/commands-posix.c| 10 +-
> >  qga/commands-win32.c| 10 +-
> >  qga/commands.c  |  6 +++---
> >  qga/guest-agent-command-state.c |  4 ++--
> >  6 files changed, 17 insertions(+), 17 deletions(-)
> >
> > diff --git a/qga/channel-posix.c b/qga/channel-posix.c
> > index 8aad4fe..61aa3cb 100644
> > --- a/qga/channel-posix.c
> > +++ b/qga/channel-posix.c
> > @@ -249,7 +249,7 @@ GIOStatus ga_channel_read(GAChannel *c, gchar *buf, 
> > gsize size, gsize *count)
> >  GAChannel *ga_channel_new(GAChannelMethod method, const gchar *path,
> >GAChannelCallback cb, gpointer opaque)
> >  {
> > -GAChannel *c = g_malloc0(sizeof(GAChannel));
> > +GAChannel *c = g_new0(GAChannel, 1);
> >  c->event_cb = cb;
> >  c->user_data = opaque;
> >  
> > diff --git a/qga/channel-win32.c b/qga/channel-win32.c
> > index 04fa5e4..215b15a 100644
> > --- a/qga/channel-win32.c
> > +++ b/qga/channel-win32.c
> > @@ -322,7 +322,7 @@ static gboolean ga_channel_open(GAChannel *c, 
> > GAChannelMethod method,
> >  GAChannel *ga_channel_new(GAChannelMethod method, const gchar *path,
> >GAChannelCallback cb, gpointer opaque)
> >  {
> > -GAChannel *c = g_malloc0(sizeof(GAChannel));
> > +GAChannel *c = g_new0(GAChannel, 1);
> >  SECURITY_ATTRIBUTES sec_attrs;
> >  
> >  if (!ga_channel_open(c, method, path)) {
> > diff --git a/qga/commands-posix.c b/qga/commands-posix.c
> > index b03c316..a932809 100644
> > --- a/qga/commands-posix.c
> > +++ b/qga/commands-posix.c
> > @@ -235,7 +235,7 @@ static int64_t guest_file_handle_add(FILE *fh, Error 
> > **errp)
> >  return -1;
> >  }
> >  
> > -gfh = g_malloc0(sizeof(GuestFileHandle));
> > +gfh = g_new0(GuestFileHandle, 1);
> >  gfh->id = handle;
> >  gfh->fh = fh;
> >  QTAILQ_INSERT_TAIL(&guest_file_state.filehandles, gfh, next);
> > @@ -488,7 +488,7 @@ struct GuestFileRead *qmp_guest_file_read(int64_t 
> > handle, bool has_count,
> >  slog("guest-file-read failed, handle: %" PRId64, handle);
> >  } else {
> >  buf[read_count] = 0;
> > -read_data = g_malloc0(sizeof(GuestFileRead));
> > +read_data = g_new0(GuestFileRead, 1);
> >  read_data->count = read_count;
> >  read_data->eof = feof(fh);
> >  if (read_count) {
> > @@ -533,7 +533,7 @@ GuestFileWrite *qmp_guest_file_write(int64_t handle, 
> > const char *buf_b64,
> >  error_setg_errno(errp, errno, "failed to write to file");
> >  slog("guest-file-write failed, handle: %" PRId64, handle);
> >  } else {
> > -write_data = g_malloc0(sizeof(GuestFileWrite));
> > +write_data = g_new0(GuestFileWrite, 1);
> >  write_data->count = write_count;
> >  write_data->eof = feof(fh);
> >  }
> > @@ -678,7 +678,7 @@ static void build_fs_mount_list_from_mtab(FsMountList 
> > *mounts, Error **errp)
> >  continue;
> >  }
> >  
> > -mount = g_malloc0(sizeof(FsMount));
> > +mount = g_new0(FsMount, 1);
> >  mount->dirname = g_strdup(ment->mnt_dir);
> >  mount->devtype = g_strdup(ment->mnt_type);
> >  mount->devmajor = devmajor;
> > @@ -757,7 +757,7 @@ static void build_fs_mount_list(FsMountList *mounts, 
> > Error **errp)
> >  }
> >  }
> >  
> > -mount = g_malloc0(sizeof(FsMount));
> > +mount = g_new0(FsMount, 1);
> >  mount->dirname = g_strdup(line + dir_s);
> >  mount->devtype = g_strdup(dash + type_s);
> >  mount->devmajor = devmajor;
> > diff --git a/qga/commands-win32.c b/qga/commands-win32.c
> > index 41bdd3f..0481646 100644
> > --- a/qga/commands-win32.c
> > +++ b/qga/commands-win32.c
> > @@ -106,7 +106,7 @@ static int64_t guest_file_handle_add(HANDLE fh, Error 
> > **errp)
> >  if (handle < 0) {
> >  return -1;
> >  }
> > -gfh = g_malloc0(sizeof(GuestFileHandle));
> > +gfh = g_new0(GuestFileHandle, 1);
> >  gfh->id = handle;
> >  gfh->fh = fh;
> >  QTAILQ_INSERT_TAIL(&guest_file_state.filehandles, gfh, next);
> > @@ -298,7 +298,7 @@ GuestFileRead *qmp_guest_file_read(int64_t handle, bool 
> > has_count,
> >  slog("guest-file-read failed,

[Qemu-devel] [RFC PATCH v1 1/2] spapr: Allocate HTAB from machine init

[Bharata B Rao]

Allocate HTAB from ppc_spapr_init() so that we can abort the guest
if requested HTAB size is't allocated by the host. However retain the
htab reset call in spapr_reset_htab() so that HTAB gets reset (and
not allocated) during machine reset.

Signed-off-by: Bharata B Rao 
---
 hw/ppc/spapr.c | 34 +++---
 1 file changed, 27 insertions(+), 7 deletions(-)

diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
index 7f4f196..f6a5c29 100644
--- a/hw/ppc/spapr.c
+++ b/hw/ppc/spapr.c
@@ -979,7 +979,7 @@ static void emulate_spapr_hypercall(PowerPCCPU *cpu)
 #define CLEAN_HPTE(_hpte)  ((*(uint64_t *)(_hpte)) &= 
tswap64(~HPTE64_V_HPTE_DIRTY))
 #define DIRTY_HPTE(_hpte)  ((*(uint64_t *)(_hpte)) |= 
tswap64(HPTE64_V_HPTE_DIRTY))
 
-static void spapr_reset_htab(sPAPRMachineState *spapr)
+static void spapr_alloc_htab(sPAPRMachineState *spapr)
 {
 long shift;
 int index;
@@ -994,18 +994,37 @@ static void spapr_reset_htab(sPAPRMachineState *spapr)
 /* Kernel handles htab, we don't need to allocate one */
 spapr->htab_shift = shift;
 kvmppc_kern_htab = true;
+} else {
+/* Allocate htab */
+spapr->htab = qemu_memalign(HTAB_SIZE(spapr), HTAB_SIZE(spapr));
+
+/* And clear it */
+memset(spapr->htab, 0, HTAB_SIZE(spapr));
+
+for (index = 0; index < HTAB_SIZE(spapr) / HASH_PTE_SIZE_64; index++) {
+DIRTY_HPTE(HPTE(spapr->htab, index));
+}
+}
+}
+
+/*
+ * Clear HTAB entries during reset.
+ *
+ * If host kernel has allocated HTAB, KVM_PPC_ALLOCATE_HTAB ioctl is
+ * used to clear HTAB. Otherwise QEMU-allocated HTAB is cleared manually.
+ */
+static void spapr_reset_htab(sPAPRMachineState *spapr)
+{
+long shift;
+int index;
 
+shift = kvmppc_reset_htab(spapr->htab_shift);
+if (shift > 0) {
 /* Tell readers to update their file descriptor */
 if (spapr->htab_fd >= 0) {
 spapr->htab_fd_stale = true;
 }
 } else {
-if (!spapr->htab) {
-/* Allocate an htab if we don't yet have one */
-spapr->htab = qemu_memalign(HTAB_SIZE(spapr), HTAB_SIZE(spapr));
-}
-
-/* And clear it */
 memset(spapr->htab, 0, HTAB_SIZE(spapr));
 
 for (index = 0; index < HTAB_SIZE(spapr) / HASH_PTE_SIZE_64; index++) {
@@ -1709,6 +1728,7 @@ static void ppc_spapr_init(MachineState *machine)
 }
 spapr->htab_shift++;
 }
+spapr_alloc_htab(spapr);
 
 /* Set up Interrupt Controller before we create the VCPUs */
 spapr->icp = xics_system_init(machine,
-- 
2.1.0

[Qemu-devel] [RFC PATCH v1 2/2] spapr: Abort when HTAB of requested size isn't allocated

[Bharata B Rao]

Terminate the guest when HTAB of requested size isn't allocated by
the host.

When memory hotplug is attempted on a guest that has booted with
less than requested HTAB size, the guest kernel will not be able
to gracefully fail the hotplug request. This patch will ensure that
we never end up in a situation where memory hotplug fails due to
less than requested HTAB size.

Signed-off-by: Bharata B Rao 
---
 hw/ppc/spapr.c | 8 
 1 file changed, 8 insertions(+)

diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
index f6a5c29..3c60df2 100644
--- a/hw/ppc/spapr.c
+++ b/hw/ppc/spapr.c
@@ -992,6 +992,10 @@ static void spapr_alloc_htab(sPAPRMachineState *spapr)
 
 if (shift > 0) {
 /* Kernel handles htab, we don't need to allocate one */
+if (shift != spapr->htab_shift) {
+error_setg(&error_abort, "Failed to allocate HTAB of requested 
size, try with smaller maxmem");
+}
+
 spapr->htab_shift = shift;
 kvmppc_kern_htab = true;
 } else {
@@ -1020,6 +1024,10 @@ static void spapr_reset_htab(sPAPRMachineState *spapr)
 
 shift = kvmppc_reset_htab(spapr->htab_shift);
 if (shift > 0) {
+if (shift != spapr->htab_shift) {
+error_setg(&error_abort, "Requested HTAB allocation failed during 
reset");
+}
+
 /* Tell readers to update their file descriptor */
 if (spapr->htab_fd >= 0) {
 spapr->htab_fd_stale = true;
-- 
2.1.0

Re: [Qemu-devel] [PULL 6/7] vmxnet3: Drop net_vmxnet3_info.can_receive

[Shmulik Ladkani]

Hi,

On Thu, 3 Sep 2015 10:19:19 +0300, shmulik.ladk...@ravellosystems.com wrote:
> On Wed,  2 Sep 2015 17:14:52 +0100, stefa...@redhat.com wrote:
> > From: Fam Zheng 
> > 
> > Commit 6e99c63 ("net/socket: Drop net_socket_can_send") changed the
> > semantics around .can_receive for sockets to now require the device to
> > flush queued pkts when transitioning to a .can_receive=true state. But
> > it's OK to drop incoming packets when the link is not active.
> > 
> > Signed-off-by: Fam Zheng 
> > Signed-off-by: Stefan Hajnoczi 
> 
> Tested-by: Shmulik Ladkani 

Ping...

I'm not sure what's the exact policy, but shouldn't
'2734a20 vmxnet3: Drop net_vmxnet3_info.can_receive' go into stable-2.4?

Otherwise, vmxnet3 has no rx connectivity in 2.4 based releases.

See here: 
https://lists.gnu.org/archive/html/qemu-devel/2015-08/msg02233.html

Regards,
Shmulik

Re: [Qemu-devel] [PATCH v3] docs: describe the QEMU build system structure / design

[Laszlo Ersek]

On 09/24/15 11:49, Daniel P. Berrange wrote:
> Developers who are new to QEMU, or have a background familiarity
> with GNU autotools, can have trouble getting their head around the
> home-grown QEMU build system. This document attempts to explain
> the structure / design of the configure script and the various
> Makefile pieces that live across the source tree.
> 
> Signed-off-by: Daniel P. Berrange 
> ---
> 
> Changed in v3:
> 
>  - More speling eror fixes
>  - Rephrased more paragraphs as suggested
> 
> Changed in v2:
> 
>  - Misc speling eror fixes
>  - Rephrased some paragraphs as suggested
>  - Added note about config-host.h file generation & use
> 
>  docs/build-system.txt | 507 
> ++
>  1 file changed, 507 insertions(+)
>  create mode 100644 docs/build-system.txt

Diffed this against v2; I have only one comment:

> +the userspace emulator targets as the global $(LIBS), or more targetted

This is new in v3, and it seems to reintroduce the (otherwise correct
en_GB?) spelling of "targetted". If I recall correctly, it has been
suggested to spell it as "targeted" (and that form can in fact be found
in the same paragraph).

Otherwise, ACK.

Thanks
Laszlo

Re: [Qemu-devel] [PATCH v11 2/7] vhost-user: add protocol feature negotiation

[Yuanhan Liu]

On Thu, Sep 24, 2015 at 01:13:24PM +0300, Marcel Apfelbaum wrote:
> On 09/23/2015 07:19 AM, Yuanhan Liu wrote:
> >From: "Michael S. Tsirkin" 
> >
> >Support a separate bitmask for vhost-user protocol features,
> >and messages to get/set protocol features.
> >
> >Invoke them at init.
> >
> >No features are defined yet.
> >
> >[ leverage vhost_user_call for request handling -- Yuanhan Liu ]
> >
> >Signed-off-by: Michael S. Tsirkin 
> >Signed-off-by: Yuanhan Liu 
> >---
> >  docs/specs/vhost-user.txt | 37 +
> >  hw/net/vhost_net.c|  2 ++
> >  hw/virtio/vhost-user.c| 31 +++
> >  include/hw/virtio/vhost.h |  1 +
> >  4 files changed, 71 insertions(+)
> >
> >diff --git a/docs/specs/vhost-user.txt b/docs/specs/vhost-user.txt
> >index 650bb18..70da3b1 100644
> >--- a/docs/specs/vhost-user.txt
> >+++ b/docs/specs/vhost-user.txt
> >@@ -113,6 +113,7 @@ message replies. Most of the requests don't require 
> >replies. Here is a list of
> >  the ones that do:
> >
> >   * VHOST_GET_FEATURES
> >+ * VHOST_GET_PROTOCOL_FEATURES
> >   * VHOST_GET_VRING_BASE
> >
> >  There are several messages that the master sends with file descriptors 
> > passed
> >@@ -127,6 +128,13 @@ in the ancillary data:
> >  If Master is unable to send the full message or receives a wrong reply it 
> > will
> >  close the connection. An optional reconnection mechanism can be 
> > implemented.
> >
> >+Any protocol extensions are gated by protocol feature bits,
> >+which allows full backwards compatibility on both master
> >+and slave.
> >+As older slaves don't support negotiating protocol features,
> >+a feature bit was dedicated for this purpose:
> >+#define VHOST_USER_F_PROTOCOL_FEATURES 30
> >+
> >  Message types
> >  -
> >
> >@@ -138,6 +146,8 @@ Message types
> >Slave payload: u64
> >
> >Get from the underlying vhost implementation the features bitmask.
> >+  Feature bit VHOST_USER_F_PROTOCOL_FEATURES signals slave support for
> >+  VHOST_USER_GET_PROTOCOL_FEATURES and VHOST_USER_SET_PROTOCOL_FEATURES.
> >
> >   * VHOST_USER_SET_FEATURES
> >
> >@@ -146,6 +156,33 @@ Message types
> >Master payload: u64
> >
> >Enable features in the underlying vhost implementation using a 
> > bitmask.
> >+  Feature bit VHOST_USER_F_PROTOCOL_FEATURES signals slave support for
> >+  VHOST_USER_GET_PROTOCOL_FEATURES and VHOST_USER_SET_PROTOCOL_FEATURES.
> >+
> >+ * VHOST_USER_GET_PROTOCOL_FEATURES
> >+
> >+  Id: 15
> >+  Equivalent ioctl: VHOST_GET_FEATURES
> 
> VHOST_USER_GET_PROTOCOL_FEATURES does not have an equivalent ioctl

Oops, I didn't notice that. Will fix it soon, and thank you for the review!


--yliu

> 
> >+  Master payload: N/A
> >+  Slave payload: u64
> >+
> >+  Get the protocol feature bitmask from the underlying vhost 
> >implementation.
> >+  Only legal if feature bit VHOST_USER_F_PROTOCOL_FEATURES is present in
> >+  VHOST_USER_GET_FEATURES.
> >+  Note: slave that reported VHOST_USER_F_PROTOCOL_FEATURES must support
> >+  this message even before VHOST_USER_SET_FEATURES was called.
> >+
> >+ * VHOST_USER_SET_PROTOCOL_FEATURES
> >+
> >+  Id: 16
> >+  Ioctl: VHOST_SET_FEATURES
> 
> Same here
> 
> >+  Master payload: u64
> >+
> >+  Enable protocol features in the underlying vhost implementation.
> >+  Only legal if feature bit VHOST_USER_F_PROTOCOL_FEATURES is present in
> >+  VHOST_USER_GET_FEATURES.
> >+  Note: slave that reported VHOST_USER_F_PROTOCOL_FEATURES must support
> >+  this message even before VHOST_USER_SET_FEATURES was called.
> >
> >   * VHOST_USER_SET_OWNER
> >
> >diff --git a/hw/net/vhost_net.c b/hw/net/vhost_net.c
> >index 1d76b94..9d32d76 100644
> >--- a/hw/net/vhost_net.c
> >+++ b/hw/net/vhost_net.c
> >@@ -152,8 +152,10 @@ struct vhost_net *vhost_net_init(VhostNetOptions 
> >*options)
> >  net->dev.backend_features = qemu_has_vnet_hdr(options->net_backend)
> >  ? 0 : (1ULL << VHOST_NET_F_VIRTIO_NET_HDR);
> >  net->backend = r;
> >+net->dev.protocol_features = 0;
> >  } else {
> >  net->dev.backend_features = 0;
> >+net->dev.protocol_features = 0;
> >  net->backend = -1;
> >  }
> 
> Maybe protocol_features assignment should be outside the if clause.
> (assigned to 0 in both cases)
> 
> >  net->nc = options->net_backend;
> >diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c
> >index 13677ac..7fe35c6 100644
> >--- a/hw/virtio/vhost-user.c
> >+++ b/hw/virtio/vhost-user.c
> >@@ -24,6 +24,8 @@
> >  #include 
> >
> >  #define VHOST_MEMORY_MAX_NREGIONS8
> >+#define VHOST_USER_F_PROTOCOL_FEATURES 30
> >+#define VHOST_USER_PROTOCOL_FEATURE_MASK 0x0ULL
> >
> >  typedef enum VhostUserRequest {
> >  VHOST_USER_NONE = 0,
> >@@ -41,6 +43,8 @@ typedef enum VhostUserRequest {
> >  VHOST_USER_SET_VRING_KICK = 12,
> >  VHOST_USER_SET_VRING_CALL = 

Re: [Qemu-devel] [PATCH v11 2/7] vhost-user: add protocol feature negotiation

[Yuanhan Liu]

On Thu, Sep 24, 2015 at 01:13:24PM +0300, Marcel Apfelbaum wrote:
> >diff --git a/hw/net/vhost_net.c b/hw/net/vhost_net.c
> >index 1d76b94..9d32d76 100644
> >--- a/hw/net/vhost_net.c
> >+++ b/hw/net/vhost_net.c
> >@@ -152,8 +152,10 @@ struct vhost_net *vhost_net_init(VhostNetOptions 
> >*options)
> >  net->dev.backend_features = qemu_has_vnet_hdr(options->net_backend)
> >  ? 0 : (1ULL << VHOST_NET_F_VIRTIO_NET_HDR);
> >  net->backend = r;
> >+net->dev.protocol_features = 0;
> >  } else {
> >  net->dev.backend_features = 0;
> >+net->dev.protocol_features = 0;
> >  net->backend = -1;
> >  }
> 
> Maybe protocol_features assignment should be outside the if clause.
> (assigned to 0 in both cases)

Yeah, we could do that. However, it seems that it will take more effort,
for handling patch conflicts while rebase, than it worths. Therefore
I will keep it.

--yliu

Re: [Qemu-devel] [PATCH v11 01/12] qmp: delete qemu opts when delete an object

[Markus Armbruster]

Yang Hongyang  writes:

> On 09/24/2015 05:42 PM, Markus Armbruster wrote:
>> Yang Hongyang  writes:
>>
>>> On 09/24/2015 03:43 PM, Markus Armbruster wrote:
 This has finally reached the front of my review queue.  I apologize for
 the lng delay.

 Copying Paolo for another pair of eyeballs (he wrote this code).

>>> [...]
> +
> +opts = qemu_opts_find(qemu_find_opts_err("object", NULL), id);
> +qemu_opts_del(opts);

 qemu_find_opts_err("object", &error_abort) please, because when it
 fails, we want to die right away, not when the null pointer it returns
 gets dereferenced.
>>>
>>> Thanks for the review.
>>> Jason, do you want me to propose a fix on top of this series or simply drop
>>> this for now because this patch is an independent bug fix and won't
>>> affect the
>>> other filter patch series.
>>>

 Same sloppiness in netdev_del_completion() and qmp_netdev_del(), not
 your patch's fault.

 Elsewhere, we store the QemuOpts in the object just so we can delete it:
 DeviceState, DriveInfo.  Paolo, what do you think?
>>>
>>> I don't get it. Currently, only objects created at the beginning through
>>> QEMU command line will be stored in the QemuOpts, objects that created
>>> with object_add won't stored in QemuOpts. Do you mean for DeviceState,
>>> DriveInfo they store there QemuOpts explicity so that they can delete it?
>>> Why don't we just delete it from objects directly instead?
>>
>> Let me elaborate.
>
> Thanks very much for the elaboration.
>
>>
>> We have the same pattern in multiple places: some kind of object gets
>> configured via QemuOpts, and an object's QemuOpts need to stay around
>> until the object dies.
>>
>> Example 1: Block device backends
>>
>>  DriveInfo has a member opts.
>>
>>  drive_new() stores the QemuOpts in dinfo->opts.
>>
>>  drive_info_del() destroys dinfo->opts.
>>
>>  Note: DriveInfo member opts is always non-null.  But not every
>>  BlockBackend has a DriveInfo.
>>
>> Example 2: Device frontends
>>
>>  DeviceState has a member opts.
>>
>>  qdev_device_add() stores the QemuOpts in dev->opts.
>>
>>  device_finalize() destroys dev->opts.
>>
>>  Note: DeviceState member opts may be null (not every device is
>>  created by qdev_device_add()).  Fine, because qemu_opts_del(NULL) is
>>  a no-op.
>>
>> Example 3: Character device backends
>>
>>  CharDriverState has a member opts.
>>
>>  qemu_chr_new_from_opts() stores the QemuOpts in chr->opts.
>>
>>  qemu_chr_delete() destroys chr->opts.
>
> 1-3 store there ops in there own state, not in global ops group right?

Both!  But keeping a pointer in their own state simplifies calling
qemu_opts_del() on destruction, and also makes it more obvious what is
keeping the QemuOpts alive.

>> Example 4: Network device backends
>>
>>  Two cases
>>
>>  A. netdev
>>
>> qmp_netdev_add() does not store the QemuOpts.
>>
>> qmp_netdev_del() still needs to destroy it.  It has to find it
>> somehow.  Here's how it does it:
>>
>> opts = qemu_opts_find(qemu_find_opts_err("netdev", NULL), id);
>> if (!opts) {
>> error_setg(errp, "Device '%s' is not a netdev", id);
>> return;
>> }
>>
>> The !opts condition is a non-obvious way to test "not created
>> with -netdev", see commit 645c949.  Note that the commit's claim
>> that qemu_opts_del(NULL) crashes is no longer true since commit
>> 4782183.
>>
>>  B. Legacy net
>>
>> hmp_host_net_add() does not store the QemuOpts.
>
> I'm afraid it does store the QemuOpts, but not in it's own state.
> net/net.c:
> 1088 qemu_opt_set(opts, "type", device, &error_abort);
> This will store the QemuOpts, or am I misunderstood it?

Doesn't store opts anywhere, actually.  It merely modifies it (adds a
parameter "type")

>>
>> hmp_host_net_remove() still needs to destroy it.  I can't see
>> where that happens, and I'm not sure it does.
>>
>> Example 5: Generic object
>>
>>  object_create() does not store the QemuOpts.
>>
>>  It still needs to be destroyed along with the object.  It isn't, and
>>  your patch fixes it.
>>
>> Personally, I find the technique in example 1-3 easier to understand
>> than the one in example 4-5.
>
> I agree that opts should not be used to determine not created something
> while there's case when something created but Opts not stored.

Re: [Qemu-devel] [PATCH v11 01/12] qmp: delete qemu opts when delete an object

[Markus Armbruster]

Yang Hongyang  writes:

> On 09/24/2015 05:42 PM, Markus Armbruster wrote:
>> Yang Hongyang  writes:
>>
>>> On 09/24/2015 03:43 PM, Markus Armbruster wrote:
 This has finally reached the front of my review queue.  I apologize for
 the lng delay.

 Copying Paolo for another pair of eyeballs (he wrote this code).

>>> [...]
> +
> +opts = qemu_opts_find(qemu_find_opts_err("object", NULL), id);
> +qemu_opts_del(opts);

 qemu_find_opts_err("object", &error_abort) please, because when it
 fails, we want to die right away, not when the null pointer it returns
 gets dereferenced.
>>>
>>> Thanks for the review.
>>> Jason, do you want me to propose a fix on top of this series or simply drop
>>> this for now because this patch is an independent bug fix and won't
>>> affect the
>>> other filter patch series.
>>>

 Same sloppiness in netdev_del_completion() and qmp_netdev_del(), not
 your patch's fault.

 Elsewhere, we store the QemuOpts in the object just so we can delete it:
 DeviceState, DriveInfo.  Paolo, what do you think?
>>>
>>> I don't get it. Currently, only objects created at the beginning through
>>> QEMU command line will be stored in the QemuOpts, objects that created
>>> with object_add won't stored in QemuOpts. Do you mean for DeviceState,
>>> DriveInfo they store there QemuOpts explicity so that they can delete it?
>>> Why don't we just delete it from objects directly instead?
>>
>> Let me elaborate.
>>
>> We have the same pattern in multiple places: some kind of object gets
>> configured via QemuOpts, and an object's QemuOpts need to stay around
>> until the object dies.
>>
>> Example 1: Block device backends
>>
>>  DriveInfo has a member opts.
>>
>>  drive_new() stores the QemuOpts in dinfo->opts.
>>
>>  drive_info_del() destroys dinfo->opts.
>>
>>  Note: DriveInfo member opts is always non-null.  But not every
>>  BlockBackend has a DriveInfo.
>>
>> Example 2: Device frontends
>>
>>  DeviceState has a member opts.
>>
>>  qdev_device_add() stores the QemuOpts in dev->opts.
>>
>>  device_finalize() destroys dev->opts.
>>
>>  Note: DeviceState member opts may be null (not every device is
>>  created by qdev_device_add()).  Fine, because qemu_opts_del(NULL) is
>>  a no-op.
>>
>> Example 3: Character device backends
>>
>>  CharDriverState has a member opts.
>>
>>  qemu_chr_new_from_opts() stores the QemuOpts in chr->opts.
>>
>>  qemu_chr_delete() destroys chr->opts.
>>
>> Example 4: Network device backends
>>
>>  Two cases
>>
>>  A. netdev
>>
>> qmp_netdev_add() does not store the QemuOpts.
>
> The QemuOpts stored by qmp_netdev_add() and also hmp_netdev_add().
> through this function:
> net/net.c: qmp_netdev_add()
> 1134 opts = qemu_opts_from_qdict(opts_list, qdict, &local_err);
>
> hmp.c: hmp_netdev_add()
> 1579 opts = qemu_opts_from_qdict(qemu_find_opts("netdev"), qdict, &err);

That's where the QemuOpts are created.  By "does not store" I mean "does
not store in its own state, unlike example 1-3".

>>
>> qmp_netdev_del() still needs to destroy it.  It has to find it
>> somehow.  Here's how it does it:
>>
>> opts = qemu_opts_find(qemu_find_opts_err("netdev", NULL), id);
>> if (!opts) {
>> error_setg(errp, "Device '%s' is not a netdev", id);
>> return;
>> }
>>
>> The !opts condition is a non-obvious way to test "not created
>> with -netdev", see commit 645c949.  Note that the commit's claim
>> that qemu_opts_del(NULL) crashes is no longer true since commit
>> 4782183.
>>
>>  B. Legacy net
>>
>> hmp_host_net_add() does not store the QemuOpts.
>>
>> hmp_host_net_remove() still needs to destroy it.  I can't see
>> where that happens, and I'm not sure it does.
>>
>> Example 5: Generic object
>>
>>  object_create() does not store the QemuOpts.
>>
>>  It still needs to be destroyed along with the object.  It isn't, and
>>  your patch fixes it.
>>
>> Personally, I find the technique in example 1-3 easier to understand
>> than the one in example 4-5.
>> .
>>

[Qemu-devel] [PATCH v4 02/47] msix: add VMSTATE_MSIX_TEST

[marcandre . lureau]

From: Marc-André Lureau 

ivshmem is going to use MSIX state conditionally.

Signed-off-by: Marc-André Lureau 
---
 include/hw/pci/msix.h | 16 ++--
 1 file changed, 10 insertions(+), 6 deletions(-)

diff --git a/include/hw/pci/msix.h b/include/hw/pci/msix.h
index 954d82b..72e5f93 100644
--- a/include/hw/pci/msix.h
+++ b/include/hw/pci/msix.h
@@ -46,12 +46,16 @@ void msix_unset_vector_notifiers(PCIDevice *dev);
 
 extern const VMStateDescription vmstate_msix;
 
-#define VMSTATE_MSIX(_field, _state) {   \
-.name   = (stringify(_field)),   \
-.size   = sizeof(PCIDevice), \
-.vmsd   = &vmstate_msix, \
-.flags  = VMS_STRUCT,\
-.offset = vmstate_offset_value(_state, _field, PCIDevice),   \
+#define VMSTATE_MSIX_TEST(_field, _state, _test) {   \
+.name = (stringify(_field)), \
+.size = sizeof(PCIDevice),   \
+.vmsd = &vmstate_msix,   \
+.flags= VMS_STRUCT,  \
+.offset   = vmstate_offset_value(_state, _field, PCIDevice), \
+.field_exists = (_test)  \
 }
 
+#define VMSTATE_MSIX(_f, _s) \
+VMSTATE_MSIX_TEST(_f, _s, NULL)
+
 #endif
-- 
2.4.3

[Qemu-devel] [PATCH v4 00/47] ivshmem improvements

[marcandre . lureau]

From: Marc-André Lureau 

Hi,

This series is mostly about adding the client/server code from David
Marchand, code cleanups, and little improvements and fixes for
ivshmem. Finally there is some ivshmem tests (they work fine without
kvm btw).

Git: https://github.com/elmarco/qemu.git ivshmem branch

v3->v4:
- remove an extra ;
- improve msix debug message
- improve some commit summary
- improve some code comment
- add a message to server to inform hugepages or shm usage
- replace function macros in test with regular functions
- replace parse_mem_size() patch with strtosz()
- add a patch to change the protocol to i64 le

v2->v3:
- add old migration fallback code
- split a few patches
- improve some commit messages
- drop a commit checking eventfd read value

v1->v2:
- add support for hugepage shm (suggested by Andrew Jones)
- add irqfd for msix notification

David Marchand (3):
  contrib: add ivshmem client and server
  docs: update ivshmem device spec
  ivshmem: add check on protocol version in QEMU

Marc-André Lureau (44):
  char: add qemu_chr_free()
  msix: add VMSTATE_MSIX_TEST
  ivhsmem: read do not accept more than sizeof(long)
  ivshmem: fix number of bytes to push to fifo
  ivshmem: factor out the incoming fifo handling
  ivshmem: remove unnecessary dup()
  ivshmem: remove superflous ivshmem_attr field
  ivshmem: remove useless doorbell field
  ivshmem: more qdev conversion
  ivshmem: remove last exit(1)
  ivshmem: limit maximum number of peers to G_MAXUINT16
  ivshmem: simplify around increase_dynamic_storage()
  ivshmem: allocate eventfds in resize_peers()
  ivshmem: remove useless ivshmem_update_irq() val argument
  ivshmem: initialize max_peer to -1
  ivshmem: remove max_peer field
  ivshmem: improve debug messages
  ivshmem: improve error handling
  ivshmem: print error on invalid peer id
  ivshmem: simplify a bit the code
  ivshmem: use common return
  ivshmem: use common is_power_of_2()
  ivshmem: migrate with VMStateDescription
  ivshmem: shmfd can be 0
  ivshmem: check shm isn't already initialized
  ivshmem: add device description
  ivshmem: fix pci_ivshmem_exit()
  ivshmem: replace 'guest' for 'peer' appropriately
  ivshmem: error on too many eventfd received
  ivshmem: reset mask on device reset
  ivshmem-client: check the number of vectors
  ivshmem-server: use a uint16 for client ID
  ivshmem-server: fix hugetlbfs support
  contrib: remove unnecessary strdup()
  msix: implement pba write (but read-only)
  qtest: add qtest_add_abrt_handler()
  tests: add ivshmem qtest
  ivshmem: do not keep shm_fd open
  ivshmem: use strtosz()
  ivshmem: add hostmem backend
  ivshmem: remove EventfdEntry.vector
  ivshmem: rename MSI eventfd_table
  ivshmem: use kvm irqfd for msi notifications
  ivshmem: use little-endian int64_t for the protocol

 Makefile|   8 +
 configure   |   3 +
 contrib/ivshmem-client/ivshmem-client.c | 445 +
 contrib/ivshmem-client/ivshmem-client.h | 213 
 contrib/ivshmem-client/main.c   | 239 +
 contrib/ivshmem-server/ivshmem-server.c | 480 ++
 contrib/ivshmem-server/ivshmem-server.h | 166 +++
 contrib/ivshmem-server/main.c   | 263 ++
 docs/specs/ivshmem_device_spec.txt  | 127 +++--
 hw/misc/ivshmem.c   | 827 ++--
 hw/pci/msix.c   |   6 +
 include/hw/misc/ivshmem.h   |  25 +
 include/hw/pci/msix.h   |  16 +-
 include/sysemu/char.h   |  10 +-
 qemu-char.c |   9 +-
 qemu-doc.texi   |  10 +-
 tests/Makefile  |   3 +
 tests/ivshmem-test.c| 493 +++
 tests/libqtest.c|  37 +-
 tests/libqtest.h|   2 +
 20 files changed, 3066 insertions(+), 316 deletions(-)
 create mode 100644 contrib/ivshmem-client/ivshmem-client.c
 create mode 100644 contrib/ivshmem-client/ivshmem-client.h
 create mode 100644 contrib/ivshmem-client/main.c
 create mode 100644 contrib/ivshmem-server/ivshmem-server.c
 create mode 100644 contrib/ivshmem-server/ivshmem-server.h
 create mode 100644 contrib/ivshmem-server/main.c
 create mode 100644 include/hw/misc/ivshmem.h
 create mode 100644 tests/ivshmem-test.c

-- 
2.4.3