Re: [Qemu-devel] [PATCH 08/17] mm: madvise MADV_USERFAULT

2014-10-03 Thread Mike Hommey
On Fri, Oct 03, 2014 at 07:07:58PM +0200, Andrea Arcangeli wrote: > MADV_USERFAULT is a new madvise flag that will set VM_USERFAULT in the > vma flags. Whenever VM_USERFAULT is set in an anonymous vma, if > userland touches a still unmapped virtual address, a sigbus signal is > sent instead of allo

Re: [Qemu-devel] [PATCH 5/6] qemu-char: Add reconnecting to client sockets

2014-10-03 Thread Corey Minyard
On 10/03/2014 05:22 PM, Paolo Bonzini wrote: > Il 02/10/2014 18:17, miny...@acm.org ha scritto: >> +if (!qemu_chr_open_socket_fd(chr, &err)) { >> +error_report("Unable to connect to char device %s\n", chr->label); >> +qemu_chr_socket_restart_timer(chr); >> +} > Can you pleas

Re: [Qemu-devel] [PATCH 5/6] qemu-char: Add reconnecting to client sockets

2014-10-03 Thread Paolo Bonzini
Il 02/10/2014 18:17, miny...@acm.org ha scritto: > +if (!qemu_chr_open_socket_fd(chr, &err)) { > +error_report("Unable to connect to char device %s\n", chr->label); > +qemu_chr_socket_restart_timer(chr); > +} Can you please add a follow-up patch that only prints this messag

Re: [Qemu-devel] [PATCH v3 0/6] target-i386: Make most CPU models work with "enforce" out of the box

2014-10-03 Thread Paolo Bonzini
Il 03/10/2014 21:39, Eduardo Habkost ha scritto: > Changes v2 -> v3: > * None. This is just a rebase against latest qemu.git master (commit b00a0dd) > > Changes v1 -> v2: > * Commit message and comment changes. > * Update compat code to change pc-*-2.1, not pc-*-2.0. > * Added patch to disable

[Qemu-devel] [PATCH v2 1/1] -machine vmport=off: Allow disabling of VMWare ioport emulation

2014-10-03 Thread Don Slutz
From: "Dr. David Alan Gilbert" This is a pc & q35 only machine opt. VMWare apparently doesn't like running under QEMU due to our incomplete emulation of it's special IO Port. This adds a pc & q35 property to allow it to be turned off. Signed-off-by: Dr. David Alan Gilbert Signed-off-by: Don S

[Qemu-devel] [PATCH v2 0/1] -machine vmport=off: Allow disabling of VMWare ioport emulation

2014-10-03 Thread Don Slutz
Changes v1 to v2 (Don Slutz): make vmport a pc & q35 only machine opt I.E. a machine property. Dr. David Alan Gilbert (1): -machine vmport=off: Allow disabling of VMWare ioport emulation hw/i386/pc.c | 19 +++ hw/i386/pc_piix.c| 4 ++-- hw/i386/pc_q35.c | 3

Re: [Qemu-devel] [PATCH 01/17] mm: gup: add FOLL_TRIED

2014-10-03 Thread Paolo Bonzini
> This needs more explanation than that one-liner comment. Make the > commit message explain why the new FOLL_TRIED flag exists. This patch actually is extracted from a 3.18 commit in the KVM tree, https://git.kernel.org/cgit/virt/kvm/kvm.git/commit/?h=next&id=234b239b. Here is how that patch use

[Qemu-devel] [PATCH 12/17] mm: sys_remap_anon_pages

2014-10-03 Thread Andrea Arcangeli
This new syscall will move anon pages across vmas, atomically and without touching the vmas. It only works on non shared anonymous pages because those can be relocated without generating non linear anon_vmas in the rmap code. It is the ideal mechanism to handle userspace page faults. Normally the

[Qemu-devel] [Bug 1336801] Re: 12.04 guest hangs on a 14.04 host server with cirrus graphics

2014-10-03 Thread Colin
I'm also having issues with libvirt/kvm guests becoming unresponsive and they were all using cirrus as their video model. I've tried switching some to vga to see if that helps. -- You received this bug notification because you are a member of qemu- devel-ml, which is subscribed to QEMU. https://b

[Qemu-devel] [PATCH 07/17] mm: madvise MADV_USERFAULT: prepare vm_flags to allow more than 32bits

2014-10-03 Thread Andrea Arcangeli
We run out of 32bits in vm_flags, noop change for 64bit archs. Signed-off-by: Andrea Arcangeli --- fs/proc/task_mmu.c | 4 ++-- include/linux/huge_mm.h | 4 ++-- include/linux/ksm.h | 4 ++-- include/linux/mm_types.h | 2 +- mm/huge_memory.c | 2 +- mm/ksm.c |

[Qemu-devel] [PATCH 02/17] mm: gup: add get_user_pages_locked and get_user_pages_unlocked

2014-10-03 Thread Andrea Arcangeli
We can leverage the VM_FAULT_RETRY functionality in the page fault paths better by using either get_user_pages_locked or get_user_pages_unlocked. The former allow conversion of get_user_pages invocations that will have to pass a "&locked" parameter to know if the mmap_sem was dropped during the ca

[Qemu-devel] [PATCH 15/17] userfaultfd: make userfaultfd_write non blocking

2014-10-03 Thread Andrea Arcangeli
It is generally inefficient to ask the wakeup of userfault ranges where there's not a single userfault address read through userfaultfd_read earlier and in turn waiting a wakeup. However it may come handy to wakeup the same userfault range twice in case of multiple thread faulting on the same addre

Re: [Qemu-devel] [PATCH 10/17] mm: rmap preparation for remap_anon_pages

2014-10-03 Thread Linus Torvalds
On Fri, Oct 3, 2014 at 10:08 AM, Andrea Arcangeli wrote: > > Overall this looks a fairly small change to the rmap code, notably > less intrusive than the nonlinear vmas created by remap_file_pages. Considering that remap_file_pages() was an unmitigated disaster, and -mm has a patch to remove it e

[Qemu-devel] [PATCH 05/17] mm: gup: use get_user_pages_fast and get_user_pages_unlocked

2014-10-03 Thread Andrea Arcangeli
Just an optimization. Signed-off-by: Andrea Arcangeli --- drivers/dma/iovlock.c | 10 ++ drivers/iommu/amd_iommu_v2.c | 6 ++ drivers/media/pci/ivtv/ivtv-udma.c | 6 ++ drivers/scsi/st.c | 10 ++ drivers/video/fbdev/pvr2fb.c |

[Qemu-devel] [PATCH 04/17] mm: gup: make get_user_pages_fast and __get_user_pages_fast latency conscious

2014-10-03 Thread Andrea Arcangeli
This teaches gup_fast and __gup_fast to re-enable irqs and cond_resched() if possible every BATCH_PAGES. This must be implemented by other archs as well and it's a requirement before converting more get_user_pages() to get_user_pages_fast() as an optimization (instead of using get_user_pages_unloc

[Qemu-devel] [PATCH 08/17] mm: madvise MADV_USERFAULT

2014-10-03 Thread Andrea Arcangeli
MADV_USERFAULT is a new madvise flag that will set VM_USERFAULT in the vma flags. Whenever VM_USERFAULT is set in an anonymous vma, if userland touches a still unmapped virtual address, a sigbus signal is sent instead of allocating a new page. The sigbus signal handler will then resolve the page fa

[Qemu-devel] [PATCH 10/17] mm: rmap preparation for remap_anon_pages

2014-10-03 Thread Andrea Arcangeli
remap_anon_pages (unlike remap_file_pages) tries to be non intrusive in the rmap code. As far as the rmap code is concerned, rmap_anon_pages only alters the page->mapping and page->index. It does it while holding the page lock. However there are a few places that in presence of anon pages are allo

[Qemu-devel] [PATCH 17/17] userfaultfd: implement USERFAULTFD_RANGE_REGISTER|UNREGISTER

2014-10-03 Thread Andrea Arcangeli
This adds two protocol commands to the userfaultfd protocol. To register memory regions into userfaultfd you can write 16 bytes as: [ start|0x1, end ] to unregister write: [ start|0x2, end ] End is "start+len" (not start+len-1). Same as vma->vm_end. This also enforces the co

Re: [Qemu-devel] [PATCH 04/17] mm: gup: make get_user_pages_fast and __get_user_pages_fast latency conscious

2014-10-03 Thread Linus Torvalds
On Fri, Oct 3, 2014 at 10:07 AM, Andrea Arcangeli wrote: > This teaches gup_fast and __gup_fast to re-enable irqs and > cond_resched() if possible every BATCH_PAGES. This is disgusting. Many (most?) __gup_fast() users just want a single page, and the stupid overhead of the multi-page version is

[Qemu-devel] [PATCH 06/17] kvm: Faults which trigger IO release the mmap_sem

2014-10-03 Thread Andrea Arcangeli
From: Andres Lagar-Cavilla When KVM handles a tdp fault it uses FOLL_NOWAIT. If the guest memory has been swapped out or is behind a filemap, this will trigger async readahead and return immediately. The rationale is that KVM will kick back the guest with an "async page fault" and allow for some

[Qemu-devel] [PATCH 13/17] waitqueue: add nr wake parameter to __wake_up_locked_key

2014-10-03 Thread Andrea Arcangeli
Userfaultfd needs to wake all waitqueues (pass 0 as nr parameter), instead of the current hardcoded 1 (that would wake just the first waitqueue in the head list). Signed-off-by: Andrea Arcangeli --- include/linux/wait.h | 5 +++-- kernel/sched/wait.c | 7 --- net/sunrpc/sched.c | 2 +- 3

[Qemu-devel] [PATCH 09/17] mm: PT lock: export double_pt_lock/unlock

2014-10-03 Thread Andrea Arcangeli
Those two helpers are needed by remap_anon_pages. Signed-off-by: Andrea Arcangeli --- include/linux/mm.h | 4 mm/fremap.c| 29 + 2 files changed, 33 insertions(+) diff --git a/include/linux/mm.h b/include/linux/mm.h index bf3df07..71dbe03 100644 --- a/i

Re: [Qemu-devel] [PATCH 01/17] mm: gup: add FOLL_TRIED

2014-10-03 Thread Linus Torvalds
This needs more explanation than that one-liner comment. Make the commit message explain why the new FOLL_TRIED flag exists. Linus On Fri, Oct 3, 2014 at 10:07 AM, Andrea Arcangeli wrote: > From: Andres Lagar-Cavilla > > Reviewed-by: Radim Krčmář > Signed-off-by: Andres Lagar-Cavilla >

[Qemu-devel] [PATCH 00/17] RFC: userfault v2

2014-10-03 Thread Andrea Arcangeli
Hello everyone, There's a large To/Cc list for this RFC because this adds two new syscalls (userfaultfd and remap_anon_pages) and MADV_USERFAULT/MADV_NOUSERFAULT, so suggestions on changes are welcome sooner than later. The major change compared to the previous RFC I sent a few months ago is that

[Qemu-devel] [PATCH 11/17] mm: swp_entry_swapcount

2014-10-03 Thread Andrea Arcangeli
Provide a new swapfile method for remap_anon_pages to verify the swap entry is mapped only in one vma before relocating the swap entry in a different virtual address. Otherwise if the swap entry is mapped in multiple vmas, when the page is swapped back in, it could get mapped in a non linear way in

[Qemu-devel] [PATCH 14/17] userfaultfd: add new syscall to provide memory externalization

2014-10-03 Thread Andrea Arcangeli
Once an userfaultfd is created MADV_USERFAULT regions talks through the userfaultfd protocol with the thread responsible for doing the memory externalization of the process. The protocol starts by userland writing the requested/preferred USERFAULT_PROTOCOL version into the userfault fd (64bit writ

[Qemu-devel] [PATCH 01/17] mm: gup: add FOLL_TRIED

2014-10-03 Thread Andrea Arcangeli
From: Andres Lagar-Cavilla Reviewed-by: Radim Krčmář Signed-off-by: Andres Lagar-Cavilla Signed-off-by: Andrea Arcangeli --- include/linux/mm.h | 1 + mm/gup.c | 4 2 files changed, 5 insertions(+) diff --git a/include/linux/mm.h b/include/linux/mm.h index 8981cc8..0f4196a 100

[Qemu-devel] [PATCH 03/17] mm: gup: use get_user_pages_unlocked within get_user_pages_fast

2014-10-03 Thread Andrea Arcangeli
Signed-off-by: Andrea Arcangeli --- arch/mips/mm/gup.c | 8 +++- arch/powerpc/mm/gup.c| 6 ++ arch/s390/kvm/kvm-s390.c | 4 +--- arch/s390/mm/gup.c | 6 ++ arch/sh/mm/gup.c | 6 ++ arch/sparc/mm/gup.c | 6 ++ arch/x86/mm/gup.c| 7 +++ 7

[Qemu-devel] [PATCH 16/17] powerpc: add remap_anon_pages and userfaultfd

2014-10-03 Thread Andrea Arcangeli
Add the syscall numbers. Signed-off-by: Andrea Arcangeli --- arch/powerpc/include/asm/systbl.h | 2 ++ arch/powerpc/include/asm/unistd.h | 2 +- arch/powerpc/include/uapi/asm/unistd.h | 2 ++ 3 files changed, 5 insertions(+), 1 deletion(-) diff --git a/arch/powerpc/include/asm/systbl.

Re: [Qemu-devel] [Xen-devel] [RFC][PATCH v2 1/1] Add IOREQ_TYPE_VMWARE_PORT

2014-10-03 Thread Don Slutz
On 10/03/14 05:46, Paul Durrant wrote: -Original Message- From: Paul Durrant Sent: 03 October 2014 10:29 To: 'Don Slutz'; xen-de...@lists.xen.org Cc: Keir (Xen.org); Ian Campbell; Jan Beulich Subject: RE: [Xen-devel] [RFC][PATCH v2 1/1] Add IOREQ_TYPE_VMWARE_PORT -Original Message--

[Qemu-devel] [PATCH v3 5/6] target-i386: Don't enable nested VMX by default

2014-10-03 Thread Eduardo Habkost
TCG doesn't support VMX, and nested VMX is not enabled by default on the KVM kernel module. So, there's no reason to have VMX enabled by default on the core2duo and coreduo CPU models, today. Even the newer Intel CPU model definitions don't have it enabled. In this case, we need machine-type comp

[Qemu-devel] [PATCH v3 1/6] pc: Create pc_compat_2_1() functions

2014-10-03 Thread Eduardo Habkost
We will need new compat code for the 2.1 machine-types. Signed-off-by: Eduardo Habkost --- hw/i386/pc_piix.c | 13 - hw/i386/pc_q35.c | 13 - 2 files changed, 24 insertions(+), 2 deletions(-) diff --git a/hw/i386/pc_piix.c b/hw/i386/pc_piix.c index 103d756..0a4f4d3 1006

[Qemu-devel] [PATCH v3 3/6] target-i386: Disable CPUID_ACPI by default on KVM mode

2014-10-03 Thread Eduardo Habkost
KVM never supported the CPUID_ACPI flag, so it doesn't make sense to have it enabled by default when KVM is enabled. The motivation here is exactly the same we had for the MONITOR flag (disabled by commit 136a7e9a85d7047461f8153f7d12c514a3d68f69). And like on the MONITOR flag case, we don't need

[Qemu-devel] [PATCH v3 6/6] target-i386: Disable SVM by default in KVM mode

2014-10-03 Thread Eduardo Habkost
Make SVM be disabled by default on all CPU models when in KVM mode. Nested SVM is enabled by default in the KVM kernel module, but it is probably less stable than nested VMX (which is already disabled by default). Add a new compat function, x86_cpu_compat_kvm_no_autodisable(), to keep compatibilit

[Qemu-devel] [PATCH v3 0/6] target-i386: Make most CPU models work with "enforce" out of the box

2014-10-03 Thread Eduardo Habkost
Changes v2 -> v3: * None. This is just a rebase against latest qemu.git master (commit b00a0dd) Changes v1 -> v2: * Commit message and comment changes. * Update compat code to change pc-*-2.1, not pc-*-2.0. * Added patch to disable SVM by default in KVM mode. Most of the bits that make "enfor

[Qemu-devel] [PATCH v3 2/6] target-i386: Rename KVM auto-feature-enable compat function

2014-10-03 Thread Eduardo Habkost
The x86_cpu_compat_disable_kvm_features() name was a bit confusing, as it won't forcibly disable the feature for all CPU models (i.e. add it to kvm_default_unset_features), but it will instead turn off the KVM auto-enabling of the feature (i.e. remove it from kvm_default_features), meaning the feat

[Qemu-devel] [PATCH v3 4/6] target-i386: Remove unsupported bits from all CPU models

2014-10-03 Thread Eduardo Habkost
The following CPU features were never supported by neither TCG or KVM, so they are useless on the CPU model definitions, today: * CPUID_DTS (DS) * CPUID_HT * CPUID_TM * CPUID_PBE * CPUID_EXT_DTES64 * CPUID_EXT_DSCPL * CPUID_EXT_EST * CPUID_EXT_TM2 * CPUID_EXT_XTPR * CPUID_EXT_PDCM * CPU

Re: [Qemu-devel] [PATCH v4 00/47] Postcopy implementation

2014-10-03 Thread Dr. David Alan Gilbert
I've updated our github at: https://github.com/orbitfp7/qemu/tree/wp3-postcopy to have this version. and it corresponds to the tag: https://github.com/orbitfp7/qemu/releases/tag/wp3-postcopy-v4 Dave * Dr. David Alan Gilbert (git) (dgilb...@redhat.com) wrote: > From: "Dr. David Alan Gilbert" >

[Qemu-devel] [PATCH v4 39/47] postcopy_ram.c: place_page and helpers

2014-10-03 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" postcopy_place_page (etc) provide a way for postcopy to place a page into guests memory atomically (using the new remap_anon_pages syscall). Signed-off-by: Dr. David Alan Gilbert --- include/migration/migration.h| 2 + include/migration/postcopy-ram.h | 23

[Qemu-devel] [PATCH v4 34/47] Postcopy: Create a fault handler thread before marking the ram as userfault

2014-10-03 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Signed-off-by: Dr. David Alan Gilbert --- include/migration/migration.h | 3 +++ postcopy-ram.c| 23 +++ 2 files changed, 26 insertions(+) diff --git a/include/migration/migration.h b/include/migration/migration.h index f40177

Re: [Qemu-devel] [PATCH v2 1/1] xen-hvm.c: Add support for Xen access to vmport

2014-10-03 Thread Don Slutz
On 10/03/14 12:23, Stefano Stabellini wrote: On Fri, 3 Oct 2014, Don Slutz wrote: On 10/03/14 05:52, Stefano Stabellini wrote: On Thu, 2 Oct 2014, Don Slutz wrote: This adds synchronisation of the 6 vcpu registers (only 32bits of them) that vmport.c needs between Xen and QEMU. ... } -

[Qemu-devel] [PATCH v4 46/47] postcopy: Wire up loadvm_postcopy_ram_handle_{run, end} commands

2014-10-03 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Wire up more of the handlers for the commands on the destination side, in particular loadvm_postcopy_ram_handle_run now has enough to start the guest running. Signed-off-by: Dr. David Alan Gilbert --- savevm.c | 63

[Qemu-devel] [PATCH v4 42/47] Don't sync dirty bitmaps in postcopy

2014-10-03 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Once we're in postcopy the source processors are stopped and memory shouldn't change any more, so there's no need to look at the dirty map. There are two notes to this: 1) If we do resync and a page had changed then the page would get sent again, which the d

[Qemu-devel] [PATCH v4 33/47] Postcopy: Postcopy startup in migration thread

2014-10-03 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Rework the migration thread to setup and start postcopy. Signed-off-by: Dr. David Alan Gilbert --- include/migration/migration.h | 3 + migration.c | 201 ++ 2 files changed, 185 insertions(+), 19 deleti

[Qemu-devel] [PATCH v4 32/47] postcopy: ram_enable_notify to switch on userfault

2014-10-03 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Signed-off-by: Dr. David Alan Gilbert --- include/migration/migration.h| 2 ++ include/migration/postcopy-ram.h | 6 + postcopy-ram.c | 49 +++- savevm.c | 9 4 files

[Qemu-devel] [PATCH v4 29/47] Postcopy page-map-incoming (PMI) structure

2014-10-03 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" The PMI holds the state of each page on the incoming side, so that we can tell if the page is missing, already received or there is a request outstanding for it. Signed-off-by: Dr. David Alan Gilbert --- include/migration/migration.h| 19 include/migrat

[Qemu-devel] [PATCH v4 44/47] Postcopy; Handle userfault requests

2014-10-03 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" userfaultfd is a Linux syscall that gives an fd that receives a stream of notifications of accesses to pages marked as MADV_USERFAULT, and allows the program to acknowledge those stalls and tell the accessing thread to carry on. Signed-off-by: Dr. David Alan Gilber

[Qemu-devel] [PATCH v4 28/47] qemu_savevm_state_complete: Postcopy changes

2014-10-03 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" When postcopy calls qemu_savevm_state_complete it's not really the end of migration, so skip: a) Finishing postcopiable iterative devices - they'll carry on b) The termination byte on the end of the stream. We then also add: qemu_savevm_state_postcopy_compl

[Qemu-devel] [PATCH v4 41/47] qemu_ram_block_from_host

2014-10-03 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Postcopy sends RAMBlock names and offsets over the wire (since it can't rely on the order of ramaddr being the same), and it starts out with HVA fault addresses from the kernel. qemu_ram_block_from_host translates a HVA into a RAMBlock, an offset in the RAMBlock, t

[Qemu-devel] [PATCH v4 25/47] postcopy: OS support test

2014-10-03 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Provide a check to see if the OS we're running on has all the bits needed for postcopy. Creates postcopy-ram.c which will get most of the other helpers we need. Signed-off-by: Dr. David Alan Gilbert --- Makefile.objs| 2 +- include/migratio

[Qemu-devel] [PATCH v4 35/47] Page request: Add MIG_RPCOMM_REQPAGES reverse command

2014-10-03 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Add MIG_RPCOMM_REQPAGES command on Return path for the postcopy destination to request a page from the source. Signed-off-by: Dr. David Alan Gilbert --- include/migration/migration.h | 3 ++ migration.c | 74

[Qemu-devel] [PATCH v4 40/47] Postcopy: Use helpers to map pages during migration

2014-10-03 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" In postcopy, the destination guest is running at the same time as it's receiving pages; as we receive new pages we must put them into the guests address space atomically to avoid a running CPU accessing a partially written page. Use the helpers in postcopy-ram.c to

[Qemu-devel] [PATCH v4 23/47] migrate_init: Call from savevm

2014-10-03 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Suspend to file is very much like a migrate, and it makes life easier if we have the Migration state available, so initialise it in the savevm.c code for suspending. Signed-off-by: Dr. David Alan Gilbert --- include/migration/migration.h | 1 + include/qemu/typed

[Qemu-devel] [PATCH v4 47/47] End of migration for postcopy

2014-10-03 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Tweak the end of migration cleanup; we don't want to close stuff down at the end of the main stream, since the postcopy is still sending pages on the other thread. Signed-off-by: Dr. David Alan Gilbert --- migration.c | 23 ++- 1 file changed,

[Qemu-devel] [PATCH v4 30/47] Postcopy: Maintain sentmap and calculate discard

2014-10-03 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Where postcopy is preceeded by a period of precopy, the destination will have received pages that may have been dirtied on the source after the page was sent. The destination must throw these pages away before starting it's CPUs. Maintain a 'sentmap' of pages that

[Qemu-devel] [PATCH v4 37/47] Page request: Consume pages off the post-copy queue

2014-10-03 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" When transmitting RAM pages, consume pages that have been queued by MIG_RPCOMM_REQPAGE commands and send them ahead of normal page scanning. Note: a) After a queued page the linear walk carries on from after the unqueued page; there is a reasonable chance that th

[Qemu-devel] [PATCH v4 45/47] Start up a postcopy/listener thread ready for incoming page data

2014-10-03 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" The loading of a device state (during postcopy) may access guest memory that's still on the source machine and thus might need a page fill; split off a separate thread that handles the incoming page data so that the original incoming migration code can finish off th

[Qemu-devel] [PATCH v4 20/47] Add migration-capability boolean for postcopy-ram.

2014-10-03 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Signed-off-by: Dr. David Alan Gilbert Reviewed-by: Eric Blake --- include/migration/migration.h | 1 + migration.c | 9 + qapi-schema.json | 6 +- 3 files changed, 15 insertions(+), 1 deletion(-) diff --git a/include/mi

[Qemu-devel] [PATCH v4 27/47] MIG_STATE_POSTCOPY_ACTIVE: Add new migration state

2014-10-03 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" 'MIG_STATE_POSTCOPY_ACTIVE' is entered after migrate_start_postcopy 'migration_postcopy_phase' is provided for other sections to know if they're in postcopy. Signed-off-by: Dr. David Alan Gilbert --- include/migration/migration.h | 2 ++ migration.c

[Qemu-devel] [PATCH v4 31/47] postcopy: Incoming initialisation

2014-10-03 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Signed-off-by: Dr. David Alan Gilbert --- arch_init.c | 11 include/migration/migration.h| 1 + include/migration/postcopy-ram.h | 12 + migration.c | 1 + postcopy-ram.c | 110

[Qemu-devel] [PATCH v4 19/47] Rework loadvm path for subloops

2014-10-03 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Postcopy needs to have two migration streams loading concurrently; one from memory (with the device state) and the other from the fd with the memory transactions. Split the core of qemu_loadvm_state out so we can use it for both. Allow the inner loadvm loop to qui

[Qemu-devel] [PATCH v4 43/47] Host page!=target page: Cleanup bitmaps

2014-10-03 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Prior to the start of postcopy, ensure that everything that will be transferred later is a whole host-page in size. This is accomplished by discarding partially transferred host pages and marking any that are partially dirty as fully dirty. Signed-off-by: Dr. Davi

[Qemu-devel] [PATCH v4 26/47] migrate_start_postcopy: Command to trigger transition to postcopy

2014-10-03 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Once postcopy is enabled (with migrate_set_capability), the migration will still start on precopy mode. To cause a transition into postcopy the: migrate_start_postcopy command must be issued. Postcopy will start sometime after this (when it's next checked in t

[Qemu-devel] [PATCH v4 38/47] Add assertion to check migration_dirty_pages

2014-10-03 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" I've seen it go negative once during dev, it shouldn't happen. Signed-off-by: Dr. David Alan Gilbert --- arch_init.c | 1 + 1 file changed, 1 insertion(+) diff --git a/arch_init.c b/arch_init.c index a945990..2f4345a 100644 --- a/arch_init.c +++ b/arch_init.c @@

[Qemu-devel] [PATCH v4 24/47] Allow savevm handlers to state whether they could go into postcopy

2014-10-03 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Use that to split the qemu_savevm_state_pending counts into postcopiable and non-postcopiable amounts Signed-off-by: Dr. David Alan Gilbert --- arch_init.c | 7 +++ include/migration/vmstate.h | 2 +- include/sysemu/sysemu.h | 4 +++- m

[Qemu-devel] [PATCH v4 17/47] qemu_loadvm errors and debug

2014-10-03 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Flip many fprintf's to error_report Add lots of DPRINTF debug in qemu_loadvm* Signed-off-by: Dr. David Alan Gilbert --- savevm.c | 23 +-- 1 file changed, 17 insertions(+), 6 deletions(-) diff --git a/savevm.c b/savevm.c index 8eebbfd..2c0d61

[Qemu-devel] [PATCH v4 22/47] QEMU_VM_CMD_PACKAGED: Send a packaged chunk of migration stream

2014-10-03 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" QEMU_VM_CMD_PACKAGED is a migration command that allows a chunk of migration stream to be sent in one go, and be received by a separate instance of the loadvm loop while not interacting with the migration stream. This is used by postcopy to load device state (from

[Qemu-devel] [PATCH v4 36/47] Page request: Process incoming page request

2014-10-03 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" On receiving MIG_RPCOMM_REQPAGES look up the address and queue the page. Signed-off-by: Dr. David Alan Gilbert --- arch_init.c | 52 +++ include/migration/migration.h | 21 + include/qemu/t

[Qemu-devel] [PATCH v4 18/47] ram_debug_dump_bitmap: Dump a migration bitmap as text

2014-10-03 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Misses out lines that are all the expected value so the output can be quite compact depending on the circumstance. Signed-off-by: Dr. David Alan Gilbert --- arch_init.c | 39 +++ include/migration/migration.h

[Qemu-devel] [PATCH v4 16/47] Return path: Source handling of return path

2014-10-03 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Open a return path, and handle messages that are received upon it. Signed-off-by: Dr. David Alan Gilbert --- include/migration/migration.h | 10 +++ migration.c | 181 +- 2 files changed, 190 insertions(+

[Qemu-devel] [PATCH v4 21/47] Add wrappers and handlers for sending/receiving the postcopy-ram migration messages.

2014-10-03 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Add state variable showing current incoming postcopy state. Signed-off-by: Dr. David Alan Gilbert --- include/migration/migration.h | 8 + include/sysemu/sysemu.h | 20 +++ savevm.c | 335 ++ 3

[Qemu-devel] [PATCH v4 15/47] Return path: Send responses from destination to source

2014-10-03 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Add migrate_send_rp_message to send a message from destination to source along the return path. (It uses a mutex to let it be called from multiple threads) Add migrate_send_rp_shut to send a 'shut' message to indicate the destination is finished with the RP. Ad

[Qemu-devel] [PATCH v4 12/47] Handle bi-directional communication for fd migration

2014-10-03 Thread Dr. David Alan Gilbert (git)
From: Cristian Klein Signed-off-by: Cristian Klein --- migration-fd.c | 24 ++-- 1 file changed, 22 insertions(+), 2 deletions(-) diff --git a/migration-fd.c b/migration-fd.c index d2e523a..129da99 100644 --- a/migration-fd.c +++ b/migration-fd.c @@ -31,13 +31,29 @@ do

[Qemu-devel] [PATCH v4 14/47] Return path: Control commands

2014-10-03 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Add two src->dest commands: * OPENRP - To request that the destination open the return path * REQACK - Request an acknowledge from the destination Signed-off-by: Dr. David Alan Gilbert --- include/migration/migration.h | 2 ++ include/sysemu/sysemu.h

[Qemu-devel] [PATCH v4 13/47] Migration commands

2014-10-03 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Create QEMU_VM_COMMAND section type for sending commands from source to destination. These commands are not intended to convey guest state but to control the migration process. For use in postcopy. Signed-off-by: Dr. David Alan Gilbert --- include/migration/mig

[Qemu-devel] [PATCH v4 10/47] Return path: Open a return path on QEMUFile for sockets

2014-10-03 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Postcopy needs a method to send messages from the destination back to the source, this is the 'return path'. Wire it up for 'socket' QEMUFile's using a dup'd fd. Signed-off-by: Dr. David Alan Gilbert --- include/migration/qemu-file.h | 7 + qemu-file.c

[Qemu-devel] [PATCH v4 11/47] Return path: socket_writev_buffer: Block even on non-blocking fd's

2014-10-03 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" The return path uses a non-blocking fd so as not to block waiting for the (possibly broken) destination to finish returning a message, however we still want outbound data to behave in the same way and block. Signed-off-by: Dr. David Alan Gilbert --- qemu-file.c |

[Qemu-devel] [PATCH v4 08/47] socket shutdown

2014-10-03 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Add QEMUFile interface to allow a socket to be 'shut down' - i.e. any reads/writes will fail (and any blocking read/write will be woken). Add qemu_socket wrapper to let OS dependencies be extracted out. Signed-off-by: Dr. David Alan Gilbert --- include/migration

[Qemu-devel] [PATCH v4 04/47] qemu_ram_foreach_block: pass up error value, and down the ramblock name

2014-10-03 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" check the return value of the function it calls and error if it's non-0 Fixup qemu_rdma_init_one_block that is the only current caller, and __qemu_rdma_add_block the only function it calls using it. Pass the name of the ramblock to the function; helps in debuggin

[Qemu-devel] [PATCH v4 09/47] Provide runtime Target page information

2014-10-03 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" The migration code generally is built target-independent, however there are a few places where knowing the target page size would avoid artificially moving stuff into arch_init. Provide 'qemu_target_page_bits()' that returns TARGET_PAGE_BITS to other bits of code s

[Qemu-devel] [PATCH v4 05/47] improve DPRINTF macros, add to savevm

2014-10-03 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Improve the existing DPRINTF macros in migration.c and arch_init by: 1) Making them go to stderr rather than stdout (so you can run with -nographic and redirect your debug to a file) 2) Making them print the ms time with each debug - useful for debugging latency

[Qemu-devel] [PATCH v4 07/47] Create MigrationIncomingState

2014-10-03 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" There are currently lots of pieces of incoming migration state scattered around, and postcopy is adding more, and it seems better to try and keep it together. allocate MIS in process_incoming_migration_co Signed-off-by: Dr. David Alan Gilbert --- include/migrati

[Qemu-devel] [PATCH v4 03/47] Start documenting how postcopy works.

2014-10-03 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Signed-off-by: Dr. David Alan Gilbert --- docs/migration.txt | 189 + 1 file changed, 189 insertions(+) diff --git a/docs/migration.txt b/docs/migration.txt index 0492a45..a07b744 100644 --- a/docs/migration.txt

[Qemu-devel] [PATCH v4 01/47] QEMUSizedBuffer based QEMUFile

2014-10-03 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" * Please comment on separate thread for this QEMUSizedBuffer patch * This is based on Stefan and Joel's patch that creates a QEMUFile that goes to a memory buffer; from: http://lists.gnu.org/archive/html/qemu-devel/2013-03/msg05036.html Using the QEMUFile interfa

[Qemu-devel] [PATCH v4 06/47] Add qemu_get_counted_string to read a string prefixed by a count byte

2014-10-03 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" and use it in loadvm_state. Signed-off-by: Dr. David Alan Gilbert --- include/migration/qemu-file.h | 2 ++ qemu-file.c | 15 +++ savevm.c | 18 ++ 3 files changed, 27 insertions(+), 8 deletions(

[Qemu-devel] [PATCH v4 02/47] Tests: QEMUSizedBuffer/QEMUBuffer

2014-10-03 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" * Please comment on separate thread for this QEMUSziedBuffer patch * Modify some of tests/test-vmstate.c to use the in memory file based on QEMUSizedBuffer to provide basic testing of QEMUSizedBuffer and the associated memory backed QEMUFile type. Only some of the

[Qemu-devel] [PATCH v4 00/47] Postcopy implementation

2014-10-03 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Hi, This is the 4th cut of my version of postcopy; it is designed for use with the Linux kernel additions just posted by Andrea Arcangeli here: http://marc.info/?l=linux-kernel&m=141235633015100&w=2 (Note: This is a new version compared to my previous postcopy p

[Qemu-devel] [Bug 1377163] Re: Does not add usb-host devices as they are hotplugged

2014-10-03 Thread Peter Wu
With some devices, I get a speed mismatch with ehci-usb: qemu-system-x86_64: Warning: speed mismatch trying to attach usb device "USB Keyboard" (( speed) to bus "ehci.0", port "1" (high speed) nec-usb-xhci fixes the keyboard+usb-storage case, but it breaks a webcam: qemu-system-x86_64: Warning: s

Re: [Qemu-devel] [PATCH v2 1/1] xen-hvm.c: Add support for Xen access to vmport

2014-10-03 Thread Stefano Stabellini
On Fri, 3 Oct 2014, Don Slutz wrote: > On 10/03/14 05:52, Stefano Stabellini wrote: > > On Thu, 2 Oct 2014, Don Slutz wrote: > > > This adds synchronisation of the 6 vcpu registers (only 32bits of > > > them) that vmport.c needs between Xen and QEMU. > > > > > > This is to avoid a 2nd and 3rd exch

Re: [Qemu-devel] is x-data-plane considered "stable" ?

2014-10-03 Thread Alexandre DERUMIER
Ok, Great :) Thanks ! - Mail original - De: "Paolo Bonzini" À: "Alexandre DERUMIER" Cc: qemu-devel@nongnu.org, "Scott Sullivan" Envoyé: Vendredi 3 Octobre 2014 17:33:00 Objet: Re: is x-data-plane considered "stable" ? Il 03/10/2014 16:26, Alexandre DERUMIER ha scritto: > Hi Paolo, >

Re: [Qemu-devel] is x-data-plane considered "stable" ?

2014-10-03 Thread Paolo Bonzini
Il 03/10/2014 16:26, Alexandre DERUMIER ha scritto: > Hi Paolo, > > do you you think it'll be possible to use block jobs with dataplane ? > > Or is it technically impossible ? I think Stefan has posted patches for that or is going to do that very soon. The goal is to let dataplane do everything

Re: [Qemu-devel] [PATCH v4 2/3] pcie: add check for ari capability of pcie devices

2014-10-03 Thread Knut Omang
On Fri, 2014-10-03 at 13:22 +0200, Knut Omang wrote: > On Wed, 2014-10-01 at 17:08 +0300, Marcel Apfelbaum wrote: > > On Wed, 2014-10-01 at 07:26 +0200, Knut Omang wrote: > > > On Tue, 2014-09-30 at 21:38 +0800, Gonglei wrote: > > > > > Subject: Re: [Qemu-devel] [PATCH v4 2/3] pcie: add check for a

Re: [Qemu-devel] is x-data-plane considered "stable" ?

2014-10-03 Thread Alexandre DERUMIER
Hi Paolo, do you you think it'll be possible to use block jobs with dataplane ? Or is it technically impossible ? - Mail original - De: "Paolo Bonzini" À: "Alexandre DERUMIER" , "Scott Sullivan" Cc: qemu-devel@nongnu.org Envoyé: Vendredi 3 Octobre 2014 15:40:59 Objet: Re: is x-data-

[Qemu-devel] [Bug 1376798] Re: Qemu does not hotplug (usb) devices

2014-10-03 Thread ManDay
*** This bug is a duplicate of bug 1377163 *** https://bugs.launchpad.net/bugs/1377163 ** This bug has been marked a duplicate of bug 1377163 Does not add usb-host devices as they are hotplugged -- You received this bug notification because you are a member of qemu- devel-ml, which is sub

[Qemu-devel] [Bug 1377163] [NEW] Does not add usb-host devices as they are hotplugged

2014-10-03 Thread ManDay
Public bug reported: A commandline such as qemu-kvm -device usb-ehci,id=USBCtrl -device host- usb,bus=USBCtrl.0,hostbus=3 should automatically add all devices on the given bus (here: 3) not only initially, but also when new devices appear on that bus while Qemu runs. Currently, all devices on th

Re: [Qemu-devel] is x-data-plane considered "stable" ?

2014-10-03 Thread Scott Sullivan
On 10/03/2014 09:40 AM, Paolo Bonzini wrote: > Il 03/10/2014 12:11, Alexandre DERUMIER ha scritto: >> I don't now about the stability, but here what is working and I have tested: >> >> -live migration >> -resizing >> -io throttling >> -hotplugging >> >> I think block jobs (mirror,backup,...) don't

Re: [Qemu-devel] is x-data-plane considered "stable" ?

2014-10-03 Thread Paolo Bonzini
Il 03/10/2014 12:11, Alexandre DERUMIER ha scritto: > I don't now about the stability, but here what is working and I have tested: > > -live migration > -resizing > -io throttling > -hotplugging > > I think block jobs (mirror,backup,...) don't work yet. > > > it's working with virtio-blk and vi

Re: [Qemu-devel] [PATCH v2 1/1] xen-hvm.c: Add support for Xen access to vmport

2014-10-03 Thread Don Slutz
On 10/03/14 05:52, Stefano Stabellini wrote: On Thu, 2 Oct 2014, Don Slutz wrote: This adds synchronisation of the 6 vcpu registers (only 32bits of them) that vmport.c needs between Xen and QEMU. This is to avoid a 2nd and 3rd exchange between QEMU and Xen to fetch and put these 6 vcpu register

Re: [Qemu-devel] [PATCH v2 1/4] pci: Make use of the devfn property when registering new devices

2014-10-03 Thread Knut Omang
Michael, I just realized this patch set might have got "stuck" because I did not respond to this? Or did I miss something else? On Tue, 2014-09-02 at 16:55 +0300, Michael S. Tsirkin wrote: > On Tue, Sep 02, 2014 at 03:44:28PM +0200, Knut Omang wrote: > > On Tue, 2014-09-02 at 16:03 +0300, Micha

Re: [Qemu-devel] [PATCH 1/3] libqos: Remove PCI assumptions in virtio driver

2014-10-03 Thread Marc Marí
El Thu, 2 Oct 2014 13:02:25 +0100 Stefan Hajnoczi escribió: > On Thu, Sep 04, 2014 at 06:24:37PM +0200, Marc Marí wrote: > > @@ -60,25 +60,25 @@ static void > > qvirtio_pci_assign_device(QVirtioDevice *d, void *data) *vpcidev = > > (QVirtioPCIDevice *)d; } > > > > -static uint8_t qvirtio_pci_con

  1   2   >