Re: [Xen-devel] [PATCH 4/5] Change x86 paging/sharing to 'Orphaned'.
On Tue, Jun 23, 2015 at 6:22 AM, Tim Deegan wrote: > This code now has no active maintainers, as neither Andres nor I > can commit the time. > > Signed-off-by: Tim Deegan > Cc: Andres Lagar-Cavilla > Wistfully, Acked-by: Andres Lagar-Cavilla > --- > MAINTAINERS | 4 +--- > 1 file changed, 1 insertion(+), 3 deletions(-) > > diff --git a/MAINTAINERS b/MAINTAINERS > index 7f0e96e..d1bec17 100644 > --- a/MAINTAINERS > +++ b/MAINTAINERS > @@ -367,9 +367,7 @@ S: Supported > F: xen/arch/x86/mm/ > > X86 MEMORY SHARING AND PAGING > -M: Andres Lagar-Cavilla > -M: Tim Deegan > -S: Supported > +S: Orphaned > F: xen/arch/x86/mm/mem_sharing.c > F: xen/arch/x86/mm/mem_paging.c > F: tools/memshr > -- > 2.3.3 > > ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH 4/5] Change x86 paging/sharing to 'Orphaned'.
I think it's amazing that you're doing this work and I'll be happy to help if you keep me on cc. It's not serious for me to claim I'm a maintainer when I don't have the time nor the resources to do serious debugging should the need arise. Andres On Tue, Jun 23, 2015 at 10:33 AM, Lengyel, Tamas wrote: > That's indeed sad. I'm actively using the sharing system in my tools and > have a couple branches laying around for improving it, like batch > memsharing for example to support flash-cloning. Now that it's orphaned, > how would I go about merging these into mainline? I'm not yet confident > enough in my understanding of the whole sharing system to volunteer as > maintainer and my tool-side stuff is also very experimental.. > > On Tue, Jun 23, 2015 at 12:56 PM, Andres Lagar Cavilla < > and...@lagarcavilla.org> wrote: > >> >> >> On Tue, Jun 23, 2015 at 6:22 AM, Tim Deegan wrote: >> >>> This code now has no active maintainers, as neither Andres nor I >>> can commit the time. >>> >>> Signed-off-by: Tim Deegan >>> Cc: Andres Lagar-Cavilla >>> >> Wistfully, Acked-by: Andres Lagar-Cavilla >> >>> --- >>> MAINTAINERS | 4 +--- >>> 1 file changed, 1 insertion(+), 3 deletions(-) >>> >>> diff --git a/MAINTAINERS b/MAINTAINERS >>> index 7f0e96e..d1bec17 100644 >>> --- a/MAINTAINERS >>> +++ b/MAINTAINERS >>> @@ -367,9 +367,7 @@ S: Supported >>> F: xen/arch/x86/mm/ >>> >>> X86 MEMORY SHARING AND PAGING >>> -M: Andres Lagar-Cavilla >>> -M: Tim Deegan >>> -S: Supported >>> +S: Orphaned >>> F: xen/arch/x86/mm/mem_sharing.c >>> F: xen/arch/x86/mm/mem_paging.c >>> F: tools/memshr >>> -- >>> 2.3.3 >>> >>> >> >> ___ >> Xen-devel mailing list >> Xen-devel@lists.xen.org >> http://lists.xen.org/xen-devel >> >> > > > -- > > [image: www.novetta.com] > > Tamas K Lengyel > > Senior Security Researcher > > 7921 Jones Branch Drive > > McLean VA 22102 > > Email tleng...@novetta.com > ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH] x86/mem_sharing: Relax sanity check for memops
On Fri, May 29, 2015 at 2:37 AM, Tamas K Lengyel
wrote:
> The sharing vm_event ring being enabled is not necessary for mem_sharing
> memops.
>
> Signed-off-by: Tamas K Lengyel
> ---
> xen/arch/x86/mm/mem_sharing.c | 4
> 1 file changed, 4 deletions(-)
>
> diff --git a/xen/arch/x86/mm/mem_sharing.c b/xen/arch/x86/mm/mem_sharing.c
> index 0700f00..16e329e 100644
> --- a/xen/arch/x86/mm/mem_sharing.c
> +++ b/xen/arch/x86/mm/mem_sharing.c
> @@ -1320,10 +1320,6 @@ int
> mem_sharing_memop(XEN_GUEST_HANDLE_PARAM(xen_mem_sharing_op_t) arg)
> if ( !hap_enabled(d) || !d->arch.hvm_domain.mem_sharing_enabled )
> goto out;
>
> -rc = -ENODEV;
> -if ( unlikely(!d->vm_event->share.ring_page) )
> -goto out;
> -
>
Reviewed-by: Andres Lagar-Cavilla
> switch ( mso.op )
> {
> case XENMEM_sharing_op_nominate_gfn:
> --
> 2.1.4
>
>
___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel
Re: [Xen-devel] gfn_lock() seems useless.
At the time, I had a patch queue for splitting the p2m locking into a tree of per-2M locks. Very similar to Linux's split locks. I scaffolded the macros to allow for that expansion. Then, I got pulled onto other things. Andres On Tue, Feb 2, 2016 at 8:26 AM, Jan Beulich wrote: > >>> On 02.02.16 at 07:54, wrote: > > Hi all: > > In patch e1e40bccee7490a01ac7d1f759ec2bbafd3c7185, it says that"many > > routines can logically assert holding the p2m *FOR A SPECIFIC GFN.*" , > > But I find out that it did nothing for locking a single gfn, in fact it > > still locked the entire p2m list. > > > > -#define p2m_lock_recursive(p) mm_lock_recursive(p2m, &(p)->lock) > > +#define gfn_lock(p,g,o) mm_lock_recursive(p2m, &(p)->lock) //'g' > is not > > used. The entire p2m list is locked. > > > > > > Do we have any plan to lock a specific gfn? > > This indeed is a long term plan, but afaik no-one is currently > doing any work in that direction. > > Jan > > ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] Xen crash with mem-sharing and cloning
On Mon, Mar 23, 2015 at 9:10 AM, Tamas K Lengyel wrote: > Hello everyone, > I'm trying to chase down a bug that reproducibly crashes Xen (tested with > 4.4.1). The problem is somewhere within the mem-sharing subsystem and how > that interacts with domains that are being actively saved. In my setup I > use the xl toolstack to rapidly create clones of HVM domains by piping "xl > save -c" into xl restore with a modified domain config which updates the > name/disk/vif. However, during such an operation Xen crashes with the > following log if there are already active clones. > > IMHO there should be no conflict between saving the domain and memsharing, > as long as the domain is actually just being checkpointed "-c" - it's > memory should remain as is. This is however clearly not the case. Any ideas? > Tamas, I'm not clear on the use of memsharing in this workflow. As described, you pipe save into restore, but the internal magic is lost on me. Are you fanning out to multiple restores? That would seem to be the case, given the need to update name/disk/vif. Anyway, I'm inferring. Instead, could you elaborate? Thanks Andres > > Thanks, > Tamas > > -- > (XEN) HVM1 save: CPU > (XEN) HVM1 save: PIC > (XEN) HVM1 save: IOAPIC > (XEN) HVM1 save: LAPIC > (XEN) HVM1 save: LAPIC_REGS > (XEN) HVM1 save: PCI_IRQ > (XEN) HVM1 save: ISA_IRQ > (XEN) HVM1 save: PCI_LINK > (XEN) HVM1 save: PIT > (XEN) HVM1 save: RTC > (XEN) HVM1 save: HPET > (XEN) HVM1 save: PMTIMER > (XEN) HVM1 save: MTRR > (XEN) HVM1 save: VIRIDIAN_DOMAIN > (XEN) HVM1 save: CPU_XSAVE > (XEN) HVM1 save: VIRIDIAN_VCPU > (XEN) HVM1 save: VMCE_VCPU > (XEN) HVM1 save: TSC_ADJUST > (XEN) HVM19 restore: CPU 0 > (XEN) HVM19 restore: PIC 0 > (XEN) HVM19 restore: PIC 1 > (XEN) HVM19 restore: IOAPIC 0 > (XEN) HVM19 restore: LAPIC 0 > (XEN) HVM19 restore: LAPIC_REGS 0 > (XEN) HVM19 restore: PCI_IRQ 0 > (XEN) HVM19 restore: ISA_IRQ 0 > (XEN) HVM19 restore: PCI_LINK 0 > (XEN) HVM19 restore: PIT 0 > (XEN) HVM19 restore: RTC 0 > (XEN) HVM19 restore: HPET 0 > (XEN) HVM19 restore: PMTIMER 0 > (XEN) HVM19 restore: MTRR 0 > (XEN) HVM19 restore: CPU_XSAVE 0 > (XEN) HVM19 restore: VMCE_VCPU 0 > (XEN) HVM19 restore: TSC_ADJUST 0 > (XEN) [ Xen-4.4.1 x86_64 debug=n Not tainted ] > (XEN) CPU:2 > (XEN) RIP:e008:[] > __mem_sharing_unshare_page+0x1ab/0xb10 > (XEN) RFLAGS: 00010283 CONTEXT: hypervisor > (XEN) rax: rbx: 83021fa92000 rcx: 000d > (XEN) rdx: 8302ea96cb90 rsi: 0001 rdi: 002acd06 > (XEN) rbp: 6306 rsp: 83042fdffca8 r8: 007f > (XEN) r9: 0400 r10: 0081 r11: 0002 > (XEN) r12: r13: 82e00559a0c0 r14: 002acd06 > (XEN) r15: cr0: 80050033 cr4: 000426f0 > (XEN) cr3: 000253b45000 cr2: > (XEN) ds: es: fs: gs: ss: e010 cs: e008 > (XEN) Xen stack trace from rsp=83042fdffca8: > (XEN)83040001 83031010e810 83042fdf8000 > 82d08030110c > (XEN)0033b2f75d80 002acd06 0001 > > (XEN)830227b19000 0007000c > 0003 > (XEN)83031010e810 6306 83042fdffec0 > 83042fdffdb0 > (XEN) 82d0801e45b3 82d080301108 > 83042fdf8000 > (XEN) 83042fdffec0 83031010e810 > 83042fdffdb0 > (XEN)6306 0002 83021fa92000 > 82d0801e483b > (XEN)83020001 82d080173f5d 00fd00ff > 002acd06 > (XEN)0100 83040007 83042fdb77e0 > 0007 > (XEN)83027f86d1a8 > 0027f86d > (XEN)880056313bd8 82d08017e190 000a > > (XEN) 82d08017bc23 82d08012aa4b > 83040002 > (XEN)83021fa92000 83042fdaf000 8300aecea000 > 00012fdaf0b0 > (XEN)83042fdf8000 82e004ff0da0 83042fdaf000 > 0007 > (XEN)83042fdf8000 86306627 83027f86d000 > 8300aecea000 > (XEN)0027f86d 000d0003 > 6306 > (XEN)00253b45 0206 > 7f8f000c > (XEN)00027f86d1a8 86306627 0033 > 8300aecea000 > (XEN)0001 7f8f84a35000 0001 > 880056313bd8 > (XEN)880061468d48 82d080220019 0091c9d82304 > 0003 > (XEN) Xen call trace: > (XEN)[] __mem_sharing_unshare_page+0x1ab/0xb10 > (XEN)[] __get_gfn_type_access+0xd3/0x200 > (XEN)[] get_page_from_gfn_p2m+0xfb/0x2a0 > (XEN)[] get_page+0x2d/0x100 > (XEN)[] do_mmu_update+0x1080/0x1bb0 > (XEN)[] do_mmuext_op+0x803/0x1
Re: [Xen-devel] Xen crash with mem-sharing and cloning
On Mon, Mar 23, 2015 at 11:25 AM, Tamas K Lengyel wrote: > On Mon, Mar 23, 2015 at 6:59 PM, Andres Lagar Cavilla < > and...@lagarcavilla.org> wrote: > >> On Mon, Mar 23, 2015 at 9:10 AM, Tamas K Lengyel > > wrote: >> >>> Hello everyone, >>> I'm trying to chase down a bug that reproducibly crashes Xen (tested >>> with 4.4.1). The problem is somewhere within the mem-sharing subsystem and >>> how that interacts with domains that are being actively saved. In my setup >>> I use the xl toolstack to rapidly create clones of HVM domains by piping >>> "xl save -c" into xl restore with a modified domain config which updates >>> the name/disk/vif. However, during such an operation Xen crashes with the >>> following log if there are already active clones. >>> >>> IMHO there should be no conflict between saving the domain and >>> memsharing, as long as the domain is actually just being checkpointed "-c" >>> - it's memory should remain as is. This is however clearly not the case. >>> Any ideas? >>> >> >> Tamas, I'm not clear on the use of memsharing in this workflow. As >> described, you pipe save into restore, but the internal magic is lost on >> me. Are you fanning out to multiple restores? That would seem to be the >> case, given the need to update name/disk/vif. >> >> Anyway, I'm inferring. Instead, could you elaborate? >> >> Thanks >> Andre >> > > Hi Andre, > thanks for getting back on this issue. The script I'm using is at > https://github.com/tklengyel/drakvuf/blob/master/tools/clone.pl. The > script simply creates a FIFO pipe (mkfifo) and saves the domain into that > pipe which is immediately read by xl restore with the updated configuration > file. This mainly just to eliminate having to read the memory dump from > disk. That part of the system works as expected and multiple save/restores > running at the same time don't cause any side-effects. Once the domain has > thus been cloned, I run memshare on every page which also works as > expected. This problem only occurs when the cloning procedure runs when a > page unshare operation kicks in on a already active clone (as you see in > the log). > Sorry Tamas, I'm a bit slow here, I looked at your script -- looks allright, no mention of memsharing in there. Re-reading ... memsharing? memshare? Is this memshrtool in tools/testing? How are you running it? Certainly no xen crash should happen with user-space input. I'm just trying to understand what you're doing. The unshare code is not, uhmm, brief, so a NULL deref could happen in half a dozen places at first glance. Thanks Andres > > Tamas > ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] Xen crash with mem-sharing and cloning
On Tue, Mar 24, 2015 at 1:48 AM, Tamas K Lengyel wrote: > > > On Tue, Mar 24, 2015 at 4:54 AM, Andres Lagar Cavilla < > and...@lagarcavilla.org> wrote: > >> >> >> On Mon, Mar 23, 2015 at 11:25 AM, Tamas K Lengyel < >> tkleng...@sec.in.tum.de> wrote: >> >>> On Mon, Mar 23, 2015 at 6:59 PM, Andres Lagar Cavilla < >>> and...@lagarcavilla.org> wrote: >>> >>>> On Mon, Mar 23, 2015 at 9:10 AM, Tamas K Lengyel < >>>> tkleng...@sec.in.tum.de> wrote: >>>> >>>>> Hello everyone, >>>>> I'm trying to chase down a bug that reproducibly crashes Xen (tested >>>>> with 4.4.1). The problem is somewhere within the mem-sharing subsystem and >>>>> how that interacts with domains that are being actively saved. In my setup >>>>> I use the xl toolstack to rapidly create clones of HVM domains by piping >>>>> "xl save -c" into xl restore with a modified domain config which updates >>>>> the name/disk/vif. However, during such an operation Xen crashes with the >>>>> following log if there are already active clones. >>>>> >>>>> IMHO there should be no conflict between saving the domain and >>>>> memsharing, as long as the domain is actually just being checkpointed "-c" >>>>> - it's memory should remain as is. This is however clearly not the case. >>>>> Any ideas? >>>>> >>>> >>>> Tamas, I'm not clear on the use of memsharing in this workflow. As >>>> described, you pipe save into restore, but the internal magic is lost on >>>> me. Are you fanning out to multiple restores? That would seem to be the >>>> case, given the need to update name/disk/vif. >>>> >>>> Anyway, I'm inferring. Instead, could you elaborate? >>>> >>>> Thanks >>>> Andre >>>> >>> >>> Hi Andre, >>> thanks for getting back on this issue. The script I'm using is at >>> https://github.com/tklengyel/drakvuf/blob/master/tools/clone.pl. The >>> script simply creates a FIFO pipe (mkfifo) and saves the domain into that >>> pipe which is immediately read by xl restore with the updated configuration >>> file. This mainly just to eliminate having to read the memory dump from >>> disk. That part of the system works as expected and multiple save/restores >>> running at the same time don't cause any side-effects. Once the domain has >>> thus been cloned, I run memshare on every page which also works as >>> expected. This problem only occurs when the cloning procedure runs when a >>> page unshare operation kicks in on a already active clone (as you see in >>> the log). >>> >> >> Sorry Tamas, I'm a bit slow here, I looked at your script -- looks >> allright, no mention of memsharing in there. >> >> Re-reading ... memsharing? memshare? Is this memshrtool in tools/testing? >> How are you running it? >> > > > Hi Andre, > the memsharing happens here > https://github.com/tklengyel/drakvuf/blob/master/src/main.c#L144 after > the clone script finished. This is effectively the same approach as in > tools/testing, just automatically looping from 0 to max_gpfn. Afterwards > all unsharing happens automatically either induced by the guest itself, or > when I map pages into the my app with xc_map_foreign_range PROT_WRITE. > Thanks. Couple of observations on your script 1. sharing all gfns from zero to max is inefficient. There are non trivial holes in the physmap space that you want to jump over. (Holes are not the cause of the crash) 2. xc_memshr_add_to_physmap was created exactly for this case. Rather than deduplicating two pages into one, it grafts a sharing-nominated page directly onto an otherwise empty p2m entry. Apart from the obvious overhead reduction benefit, it does not require you to have 2x memory capacity in order to clone a VM. > > >> >> Certainly no xen crash should happen with user-space input. I'm just >> trying to understand what you're doing. The unshare code is not, uhmm, >> brief, so a NULL deref could happen in half a dozen places at first glance. >> > > Well let me know what I could do help tracing it down. I don't think > (potentially buggy) userspace tools should crash Xen either =) > >From the crash a writable foreign map (qemu -- assuming you run your memshare tool strictly after xl restore has finished) is triggering the unshare NULL deref. My main suspicion is the rmap becoming racy. I would liberally sprinkle printks, retry, see how far printks say you got. Andres > > Tamas > > >> >> Thanks >> Andres >> > > > ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
