Hey Julien,
Would you mind sharing the latest version of your code?
>
>
Of course not. This is the current version:
asmlinkage void leave_hypervisor_tail(void)
> {
> + /*This methode will be called after the 'guest_entry' macro in
> /arch/arm64/entry.S set guest registers
> + Check single_step_enabled flag in domain struct here and set needed
> registers
> + */
> +
> + struct vcpu *v = current;
> +
> + if ( unlikely(v->domain->arch.monitor.singlestep_enabled ) )
> + {
> +
> +
> +
> + if(!(guest_cpu_user_regs()->cpsr & 0b1000))
> + {
> + WRITE_SYSREG(READ_SYSREG(MDSCR_EL1) | 0x1, MDSCR_EL1);
> + WRITE_SYSREG(READ_SYSREG(MDCR_EL2) | HDCR_TDE, MDCR_EL2);
> + guest_cpu_user_regs()->cpsr = guest_cpu_user_regs()->cpsr |
> 0x200000;
> + WRITE_SYSREG( READ_SYSREG(DAIF) & ~0x200, DAIF);
> + isb();
> + v->arch.single_step = 1;
> +
> + }
> +
> +
> + }else
> + {
> + //single_step Domain flag not set
> + if( v->arch.single_step )
> + {
> + gprintk(XENLOG_ERR, "Domain flag not set, but vcpu flag is
> set\n");
> + WRITE_SYSREG(READ_SYSREG(MDSCR_EL1) & ~0x1, MDSCR_EL1);
> + guest_cpu_user_regs()->cpsr = guest_cpu_user_regs()->cpsr &
> ~0x200000;
> + //WRITE_SYSREG(READ_SYSREG(SPSR_EL2) | 0x200000, SPSR_EL2 );
> + WRITE_SYSREG( READ_SYSREG(DAIF) & ~0x200, DAIF);
> + v->arch.single_step = 0;
> + }
> +
> + }
> +
> while (1)
> {
> local_irq_disable();
>
This code does still the same. Check if domain flag is set, if so check if
SS bit has to be set. If Domain flag is not set, clear SS related bits.
Did you look where the PCs point to? Is it your kernel module?
>
> Yes. I compared it with my Linux Image (using objdump) and found that
these instructions are within a spinlock (the function is called
_raw_spin_lock to be exact). My module is always loaded around the address
0xffff0000008e0000. Also I could see that its not the case that the
addresses are printed twice but the VM just keeps within the spinlock
(which created problems with printing).
By tracing every single step I could determine the function sequence that
is called. It starts out with an <el1_irq> and stops with an
<hrtimer_interrupt> that keeps getting locked in <_raw_spin_lock>.
In order to solve this, I compared my solution with the KVM one, where I
saw (at least for my understanding) that they disable Interrupts for the
VM. In the KVM file: /kvm/virt/arm/arm.c the function
"kvm_arch_vcpu_ioctl_run" handles the running of the VM. The
"kvm_arm_setup_debug" function does the same steps as I do in order to
enable software step exceptions.
So I can't see any difference there.
I also modified xen-access so that the singlestepp will be startet with an
SMC from the Guest. Additionally i wrote a second test kernel module which
only executes an SMC and than will be stopped. I can trace the two SMCs and
than the course to the spinlock (I put the trace below). While in the
spinlock the VM won't response to anything. But after disabling singlestep
it starts to work again.
Enabled singlestep directly (not with an SMC) results in the VM to be
locked in the spinlock immediately.
There is also a problem with using printk within my module for the same
reason. It will always end in the spinlock.
One reason I could imaging is that because I'm singlestepping everything,
including timer interrupts, there will be problems with the scheduling of
the VM. This results in, not meeting the conditions to exit the spinlock.
I hope i made the situation as clear as possible.
Thank you for your help
Florian
This is the function trace obtained by singlestepping every instruction
until the system reached the spinlock below. The two SMCs from my Module
are needed for my xen-access implementation to ensure that the SS starts
withing my module
root@avocet:~# ./xen-access -m 1 singlestep
> Singlestep request found
> xenaccess init
> max_gpfn = 60000
> Run singlestep with commands? (y/n):y
> Starting loop
> Privileged call: pc=ffff0000008e0000 (vcpu 0)
> Privileged call: pc=ffff0000008e0004 (vcpu 0)
>
> Singlestep: PC=ffff000008081a80 b ffff000008082700 <el1_irq>
> Singlestep: PC=ffff000008082700 <el1_irq>
> [...]
> Singlestep: PC=ffff0000080814e8 <gic_handle_irq>:
> [...]
> Singlestep: PC=ffff000008100f60 <__handle_domain_irq>:
> [...]
> Singlestep: PC=ffff0000080c1708 <irq_enter>:
> [...]
> Singlestep: PC=ffff00000810fbf8 <rcu_irq_enter>:
> [...]
> Singlestep: PC=ffff0000080c1714 return to <irq_enter>
> [...]
> Singlestep: PC=ffff000008100f98 return to <__handle_domain_irq>:
> [...]
> Singlestep: PC=ffff000008108238 <irq_find_mapping>:
> [...]
> Singlestep: PC=ffff000008100ff4 return to <__handle_domain_irq>:
> [...]
> Singlestep: PC=ffff000008100928 <generic_handle_irq>:
> [...]
> Singlestep: PC=ffff00000837d0f8 <radix_tree_lookup>:
> [...]
> Singlestep: PC=ffff00000837cfd8 return to <generic_handle_irq>:
> [...]
> Singlestep: PC=ffff00000837d000 return to <radix_tree_lookup>:
> [...]
> Singlestep: PC=ffff000008100940 return to <generic_handle_irq>:
> [...]
> Singlestep: PC=ffff000008105b18 handle_percpu_devid_irq
> [...]
> Singlestep: PC=ffff0000087788c8 <arch_timer_handler_virt>:
> /linux/driver/clocksource/arm_arch_timer.c
> [...]
> Singlestep: PC=ffff000008115108 <hrtimer_interrupt>:
> [...]
> Singlestep: PC=ffff0000088cbd50 <_raw_spin_lock>:
> Singlestep: PC=ffff0000088cbd54
> Singlestep: PC=ffff0000088cbd58
> Singlestep: PC=ffff0000088cbd5c
> Singlestep: PC=ffff0000088cbd60
>
> Singlestep: PC=ffff0000088cbd64
> Singlestep: PC=ffff0000088cbd68 This code block gets repeated until
> SS disabled
> Singlestep: PC=ffff0000088cbd6c
> Singlestep: PC=ffff0000088cbd70
>
> Singlestep: PC=ffff0000088cbd64
> Singlestep: PC=ffff0000088cbd68
> Singlestep: PC=ffff0000088cbd6c
> Singlestep: PC=ffff0000088cbd70
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
