Re: [Xen-devel] [VMI] Possible race-condition in altp2m APIs

2019-06-11 Thread Mathieu Tarral
Hi, > Yes I missed that PatchGuard would eventually check those shadow pages anyway. > I was already happy to see that my breakpoints were working, and I proceeded > to the tests > hoping to have a quick reproduction of the bug. > > I implemented a basic mem_access event on the restricting to --X

Re: [Xen-devel] [VMI] Possible race-condition in altp2m APIs

2019-06-11 Thread Mathieu Tarral
Hi Andrew, Tamas, Le mercredi, mai 29, 2019 2:49 AM, Andrew Cooper a écrit : > After a series of tests on 1 or 4 VCPUs, my domain end up in 2 possible > states: - frozen: the mouse doesn't move: so I would guess the VCPU are > blocked. I'm calling the xc_(un)pause_domain APIs multiple times w

Re: [Xen-devel] [VMI] Possible race-condition in altp2m APIs

2019-05-29 Thread Tamas K Lengyel
> After a series of tests on 1 or 4 VCPUs, my domain end up in 2 possible > states: > - frozen: the mouse doesn't move: so I would guess the VCPU are blocked. You probably have events pending on the ring that you didn't process. After you pause the domain and remove the registered evenst, breakpo

Re: [Xen-devel] [VMI] Possible race-condition in altp2m APIs

2019-05-29 Thread Tamas K Lengyel
> > There are three views being used: the default (hostp2m); the > > execute-view which is active by default and has the remapped > > shadow-copies of the pages with breakpoints injected at the very end > > of the guests' physmap; and the read-only view that is only used when > > someone is trying

Re: [Xen-devel] [VMI] Possible race-condition in altp2m APIs

2019-05-28 Thread Andrew Cooper
On 29/05/2019 02:34, Tamas K Lengyel wrote: >> And some questions. >> >> 1) I'm guessing the drakvuf_inject_trap(drakvuf, 0x293e6a0, 0) call is >> specific to the exact windows kernel in use? >> >> 2) In vmi_init(), what is the purpose of fmask and zero_page_gfn? You add >> one extra gfn to the

Re: [Xen-devel] [VMI] Possible race-condition in altp2m APIs

2019-05-28 Thread Tamas K Lengyel
> @Tamas, if you could check the traps implementation. I had a quick look and it seems like you forgot to set the mem_access permissions on the pages. You need the remapped gfn's to be marked execute-only in the altp2m_idx view, and their actual gfn completely inaccessible in altp2m_idx. You need

Re: [Xen-devel] [VMI] Possible race-condition in altp2m APIs

2019-05-28 Thread Tamas K Lengyel
> And some questions. > > 1) I'm guessing the drakvuf_inject_trap(drakvuf, 0x293e6a0, 0) call is > specific to the exact windows kernel in use? > > 2) In vmi_init(), what is the purpose of fmask and zero_page_gfn? You add > one extra gfn to the guest, called zero_page, and fill it with 1's from

Re: [Xen-devel] [VMI] Possible race-condition in altp2m APIs

2019-05-28 Thread Andrew Cooper
On 28/05/2019 13:33, Mathieu Tarral wrote: > Hi Andrew, > >>> The bug is still here, so we can exclude a microcode issue. >> Good - that is one further angle excluded.  Always make sure you are >> running with up-to-date microcode, but it looks like we back to >> investigating a logical bug in libv

Re: [Xen-devel] [VMI] Possible race-condition in altp2m APIs

2019-05-28 Thread Mathieu Tarral
Hi Andrew, > > The bug is still here, so we can exclude a microcode issue. > > Good - that is one further angle excluded.  Always make sure you are > running with up-to-date microcode, but it looks like we back to > investigating a logical bug in libvmi or Xen. I reimplemented a small test, with

Re: [Xen-devel] [VMI] Possible race-condition in altp2m APIs

2019-05-13 Thread Razvan Cojocaru
On 5/13/19 7:18 PM, Mathieu Tarral wrote: > Le vendredi, mai 10, 2019 5:21 PM, Andrew Cooper > a écrit : > >> On 10/05/2019 16:17, Mathieu Tarral wrote: >> >>> Le jeudi, mai 9, 2019 6:42 PM, Andrew Cooper andrew.coop...@citrix.com a >>> écrit : >>> Therefore, the conclusion to draw is that

Re: [Xen-devel] [VMI] Possible race-condition in altp2m APIs

2019-05-13 Thread Mathieu Tarral
Le vendredi, mai 10, 2019 5:21 PM, Andrew Cooper a écrit : > On 10/05/2019 16:17, Mathieu Tarral wrote: > > > Le jeudi, mai 9, 2019 6:42 PM, Andrew Cooper andrew.coop...@citrix.com a > > écrit : > > > > > Therefore, the conclusion to draw is that it is a logical bug somewhere. > > The bug is st

Re: [Xen-devel] [VMI] Possible race-condition in altp2m APIs

2019-05-10 Thread Andrew Cooper
On 10/05/2019 16:17, Mathieu Tarral wrote: > Le jeudi, mai 9, 2019 6:42 PM, Andrew Cooper a > écrit : >> Therefore, the conclusion to draw is that it is a logical bug somewhere. >> >> First of all - ensure you are using up-to-date microcode.  The number of >> errata which have been discovered by

Re: [Xen-devel] [VMI] Possible race-condition in altp2m APIs

2019-05-10 Thread Mathieu Tarral
Le jeudi, mai 9, 2019 8:08 PM, Tamas K Lengyel a écrit : > > > > I already suggested to Mathieu to try to reproduce the issue using the > > > > xen-access test tool that's in the Xen tree to cut out all that > > > > complexity. > > > > xen-access is ok, but I've never encountered a situation where

Re: [Xen-devel] [VMI] Possible race-condition in altp2m APIs

2019-05-10 Thread Mathieu Tarral
Le jeudi, mai 9, 2019 6:42 PM, Andrew Cooper a écrit : > Therefore, the conclusion to draw is that it is a logical bug somewhere. > > First of all - ensure you are using up-to-date microcode.  The number of > errata which have been discovered by people associated with the Xen > community is large

Re: [Xen-devel] [VMI] Possible race-condition in altp2m APIs

2019-05-10 Thread Andrew Cooper
On 10/05/2019 11:52, Petre Ovidiu PIRCALABU wrote: > On Thu, 2019-05-09 at 19:00 +0100, Andrew Cooper wrote: >> On 09/05/2019 18:46, Tamas K Lengyel wrote: >> I have some plans to replace it with something far more usable, as >> part >> of tying together some XTF-based VMI testing, but none of that

Re: [Xen-devel] [VMI] Possible race-condition in altp2m APIs

2019-05-10 Thread Petre Ovidiu PIRCALABU
On Thu, 2019-05-09 at 19:00 +0100, Andrew Cooper wrote: > > > On 09/05/2019 18:46, Tamas K Lengyel wrote: > > > > I have some plans to replace it with something far more usable, as > part > of tying together some XTF-based VMI testing, but none of that is > remotely ready yet. Hi Andrew, Did yo

Re: [Xen-devel] [VMI] Possible race-condition in altp2m APIs

2019-05-09 Thread Tamas K Lengyel
On Thu, May 9, 2019 at 12:00 PM Andrew Cooper wrote: > > On 09/05/2019 18:46, Tamas K Lengyel wrote: > > On Thu, May 9, 2019 at 10:43 AM Andrew Cooper > > wrote: > >> On 09/05/2019 17:19, Mathieu Tarral wrote: > >>> Le mardi, mai 7, 2019 2:01 PM, Mathieu Tarral > >>> a écrit : > >>> > > Gi

Re: [Xen-devel] [VMI] Possible race-condition in altp2m APIs

2019-05-09 Thread Andrew Cooper
On 09/05/2019 18:46, Tamas K Lengyel wrote: > On Thu, May 9, 2019 at 10:43 AM Andrew Cooper > wrote: >> On 09/05/2019 17:19, Mathieu Tarral wrote: >>> Le mardi, mai 7, 2019 2:01 PM, Mathieu Tarral >>> a écrit : >>> > Given how many EPT flushing bugs I've already found in this area, I >

Re: [Xen-devel] [VMI] Possible race-condition in altp2m APIs

2019-05-09 Thread Tamas K Lengyel
> I don't feel comfortable digging into Xen's code, especially for something as > complicated as page table and memory management, > increased by the complexity of altp2m. Unfortunately the reality is that you might very well have to become familiar with these areas if you hope to debug some of t

Re: [Xen-devel] [VMI] Possible race-condition in altp2m APIs

2019-05-09 Thread Tamas K Lengyel
On Thu, May 9, 2019 at 10:43 AM Andrew Cooper wrote: > > On 09/05/2019 17:19, Mathieu Tarral wrote: > > Le mardi, mai 7, 2019 2:01 PM, Mathieu Tarral > > a écrit : > > > >>> Given how many EPT flushing bugs I've already found in this area, I > >>> wouldn't be surprised if there are further ones

Re: [Xen-devel] [VMI] Possible race-condition in altp2m APIs

2019-05-09 Thread Andrew Cooper
On 09/05/2019 17:19, Mathieu Tarral wrote: > Le mardi, mai 7, 2019 2:01 PM, Mathieu Tarral > a écrit : > >>> Given how many EPT flushing bugs I've already found in this area, I >>> wouldn't be surprised if there are further ones lurking.  If it is an EPT >>> flushing bug, this delta should make

Re: [Xen-devel] [VMI] Possible race-condition in altp2m APIs

2019-05-09 Thread Mathieu Tarral
Le mardi, mai 7, 2019 2:01 PM, Mathieu Tarral a écrit : > > Given how many EPT flushing bugs I've already found in this area, I > > wouldn't be surprised if there are further ones lurking.  If it is an EPT > > flushing bug, this delta should make it go away, but it will come with a > > hefty

Re: [Xen-devel] [VMI] Possible race-condition in altp2m APIs

2019-05-07 Thread Mathieu Tarral
Le lundi, mai 6, 2019 7:07 PM, Andrew Cooper a écrit : > There is a lot in here. I wanted to gather enough data before making a bug report on such a complicated issue. > As for your BSOD analysis, the first thing to be aware of is that Double > Fault is not necessarily precise, which means y

Re: [Xen-devel] [VMI] Possible race-condition in altp2m APIs

2019-05-06 Thread Tamas K Lengyel
On Mon, May 6, 2019 at 12:30 PM Andrew Cooper wrote: > > On 06/05/2019 18:41, Tamas K Lengyel wrote: > > Hi Andrew, > > thanks for helping brainstorming on this. > > > >> How exactly does DRAKVUF go about injecting silent breakpoints? It > >> obviously has to allocate a new gfn from somewhere to

Re: [Xen-devel] [VMI] Possible race-condition in altp2m APIs

2019-05-06 Thread Andrew Cooper
On 06/05/2019 18:41, Tamas K Lengyel wrote: > Hi Andrew, > thanks for helping brainstorming on this. > >> How exactly does DRAKVUF go about injecting silent breakpoints? It >> obviously has to allocate a new gfn from somewhere to begin with. Do the >> bifurcated frames end up in two different a

Re: [Xen-devel] [VMI] Possible race-condition in altp2m APIs

2019-05-06 Thread Tamas K Lengyel
Hi Andrew, thanks for helping brainstorming on this. > How exactly does DRAKVUF go about injecting silent breakpoints? It obviously > has to allocate a new gfn from somewhere to begin with. Do the bifurcated > frames end up in two different altp2ms, or one in the host p2m and one in an > alte

Re: [Xen-devel] [VMI] Possible race-condition in altp2m APIs

2019-05-06 Thread Andrew Cooper
On 06/05/2019 17:18, Mathieu Tarral wrote: > Hi, > > I would like to submit a strange bug that i'm facing while using DRAKVUF to > monitor applications from the hypervisor. > > I wanted to evaluate DRAKVUF's robustness, so I built a test suite, and began > by executing reg.exe via shellexec injecti