Re: [PATCH V2 7/9] vhost: do not use RCU to synchronize MMU notifier with worker

2019-08-06 Thread Jason Wang
On 2019/8/6 下午8:04, Jason Gunthorpe wrote: On Mon, Aug 05, 2019 at 12:20:45PM +0800, Jason Wang wrote: On 2019/8/2 下午8:46, Jason Gunthorpe wrote: On Fri, Aug 02, 2019 at 05:40:07PM +0800, Jason Wang wrote: This must be a proper barrier, like a spinlock, mutex, or synchronize_rcu. I start wi

Re: [PATCH V2 7/9] vhost: do not use RCU to synchronize MMU notifier with worker

2019-08-06 Thread Jason Gunthorpe
On Tue, Aug 06, 2019 at 09:36:58AM -0400, Michael S. Tsirkin wrote: > On Tue, Aug 06, 2019 at 08:53:17AM -0300, Jason Gunthorpe wrote: > > On Sun, Aug 04, 2019 at 04:07:17AM -0400, Michael S. Tsirkin wrote: > > > > > > Also, why can't this just permanently GUP the pages? In fact, where > > > > > >

Re: [PATCH V2 7/9] vhost: do not use RCU to synchronize MMU notifier with worker

2019-08-06 Thread Michael S. Tsirkin
On Tue, Aug 06, 2019 at 08:53:17AM -0300, Jason Gunthorpe wrote: > On Sun, Aug 04, 2019 at 04:07:17AM -0400, Michael S. Tsirkin wrote: > > > > > Also, why can't this just permanently GUP the pages? In fact, where > > > > > does it put_page them anyhow? Worrying that 7f466 adds a get_user page > > >

Re: [PATCH V2 7/9] vhost: do not use RCU to synchronize MMU notifier with worker

2019-08-06 Thread Jason Gunthorpe
On Mon, Aug 05, 2019 at 12:20:45PM +0800, Jason Wang wrote: > > On 2019/8/2 下午8:46, Jason Gunthorpe wrote: > > On Fri, Aug 02, 2019 at 05:40:07PM +0800, Jason Wang wrote: > > > > This must be a proper barrier, like a spinlock, mutex, or > > > > synchronize_rcu. > > > > > > I start with synchroniz

Re: [PATCH V2 7/9] vhost: do not use RCU to synchronize MMU notifier with worker

2019-08-06 Thread Jason Gunthorpe
On Sun, Aug 04, 2019 at 04:07:17AM -0400, Michael S. Tsirkin wrote: > > > > Also, why can't this just permanently GUP the pages? In fact, where > > > > does it put_page them anyhow? Worrying that 7f466 adds a get_user page > > > > but does not add a put_page?? > > > > You didn't answer this.. Why

Re: [PATCH V2 7/9] vhost: do not use RCU to synchronize MMU notifier with worker

2019-08-05 Thread Jason Wang
On 2019/8/5 下午2:40, Michael S. Tsirkin wrote: On Mon, Aug 05, 2019 at 12:41:45PM +0800, Jason Wang wrote: On 2019/8/5 下午12:36, Jason Wang wrote: On 2019/8/2 下午10:27, Michael S. Tsirkin wrote: On Fri, Aug 02, 2019 at 09:46:13AM -0300, Jason Gunthorpe wrote: On Fri, Aug 02, 2019 at 05:40:07PM

Re: [PATCH V2 7/9] vhost: do not use RCU to synchronize MMU notifier with worker

2019-08-05 Thread Jason Wang
On 2019/8/5 下午2:30, Michael S. Tsirkin wrote: On Mon, Aug 05, 2019 at 12:36:40PM +0800, Jason Wang wrote: On 2019/8/2 下午10:27, Michael S. Tsirkin wrote: On Fri, Aug 02, 2019 at 09:46:13AM -0300, Jason Gunthorpe wrote: On Fri, Aug 02, 2019 at 05:40:07PM +0800, Jason Wang wrote: This must be

Re: [PATCH V2 7/9] vhost: do not use RCU to synchronize MMU notifier with worker

2019-08-05 Thread Jason Wang
On 2019/8/5 下午2:28, Michael S. Tsirkin wrote: On Mon, Aug 05, 2019 at 12:33:45PM +0800, Jason Wang wrote: On 2019/8/2 下午10:03, Michael S. Tsirkin wrote: On Fri, Aug 02, 2019 at 05:40:07PM +0800, Jason Wang wrote: Btw, I come up another idea, that is to disable preemption when vhost thread ne

Re: [PATCH V2 7/9] vhost: do not use RCU to synchronize MMU notifier with worker

2019-08-05 Thread Jason Wang
On 2019/8/4 上午5:54, Michael S. Tsirkin wrote: On Thu, Aug 01, 2019 at 04:06:13AM -0400, Jason Wang wrote: On 2019/8/1 上午2:29, Michael S. Tsirkin wrote: On Wed, Jul 31, 2019 at 04:46:53AM -0400, Jason Wang wrote: We used to use RCU to synchronize MMU notifier with worker. This leads calling s

Re: [PATCH V2 7/9] vhost: do not use RCU to synchronize MMU notifier with worker

2019-08-04 Thread Michael S. Tsirkin
On Mon, Aug 05, 2019 at 12:41:45PM +0800, Jason Wang wrote: > > On 2019/8/5 下午12:36, Jason Wang wrote: > > > > On 2019/8/2 下午10:27, Michael S. Tsirkin wrote: > > > On Fri, Aug 02, 2019 at 09:46:13AM -0300, Jason Gunthorpe wrote: > > > > On Fri, Aug 02, 2019 at 05:40:07PM +0800, Jason Wang wrote:

Re: [PATCH V2 7/9] vhost: do not use RCU to synchronize MMU notifier with worker

2019-08-04 Thread Michael S. Tsirkin
On Mon, Aug 05, 2019 at 12:36:40PM +0800, Jason Wang wrote: > > On 2019/8/2 下午10:27, Michael S. Tsirkin wrote: > > On Fri, Aug 02, 2019 at 09:46:13AM -0300, Jason Gunthorpe wrote: > > > On Fri, Aug 02, 2019 at 05:40:07PM +0800, Jason Wang wrote: > > > > > This must be a proper barrier, like a spin

Re: [PATCH V2 7/9] vhost: do not use RCU to synchronize MMU notifier with worker

2019-08-04 Thread Michael S. Tsirkin
On Mon, Aug 05, 2019 at 12:33:45PM +0800, Jason Wang wrote: > > On 2019/8/2 下午10:03, Michael S. Tsirkin wrote: > > On Fri, Aug 02, 2019 at 05:40:07PM +0800, Jason Wang wrote: > > > Btw, I come up another idea, that is to disable preemption when vhost > > > thread > > > need to access the memory.

Re: [PATCH V2 7/9] vhost: do not use RCU to synchronize MMU notifier with worker

2019-08-04 Thread Jason Wang
On 2019/8/5 下午12:36, Jason Wang wrote: On 2019/8/2 下午10:27, Michael S. Tsirkin wrote: On Fri, Aug 02, 2019 at 09:46:13AM -0300, Jason Gunthorpe wrote: On Fri, Aug 02, 2019 at 05:40:07PM +0800, Jason Wang wrote: This must be a proper barrier, like a spinlock, mutex, or synchronize_rcu. I s

Re: [PATCH V2 7/9] vhost: do not use RCU to synchronize MMU notifier with worker

2019-08-04 Thread Jason Wang
On 2019/8/4 下午4:07, Michael S. Tsirkin wrote: On Sat, Aug 03, 2019 at 09:14:00PM -0300, Jason Gunthorpe wrote: On Sat, Aug 03, 2019 at 05:36:13PM -0400, Michael S. Tsirkin wrote: On Fri, Aug 02, 2019 at 02:24:18PM -0300, Jason Gunthorpe wrote: On Fri, Aug 02, 2019 at 10:27:21AM -0400, Michae

Re: [PATCH V2 7/9] vhost: do not use RCU to synchronize MMU notifier with worker

2019-08-04 Thread Jason Wang
On 2019/8/2 下午10:27, Michael S. Tsirkin wrote: On Fri, Aug 02, 2019 at 09:46:13AM -0300, Jason Gunthorpe wrote: On Fri, Aug 02, 2019 at 05:40:07PM +0800, Jason Wang wrote: This must be a proper barrier, like a spinlock, mutex, or synchronize_rcu. I start with synchronize_rcu() but both you

Re: [PATCH V2 7/9] vhost: do not use RCU to synchronize MMU notifier with worker

2019-08-04 Thread Jason Wang
On 2019/8/2 下午10:03, Michael S. Tsirkin wrote: On Fri, Aug 02, 2019 at 05:40:07PM +0800, Jason Wang wrote: Btw, I come up another idea, that is to disable preemption when vhost thread need to access the memory. Then register preempt notifier and if vhost thread is preempted, we're sure no one

Re: [PATCH V2 7/9] vhost: do not use RCU to synchronize MMU notifier with worker

2019-08-04 Thread Jason Wang
On 2019/8/2 下午8:46, Jason Gunthorpe wrote: On Fri, Aug 02, 2019 at 05:40:07PM +0800, Jason Wang wrote: This must be a proper barrier, like a spinlock, mutex, or synchronize_rcu. I start with synchronize_rcu() but both you and Michael raise some concern. I've also idly wondered if calling sy

Re: [PATCH V2 7/9] vhost: do not use RCU to synchronize MMU notifier with worker

2019-08-04 Thread Michael S. Tsirkin
On Sat, Aug 03, 2019 at 09:14:00PM -0300, Jason Gunthorpe wrote: > On Sat, Aug 03, 2019 at 05:36:13PM -0400, Michael S. Tsirkin wrote: > > On Fri, Aug 02, 2019 at 02:24:18PM -0300, Jason Gunthorpe wrote: > > > On Fri, Aug 02, 2019 at 10:27:21AM -0400, Michael S. Tsirkin wrote: > > > > On Fri, Aug 0

Re: [PATCH V2 7/9] vhost: do not use RCU to synchronize MMU notifier with worker

2019-08-03 Thread Jason Gunthorpe
On Sat, Aug 03, 2019 at 05:36:13PM -0400, Michael S. Tsirkin wrote: > On Fri, Aug 02, 2019 at 02:24:18PM -0300, Jason Gunthorpe wrote: > > On Fri, Aug 02, 2019 at 10:27:21AM -0400, Michael S. Tsirkin wrote: > > > On Fri, Aug 02, 2019 at 09:46:13AM -0300, Jason Gunthorpe wrote: > > > > On Fri, Aug 0

Re: [PATCH V2 7/9] vhost: do not use RCU to synchronize MMU notifier with worker

2019-08-03 Thread Michael S. Tsirkin
On Thu, Aug 01, 2019 at 04:06:13AM -0400, Jason Wang wrote: > On 2019/8/1 上午2:29, Michael S. Tsirkin wrote: > > On Wed, Jul 31, 2019 at 04:46:53AM -0400, Jason Wang wrote: > >> We used to use RCU to synchronize MMU notifier with worker. This leads > >> calling synchronize_rcu() in invalidate_range_

Re: [PATCH V2 7/9] vhost: do not use RCU to synchronize MMU notifier with worker

2019-08-03 Thread Michael S. Tsirkin
On Fri, Aug 02, 2019 at 02:24:18PM -0300, Jason Gunthorpe wrote: > On Fri, Aug 02, 2019 at 10:27:21AM -0400, Michael S. Tsirkin wrote: > > On Fri, Aug 02, 2019 at 09:46:13AM -0300, Jason Gunthorpe wrote: > > > On Fri, Aug 02, 2019 at 05:40:07PM +0800, Jason Wang wrote: > > > > > This must be a prop

Re: [PATCH V2 7/9] vhost: do not use RCU to synchronize MMU notifier with worker

2019-08-02 Thread Jason Gunthorpe
On Fri, Aug 02, 2019 at 10:27:21AM -0400, Michael S. Tsirkin wrote: > On Fri, Aug 02, 2019 at 09:46:13AM -0300, Jason Gunthorpe wrote: > > On Fri, Aug 02, 2019 at 05:40:07PM +0800, Jason Wang wrote: > > > > This must be a proper barrier, like a spinlock, mutex, or > > > > synchronize_rcu. > > > >

Re: [PATCH V2 7/9] vhost: do not use RCU to synchronize MMU notifier with worker

2019-08-02 Thread Michael S. Tsirkin
On Fri, Aug 02, 2019 at 09:46:13AM -0300, Jason Gunthorpe wrote: > On Fri, Aug 02, 2019 at 05:40:07PM +0800, Jason Wang wrote: > > > This must be a proper barrier, like a spinlock, mutex, or > > > synchronize_rcu. > > > > > > I start with synchronize_rcu() but both you and Michael raise some > >

Re: [PATCH V2 7/9] vhost: do not use RCU to synchronize MMU notifier with worker

2019-08-02 Thread Michael S. Tsirkin
On Fri, Aug 02, 2019 at 05:40:07PM +0800, Jason Wang wrote: > Btw, I come up another idea, that is to disable preemption when vhost thread > need to access the memory. Then register preempt notifier and if vhost > thread is preempted, we're sure no one will access the memory and can do the > cleanu

Re: [PATCH V2 7/9] vhost: do not use RCU to synchronize MMU notifier with worker

2019-08-02 Thread Jason Gunthorpe
On Fri, Aug 02, 2019 at 05:40:07PM +0800, Jason Wang wrote: > > This must be a proper barrier, like a spinlock, mutex, or > > synchronize_rcu. > > > I start with synchronize_rcu() but both you and Michael raise some > concern. I've also idly wondered if calling synchronize_rcu() under the variou

Re: [PATCH V2 7/9] vhost: do not use RCU to synchronize MMU notifier with worker

2019-08-02 Thread Jason Wang
On 2019/8/1 下午10:15, Jason Gunthorpe wrote: On Thu, Aug 01, 2019 at 01:02:18PM +0800, Jason Wang wrote: On 2019/8/1 上午3:30, Jason Gunthorpe wrote: On Wed, Jul 31, 2019 at 09:28:20PM +0800, Jason Wang wrote: On 2019/7/31 下午8:39, Jason Gunthorpe wrote: On Wed, Jul 31, 2019 at 04:46:53AM -0400

Re: [PATCH V2 7/9] vhost: do not use RCU to synchronize MMU notifier with worker

2019-08-01 Thread Jason Gunthorpe
On Thu, Aug 01, 2019 at 01:02:18PM +0800, Jason Wang wrote: > > On 2019/8/1 上午3:30, Jason Gunthorpe wrote: > > On Wed, Jul 31, 2019 at 09:28:20PM +0800, Jason Wang wrote: > > > On 2019/7/31 下午8:39, Jason Gunthorpe wrote: > > > > On Wed, Jul 31, 2019 at 04:46:53AM -0400, Jason Wang wrote: > > > > >

Re: [PATCH V2 7/9] vhost: do not use RCU to synchronize MMU notifier with worker

2019-08-01 Thread Jason Wang
On 2019/8/1 上午2:29, Michael S. Tsirkin wrote: > On Wed, Jul 31, 2019 at 04:46:53AM -0400, Jason Wang wrote: >> We used to use RCU to synchronize MMU notifier with worker. This leads >> calling synchronize_rcu() in invalidate_range_start(). But on a busy >> system, there would be many factors that m

Re: [PATCH V2 7/9] vhost: do not use RCU to synchronize MMU notifier with worker

2019-07-31 Thread Jason Wang
On 2019/8/1 上午3:30, Jason Gunthorpe wrote: On Wed, Jul 31, 2019 at 09:28:20PM +0800, Jason Wang wrote: On 2019/7/31 下午8:39, Jason Gunthorpe wrote: On Wed, Jul 31, 2019 at 04:46:53AM -0400, Jason Wang wrote: We used to use RCU to synchronize MMU notifier with worker. This leads calling synchr

Re: [PATCH V2 7/9] vhost: do not use RCU to synchronize MMU notifier with worker

2019-07-31 Thread Jason Gunthorpe
On Wed, Jul 31, 2019 at 09:28:20PM +0800, Jason Wang wrote: > > On 2019/7/31 下午8:39, Jason Gunthorpe wrote: > > On Wed, Jul 31, 2019 at 04:46:53AM -0400, Jason Wang wrote: > > > We used to use RCU to synchronize MMU notifier with worker. This leads > > > calling synchronize_rcu() in invalidate_ran

Re: [PATCH V2 7/9] vhost: do not use RCU to synchronize MMU notifier with worker

2019-07-31 Thread Michael S. Tsirkin
On Wed, Jul 31, 2019 at 04:46:53AM -0400, Jason Wang wrote: > We used to use RCU to synchronize MMU notifier with worker. This leads > calling synchronize_rcu() in invalidate_range_start(). But on a busy > system, there would be many factors that may slow down the > synchronize_rcu() which makes it

Re: [PATCH V2 7/9] vhost: do not use RCU to synchronize MMU notifier with worker

2019-07-31 Thread Jason Wang
On 2019/7/31 下午8:39, Jason Gunthorpe wrote: On Wed, Jul 31, 2019 at 04:46:53AM -0400, Jason Wang wrote: We used to use RCU to synchronize MMU notifier with worker. This leads calling synchronize_rcu() in invalidate_range_start(). But on a busy system, there would be many factors that may slow

Re: [PATCH V2 7/9] vhost: do not use RCU to synchronize MMU notifier with worker

2019-07-31 Thread Jason Gunthorpe
On Wed, Jul 31, 2019 at 04:46:53AM -0400, Jason Wang wrote: > We used to use RCU to synchronize MMU notifier with worker. This leads > calling synchronize_rcu() in invalidate_range_start(). But on a busy > system, there would be many factors that may slow down the > synchronize_rcu() which makes it

Re: [PATCH V2 7/9] vhost: do not use RCU to synchronize MMU notifier with worker

2019-07-31 Thread Jason Wang
On 2019/7/31 下午4:46, Jason Wang wrote: We used to use RCU to synchronize MMU notifier with worker. This leads calling synchronize_rcu() in invalidate_range_start(). But on a busy system, there would be many factors that may slow down the synchronize_rcu() which makes it unsuitable to be called

[PATCH V2 7/9] vhost: do not use RCU to synchronize MMU notifier with worker

2019-07-31 Thread Jason Wang
We used to use RCU to synchronize MMU notifier with worker. This leads calling synchronize_rcu() in invalidate_range_start(). But on a busy system, there would be many factors that may slow down the synchronize_rcu() which makes it unsuitable to be called in MMU notifier. A solution is SRCU but it