Good Catch.
The original code looks too old, use the ksm_mmlist_lock to protect the mm_list
looks will affect the performance,
Should we use the RCU to protect the list and not free the mm until out of the
rcu critical period?
On 2016/5/6 5:57, Andrea Arcangeli wrote:
> Hello Zhou,
>
> Great catch.
>
> On Thu, May 05, 2016 at 08:42:56PM +0800, Zhou Chengming wrote:
>> remove_trailing_rmap_items(slot, ksm_scan.rmap_list);
>> + up_read(&mm->mmap_sem);
>>
>> spin_lock(&ksm_mmlist_lock);
>> ksm_scan.mm_slot = list_entry(slot->mm_list.next,
>> @@ -1666,16 +1667,12 @@ next_mm:
>> */
>> hash_del(&slot->link);
>> list_del(&slot->mm_list);
>> - spin_unlock(&ksm_mmlist_lock);
>>
>> free_mm_slot(slot);
>> clear_bit(MMF_VM_MERGEABLE, &mm->flags);
>> - up_read(&mm->mmap_sem);
>> mmdrop(mm);
>
> I thought the mmap_sem for reading prevented a race of the above
> clear_bit against a concurrent madvise(MADV_MERGEABLE) which takes the
> mmap_sem for writing. After this change can't __ksm_enter run
> concurrently with the clear_bit above introducing a different SMP race
> condition?
>
>> - } else {
>> - spin_unlock(&ksm_mmlist_lock);
>> - up_read(&mm->mmap_sem);
>
> The strict obviously safe fix is just to invert the above two,
> up_read; spin_unlock.
>
> Then I found another instance of this same SMP race condition in
> unmerge_and_remove_all_rmap_items() that you didn't fix.
>
> Actually for the other instance of the bug the implementation above
> that releases the mmap_sem early sounds safe, because it's a
> ksm_text_exit that takes the clear_bit path, not just the fact we
> didn't find a vma with VM_MERGEABLE set and we garbage collect the
> mm_slot, while the "mm" may still alive. In the other case the "mm"
> isn't alive anymore so the race with MADV_MERGEABLE shouldn't be
> possible to materialize.
>
> Could you fix it by just inverting the up_read/spin_unlock order, in
> the place you patched, and add this comment:
>
> } else {
> /*
> * up_read(&mm->mmap_sem) first because after
> * spin_unlock(&ksm_mmlist_lock) run, the "mm" may
> * already have been freed under us by __ksm_exit()
> * because the "mm_slot" is still hashed and
> * ksm_scan.mm_slot doesn't point to it anymore.
> */
> up_read(&mm->mmap_sem);
> spin_unlock(&ksm_mmlist_lock);
> }
>
> And in unmerge_and_remove_all_rmap_items() same thing, except there
> you can apply your up_read() early and you can just drop the "else"
> clause.
>
> .
>
