On 2012/4/5 9:54, Sushanth Rai wrote:
I have a multithreaded user space program that basically runs at realtime
priority. Synchronization between threads are done using spinlock. When running
this program on a SMP system under heavy memory pressure I see that thread
holding the spinlock is starved out of cpu. The cpus are effectively consumed
by other threads that are spinning for lock to become available.
After instrumenting the kernel a little bit what I found was that under memory
pressure, when the user thread holding the spinlock traps into the kernel due
to page fault, that thread sleeps until the free pages are available. The
thread sleeps PUSER priority (within vm_waitpfault()). When it is ready to run,
it is queued at PUSER priority even thought it's base priority is realtime. The
other siblings threads that are spinning at realtime priority to acquire the
spinlock starves the owner of spinlock.
I was wondering if the sleep in vm_waitpfault() should be a MAX(td_user_pri,
PUSER) instead of just PUSER. I'm running on 7.2 and it looks like this logic
is the same in the trunk.
Thanks,
Sushanth
I think 7.2 still has libkse which supports static priority scheduling,
if performance is not important
but correctness, you may try libkse with process-scope threads, and use
priority-inherit mutex to
do locking.
Kernel is known to be vulnerable to support user realtime threads. I
think not every-locking primitive
can support priority propagation, this is an issue.
In userland, internal library mutexes are not priority-inherit, so
starvation may happen too. If you
know what you are doing, don't call such functions which uses internal
mutexes, but this is rather
difficult.
Regards,
David Xu
_______________________________________________
[email protected] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[email protected]"
