> > But the coalescing you describe would also apply to those implicit > > timers (created from select(timeout = ..) within the Linux kernel)? > > It applies to all kernel timers not created by realtime processes. >
Alright. I see. > > > This is all fine. But how do I _explicitly_ limit the rate at which > > select() is called to say 1000Hz (at the expense of timer precision)? > > > I don't want to let the box hit it's syscall rate limit. Because the > > box will spend a fair amount of resources for context switching all > > the time with to real gain. > > From a reading of http://lwn.net/Articles/296578/, at time of writing the > default select() implementation coalesces sub-second timeouts to 50us > boundaries, and this can be adjusted via prctl(PR_SET_TIMERSLACK) > (http://linux.die.net/man/2/prctl) on a per-process basis. Even more interesting! This might be what I'm looking for .. need to read through. > > That article is from 2008, and though the relevant kernel code seems to match > the article content, a huge amount of power-efficiency related changes went > into the kernel since that time. My assumption is nowadays the kernel rounds > more aggressively than the default of 50us documented by that article. Ah, ok. 50us corresponds to 20k syscalls/sec .. which seems well below the syscall rate limit on modern boxes. > > Short answer is yes, you can set the max hz of select(), but in all > likelihood you > won't have to. As always, benchmarking and profiling real code might reveal > this to be a non-issue. Thanks a lot for those hints! I will read into this material. /Tobias > > > David > > > > > Thanks for your hints and patience, > > Tobias > > > > > > > > > > > On Sun, Aug 10, 2014 at 05:16:51AM -0700, Tobias Oberstein wrote: > > > > Hi, > > > > > > > > I have a question regarding scalability of timers in Twisted. > > > > > > > > Say I have a massive number of periodic timers (lets say each with > > > > period 1s, > > > but all slightly time shifted to each other). > > > > > > > > As far as I understand, timers are implemented ultimately by > > > > setting the > > > timeout parameter when calling into OS select/poll/epoll/kqueue. > > > > > > > > If this is true, then the number of timers scales linearly with > > > > the number of > > > syscalls. This can get limiting (the total number of syscalls a > > > Linux box can sustain is a couple of 100k's per second). As more and > > > more timers are setup, the timeout essentially will approach 0. On > > > the upside, timers will fire precisely. > > > > > > > > However, say I am fine with a precision of 1ms. > > > > > > > > Is there a way that limits the syscall rate to 1000/s (given no FD > > > > activity > > > happens) _independently_ of the number of timers setup? > > > > > > > > Timers that fall into a certain ms slice would all fire roughly at > > > > the same time > > > (still ordered). > > > > > > > > Is that possible? > > > > > > > > Thanks, > > > > Tobias > > > > > > > > _______________________________________________ > > > > Twisted-Python mailing list > > > > Twisted-Python@twistedmatrix.com > > > > http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python > > > > > > _______________________________________________ > > > Twisted-Python mailing list > > > Twisted-Python@twistedmatrix.com > > > http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python > > > > _______________________________________________ > > Twisted-Python mailing list > > Twisted-Python@twistedmatrix.com > > http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python > > _______________________________________________ > Twisted-Python mailing list > Twisted-Python@twistedmatrix.com > http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python _______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
