On Dec 23 14:55:30,c...@schulte.it wrote:
things. I am searching for some kind of system API allowing an
application to reserve a certain amount of CPUs exclusively - not shared
with any other application - maybe not even the OS.
Let's back off before blaming the OS
or declaring CPU affinity to be the biggest problem.
IMnsHO, that's rarely the limiting issue.
I apologize in advance if you have already done this analysis.
Warning: The solution may be refactoring an existing program suite.
What results do you want?
What tasks are necessary to do that?
i.e. math, database lookups, text/data processing
not any specific program
Is your problem massively SIMD (video, weather)?
Don't try to use general purpose hardware.
Skip the rest of this, buy video cards and use coin mining software.
Have you profiled your current implementation?
What limits accomplishing the tasks to your satisfaction?
time - latency, quantity of work done, fairness,
bandwidth of some resource
space = resources - CPU speed, count, memory, mass storage, network
How independent and sequential are the subtasks?
If you are trying to parallelize your operations
there is a point of diminishing returns.
That point is usually surprisingly low.
Scheduling has a cost. It often is very hard to do in parallel.
Intertask synchronization is costly and necessary in unexpected places.
CPUs are often not the limit.
A surprising amount of other hardware has limits, often far lower than
you'd like.
In your current implementation what are you asking the OS to do?
If the OS scheduling is a problem, refactoring the application
very frequently solves that.
And, of course, full analysis of an existing program suite
is both costly and painful.
Side note: matching compute resources to an input data stream
requires either resources capable of the largest data stream leaving
some idle most of the time or pushing back on the input stream
potentially causing an indefinite processing delay.
hth
Geoff Steckel
