On Mon, Nov 05, 2018 at 01:19:50PM -0800, Randy Dunlap wrote: > On 11/5/18 8:55 AM, Daniel Jordan wrote: > > Hi, > > > +Resource Limits > > +=============== > > + > > +ktask has resource limits on the number of work items it sends to > > workqueue. > > to a > workqueue. > or: to > workqueues.
Ok, I'll do "to workqueues" since ktask uses two internally (NUMA-aware and non-NUMA-aware). > > > +In ktask, a workqueue item is a thread that runs chunks of the task until > > the > > +task is finished. > > + > > +These limits support the different ways ktask uses workqueues: > > + - ktask_run to run threads on the calling thread's node. > > + - ktask_run_numa to run threads on the node(s) specified. > > + - ktask_run_numa with nid=NUMA_NO_NODE to run threads on any node in the > > + system. > > + > > +To support these different ways of queueing work while maintaining an > > efficient > > +concurrency level, we need both system-wide and per-node limits on the > > number > > I would prefer to refer to ktask as ktask instead of "we", so > s/we need/ktask needs/ Good idea, I'll change it. > > +of threads. Without per-node limits, a node might become oversubscribed > > +despite ktask staying within the system-wide limit, and without a > > system-wide > > +limit, we can't properly account for work that can run on any node. > > s/we/ktask/ Ok. > > + > > +The system-wide limit is based on the total number of CPUs, and the > > per-node > > +limit on the CPU count for each node. A per-node work item counts against > > the > > +system-wide limit. Workqueue's max_active can't accommodate both types of > > +limit, no matter how many workqueues are used, so ktask implements its own. > > + > > +If a per-node limit is reached, the work item is allowed to run anywhere > > on the > > +machine to avoid overwhelming the node. If the global limit is also > > reached, > > +ktask won't queue additional work items until we fall below the limit > > again. > > s/we fall/ktask falls/ > or s/we fall/it falls/ 'ktask.' Will change. > > + > > +These limits apply only to workqueue items--that is, helper threads beyond > > the > > +one starting the task. That way, one thread per task is always allowed to > > run. > > > thanks. Appreciate the feedback!
