On Wed, Jul 01 2026 at 16:22, Frederic Weisbecker wrote:
> Le Thu, Jun 25, 2026 at 01:27:54AM -0400, Waiman Long a écrit :
>> That will require some adjustments to the nohz_full related hotplug
>> functions. I have some ideas of what needs to be done. However, I haven't
>> looked into RCU yet. I know RCU support changing the nocb mask for fully
>> offline CPUs, I will need to find out if it possible to do that for
>> partially offline CPUs.
>
> No because callbacks can still be enqueued at this stage. But we could
> manage to make it work with CPUHP_AP_IDLE_DEAD.
Well, if you go down to CPUHP_AP_IDLE_DEAD then that's not any different
from going down all the way because the latency spike of stomp_machine()
for bringing it down is the same.
You are right that with the current code this is not possible, but it
should be possible to avoid that alltogether.
The only critical path is when a CPU switches to offload mode. Switching
to 'yes queue callbacks here' mode is not really interesting.
Let's look how RCU hot-unplug works:
1) CPU is marked !active
2) rcutree_offline_cpu() removes the CPU from the fully functional CPU
mask
3) stomp_machine()
4) rcutree_cpu_dying() just traces that the CPU is about to vanish
5) Wait for the CPU to report DEAD
6) rcutree_migrate_callbacks() mops up the leftover callbacks on the
dead CPU
So if the whole machinery changes to:
1) CPU is marked !active
2) rcutree_offline_cpu() removes the CPU from the fully functional CPU
mask _AND_ marks the CPU as "lightweight offloaded", which means:
- no new callbacks can be queued on it anymore neither from the
CPU itself nor from truly offloaded CPUs
- the CPU is still processing already queued callbacks and
participates in the GP magic
3) Before CPUHP_AP_SCHED_WAIT_EMPTY add a new CPUHP_AP_RCU_SYNC state,
which does:
- a full RCU synchronization to end all outstanding read side
critical sections
- drain the now ready callbacks on this CPU
4) Proceed to CPUHP_TEARDOWN_CPU, where the operation stops
5) Do the magic cpuset changes for the CPU
6) Bring CPU back up
At #4 the half unplugged CPU is not in NOHZ full mode and the tick keeps
running so all GP processing work as before except that the CPU itself
is not handling any callbacks because all queued ones are drained and no
new ones can be queued. When it comes back up it turns into a fully
offloaded one.
There are obviously a gazillion of details and cornercases to handle,
but I don't see why this can't be made work in principle.
Thanks,
tglx
