On Tue, Jun 10, 2025 at 4:53 AM Alexander Graf <g...@amazon.com> wrote: > > When booting a new kernel with kexec_file, the kernel picks a target > location that the kernel should live at, then allocates random pages, > checks whether any of those patches magically happens to coincide with > a target address range and if so, uses them for that range. > > For every page allocated this way, it then creates a page list that the > relocation code - code that executes while all CPUs are off and we are > just about to jump into the new kernel - copies to their final memory > location. We can not put them there before, because chances are pretty > good that at least some page in the target range is already in use by > the currently running Linux environment. Copying is happening from a > single CPU at RAM rate, which takes around 4-50 ms per 100 MiB. > > All of this is inefficient and error prone. > > To successfully kexec, we need to quiesce all devices of the outgoing > kernel so they don't scribble over the new kernel's memory. We have seen > cases where that does not happen properly (*cough* GIC *cough*) and hence > the new kernel was corrupted. This started a month long journey to root > cause failing kexecs to eventually see memory corruption, because the new > kernel was corrupted severely enough that it could not emit output to > tell us about the fact that it was corrupted. By allocating memory for the > next kernel from a memory range that is guaranteed scribbling free, we can > boot the next kernel up to a point where it is at least able to detect > corruption and maybe even stop it before it becomes severe. This increases > the chance for successful kexecs. > > Since kexec got introduced, Linux has gained the CMA framework which > can perform physically contiguous memory mappings, while keeping that > memory available for movable memory when it is not needed for contiguous > allocations. The default CMA allocator is for DMA allocations. > > This patch adds logic to the kexec file loader to attempt to place the > target payload at a location allocated from CMA. If successful, it uses > that memory range directly instead of creating copy instructions during > the hot phase. To ensure that there is a safety net in case anything goes > wrong with the CMA allocation, it also adds a flag for user space to force > disable CMA allocations. > > Using CMA allocations has two advantages: > > 1) Faster by 4-50 ms per 100 MiB. There is no more need to copy in the > hot phase. > 2) More robust. Even if by accident some page is still in use for DMA, > the new kernel image will be safe from that access because it resides > in a memory region that is considered allocated in the old kernel and > has a chance to reinitialize that component. > > Signed-off-by: Alexander Graf <g...@amazon.com> > Acked-by: Baoquan He <b...@redhat.com>
Reviewed-by: Pasha Tatashin <pasha.tatas...@soleen.com>
