Hi Bibo, > From: bibo mao <maob...@loongson.cn> > Sent: Tuesday, July 25, 2023 7:24 AM > To: Salil Mehta <salil.me...@huawei.com>
[...] > 在 2023/7/25 13:45, Salil Mehta 写道: > > Hello, > > > >> From: bibo mao <maob...@loongson.cn> > >> Sent: Tuesday, July 25, 2023 2:14 AM > >> To: Salil Mehta <salil.me...@huawei.com> [...] > >> 在 2023/7/25 08:56, Salil Mehta 写道: > >>> > >>>> From: bibo mao <maob...@loongson.cn> > >>>> Sent: Tuesday, July 25, 2023 1:29 AM > >>>> To: Salil Mehta <salil.me...@huawei.com> > >>>> Cc: Catalin Marinas <catalin.mari...@arm.com>; Jonathan Cameron > >>>> <jonathan.came...@huawei.com>; Marc Zyngier <m...@kernel.org>; Will > >>>> Deacon > >>>> <w...@kernel.org>; christoffer.d...@arm.com; oliver.up...@linux.dev; > >>>> mark.rutl...@arm.com; pbonz...@redhat.com; Salil Mehta > >>>> <salil.me...@opnsrc.net>; andrew.jo...@linux.dev; yuzenghui > >>>> <yuzeng...@huawei.com>; kvm...@lists.cs.columbia.edu; linux-arm- > >>>> ker...@lists.infradead.org; linux-ker...@vger.kernel.org; > >>>> k...@vger.kernel.org; qemu-devel@nongnu.org; james.mo...@arm.com; > >>>> steven.pr...@arm.com; Suzuki K Poulose <suzuki.poul...@arm.com>; Jean- > >>>> Philippe Brucker <jean-phili...@linaro.org>; kvm...@lists.linux.dev; > >>>> linux-c...@lists.linux.dev > >>>> Subject: Re: [Question - ARM CCA] vCPU Hotplug Support in ARM Realm > >>>> world might require ARM spec change? > >>>> > >>>> Is vcpu hotplug supported in arm virt-machine now? > >>> > >>> Not yet. We are working on it. Please check the RFCs being tested. > >>> > >>> > >>> [1] Pre-RFC V2 Changes: Support of Virtual CPU Hotplug for ARMv8 Arch > >>> (WIP) > >>> https://github.com/salil-mehta/qemu.git > >>> virt-cpuhp-armv8/rfc-v1-port11052023.dev-1 > >>> [2] [RFC PATCH 00/32] ACPI/arm64: add support for virtual cpuhotplug > >>> https://git.gitlab.arm.com/linux-arm/linux-jm.git > >>> virtual_cpu_hotplug/rfc/v2 > >>> > >>> > >>>> There is arm64 vcpu hotplug patch in qemu mailing list, however it is > >>>> not merged. > >>>> I do not know why it is not merged. > >>> > >>> > >>> I think you are referring to patches [3], [4]? Please follow the > >>> discussion > >>> for details. > >> > >> yeap, we reference the patch [3], [4] and benefit from them greatly -:) > > > > > > I am glad that our current work is useful to more than one architecture and > > it > > was one of the aim of our work as well but... > > > >> The patch for LoongArch vcpu hotplug link is: > >> https://lore.kernel.org/qemu-devel/cover.1689837093.git.lixiang...@loongson.cn/T/#t > > > > > > I quickly went through above patches and it looks like this patch-set is > > mostly > > based on our latest patches which are at [1], [2] and not just at [3], [4]. > > As I > > could see most of the functions which you have ported to your architecture > > are > > part of our Qemu repositories [2] which we have yet to push to community. > > As I > > am working towards RFC V2 patches and which shall be floated soon. It does > > not > > makes sense for you to duplicate the GED/ACPI changes which are common > > across > > architectures and which have been derived from the ARM64 vCPU Hotplug > > support > > original patches. > > > > This will create merge conflicts, will break large part of our original > > patch-set. > > > > Hence, I would request you to drop patches 1-4 from your patch-set or rebase > > it over ARM64 original patches in a week or 2 week of time. This is to avoid > > spoiling our previous years of hard work for the topic we have been > > persistently > > making efforts as you can see through the code and our detailed > > presentations. > > Do you have a plan to post new vcpu hotplug patch soon ? If there is, we can > postpone our patch for reviewing and wait for your arm64 vcpu Hotplug patch, > and hope that your patch can merge into qemu asap. Yes, we are working on it and plan to post it soon in 2 weeks of time. > > We always rebase on the mainline qemu version, rather than personal private > tree -:) Ok. I thought if you could take code from private branch then technically you could rebase from it as well :) Anyways, I will be soon posting the changes and it will have those ACPI/GED changes as well which we can try to get accepted quickly. Many thanks! Best wishes, Salil > Regards > Bibo Mao > > > > Hope you will agree we all need to respect others efforts and time in this > > mode of open-source collaboration. > > > > Rest assured I will help you in review of your architecture specific > > patch-set > > as it is a work of mutual interest. > > > > Thanks for understanding! > > > > > > Best Wishes, > > Salil. > > > >> Regards > >> Bibo Mao > >> > >>> > >>> > >>> [3] [PATCH RFC 00/22] Support of Virtual CPU Hotplug for ARMv8 Arch > >>> > >>> https://lore.kernel.org/all/20200613213629.21984-1-salil.me...@huawei.com/ > >>> [4] [PATCH RFC 0/4] Changes to Support *Virtual* CPU Hotplug for ARM64 > >>> > >>> https://lore.kernel.org/all/20200625133757.22332-1-salil.me...@huawei.com/#r > >>> > >>> > >>> In summary, there were some ARM64 Architecture constraints which were > >>> being > >>> violated in the earlier patches of the kernel [4] so we had to re-think > >>> of the > >>> kernel changes. The Qemu part mostly remains same with some new > >>> introductions > >>> of Guest HVC/SMC hyper call exit handling in user space etc. for policy > >>> checks > >>> in VMM/Qemu. > >>> > >>> > >>> You can follow the KVMForum conference presentations [5], [6] delivered > >>> in the > >>> year 2020 and 2023 to get hold of more details related to this. > >>> > >>> > >>> [5] KVMForum 2023: Challenges Revisited in Supporting Virt CPU Hotplug on > >>> architectures that don't Support CPU Hotplug (like ARM64) > >>> https://kvm-forum.qemu.org/2023/talk/9SMPDQ/ > >>> [6] KVMForum 2020: Challenges in Supporting Virtual CPU Hotplug on SoC > >>> Based Systems (like ARM64) > >>> https://kvmforum2020.sched.com/event/eE4m > >>> > >>> > >>>> I ask this question because we propose > >>>> similar patch about LoongArch system in qemu mailing list, and kernel > >>>> need not be > >>>> modified for vcpu hotplug. > >>> > >>> > >>> Could you please share the link of your patches so that we can have a > >>> look and > >>> draw a comparison? > >>> > >>> > >>> Thanks > >>> Salil. > >>> > >>>> > >>>> Regards > >>>> Bibo, mao > >>>> > >>>> 在 2023/7/19 10:35, Salil Mehta 写道: > >>>>> [Reposting it here from Linaro Open Discussion List for more eyes to > >>>>> look at] > >>>>> > >>>>> Hello, > >>>>> I have recently started to dabble with ARM CCA stuff and check if our > >>>>> recent changes to support vCPU Hotplug in ARM64 can work in the realm > >>>>> world. I have realized that in the RMM specification[1] PSCI_CPU_ON > >>>>> command(B5.3.3) does not handles the PSCI_DENIED return code(B5.4.2), > >>>>> from the host. This might be required to support vCPU Hotplug feature > >>>>> in the realm world in future. vCPU Hotplug is an important feature to > >>>>> support kata-containers in realm world as it reduces the VM boot time > >>>>> and facilitates dynamic adjustment of vCPUs (which I think should be > >>>>> true even with Realm world as current implementation only makes use > >>>>> of the PSCI_ON/OFF to realize the Hotplug look-like effect?) > >>>>> > >>>>> > >>>>> As per our recent changes [2], [3] related to support vCPU Hotplug on > >>>>> ARM64, we handle the guest exits due to SMC/HVC Hypercall in the > >>>>> user-space i.e. VMM/Qemu. In realm world, REC Exits to host due to > >>>>> PSCI_CPU_ON should undergo similar policy checks and I think, > >>>>> > >>>>> 1. Host should *deny* to online the target vCPUs which are NOT plugged > >>>>> 2. This means target REC should be denied by host. Can host call > >>>>> RMI_PSCI_COMPETE in such s case? > >>>>> 3. The *return* value (B5.3.3.1.3 Output values) should be PSCI_DENIED > >>>>> 4. Failure condition (B5.3.3.2) should be amended with > >>>>> runnable pre: target_rec.flags.runnable == NOT_RUNNABLE (?) > >>>>> post: result == PSCI_DENIED (?) > >>>>> 5. Change would also be required in the flow (D1.4 PSCI flows) depicting > >>>>> PSCI_CPU_ON flow (D1.4.1) > >>>>> > >>>>> > >>>>> I do understand that ARM CCA support is in its infancy stage and > >>>>> discussing about vCPU Hotplug in realm world seem to be a far-fetched > >>>>> idea right now. But specification changes require lot of time and if > >>>>> this change is really required then it should be further discussed > >>>>> within ARM. > >>>>> > >>>>> Many thanks! > >>>>> > >>>>> > >>>>> Bes regards > >>>>> Salil > >>>>> > >>>>> > >>>>> References: > >>>>> > >>>>> [1] https://developer.arm.com/documentation/den0137/latest/ > >>>>> [2] https://github.com/salil-mehta/qemu.git > >>>>> virt-cpuhp-armv8/rfc-v1-port11052023.dev-1 > >>>>> [3] https://git.gitlab.arm.com/linux-arm/linux-jm.git > >>>>> virtual_cpu_hotplug/rfc/v2
