Hi Gavin,
> From: Gavin Shan <gs...@redhat.com>
> Sent: Tuesday, October 3, 2023 12:18 AM
> To: Salil Mehta <salil.me...@huawei.com>; qemu-devel@nongnu.org; qemu-
> a...@nongnu.org
> Cc: m...@kernel.org; jean-phili...@linaro.org; Jonathan Cameron
> <jonathan.came...@huawei.com>; lpieral...@kernel.org;
> peter.mayd...@linaro.org; richard.hender...@linaro.org;
> imamm...@redhat.com; andrew.jo...@linux.dev; da...@redhat.com;
> phi...@linaro.org; eric.au...@redhat.com; oliver.up...@linux.dev;
> pbonz...@redhat.com; m...@redhat.com; w...@kernel.org; raf...@kernel.org;
> alex.ben...@linaro.org; li...@armlinux.org.uk;
> dar...@os.amperecomputing.com; il...@os.amperecomputing.com;
> vis...@os.amperecomputing.com; karl.heub...@oracle.com;
> miguel.l...@oracle.com; salil.me...@opnsrc.net; zhukeqian
> <zhukeqi...@huawei.com>; wangxiongfeng (C) <wangxiongfe...@huawei.com>;
> wangyanan (Y) <wangyana...@huawei.com>; jiakern...@gmail.com;
> maob...@loongson.cn; lixiang...@loongson.cn; Linuxarm <linux...@huawei.com>
> Subject: Re: [PATCH V2 01/10] accel/kvm: Extract common KVM vCPU
> {creation,parking} code
>
> On 9/30/23 10:19, Salil Mehta wrote:
> > KVM vCPU creation is done once during the initialization of the VM when Qemu
> > threads are spawned. This is common to all the architectures.
> ^^^^^^^^^^^^^^^^^^^
> thread is spawned.
Yes, will fix.
Thanks
Salil.
> > Hot-unplug of vCPU results in destruction of the vCPU objects in QOM but
> > the KVM vCPU objects in the Host KVM are not destroyed and their
> representative
> > KVM vCPU objects/context in Qemu are parked.
> >
> > Refactor common logic so that some APIs could be reused by vCPU Hotplug
> > code.
> >
> > Signed-off-by: Salil Mehta <salil.me...@huawei.com>
> > ---
> > accel/kvm/kvm-all.c | 63 +++++++++++++++++++++++++++++++++-----------
> > include/sysemu/kvm.h | 14 ++++++++++
> > 2 files changed, 61 insertions(+), 16 deletions(-)
> >
> > diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c
> > index ff1578bb32..b8c36ba50a 100644
> > --- a/accel/kvm/kvm-all.c
> > +++ b/accel/kvm/kvm-all.c
> > @@ -80,7 +80,7 @@
> > #endif
> >
> > struct KVMParkedVcpu {
> > - unsigned long vcpu_id;
> > + int vcpu_id;
>
> @vcpu_id represents the vCPU index (CPUState::cpu_index) instead of the
> architectural CPU ID any more. However, I don't understand how it works
> for x86, and more comments regarding it can be seen below.
I missed the reason why I had used 'unsigned long' everywhere in the
first change. You can check the PowerPC, it returns vcpu_id which might
not be an 'integer'. Hence, this change could actually create a problem.
> > int kvm_fd;
> > QLIST_ENTRY(KVMParkedVcpu) node;
> > };
> > @@ -137,6 +137,7 @@ static QemuMutex kml_slots_lock;
> > #define kvm_slots_unlock() qemu_mutex_unlock(&kml_slots_lock)
> >
> > static void kvm_slot_init_dirty_bitmap(KVMSlot *mem);
> > +static int kvm_get_vcpu(KVMState *s, int vcpu_id);
> >
> > static inline void kvm_resample_fd_remove(int gsi)
> > {
> > @@ -320,11 +321,49 @@ err:
> > return ret;
> > }
> >
> > +void kvm_park_vcpu(CPUState *cpu)
> > +{
> > + int vcpu_id = cpu->cpu_index;
> > + struct KVMParkedVcpu *vcpu;
> > +
> > + vcpu = g_malloc0(sizeof(*vcpu));
> > + vcpu->vcpu_id = vcpu_id;
> > + vcpu->kvm_fd = cpu->kvm_fd;
> > + QLIST_INSERT_HEAD(&kvm_state->kvm_parked_vcpus, vcpu, node);
> > +}
> > +
>
> @vcpu_id can be dropped as suggested previously.
>
> vcpu->vcpu_id = cpu->cpu_index;
Yes, agreed.
Thanks
Salil.
> > +int kvm_create_vcpu(CPUState *cpu)
> > +{
> > + int vcpu_id = cpu->cpu_index;
> > + KVMState *s = kvm_state;
> > + int kvm_fd;
> > +
> > + DPRINTF("kvm_create_vcpu\n");
> > +
> > + /* check if the KVM vCPU already exist but is parked */
> > + kvm_fd = kvm_get_vcpu(s, vcpu_id);
> > + if (kvm_fd < 0) {
> > + /* vCPU not parked: create a new KVM vCPU */
> > + kvm_fd = kvm_vm_ioctl(s, KVM_CREATE_VCPU, vcpu_id);
> > + if (kvm_fd < 0) {
> > + error_report("KVM_CREATE_VCPU IOCTL failed for vCPU %d",
> > vcpu_id);
> > + return kvm_fd;
> > + }
> > + }
> > +
> > + cpu->vcpu_dirty = true;
> > + cpu->kvm_fd = kvm_fd;
> > + cpu->kvm_state = s;
> > + cpu->dirty_pages = 0;
> > + cpu->throttle_us_per_full = 0;
> > +
> > + return 0;
> > +}
> > +
>
> The comments here can be dropped since the code is self-explaining.
>
> @vcpu_id represents vCPU index, instead of the architecrual vCPU ID any
> more.
> @vcpu_id is passed to host through ioctl(KVM_CREATE_VCPU), which is
> expected
> as an architecrual vCPU ID instead of a vCPU index by host. It's indicated
> by 'struct kvm_vcpu' as below.
That should not be part of this change. I think.
> struct kvm_vcpu {
> :
> int vcpu_id; /* id given by userspace at creation */
> int vcpu_idx; /* index into kvm->vcpu_array */
> };
>
> Function kvm_arch_vcpu_id() converts the vCPU instance or vCPU index to
> the architecrual vCPU ID. All architectures except x86 simply returns
> vCPU index (CPUState::cpu_index) as the architecrural vCPU ID. x86 returns
> the APIC ID. Treating them equally seems to break x86.
I think PowerPC returns a different Id as well.
[...]
> > -static int kvm_get_vcpu(KVMState *s, unsigned long vcpu_id)
> > +static int kvm_get_vcpu(KVMState *s, int vcpu_id)
> > {
> > struct KVMParkedVcpu *cpu;
> >
> > @@ -384,7 +420,7 @@ static int kvm_get_vcpu(KVMState *s, unsigned long
> vcpu_id)
> > }
> > }
> >
> > - return kvm_vm_ioctl(s, KVM_CREATE_VCPU, (void *)vcpu_id);
> > + return -1;
> > }
> >
>
> Why we have -1 here. -ENOENT seems more descriptive?
Agreed.
Thanks
Salil.
