[Xen-devel] [RFC 2/5] xen: cpupool: introduce cpupool_arch_info
From: Peng Fan
Intrdouce cpupool_arch_info.
To ARM, add a 'midr' entry to hold the MIDR info of the cpupool.
Signed-off-by: Peng Fan
Cc: Stefano Stabellini
Cc: Julien Grall
Cc: Jan Beulich
Cc: Andrew Cooper
---
xen/include/asm-arm/cpupool.h | 16
xen/include/asm-x86/cpupool.h | 16
xen/include/xen/sched-if.h| 2 ++
3 files changed, 34 insertions(+)
create mode 100644 xen/include/asm-arm/cpupool.h
create mode 100644 xen/include/asm-x86/cpupool.h
diff --git a/xen/include/asm-arm/cpupool.h b/xen/include/asm-arm/cpupool.h
new file mode 100644
index 000..f450199
--- /dev/null
+++ b/xen/include/asm-arm/cpupool.h
@@ -0,0 +1,16 @@
+/*
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+struct cpupool_arch_info
+{
+uint32_t midr; /* Hold the MIDR info of the pool */
+};
diff --git a/xen/include/asm-x86/cpupool.h b/xen/include/asm-x86/cpupool.h
new file mode 100644
index 000..3251709
--- /dev/null
+++ b/xen/include/asm-x86/cpupool.h
@@ -0,0 +1,16 @@
+/*
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+struct cpupool_arch_info
+{
+/* Nothing now.. */
+};
diff --git a/xen/include/xen/sched-if.h b/xen/include/xen/sched-if.h
index bc0e794..eb52ac7 100644
--- a/xen/include/xen/sched-if.h
+++ b/xen/include/xen/sched-if.h
@@ -8,6 +8,7 @@
#ifndef __XEN_SCHED_IF_H__
#define __XEN_SCHED_IF_H__
+#include
#include
/* A global pointer to the initial cpupool (POOL0). */
@@ -186,6 +187,7 @@ struct cpupool
unsigned int n_dom;
struct scheduler *sched;
atomic_t refcnt;
+struct cpupool_arch_info info;
};
#define cpupool_online_cpumask(_pool) \
--
2.6.6
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
[Xen-devel] [RFC 0/5] xen/arm: support big.little SoC
From: Peng Fan This patchset is to support XEN run on big.little SoC. The idea of the patch is from "https://lists.xenproject.org/archives/html/xen-devel/2016-05/msg00465.html"; There are some changes to cpupool and add x86 stub functions to avoid build break. Sending The RFC patchset out is to request for comments to see whether this implementation is acceptable or not. Patchset have been tested based on xen-4.8 unstable on NXP i.MX8. I use Big/Little CPU and cpupool to explain the idea. A pool contains Big CPUs is called Big Pool. A pool contains Little CPUs is called Little Pool. If a pool does not contains any physical cpus, Little CPUs or Big CPUs can be added to the cpupool. But the cpupool can not contain both Little and Big CPUs. The CPUs in a cpupool must have the same cpu type(midr value for ARM). CPUs can not be added to the cpupool which contains cpus that have different cpu type. Little CPUs can not be moved to Big Pool if there are Big CPUs in Big Pool, and versa. Domain in Big Pool can not be migrated to Little Pool, and versa. When XEN tries to bringup all the CPUs, only add CPUs with the same cpu type(same midr value) into cpupool0. Thinking an SoC with 4 A53(cpu[0-3]) + 2 A72(cpu[4-5]), cpu0 is the first one that boots up. When XEN tries to bringup secondary CPUs, add cpu[0-3] to cpupool0 and leave cpu[4-5] not in any cpupool. Then when Dom0 boots up, `xl cpupool-list -c` will show cpu[0-3] in Pool-0. Then use the following script to create a new cpupool and add cpu[4-5] to the cpupool. #xl cpupool-create name=\"Pool-A72\" sched=\"credit2\" #xl cpupool-cpu-add Pool-A72 4 #xl cpupool-cpu-add Pool-A72 5 #xl create -d /root/xen/domu-test pool=\"Pool-A72\" Now `xl cpupool-list -c` shows: NameCPU list Pool-0 0,1,2,3 Pool-A724,5 `xl cpupool-list` shows: Name CPUs Sched Active Domain count Pool-0 4credit y 1 Pool-A72 2 credit2 y 1 `xl cpupool-cpu-remove Pool-A72 4`, then `xl cpupool-cpu-add Pool-0 4` not success, because Pool-0 contains A53 CPUs, but CPU4 is an A72 CPU. `xl cpupool-migrate DomU Pool-0` will also fail, because DomU is created in Pool-A72 with A72 vcpu, while Pool-0 have A53 physical cpus. Patch 1/5: use "cpumask_weight(cpupool0->cpu_valid);" to replace "num_online_cpus()", because num_online_cpus() counts all the online CPUs, but now we only need Big or Little CPUs. Patch 2/5: Introduce cpupool_arch_info. To ARM SoC, need to add midr info to the cpupool. The info will be used in patch [3,4,5]/5. Patch 3/5: Need to check whether it is ok to add a physical cpu to a cpupool, When the cpupool does not contain any physical cpus, it is ok to add a cpu to the cpupool without care the cpu type. Need to check whether it is ok to move a domain to another cpupool. Patch 4/5: move vpidr from arch_domain to arch_vcpu. The vpidr in arch_domain is initialized in arch_domain_create, at this time, the domain is still in cpupool0, not moved the specified cpupool. We need to initialize vpidr later. But at the late stage, no method to initialize vpidr in arch_domain, so I move it to arch_vcpu. Patch 5/5: This is to check whether it is ok to move a domain to another cpupool. Peng Fan (5): xen/arm: domain_build: setting opt_dom0_max_vcpus according to cpupool0 info xen: cpupool: introduce cpupool_arch_info xen: cpupool: add arch cpupool hook xen/arm: move vpidr from arch_domain to arch_vcpu xen/arm: cpupool: implement arch_domain_cpupool_compatible xen/arch/arm/Makefile | 1 + xen/arch/arm/cpupool.c| 60 +++ xen/arch/arm/domain.c | 9 --- xen/arch/arm/domain_build.c | 3 ++- xen/arch/arm/traps.c | 2 +- xen/arch/x86/cpu/Makefile | 1 + xen/arch/x86/cpu/cpupool.c| 30 ++ xen/common/cpupool.c | 30 ++ xen/include/asm-arm/cpupool.h | 16 xen/include/asm-arm/domain.h | 9 --- xen/include/asm-x86/cpupool.h | 16 xen/include/xen/sched-if.h| 5 12 files changed, 173 insertions(+), 9 deletions(-) create mode 100644 xen/arch/arm/cpupool.c create mode 100644 xen/arch/x86/cpu/cpupool.c create mode 100644 xen/include/asm-arm/cpupool.h create mode 100644 xen/include/asm-x86/cpupool.h -- 2.6.6 ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
[Xen-devel] [RFC 5/5] xen/arm: cpupool: implement arch_domain_cpupool_compatible
From: Peng Fan
When migrating domain between different cpupools, need to check
whether the domain is compatible with the cpupool.
Signed-off-by: Peng Fan
Cc: Stefano Stabellini
Cc: Julien Grall
---
xen/arch/arm/cpupool.c | 17 -
1 file changed, 16 insertions(+), 1 deletion(-)
diff --git a/xen/arch/arm/cpupool.c b/xen/arch/arm/cpupool.c
index 74a5ef3..6c1c092 100644
--- a/xen/arch/arm/cpupool.c
+++ b/xen/arch/arm/cpupool.c
@@ -41,5 +41,20 @@ int arch_cpupool_cpu_add(struct cpupool *c, unsigned int cpu)
bool_t arch_domain_cpupool_compatible(struct domain *d, struct cpupool *c)
{
-return true;
+if ( !d->vcpu || !d->vcpu[0] )
+{
+/*
+ * We are in process of domain creation, vcpu not constructed or
+ * initialiszed, ok to move domain from cpupool0 to other pool
+ */
+return true;
+}
+else if ( d->vcpu[0] )
+{
+return !!( d->vcpu[0]->arch.vpidr == c->info.midr );
+}
+else
+{
+return false;
+}
}
--
2.6.6
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
[Xen-devel] [RFC 1/5] xen/arm: domain_build: setting opt_dom0_max_vcpus according to cpupool0 info
From: Peng Fan
Setting opt_dom0_max_vcpus according to cpu_valid in cpupool0.
Signed-off-by: Peng Fan
Cc: Stefano Stabellini
Cc: Julien Grall
---
xen/arch/arm/domain_build.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index 35ab08d..d171c39 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -5,6 +5,7 @@
#include
#include
#include
+#include
#include
#include
#include
@@ -59,7 +60,7 @@ custom_param("dom0_mem", parse_dom0_mem);
struct vcpu *__init alloc_dom0_vcpu0(struct domain *dom0)
{
if ( opt_dom0_max_vcpus == 0 )
-opt_dom0_max_vcpus = num_online_cpus();
+opt_dom0_max_vcpus = cpumask_weight(cpupool0->cpu_valid);
if ( opt_dom0_max_vcpus > MAX_VIRT_CPUS )
opt_dom0_max_vcpus = MAX_VIRT_CPUS;
--
2.6.6
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
[Xen-devel] [RFC 4/5] xen/arm: move vpidr from arch_domain to arch_vcpu
From: Peng Fan
Move vpidr from arch_domain to arch_vcpu.
In order to support Big.Little, Big CPUs and Little CPUs are
assigned to different cpupools.
when a new domain is to be created with cpupool specificed, the domain
is first assigned to cpupool0 and then the domain moved from cpupool0
to the specified cpupool.
In domain creation process arch_domain_create, vpidr is initialized
with boot_cpu_data.midr.bits. But Big cpupool and Little cpupool have
different midr, the guest vcpu midr should use the midr info from
cpupool which the domain runs in.
Signed-off-by: Peng Fan
Cc: Stefano Stabellini
Cc: Julien Grall
---
xen/arch/arm/domain.c| 9 +
xen/arch/arm/traps.c | 2 +-
xen/include/asm-arm/domain.h | 9 ++---
3 files changed, 12 insertions(+), 8 deletions(-)
diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
index 20bb2ba..934c112 100644
--- a/xen/arch/arm/domain.c
+++ b/xen/arch/arm/domain.c
@@ -14,6 +14,7 @@
#include
#include
#include
+#include
#include
#include
#include
@@ -150,7 +151,7 @@ static void ctxt_switch_to(struct vcpu *n)
p2m_restore_state(n);
-WRITE_SYSREG32(n->domain->arch.vpidr, VPIDR_EL2);
+WRITE_SYSREG32(n->arch.vpidr, VPIDR_EL2);
WRITE_SYSREG(n->arch.vmpidr, VMPIDR_EL2);
/* VGIC */
@@ -521,6 +522,9 @@ int vcpu_initialise(struct vcpu *v)
v->arch.actlr = READ_SYSREG32(ACTLR_EL1);
+/* The virtual ID matches the physical id of the cpu in the cpupool */
+v->arch.vpidr = v->domain->cpupool->info.midr;
+
processor_vcpu_initialise(v);
if ( (rc = vcpu_vgic_init(v)) != 0 )
@@ -562,9 +566,6 @@ int arch_domain_create(struct domain *d, unsigned int
domcr_flags,
if ( (d->shared_info = alloc_xenheap_pages(0, 0)) == NULL )
goto fail;
-/* Default the virtual ID to match the physical */
-d->arch.vpidr = boot_cpu_data.midr.bits;
-
clear_page(d->shared_info);
share_xen_page_with_guest(
virt_to_page(d->shared_info), d, XENSHARE_writable);
diff --git a/xen/arch/arm/traps.c b/xen/arch/arm/traps.c
index 683bcb2..c0ad97e 100644
--- a/xen/arch/arm/traps.c
+++ b/xen/arch/arm/traps.c
@@ -1975,7 +1975,7 @@ static void do_cp14_32(struct cpu_user_regs *regs, const
union hsr hsr)
* - Variant and Revision bits match MDIR
*/
val = (1 << 24) | (5 << 16);
-val |= ((d->arch.vpidr >> 20) & 0xf) | (d->arch.vpidr & 0xf);
+val |= ((d->vcpu[0]->arch.vpidr >> 20) & 0xf) |
(d->vcpu[0]->arch.vpidr & 0xf);
set_user_reg(regs, regidx, val);
break;
diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
index 9452fcd..b998c6d 100644
--- a/xen/include/asm-arm/domain.h
+++ b/xen/include/asm-arm/domain.h
@@ -63,9 +63,6 @@ struct arch_domain
RELMEM_done,
} relmem;
-/* Virtual CPUID */
-uint32_t vpidr;
-
struct {
uint64_t offset;
} phys_timer_base;
@@ -173,6 +170,12 @@ struct arch_vcpu
uint32_t esr;
#endif
+/*
+ * Holds the value of the Virtualization Processor ID.
+ * This is the value returned by Non-secure EL1 reads of MIDR_EL1.
+ */
+uint32_t vpidr;
+
uint32_t ifsr; /* 32-bit guests only */
uint32_t afsr0, afsr1;
--
2.6.6
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
[Xen-devel] [RFC 3/5] xen: cpupool: add arch cpupool hook
From: Peng Fan
Introduce arch_cpupool_create, arch_cpupool_cpu_add,
and arch_domain_cpupool_compatible.
To X86, nothing to do, just add an empty stub functions.
To ARM, arch_cpupool_create initialize midr with value -1;
arch_cpupool_cpu_add, if there is cpu in the cpupool or the cpu
is the first one that will be added to the cpupool, assign cpu
midr to cpupool midr. If the midr already initialzed and there is
valid cpus in the pool, check whether the midr of cpu and cpupool
is compatbile or not; arch_domain_cpupool_compatible is to check
whether the domain is ok to be moved into the cpupool.
Signed-off-by: Peng Fan
Cc: Stefano Stabellini
Cc: Julien Grall
Cc: Jan Beulich
Cc: Andrew Cooper
Cc: Juergen Gross
Cc: Dario Faggioli
---
xen/arch/arm/Makefile | 1 +
xen/arch/arm/cpupool.c | 45 +
xen/arch/x86/cpu/Makefile | 1 +
xen/arch/x86/cpu/cpupool.c | 30 ++
xen/common/cpupool.c | 30 ++
xen/include/xen/sched-if.h | 3 +++
6 files changed, 110 insertions(+)
create mode 100644 xen/arch/arm/cpupool.c
create mode 100644 xen/arch/x86/cpu/cpupool.c
diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile
index 23aaf52..1b72f66 100644
--- a/xen/arch/arm/Makefile
+++ b/xen/arch/arm/Makefile
@@ -9,6 +9,7 @@ obj-y += bootfdt.o
obj-y += cpu.o
obj-y += cpuerrata.o
obj-y += cpufeature.o
+obj-y += cpupool.o
obj-y += decode.o
obj-y += device.o
obj-y += domain.o
diff --git a/xen/arch/arm/cpupool.c b/xen/arch/arm/cpupool.c
new file mode 100644
index 000..74a5ef3
--- /dev/null
+++ b/xen/arch/arm/cpupool.c
@@ -0,0 +1,45 @@
+/*
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include
+#include
+#include
+
+int arch_cpupool_create(struct cpupool *c)
+{
+c->info.midr = -1;
+
+return 0;
+}
+
+int arch_cpupool_cpu_add(struct cpupool *c, unsigned int cpu)
+{
+if (( c->info.midr == -1 ) || ( cpumask_weight(c->cpu_valid) == 0 ))
+{
+c->info.midr = cpu_data[cpu].midr.bits;
+
+return 0;
+}
+else if (( c->info.midr == cpu_data[cpu].midr.bits ))
+{
+return 0;
+}
+else
+{
+return -EINVAL;
+}
+}
+
+bool_t arch_domain_cpupool_compatible(struct domain *d, struct cpupool *c)
+{
+return true;
+}
diff --git a/xen/arch/x86/cpu/Makefile b/xen/arch/x86/cpu/Makefile
index 74f23ae..0d3036e 100644
--- a/xen/arch/x86/cpu/Makefile
+++ b/xen/arch/x86/cpu/Makefile
@@ -4,6 +4,7 @@ subdir-y += mtrr
obj-y += amd.o
obj-y += centaur.o
obj-y += common.o
+obj-y += cpupool.o
obj-y += intel.o
obj-y += intel_cacheinfo.o
obj-y += mwait-idle.o
diff --git a/xen/arch/x86/cpu/cpupool.c b/xen/arch/x86/cpu/cpupool.c
new file mode 100644
index 000..d897a1f
--- /dev/null
+++ b/xen/arch/x86/cpu/cpupool.c
@@ -0,0 +1,30 @@
+/*
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include
+#include
+#include
+
+int arch_cpupool_create(struct cpupool *c)
+{
+return 0;
+}
+
+int arch_cpupool_cpu_add(struct cpupool *c, unsigned int cpu)
+{
+return 0;
+}
+
+bool_t arch_domain_cpupool_compatible(struct domain *d, struct cpupool *c)
+{
+return true;
+}
diff --git a/xen/common/cpupool.c b/xen/common/cpupool.c
index 9998394..798322d 100644
--- a/xen/common/cpupool.c
+++ b/xen/common/cpupool.c
@@ -134,11 +134,20 @@ static struct cpupool *cpupool_create(
struct cpupool *c;
struct cpupool **q;
int last = 0;
+int ret;
*perr = -ENOMEM;
if ( (c = alloc_cpupool_struct()) == NULL )
return NULL;
+ret = arch_cpupool_create(c);
+if ( ret )
+{
+*perr = ret;
+free_cpupool_struct(c);
+return NULL;
+}
+
/* One reference for caller, one reference for cpupool_destroy(). */
atomic_set(&c->refcnt, 2);
@@ -498,6 +507,8 @@ static int cpupool_cpu_add(unsigned int cpu)
{
if ( cpumask_test_cpu(cpu, (*c)->cpu_suspended ) )
{
+if ( arch_cpupool_cpu_add(*c, cpu) )
+continue;
[Xen-devel] [PATCH V2] xen/arm: domain_build: allocate lowmem for dom0 as much as possible
From: Peng Fan
On AArch64 SoCs, some IPs may only have the capability to access
32bits address space. The physical memory assigned for Dom0 maybe
not in 4GB address space, then the IPs will not work properly.
So need to allocate memory under 4GB for Dom0.
There is no restriction that how much lowmem needs to be allocated for
Dom0. Dom0 now use 1:1 mapping, but DomU does not use 1:1 mapping,
there is no need to reserve lowmem for DomU, so allocate lowmem as much
as possible for Dom0.
Signed-off-by: Peng Fan
Cc: Stefano Stabellini
Cc: Julien Grall
---
This patch is to resolve the issue mentioned in
https://lists.xen.org/archives/html/xen-devel/2016-09/msg00235.html
V2:
Remove the bootargs dom0_lowmem introduced in V1.
Following
"https://lists.xenproject.org/archives/html/xen-devel/2016-09/msg01459.html";
to allocate as much as possible lowmem.
Tested results:
(XEN) Allocating 1:1 mappings totalling 2048MB for dom0:
(XEN) BANK[0] 0x008800-0x00f800 (1792MB)
(XEN) BANK[1] 0x09e000-0x09f000 (256MB)
1792M allocated in 4GB address space.
xen/arch/arm/domain_build.c | 20 +---
1 file changed, 13 insertions(+), 7 deletions(-)
diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index 35ab08d..0b9b85c 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -240,11 +240,11 @@ static void allocate_memory(struct domain *d, struct
kernel_info *kinfo)
const unsigned int min_low_order =
get_order_from_bytes(min_t(paddr_t, dom0_mem, MB(128)));
const unsigned int min_order = get_order_from_bytes(MB(4));
-struct page_info *pg;
+struct page_info *pg = NULL;
unsigned int order = get_11_allocation_size(kinfo->unassigned_mem);
int i;
-bool_t lowmem = is_32bit_domain(d);
+bool_t lowmem = true;
unsigned int bits;
/*
@@ -265,22 +265,28 @@ static void allocate_memory(struct domain *d, struct
kernel_info *kinfo)
*/
while ( order >= min_low_order )
{
-for ( bits = order ; bits <= (lowmem ? 32 : PADDR_BITS); bits++ )
+for ( bits = order ; bits <= 32 ; bits++ )
{
pg = alloc_domheap_pages(d, order, MEMF_bits(bits));
if ( pg != NULL )
+{
+if ( !insert_11_bank(d, kinfo, pg, order) )
+BUG(); /* Cannot fail for first bank */
+
goto got_bank0;
+ }
}
order--;
}
-panic("Unable to allocate first memory bank");
+if ( pg == NULL )
+{
+printk(XENLOG_INFO "No bank has been allocated below 4GB.\n");
+lowmem = false;
+}
got_bank0:
-if ( !insert_11_bank(d, kinfo, pg, order) )
-BUG(); /* Cannot fail for first bank */
-
/* Now allocate more memory and fill in additional banks */
order = get_11_allocation_size(kinfo->unassigned_mem);
--
2.6.6
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
