Hi,
On 2/13/20 4:22 PM, Eneko Lacunza wrote:
Hi again,
It is also documented elsewhere:
https://www.linux-kvm.org/page/Migration#Note
First of all I want to point out that the information you posted was
last updated in 2009, and since then CPUs have gotten more and more
complex (and, if Intel's security issues are anything to go by, also
come with more bugs then ever). It _might_ have been true back then, but
I have a feeling that even then it was more wishful thinking then actual
success stories.
Cheers
El 13/2/20 a las 16:12, Eneko Lacunza escribió:
Hi Stefan,
El 13/2/20 a las 15:58, Stefan Reiter escribió:
Hi,
On 2/13/20 1:27 PM, Eneko Lacunza wrote:
Hi Stefan,
Do you think this could help specifying a custom CPU model that will
migrate VMs with more than 1 core between Intel and AMD CPUs?
https://bugzilla.proxmox.com/show_bug.cgi?id=1660
as discussed on the bug report you linked, live migration between
hosts using different vendors' CPUs is very much unsupported in
general (with KVM-enabled VMs). Even if it _appears_ to work, there
are no guarantees about stability.
That said, if you find a specific CPU flag configuration that seems
to work for you (like comment #12 in the report mentions), this
series would indeed allow you to specify it as a custom CPU model
instead of having to use the '-args' parameter.
I don't understand why you insist that this isn't supported. Linux KVM
FAQ says otherwise:
https://www.linux-kvm.org/page/FAQ#Does_KVM_support_live_migration_from_an_AMD_host_to_an_Intel_host_and_back.3F
"
Does KVM support live migration from an AMD host to an Intel host and
back?
Yes. [...]
"
I understand that there can be bugs, or maybe Proxmox VE as a product
can decide no to support such a feature; but technically upstrean says
that it should work. Shall I report the issue upstream directly?
PVE _does_ allow you to attempt live-migration between different CPU
hosts, but since time and experience has told us that it seldomly works
as intended, we don't officially support it.
This is in contrast to most other virtualization providers that simply
forbid cross-vendor migrations entirely [0][1] - and I suppose they do
it for good reason too.
Of course you can always report kernel/KVM/QEMU/etc. bugs directly
upstream, this will indeed have the highest chance of getting your
specific issues fixed. Although I personally wouldn't hold my breath for
a 'complete fix' (as in, cross-vendor live-migration working just as
well as between same-vendor hosts) anytime soon.
[0]
https://kb.vmware.com/s/article/1005764#Does%20EVC%20allow%20AMD%20and%20Intel%20CPUs%20to%20be%20vMotion%20compatible?
[1]
https://docs.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-R2-and-2012/dn859550(v=ws.11)#when-to-use-processor-compatibility-mode
Thanks a lot
Eneko
Hope that helps,
Stefan
Thanks a lot
Eneko
El 12/2/20 a las 16:11, Stefan Reiter escribió:
Based on the RFC and following on- and off-list discussion about
custom CPU
models [0].
In essence, this revised patch allows a user to specify custom CPU
models in
/etc/pve/cpu-models.conf (section-config style [1]), where VMs
using that CPU
model inherit details from the definition. This removes any fragile
"auto-magical" CPU flag detection, while still giving the user a
way to create
VMs with the best possible subset of CPU features maintaining
live-migration
compatibility.
Includes the infrastructure for broadcasting supported CPU flags
for each
cluster-node via the key-value store - this is not necessary for the
custom-cpu feature in particular, but I think could prove useful for
implementing the GUI part (e.g. show the user which flags are
supported on which
nodes).
v7 -> v8:
* rebase on master, fix tests now using +pve0
* fixed nits noted by Thomas
* moved live-migration/snapshot patches forward to avoid temporary
breakage
* fix CPU hotplug with custom CPUs
* guard mkdir with check_cfs_is_mounted and also run before write
* fix rebase-error in cfg2cmd tests (getaddrinfo_all)
* dropped applied patches
I deliberately didn't add any version_guard checks, since I don't
think a bump
is the right thing to do here (I say, after I waited for the
pve-version series
to be applied for making this v8).
Doing so would break custom CPUs on older machine versions (since
I'd have to
pin it to 4.1~pve2) for no reason. The idea would have been to show
a neat error
message, but since up to this point we didn't even check the
pve-version on
migration, it wouldn't help much. Besides, migration to older
versions already
fails cleanly with a semi-readable message ('Unknown option:
force-cpu').
< see [2] for older history >
[0]: https://pve.proxmox.com/pipermail/pve-devel/2019-July/038268.html
[1]: e.g.:
cpu-model: custom-cpu-name
host-phys-bits 1
flags +aes;+avx;+avx2
reported-model kvm64
[2]
https://pve.proxmox.com/pipermail/pve-devel/2020-January/041278.html
qemu-server: Stefan Reiter (8):
Adapt CPUConfig to handle custom models
Verify VM-specific CPU configs seperately
Include "-cpu" parameter with live-migration
Include "-cpu" parameter with snapshots/suspend
Add helpers to better structure CPU option handling
Rework get_cpu_options and allow custom CPU models
fix #2318: allow phys-bits and host-phys-bits CPU settings
cfg2cmd: add test cases for custom CPU models
PVE/API2/Qemu.pm | 10 +-
PVE/QemuConfig.pm | 36 +-
PVE/QemuMigrate.pm | 22 +
PVE/QemuServer.pm | 44 +-
PVE/QemuServer/CPUConfig.pm | 477
+++++++++++++++---
test/cfg2cmd/custom-cpu-model-defaults.conf | 8 +
.../custom-cpu-model-defaults.conf.cmd | 24 +
test/cfg2cmd/custom-cpu-model.conf | 8 +
test/cfg2cmd/custom-cpu-model.conf.cmd | 27 +
test/cfg2cmd/i440fx-win10-hostpci.conf.cmd | 2 +-
test/cfg2cmd/minimal-defaults.conf.cmd | 2 +-
test/cfg2cmd/pinned-version.conf.cmd | 2 +-
.../q35-linux-hostpci-multifunction.conf.cmd | 2 +-
test/cfg2cmd/q35-linux-hostpci.conf.cmd | 2 +-
test/cfg2cmd/q35-win10-hostpci.conf.cmd | 2 +-
test/cfg2cmd/simple1.conf.cmd | 2 +-
test/cfg2cmd/spice-enhancments.conf.cmd | 2 +-
test/cfg2cmd/spice-linux-4.1.conf.cmd | 2 +-
test/cfg2cmd/spice-usb3.conf.cmd | 2 +-
test/cfg2cmd/spice-win.conf.cmd | 2 +-
test/run_config2command_tests.pl | 23 +
21 files changed, 609 insertions(+), 92 deletions(-)
create mode 100644 test/cfg2cmd/custom-cpu-model-defaults.conf
create mode 100644 test/cfg2cmd/custom-cpu-model-defaults.conf.cmd
create mode 100644 test/cfg2cmd/custom-cpu-model.conf
create mode 100644 test/cfg2cmd/custom-cpu-model.conf.cmd
_______________________________________________
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
