[pve-devel] [PATCH v3 docs 2/5] Add cluster create screenshot

2019-08-28 Thread Stefan Reiter
Signed-off-by: Stefan Reiter --- images/screenshot/gui-cluster-create.png | Bin 0 -> 18124 bytes 1 file changed, 0 insertions(+), 0 deletions(-) create mode 100644 images/screenshot/gui-cluster-create.png diff --git a/images/screenshot/gui-cluster-create.png b/images/screenshot/gui-clus

Re: [pve-devel] [PATCH i18n] Add comment about make update

2019-08-29 Thread Stefan Reiter
On 8/29/19 10:06 AM, Dominic Jäger wrote: Signed-off-by: Dominic Jäger --- I did not know that we have to run 'make update' when adding a new language to i18n and have overseen a lot of errors as a consequence [0]. We might want to give (new) developers a hint at some place so that something lik

Re: [pve-devel] [PATCH pve-docs 08/12] Fix typos in pvesm.adoc

2019-09-02 Thread Stefan Reiter
Looked through the patches, nice job catching all those. One thing inline. On 9/2/19 11:19 AM, Fabian Ebner wrote: Signed-off-by: Fabian Ebner --- pvesm.adoc | 14 +++--- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/pvesm.adoc b/pvesm.adoc index 5300f50..c2be9fe 1006

Re: [pve-devel] [PATCH pve-docs 12/12] Fix typos in vzdump.adoc

2019-09-02 Thread Stefan Reiter
On 9/2/19 11:20 AM, Fabian Ebner wrote: Signed-off-by: Fabian Ebner --- vzdump.adoc | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/vzdump.adoc b/vzdump.adoc index fb1ac3d..e990982 100644 --- a/vzdump.adoc +++ b/vzdump.adoc @@ -78,7 +78,7 @@ consistency, the use of the

[pve-devel] [PATCH qemu-server 3/7] Add QEMU CPU flag querying helpers

2019-09-02 Thread Stefan Reiter
d. This is how libvirt queries supported flags for its "host-passthrough" CPU type. query_supported_cpu_flags is thus rather slow and shouldn't be called unnecessarily. Signed-off-by: Stefan Reiter --- Changes from RFC: * Clearer regexes * Use existing QMP infrastructure * Add

[pve-devel] [PATCH 0/7] Add basics for custom CPU models

2019-09-02 Thread Stefan Reiter
ck if this approach works better than the cluster-cpu one. [0]: https://pve.proxmox.com/pipermail/pve-devel/2019-July/038268.html [1]: e.g.: cpu-model: custom-cpu-name host-phys-bits 1 flags +aes;+avx;+avx2 basemodel kvm64 qemu: Stefan Reiter (1): Trigger pve-api-updates on updat

[pve-devel] [PATCH qemu 1/7] Trigger pve-api-updates on update

2019-09-02 Thread Stefan Reiter
A QEMU update can change supported CPU flags, thus we need to restart API services (especially pvestatd) to refresh their cached values. Signed-off-by: Stefan Reiter --- debian/triggers | 1 + 1 file changed, 1 insertion(+) create mode 100644 debian/triggers diff --git a/debian/triggers b

[pve-devel] [PATCH qemu-server 7/7] Allow custom CPU types in API

2019-09-02 Thread Stefan Reiter
Custom CPU types can be specified via the API, but to prevent arbitrary ones we have to manually check if the given model exists (as default or custom). Signed-off-by: Stefan Reiter --- PVE/QemuServer.pm | 31 +-- 1 file changed, 29 insertions(+), 2 deletions

[pve-devel] [PATCH manager 2/7] Broadcast supported CPU flags

2019-09-02 Thread Stefan Reiter
pvestatd will read supported CPU flags once on startup (since these never change during runtime, and QEMU updates trigger a service restart), then broadcasts them as a key-value pair to the cluster. Signed-off-by: Stefan Reiter --- Changes from RFC: * Cache joined value * Add eval around QEMU

[pve-devel] [PATCH qemu-server 6/7] Handle CPU flags defined in custom CPU type

2019-09-02 Thread Stefan Reiter
Special care is taken not to overwrite any special flags, or ones manually set on the VM by the user. We warn if a flag is overruled. Signed-off-by: Stefan Reiter --- PVE/QemuServer.pm | 26 +- 1 file changed, 25 insertions(+), 1 deletion(-) diff --git a/PVE

[pve-devel] [PATCH qemu-server 5/7] Support custom CPU types in get_cpu_options

2019-09-02 Thread Stefan Reiter
Supports custom basemodels (model shown to QEMU, i.e. must be a default model), vendors and (host-)phys-bits for VMs with large amounts of RAM (see bug #2318). Signed-off-by: Stefan Reiter --- PVE/QemuServer.pm | 32 +++- 1 file changed, 31 insertions(+), 1 deletion

[pve-devel] [PATCH qemu-server 4/7] Add CustomCPUConfig for storing/parsing custom CPU models

2019-09-02 Thread Stefan Reiter
Signed-off-by: Stefan Reiter --- This will definitely require some sort of versioning mechanism, otherwise CPU definitions could be changed after starting a VM, thus breaking live-migration by starting the migration-target with different parameters. Hints, ideas, recommendations?

Re: [pve-devel] applied: [PATCH v2 manager 2/2] Show supported file types in upload file selector

2019-09-03 Thread Stefan Reiter
On 9/3/19 9:53 AM, Thomas Lamprecht wrote: On 22.08.19 14:39, Stefan Reiter wrote: By default, all file types are shown, but the user now has the option of filtering only by supported types in the file selector dialog. Signed-off-by: Stefan Reiter Acked-by: Dominik Csapak --- I found it

[pve-devel] [PATCH manager] Split CPU Model Selector into visual sections

2019-09-04 Thread Stefan Reiter
before. Signed-off-by: Stefan Reiter --- The list is only going to grow I suspect, especially with custom models potentially being available in the future. Let's give the users some eye-candy to more easily parse what they see :) www/css/ext6-pve.css | 11 ++ www/manager6

[pve-devel] [PATCH v2 manager] Split CPU Model Selector into visual sections

2019-09-04 Thread Stefan Reiter
Most of the code is adapted from KVComboBox, from which it inherited before, with some generic code removed. The templating code adds a header for every group transition it encounters, thus providing support for arbitrary groups (e.g. custom models in the future). Signed-off-by: Stefan Reiter

Re: [pve-devel] [PATCH qemu-server 3/7] Add QEMU CPU flag querying helpers

2019-09-09 Thread Stefan Reiter
Will implement all of this for v2, just a note inline. On 9/6/19 1:42 PM, Fabian Grünbichler wrote: On September 2, 2019 4:27 pm, Stefan Reiter wrote: * query_understood_cpu_flags returns all flags that QEMU/KVM knows about * query_supported_cpu_flags returns all flags that QEMU/KVM can use on

Re: [pve-devel] [PATCH qemu-server 4/7] Add CustomCPUConfig for storing/parsing custom CPU models

2019-09-09 Thread Stefan Reiter
Thanks again for the thorough review, I'll try to address everything mentioned in v2. Some stuff inline for this patch in particular. On 9/9/19 11:53 AM, Fabian Grünbichler wrote: On September 2, 2019 4:27 pm, Stefan Reiter wrote: Inherits from SectionConfig to provide base pa

[pve-devel] [PATCH v3 manager] Make CPU Model Selector a grouped grid view

2019-09-10 Thread Stefan Reiter
ct too). Signed-off-by: Stefan Reiter --- I'm personally not 100% convinced this grid approach makes it easier to parse for a user, but I'd be fine with this being applied, if consensus is otherwise. A search function like with our ISO selector for example would be nice too I think, bu

[pve-devel] [PATCH v4 0/3] CPUModelSelector rework to grid

2019-09-10 Thread Stefan Reiter
ot;AMD" or "intel" and see all related CPUs. v4: Regular GridPanel with searching improvements v3: Grouped grid-panel v1/2: ComboBox with headers manager: Stefan Reiter (2): Make CPU Model Selector a searchable grid view Allow CPUModelSelector to be searched by vendor as

[pve-devel] [PATCH v4 manager 1/3] Make CPU Model Selector a searchable grid view

2019-09-10 Thread Stefan Reiter
work with value set in widget definition (would need to be preferredValue), but we always call setValue() anyway (and if we don't, value will be '', aka the default, which is correct too), so just remove that from ProcessorEdit.js. Signed-off-by: Stefan Reiter --- www/manager6/

[pve-devel] [PATCH v4 widget-toolkit 2/3] Allow searching for multiple columns in ComboGrid

2019-09-10 Thread Stefan Reiter
Uses code from ExtJS 6.0.1 ComboBox.js to overwrite 'doLocalQuery', adding a new property 'searchFields' which allows to specify fields from the bound store which will all be searched (OR filter, i.e. the searched value can appear in any column specified). Signed-off-by: Stef

[pve-devel] [PATCH v4 manager 3/3] Allow CPUModelSelector to be searched by vendor as well

2019-09-10 Thread Stefan Reiter
Signed-off-by: Stefan Reiter --- www/manager6/form/CPUModelSelector.js | 1 + 1 file changed, 1 insertion(+) diff --git a/www/manager6/form/CPUModelSelector.js b/www/manager6/form/CPUModelSelector.js index dea6c44c..55532a0d 100644 --- a/www/manager6/form/CPUModelSelector.js +++ b/www/manager6

Re: [pve-devel] [PATCH] Check if VM is running when migrating with --online 1

2019-09-16 Thread Stefan Reiter
On 9/16/19 9:46 AM, Fabian Ebner wrote: Signed-off-by: Fabian Ebner --- PVE/API2/Qemu.pm | 3 +++ 1 file changed, 3 insertions(+) diff --git a/PVE/API2/Qemu.pm b/PVE/API2/Qemu.pm index b30931d..ab5912c 100644 --- a/PVE/API2/Qemu.pm +++ b/PVE/API2/Qemu.pm @@ -3341,6 +3341,9 @@ __PACKAGE__->re

Re: [pve-devel] [PATCH manager 4/4] spice: Add enhancements to VM Creation wizard

2019-09-16 Thread Stefan Reiter
Akin to what you mentioned on the cover, there's a connection missing between the "Graphic card" field and the SPICE ones - there is no clear indication of *why* they are disabled by default in a new VM. Maybe make it a section of some sort titled "SPICE"? On 9/13/19 3:16 PM, Aaron Lauterer wr

Re: [pve-devel] [PATCH manager 3/4] spice: Add enhancements to VM Options panel

2019-09-16 Thread Stefan Reiter
On 9/13/19 3:16 PM, Aaron Lauterer wrote: Signed-off-by: Aaron Lauterer --- www/manager6/Utils.js| 18 ++ www/manager6/qemu/Options.js | 13 + 2 files changed, 31 insertions(+) diff --git a/www/manager6/Utils.js b/www/manager6/Utils.js index 6a489e7e..139

Re: [pve-devel] [PATCH manager 2/4] spice: Add enhancements form component

2019-09-16 Thread Stefan Reiter
Tried my hand at a proper code review, I like the patches in general. On 9/13/19 3:16 PM, Aaron Lauterer wrote: Signed-off-by: Aaron Lauterer --- www/manager6/Makefile | 1 + www/manager6/form/SpiceEnhancementSelector.js | 66 +++ 2 files changed, 67

[pve-devel] [PATCH v2 00/12] Add basics for custom CPU models

2019-09-30 Thread Stefan Reiter
pu-name host-phys-bits 1 flags +aes;+avx;+avx2 reported-model kvm64 manager: Stefan Reiter (1): Broadcast supported CPU flags PVE/Service/pvestatd.pm | 40 ++-- 1 file changed, 38 insertions(+), 2 deletions(-) qemu-server: Stefan Reiter (11)

[pve-devel] [PATCH v2 manager 01/12] Broadcast supported CPU flags

2019-09-30 Thread Stefan Reiter
query_supported_cpu_flags fails, we try to retain the previously detected flags). Signed-off-by: Stefan Reiter --- v1 -> v2: * broadcast directly in update_supported_cpuflags * use kvm_user_version to determine when to re-query CPU flags * don't broadcast flags when unchanged or empty PVE

[pve-devel] [PATCH v2 qemu-server 02/12] Add QEMU CPU flag querying helpers

2019-09-30 Thread Stefan Reiter
d. This is how libvirt queries supported flags for its "host-passthrough" CPU type. query_supported_cpu_flags is thus rather slow and shouldn't be called unnecessarily. Currently only supports x86_64, because QEMU-aarch64 doesn't provide the necessary querying functions.

[pve-devel] [PATCH v2 qemu-server 04/12] Adapt CPUConfig to handle custom models

2019-09-30 Thread Stefan Reiter
Turn CPUConfig into a SectionConfig with parsing/writing support for custom CPU models. IO is handled using cfs. The "custom" parameter provides differentiation between custom and default types, even if the name is the same ('namespacing'). Signed-off-by: Stefan Reiter

[pve-devel] [PATCH v2 qemu-server 09/12] fix #2318: allow phys-bits and host-phys-bits CPU settings

2019-09-30 Thread Stefan Reiter
Can be specified for a particular VM or via a custom CPU model (VM takes precedence). Signed-off-by: Stefan Reiter --- PVE/QemuServer/CPUConfig.pm | 24 1 file changed, 24 insertions(+) diff --git a/PVE/QemuServer/CPUConfig.pm b/PVE/QemuServer/CPUConfig.pm index

[pve-devel] [PATCH v2 qemu-server 12/12] cfg2cmd: fix descriptions of cfg2cmd test cases

2019-09-30 Thread Stefan Reiter
Signed-off-by: Stefan Reiter --- Independant from the rest of the series. test/cfg2cmd/i440fx-win10-hostpci.conf | 2 +- test/cfg2cmd/q35-linux-hostpci.conf| 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/test/cfg2cmd/i440fx-win10-hostpci.conf b/test/cfg2cmd/i440fx

[pve-devel] [PATCH v2 qemu-server 06/12] Verify VM-specific CPU configs seperately

2019-09-30 Thread Stefan Reiter
$cpu_fmt is being reused for custom CPUs as well as VM-specific CPU settings. The "pve-vm-cpu-conf" format is introduced to verify a config specifically for use as VM-specific settings. Signed-off-by: Stefan Reiter --- PVE/QemuServer.pm | 2 +- PVE/QemuServer/CPUConf

[pve-devel] [PATCH v2 qemu-server 11/12] cfg2cmd: add test case for custom CPU model

2019-09-30 Thread Stefan Reiter
Requires a mock CPU-model config, which is given as a raw string to also test parsing capabilities. Signed-off-by: Stefan Reiter --- test/cfg2cmd/custom-cpu-model.conf | 8 test/run_config2command_tests.pl | 21 + 2 files changed, 29 insertions(+) create mode

[pve-devel] [PATCH v2 qemu-server 10/12] cfg2cmd: fix tests for new CPU flag resolving

2019-09-30 Thread Stefan Reiter
The new flag resolving outputs flags in sorted order for consistency, adapt the test cases to not break. Only the order is changed, not which flags are present. Signed-off-by: Stefan Reiter --- test/cfg2cmd/i440fx-win10-hostpci.conf.cmd | 2 +- test/cfg2cmd/minimal-defaults.conf.cmd | 2

[pve-devel] [PATCH v2 qemu-server 07/12] Add helpers to better structure CPU option handling

2019-09-30 Thread Stefan Reiter
roviding a reason why specific CPU flags have been added, and thus allows for useful warning messages should a flag be overwritten by another. Signed-off-by: Stefan Reiter --- PVE/QemuServer/CPUConfig.pm | 67 + 1 file changed, 67 insertions(+) diff --git a/PVE/

[pve-devel] [PATCH v2 qemu-server 03/12] Add CPUConfig file and migrate some CPU helpers

2019-09-30 Thread Stefan Reiter
The package will be used for custom CPU models as a SectionConfig, hence the name. For now we simply move some CPU related helper functions and declarations over from QemuServer to reduce clutter there. Signed-off-by: Stefan Reiter --- PVE/QemuServer.pm | 242

[pve-devel] [PATCH v2 qemu-server 08/12] Rework get_cpu_options and allow custom CPU models

2019-09-30 Thread Stefan Reiter
If the "custom" property on the cpu config is set, try to load the cputype from the custom CPU model config, and set values accordingly. While at it, extract currently hardcoded values into seperate sub and add reasonings. Signed-off-by: Stefan Reiter --- It was quite interest

[pve-devel] [PATCH v2 qemu-server 05/12] Add overrides and convenience functions to CPUConfig

2019-09-30 Thread Stefan Reiter
Add two overrides to avoid writing redundant information to the config file. get_model_by_name is used to return a cpu config with default values filled out. Signed-off-by: Stefan Reiter --- PVE/QemuServer/CPUConfig.pm | 48 + 1 file changed, 48 insertions

Re: [pve-devel] [PATCH v2 qemu-server 04/12] Adapt CPUConfig to handle custom models

2019-10-02 Thread Stefan Reiter
Thanks for the review, I'll prepare a v3 :) Just some clarifications inline for this patch. On 10/2/19 7:49 AM, Thomas Lamprecht wrote: On 9/30/19 12:58 PM, Stefan Reiter wrote: Turn CPUConfig into a SectionConfig with parsing/writing support for custom CPU models. IO is handled usin

Re: [pve-devel] [PATCH v2 qemu-server 03/12] Add CPUConfig file and migrate some CPU helpers

2019-10-02 Thread Stefan Reiter
On 10/1/19 6:12 PM, Thomas Lamprecht wrote: On 9/30/19 12:58 PM, Stefan Reiter wrote: The package will be used for custom CPU models as a SectionConfig, hence the name. For now we simply move some CPU related helper functions and declarations over from QemuServer to reduce clutter there

[pve-devel] [PATCH v3 qemu-server 08/11] Add helpers to better structure CPU option handling

2019-10-03 Thread Stefan Reiter
roviding a reason why specific CPU flags have been added, and thus allows for useful warning messages should a flag be overwritten by another. Signed-off-by: Stefan Reiter --- PVE/QemuServer/CPUConfig.pm | 67 + 1 file changed, 67 insertions(+) diff --git a/PVE/

[pve-devel] [PATCH v3 qemu 02/11] Write understood CPU flags into static file

2019-10-03 Thread Stefan Reiter
located at /usr/share/kvm/cpu-flags-understood-$arch This file can be read by qemu-server's "query_understood_cpu_flags" function, avoiding a more expensive call to QEMU. For now, only x86_64 is implemented, since aarch64 doesn't print any flags when called this way. S

[pve-devel] [PATCH v3 00/11] Add basics for custom CPU models

2019-10-03 Thread Stefan Reiter
solving infrastructure * Add and fix test cases Does not include any fix for versioning/live-migration with custom models for now, felt these changes were big enough. [0]: https://pve.proxmox.com/pipermail/pve-devel/2019-July/038268.html [1]: e.g.: cpu-model: custom-cpu-name host-phys-bits

[pve-devel] [PATCH v3 qemu-server 10/11] fix #2318: allow phys-bits and host-phys-bits CPU settings

2019-10-03 Thread Stefan Reiter
Can be specified for a particular VM or via a custom CPU model (VM takes precedence). QEMU's default limit only allows up to 1TB of RAM per VM. Increasing the physical address bits available to a VM can fix this. Signed-off-by: Stefan Reiter --- v2 -> v3: * add minimum to phys-bits (8

[pve-devel] [PATCH v3 qemu-server 11/11] cfg2cmd: add test case for custom CPU model

2019-10-03 Thread Stefan Reiter
Requires a mock CPU-model config, which is given as a raw string to also test parsing capabilities. Signed-off-by: Stefan Reiter --- test/cfg2cmd/custom-cpu-model.conf | 8 test/run_config2command_tests.pl | 21 + 2 files changed, 29 insertions(+) create mode

[pve-devel] [PATCH v3 qemu-server 04/11] Add CPUConfig file and migrate some helpers

2019-10-03 Thread Stefan Reiter
: Stefan Reiter --- v3: * mention qemu_machine_feature_enabled in commit msg PVE/QemuServer.pm | 245 +- PVE/QemuServer/CPUConfig.pm | 254 PVE/QemuServer/Makefile | 1 + 3 files changed, 259 insertions(+), 241

[pve-devel] [PATCH v3 qemu-server 07/11] Verify VM-specific CPU configs seperately

2019-10-03 Thread Stefan Reiter
$cpu_fmt is being reused for custom CPUs as well as VM-specific CPU settings. The "pve-vm-cpu-conf" format is introduced to verify a config specifically for use as VM-specific settings. Signed-off-by: Stefan Reiter --- v2 -> v3: * move $cpu_fmt->{flags} changes here, to avo

[pve-devel] [PATCH v3 qemu-server 03/11] Add QEMU CPU flag querying helpers

2019-10-03 Thread Stefan Reiter
uery both. Currently only supports x86_64, because QEMU-aarch64 doesn't provide the necessary querying functions. Signed-off-by: Stefan Reiter --- query_understood_cpu_flags is currently not used, but will be very useful for the later UI part. I think it thematically fits best with thi

[pve-devel] [PATCH v3 qemu-server 06/11] Add overrides and convenience functions to CPUConfig

2019-10-03 Thread Stefan Reiter
Add two overrides to avoid writing redundant information to the config file. get_model_by_name is used to return a cpu config with default values filled out. Signed-off-by: Stefan Reiter --- v2 -> v3: * add validity checks to write_config PVE/QemuServer/CPUConfig.pm |

[pve-devel] [PATCH v3 manager 01/11] Broadcast supported CPU flags

2019-10-03 Thread Stefan Reiter
), to not try again too quickly. Signed-off-by: Stefan Reiter --- v2 -> v3: * broadcast tcg and kvm flags if available * clear kv-store on error, wait a bit until retry v1 -> v2: * broadcast directly in update_supported_cpuflags * use kvm_user_version to determine when to re-query CPU

[pve-devel] [PATCH v3 qemu-server 05/11] Adapt CPUConfig to handle custom models

2019-10-03 Thread Stefan Reiter
Turn CPUConfig into a SectionConfig with parsing/writing support for custom CPU models. IO is handled using cfs. The "built-in" parameter provides differentiation between custom and default types, even if the name is the same ('namespacing'). Signed-off-by: Stefan Reiter --

[pve-devel] [PATCH v3 qemu-server 09/11] Rework get_cpu_options and allow custom CPU models

2019-10-03 Thread Stefan Reiter
order for consistency, adapt the test cases to not break. Only the order is changed, not which flags are present. Signed-off-by: Stefan Reiter --- v3: Since it's just a few lines, I included the test changes into this patch directly. This way all patches in the series should be buildable withou

Re: [pve-devel] [PATCH v3 qemu-server 05/11] Adapt CPUConfig to handle custom models

2019-10-03 Thread Stefan Reiter
On 10/3/19 5:10 PM, Thomas Lamprecht wrote: On 10/3/19 4:56 PM, Stefan Reiter wrote: "built-in", you have to explicitly set a value in the config to 0, which looks weird - basically "unsetting" something to use a feature), but I'm very much fine with either. but it do

[pve-devel] [PATCH v4 cluster 03/12] Add "cpu-models.conf" to observed files

2019-10-07 Thread Stefan Reiter
Holds information about custom CPU models, section config style. Used in qemu-server. Signed-off-by: Stefan Reiter --- data/PVE/Cluster.pm | 1 + data/src/status.c | 1 + 2 files changed, 2 insertions(+) diff --git a/data/PVE/Cluster.pm b/data/PVE/Cluster.pm index 9cb68d8..f3f1704 100644

[pve-devel] [PATCH v4 manager 01/12] Broadcast supported CPU flags

2019-10-07 Thread Stefan Reiter
), to not try again too quickly. Signed-off-by: Stefan Reiter --- v2 -> v3: * broadcast tcg and kvm flags if available * clear kv-store on error, wait a bit until retry v1 -> v2: * broadcast directly in update_supported_cpuflags * use kvm_user_version to determine when to re-query CPU

[pve-devel] [PATCH v4 qemu-server 06/12] Adapt CPUConfig to handle custom models

2019-10-07 Thread Stefan Reiter
Turn CPUConfig into a SectionConfig with parsing/writing support for custom CPU models. IO is handled using cfs. Namespacing will be provided using "custom-" prefix for custom model names (in VM config only, cpu-models.conf will contain unprefixed names). Signed-off-by: Stefan Reite

[pve-devel] [PATCH v4 qemu-server 07/12] Add overrides and convenience functions to CPUConfig

2019-10-07 Thread Stefan Reiter
Add two overrides to avoid writing redundant information to the config file. get_model_by_name is used to return a cpu config with default values filled out. Signed-off-by: Stefan Reiter --- v3 -> v4: * add is_custom_model v2 -> v3: * add validity checks to write_config PVE/Qemu

[pve-devel] [PATCH v4 qemu-server 04/12] Add QEMU CPU flag querying helpers

2019-10-07 Thread Stefan Reiter
uery both. Currently only supports x86_64, because QEMU-aarch64 doesn't provide the necessary querying functions. Signed-off-by: Stefan Reiter --- query_understood_cpu_flags is currently not used, but will be very useful for the later UI part. I think it thematically fits best with thi

[pve-devel] [PATCH v4 qemu-server 09/12] Add helpers to better structure CPU option handling

2019-10-07 Thread Stefan Reiter
roviding a reason why specific CPU flags have been added, and thus allows for useful warning messages should a flag be overwritten by another. Signed-off-by: Stefan Reiter --- PVE/QemuServer/CPUConfig.pm | 67 + 1 file changed, 67 insertions(+) diff --git a/PVE/

[pve-devel] [PATCH v4 qemu-server 05/12] Add CPUConfig file and migrate some helpers

2019-10-07 Thread Stefan Reiter
: Stefan Reiter --- v3: * mention qemu_machine_feature_enabled in commit msg PVE/QemuServer.pm | 245 +- PVE/QemuServer/CPUConfig.pm | 254 PVE/QemuServer/Makefile | 1 + 3 files changed, 259 insertions(+), 241

[pve-devel] [PATCH v4 qemu-server 10/12] Rework get_cpu_options and allow custom CPU models

2019-10-07 Thread Stefan Reiter
test cases to not break. Only the order is changed, not which flags are present. Signed-off-by: Stefan Reiter --- v3 -> v4: * use is_custom_model v3: Since it's just a few lines, I included the test changes into this patch directly. This way all patches in the series should be buildable

[pve-devel] [PATCH v4 qemu 02/12] Write understood CPU flags into static file

2019-10-07 Thread Stefan Reiter
located at /usr/share/kvm/cpu-flags-understood-$arch This file can be read by qemu-server's "query_understood_cpu_flags" function, avoiding a more expensive call to QEMU. For now, only x86_64 is implemented, since aarch64 doesn't print any flags when called this way. S

[pve-devel] [PATCH v4 qemu-server 08/12] Verify VM-specific CPU configs seperately

2019-10-07 Thread Stefan Reiter
$cpu_fmt is being reused for custom CPUs as well as VM-specific CPU settings. The "pve-vm-cpu-conf" format is introduced to verify a config specifically for use as VM-specific settings. Signed-off-by: Stefan Reiter --- v3 -> v4: * use is_custom_model v2 -> v3: * move $cpu_fmt-

[pve-devel] [PATCH v4 qemu-server 11/12] fix #2318: allow phys-bits and host-phys-bits CPU settings

2019-10-07 Thread Stefan Reiter
Can be specified for a particular VM or via a custom CPU model (VM takes precedence). QEMU's default limit only allows up to 1TB of RAM per VM. Increasing the physical address bits available to a VM can fix this. Signed-off-by: Stefan Reiter --- v2 -> v3: * add minimum to phys-bits (8

[pve-devel] [PATCH v4 qemu-server 12/12] cfg2cmd: add test case for custom CPU model

2019-10-07 Thread Stefan Reiter
Requires a mock CPU-model config, which is given as a raw string to also test parsing capabilities. Signed-off-by: Stefan Reiter --- test/cfg2cmd/custom-cpu-model.conf | 8 test/run_config2command_tests.pl | 21 + 2 files changed, 29 insertions(+) create mode

[pve-devel] [PATCH v4 00/12] Add basics for custom CPU models

2019-10-07 Thread Stefan Reiter
ure * Add and fix test cases Does not include any fix for versioning/live-migration with custom models for now, felt these changes were big enough. [0]: https://pve.proxmox.com/pipermail/pve-devel/2019-July/038268.html [1]: e.g.: cpu-model: custom-cpu-name host-ph

[pve-devel] [PATCH] fix #2402: allow 1GB hugepages if 2MB is unavailable

2019-10-10 Thread Stefan Reiter
ly check if the requested hugepage size is available in hugepages_size instead. Signed-off-by: Stefan Reiter --- PVE/QemuServer/Memory.pm | 10 ++ 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/PVE/QemuServer/Memory.pm b/PVE/QemuServer/Memory.pm index b579955..f311dce 1

[pve-devel] [PATCH v2] fix #2402: allow 1GB hugepages if 2MB is unavailable

2019-10-10 Thread Stefan Reiter
d hugepage size is available in hugepages_size instead. Signed-off-by: Stefan Reiter --- v2: Correct error message if 1GB hugepages would be supported, but memory is not a multiple of 1024 PVE/QemuServer/Memory.pm | 16 +++- 1 file changed, 11 insertions(+), 5 deletions(-) diff -

[pve-devel] [PATCH v3 qemu-server 07/12] Add overrides and convenience functions to CPUConfig

2019-10-15 Thread Stefan Reiter
Add two overrides to avoid writing redundant information to the config file. get_custom_model is used to retrieve a custom model configuration by name. Signed-off-by: Stefan Reiter --- v4 -> v5: * change get_model_by_name to get_custom_model v3 -> v4: * add is_custom_model v2 ->

[pve-devel] [PATCH v3 00/12] Add basics for custom CPU models

2019-10-15 Thread Stefan Reiter
tom models for now, felt these changes were big enough. Also applies to CPU hotplug with custom models. [0]: https://pve.proxmox.com/pipermail/pve-devel/2019-July/038268.html [1]: e.g.: cpu-model: custom-cpu-name host-phys-bits 1 flags +aes;+avx;+avx2 reported-model kvm64 manager: Stefan

[pve-devel] [PATCH v3 qemu 02/12] Write understood CPU flags into static file

2019-10-15 Thread Stefan Reiter
located at /usr/share/kvm/cpu-flags-understood-$arch This file can be read by qemu-server's "query_understood_cpu_flags" function, avoiding a more expensive call to QEMU. For now, only x86_64 is implemented, since aarch64 doesn't print any flags when called this way. S

[pve-devel] [PATCH v3 qemu-server 06/12] Adapt CPUConfig to handle custom models

2019-10-15 Thread Stefan Reiter
Turn CPUConfig into a SectionConfig with parsing/writing support for custom CPU models. IO is handled using cfs. Namespacing will be provided using "custom-" prefix for custom model names (in VM config only, cpu-models.conf will contain unprefixed names). Signed-off-by: Stefan Reite

[pve-devel] [PATCH v3 qemu-server 11/12] fix #2318: allow phys-bits and host-phys-bits CPU settings

2019-10-15 Thread Stefan Reiter
Can be specified for a particular VM or via a custom CPU model (VM takes precedence). QEMU's default limit only allows up to 1TB of RAM per VM. Increasing the physical address bits available to a VM can fix this. Signed-off-by: Stefan Reiter --- v2 -> v3: * add minimum to phys-bits (8

[pve-devel] [PATCH v3 manager 01/12] Broadcast supported CPU flags

2019-10-15 Thread Stefan Reiter
), to not try again too quickly. Signed-off-by: Stefan Reiter --- v4 -> v5: * fix error detection ($supported_cpuflags can contain { kvm => undef, tcg => undef} but is still invalid) v2 -> v3: * broadcast tcg and kvm flags if available * clear kv-store on error, wait a bit until ret

[pve-devel] [PATCH v3 qemu-server 05/12] Add CPUConfig file and migrate some helpers

2019-10-15 Thread Stefan Reiter
: Stefan Reiter --- v5: remove unnecessary reformatting v3: mention qemu_machine_feature_enabled in commit msg PVE/QemuServer.pm | 242 +- PVE/QemuServer/CPUConfig.pm | 255 PVE/QemuServer/Makefile | 1 + 3 files

[pve-devel] [PATCH v3 qemu-server 12/12] cfg2cmd: add test case for custom CPU model

2019-10-15 Thread Stefan Reiter
Requires a mock CPU-model config, which is given as a raw string to also test parsing capabilities. Signed-off-by: Stefan Reiter --- v5: * include .cmd file test/cfg2cmd/custom-cpu-model.conf | 8 test/cfg2cmd/custom-cpu-model.conf.cmd | 27 ++ test

[pve-devel] [PATCH v3 qemu-server 09/12] Add helpers to better structure CPU option handling

2019-10-15 Thread Stefan Reiter
roviding a reason why specific CPU flags have been added, and thus allows for useful warning messages should a flag be overwritten by another. Signed-off-by: Stefan Reiter --- v5: * change $value_changed to a single calculation PVE/QemuServer/CPUConfig.pm | 64 +++

[pve-devel] [PATCH v3 qemu-server 04/12] Add QEMU CPU flag querying helpers

2019-10-15 Thread Stefan Reiter
uery both. Currently only supports x86_64, because QEMU-aarch64 doesn't provide the necessary querying functions. Signed-off-by: Stefan Reiter --- query_understood_cpu_flags is currently not used, but will be very useful for the later UI part. I think it thematically fits best with thi

[pve-devel] [PATCH v3 qemu-server 08/12] Verify VM-specific CPU configs seperately

2019-10-15 Thread Stefan Reiter
e required). Signed-off-by: Stefan Reiter --- v4 -> v5: * extract both regexes into vars * add 'pve-cpu-conf' format for future use * rename verify_vm_cpu_conf to parse_vm_cpu_conf (since it's used for parsing in a later patch, and also to make it consistent with parse

[pve-devel] [PATCH v3 qemu-server 10/12] Rework get_cpu_options and allow custom CPU models

2019-10-15 Thread Stefan Reiter
test cases to not break. Only the order is changed, not which flags are present. Signed-off-by: Stefan Reiter --- v4 -> v5: * variable renaming * extract parse_cpuflag_list sub to enable better code reuse v3 -> v4: * use is_custom_model v3: Since it's just a few lines, I includ

[pve-devel] [PATCH v3 cluster 03/12] Add "cpu-models.conf" to observed files

2019-10-15 Thread Stefan Reiter
Holds information about custom CPU models, section config style. Used in qemu-server. Signed-off-by: Stefan Reiter --- data/PVE/Cluster.pm | 1 + data/src/status.c | 1 + 2 files changed, 2 insertions(+) diff --git a/data/PVE/Cluster.pm b/data/PVE/Cluster.pm index 9cb68d8..f3f1704 100644

Re: [pve-devel] [PATCH v3 00/12] Add basics for custom CPU models

2019-10-15 Thread Stefan Reiter
Sorry, that's supposed to be v5 in the subject... On 10/15/19 4:12 PM, Stefan Reiter wrote: Based on the RFC and following on- and off-list discussion about custom CPU models [0]. In essence, this revised patch allows a user to specify custom CPU models in /etc/pve/cpu-models.conf (se

Re: [pve-devel] [PATCH v3 qemu 02/12] Write understood CPU flags into static file

2019-10-17 Thread Stefan Reiter
On 10/17/19 11:07 AM, Thomas Lamprecht wrote: On 10/15/19 4:12 PM, Stefan Reiter wrote: located at /usr/share/kvm/cpu-flags-understood-$arch This file can be read by qemu-server's "query_understood_cpu_flags" function, avoiding a more expensive call to QEMU. For now,

Re: [pve-devel] [PATCH qemu-server 2/2] fix #2367: do not allow snapshot with name PENDING

2019-10-22 Thread Stefan Reiter
On 10/22/19 12:12 PM, Oguz Bektas wrote: or any other variant of the word 'pending' note that we can actually allow this snapshot after PVE 7.0, since pending section and snapshots will be properly namespaced. ([pve:pending] and [snap:$snapname] or similar) Signed-off-by: Oguz Bektas --- PVE

[pve-devel] [PATCH qemu-server] fix #2408: use scsi-hd backend for iSCSI as well

2019-10-22 Thread Stefan Reiter
between the two anyway. scsi-block (which potentially also breaks) requires a flag to be manually set on the disk, so we can assume the user knows what they're doing. Signed-off-by: Stefan Reiter --- See also recent discussion on the mailing list: https://pve.proxmox.com/pipermail/pve-devel

[pve-devel] [PATCH docs] Replace erroneous interpunct with space

2019-10-23 Thread Stefan Reiter
Signed-off-by: Stefan Reiter --- That dot was bothering me. I can safely say that, since I'm the one who mistakenly put it there :) system-booting.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/system-booting.adoc b/system-booting.adoc index d3d7813..fcfe6c4 1

Re: [pve-devel] [PATCH v3 00/12] Add basics for custom CPU models

2019-10-23 Thread Stefan Reiter
fully) one last time. Thanks for all the reviews! On October 15, 2019 4:12 pm, Stefan Reiter wrote: Based on the RFC and following on- and off-list discussion about custom CPU models [0]. In essence, this revised patch allows a user to specify custom CPU models in /etc/pve/cpu-models.conf (se

[pve-devel] [PATCH manager 11/11] refactor: vm_mon_cmd was moved to PVE::QMP

2019-10-28 Thread Stefan Reiter
Signed-off-by: Stefan Reiter --- PVE/Service/pvestatd.pm | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/PVE/Service/pvestatd.pm b/PVE/Service/pvestatd.pm index bad1b73d..d8c86886 100755 --- a/PVE/Service/pvestatd.pm +++ b/PVE/Service/pvestatd.pm @@ -18,6 +18,7 @@ use PVE

[pve-devel] [PATCH common 01/11] Make get_host_arch return raw uname entry

2019-10-28 Thread Stefan Reiter
The current version had only one user in LXC, so move the LXC-specific code there to reuse this in QemuServer. Also cache, since the host's architecture can't change during runtime. Signed-off-by: Stefan Reiter --- src/PVE/Tools.pm | 17 + 1 file changed, 5 inserti

[pve-devel] [PATCH ha-manager 08/11] refactor: check_running was moved to PVE::QemuConfig

2019-10-28 Thread Stefan Reiter
Signed-off-by: Stefan Reiter --- src/PVE/HA/Resources/PVEVM.pm | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/PVE/HA/Resources/PVEVM.pm b/src/PVE/HA/Resources/PVEVM.pm index 0a37cf6..3a4c07a 100644 --- a/src/PVE/HA/Resources/PVEVM.pm +++ b/src/PVE/HA/Resources/PVEVM.pm

[pve-devel] [PATCH container 02/11] Move LXC-specific architecture translation here

2019-10-28 Thread Stefan Reiter
This is the only time we need to do this translation, moving it here allows reuse of the PVE::Tools function. Signed-off-by: Stefan Reiter --- src/PVE/LXC/Setup.pm | 9 + 1 file changed, 9 insertions(+) diff --git a/src/PVE/LXC/Setup.pm b/src/PVE/LXC/Setup.pm index 845aced..ae42a10

[pve-devel] [PATCH qemu-server 03/11] Use get_host_arch from PVE::Tools

2019-10-28 Thread Stefan Reiter
...now that it no longer does LXC-specific stuff. Removes a FIXME. Signed-off-by: Stefan Reiter --- PVE/QemuServer.pm | 8 +--- 1 file changed, 1 insertion(+), 7 deletions(-) diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm index b635760..9af690a 100644 --- a/PVE/QemuServer.pm +++ b/PVE

[pve-devel] [PATCH qemu-server 07/11] refactor: extract QEMU machine related helpers to package

2019-10-28 Thread Stefan Reiter
...PVE::QemuServer::Machine. qemu_machine_feature_enabled is exported since it has a *lot* of users in PVE::QemuServer and a long enough name as it is. Signed-off-by: Stefan Reiter --- Not sure if PVE::QemuMachine wouldn't be a better package name. I'm fine with both (or other sugges

[pve-devel] [PATCH 00/11] Refactor QemuServer to avoid dependency cycles

2019-10-28 Thread Stefan Reiter
I also didn't want to create a new module just for this one function. Open for ideas ofc. [0] https://pve.proxmox.com/pipermail/pve-devel/2019-October/039608.html (@Thomas: I rebased the series just before sending to work with your cleanups) common: Stefan Reiter (1): Make

[pve-devel] [PATCH ha-manager 09/11] refactor: vm_qmp_command was moved to PVE::QMP

2019-10-28 Thread Stefan Reiter
Signed-off-by: Stefan Reiter --- src/PVE/HA/Resources/PVEVM.pm | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/src/PVE/HA/Resources/PVEVM.pm b/src/PVE/HA/Resources/PVEVM.pm index 3a4c07a..84c23be 100644 --- a/src/PVE/HA/Resources/PVEVM.pm +++ b/src/PVE/HA/Resources

[pve-devel] [PATCH qemu-server 06/11] refactor: create PVE::QMP for high-level QMP access

2019-10-28 Thread Stefan Reiter
tests. Signed-off-by: Stefan Reiter --- PVE/API2/Qemu.pm | 13 PVE/API2/Qemu/Agent.pm | 7 ++-- PVE/CLI/qm.pm| 11 --- PVE/Makefile | 1 + PVE/QMP.pm | 71 PVE/QemuConfig.pm| 15

[pve-devel] [PATCH manager 10/11] refactor: check_running was moved to QemuConfig

2019-10-28 Thread Stefan Reiter
Signed-off-by: Stefan Reiter --- PVE/API2/Nodes.pm | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/PVE/API2/Nodes.pm b/PVE/API2/Nodes.pm index 9e731e05..0f30a518 100644 --- a/PVE/API2/Nodes.pm +++ b/PVE/API2/Nodes.pm @@ -1729,7 +1729,7 @@ __PACKAGE__->register_met

[pve-devel] [PATCH qemu-server 04/11] refactor: create QemuSchema and move file/dir code

2019-10-28 Thread Stefan Reiter
Also merge the 'mkdir's from QemuServer and QemuConfig to reduce duplication (both modules depend on QemuSchema anyway). nodename() is still called in multiple modules, but since it's cached by the INotify module it doesn't really matter. Signed-off-by: Stefan Reiter ---

[pve-devel] [PATCH qemu-server 05/11] refactor: Move check_running to QemuConfig

2019-10-28 Thread Stefan Reiter
Also move check_cmdline, since check_running is its only user. Changes all uses of check_running in QemuServer, including mocking in snapshot tests. Signed-off-by: Stefan Reiter --- PVE/API2/Qemu.pm | 32 +++--- PVE/CLI/qm.pm| 13 +++--- PVE/QemuConfig.pm

  1   2   3   4   5   >