Re: [pve-devel] [PATCH v2 storage 1/3] fix #2467 remove duplicate volumes & tag with correct content type

On November 20, 2019 11:35 am, Tim Marx wrote: > The bugfix for #2317 introduced a kind of odd api behavior, where each volume > was returned twice from our api if a storage has both 'rootdir' & 'images' > content > types enabled. To give the content type of the volume an actual meaning, it is > n

[pve-devel] applied: [PATCH manager] api/ceph: skip merging metadata if hostname is undefined

It's a bit hard to figure out the exact constellation required for this to happen, but we saw it in live systems when one node was dead in a three node cluster. Signed-off-by: Thomas Lamprecht --- PVE/API2/Cluster/Ceph.pm | 2 ++ 1 file changed, 2 insertions(+) diff --git a/PVE/API2/Cluster/Cep

[pve-devel] [PATCH qemu-server] fix #2473: use of unitialized value

With the noerr flag set in parse_volume_id we have to check if $volname is defined before comparing it to 'cloudinit'. Signed-off-by: Mira Limbeck --- PVE/API2/Qemu.pm | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/PVE/API2/Qemu.pm b/PVE/API2/Qemu.pm index 4dd329a..ed4e5

[pve-devel] [PATCH v3 storage 3/3] change var name ct to type to prevent confusion with container

Signed-off-by: Tim Marx --- PVE/Storage/Plugin.pm | 20 ++-- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/PVE/Storage/Plugin.pm b/PVE/Storage/Plugin.pm index 2573cb7..189356a 100644 --- a/PVE/Storage/Plugin.pm +++ b/PVE/Storage/Plugin.pm @@ -938,21 +938,21 @@ su

[pve-devel] [PATCH v3 storage 1/3] fix #2467 remove duplicate volumes & tag with correct content type

The bugfix for #2317 introduced a kind of odd api behavior, where each volume was returned twice from our api if a storage has both 'rootdir' & 'images' content types enabled. To give the content type of the volume an actual meaning, it is now inferred from the associated guest, if there's no gues

[pve-devel] [PATCH v3 storage 2/3] add content type to pvesm list output

Signed-off-by: Tim Marx --- PVE/CLI/pvesm.pm | 12 +++- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/PVE/CLI/pvesm.pm b/PVE/CLI/pvesm.pm index 01ddd23..63b212a 100755 --- a/PVE/CLI/pvesm.pm +++ b/PVE/CLI/pvesm.pm @@ -124,21 +124,23 @@ my $print_content = sub { my

[pve-devel] applied: [PATCH v2 storage 2/3] add content type to pvesm list output

On 11/20/19 11:35 AM, Tim Marx wrote: > Signed-off-by: Tim Marx > --- > PVE/CLI/pvesm.pm | 12 +++- > 1 file changed, 7 insertions(+), 5 deletions(-) > applied that one, as it was not related directly to this series. ___ pve-devel mailing lis

[pve-devel] [PATCH installer 1/2] Remove unused hdsize from zfs advanced options

When a zpool is created the whole disks are used, so a user cannot set a size limit in this case. Signed-off-by: Fabian Ebner --- proxinstall | 1 - 1 file changed, 1 deletion(-) diff --git a/proxinstall b/proxinstall index 5d02b34..93a61cb 100755 --- a/proxinstall +++ b/proxinstall @@ -2877,7

[pve-devel] [PATCH installer 2/2] Fix typos

Signed-off-by: Fabian Ebner --- proxinstall | 20 ++-- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/proxinstall b/proxinstall index 93a61cb..268bc91 100755 --- a/proxinstall +++ b/proxinstall @@ -678,7 +678,7 @@ sub read_cmap { } } -# search for Harddisk

Re: [pve-devel] [PATCH installer 1/2] Remove unused hdsize from zfs advanced options

On 11/21/19 12:35 PM, Fabian Ebner wrote: > When a zpool is created the whole disks are used, so a user cannot set a size > limit in this case. > are you sure?? AFAICR, this was added to ZFS so that one can leave some free space to add a swap device outside from ZFS (which made issues in the pa

[pve-devel] applied: [PATCH v3 storage 3/3] change var name ct to type to prevent confusion with container

On 11/21/19 11:43 AM, Tim Marx wrote: > Signed-off-by: Tim Marx > --- > PVE/Storage/Plugin.pm | 20 ++-- > 1 file changed, 10 insertions(+), 10 deletions(-) > applied thanks! ___ pve-devel mailing list pve-devel@pve.proxmox.com https:/

[pve-devel] applied: [PATCH v3 storage 1/3] fix #2467 remove duplicate volumes & tag with correct content type

On 11/21/19 11:43 AM, Tim Marx wrote: > The bugfix for #2317 introduced a kind of odd api behavior, where each volume > was returned twice from our api if a storage has both 'rootdir' & 'images' > content > types enabled. To give the content type of the volume an actual meaning, it is > now inferr

[pve-devel] applied: [PATCH v4 storage] fix #2085: Handle non-default mount point in path() by introducing new mountpoint property

On 11/18/19 11:45 AM, Fabian Ebner wrote: > When adding a zfspool storage with 'pvesm add' the mount point is now added > automatically to the storage configuration if it can be determined. > path() does not assume the default mountpoint anymore, fixing 2085. > > Signed-off-by: Fabian Ebner > ---

[pve-devel] applied: [PATCH v4 docs] Add description for mountpoint property

On 11/18/19 11:45 AM, Fabian Ebner wrote: > Signed-off-by: Fabian Ebner > --- > > Changes from v3: > * 'path' renamed to 'mountpoint' > > pve-storage-zfspool.adoc | 6 ++ > 1 file changed, 6 insertions(+) > > diff --git a/pve-storage-zfspool.adoc b/pve-storage-zfspool.adoc > index f53a

[pve-devel] applied: [PATCH qemu-server] fix #2473: use of unitialized value

On 11/21/19 11:04 AM, Mira Limbeck wrote: > With the noerr flag set in parse_volume_id we have to check if > $volname is defined before comparing it to 'cloudinit'. > > Signed-off-by: Mira Limbeck > --- > PVE/API2/Qemu.pm | 4 ++-- > 1 file changed, 2 insertions(+), 2 deletions(-) > > diff --gi

Re: [pve-devel] applied: [PATCH qemu-server] fix #2473: use of unitialized value

On 11/21/19 12:58 PM, Thomas Lamprecht wrote: On 11/21/19 11:04 AM, Mira Limbeck wrote: With the noerr flag set in parse_volume_id we have to check if $volname is defined before comparing it to 'cloudinit'. Signed-off-by: Mira Limbeck --- PVE/API2/Qemu.pm | 4 ++-- 1 file changed, 2 inserti

Re: [pve-devel] [PATCH manager 1/1] renew pve-ssl.pem when it nearly expires

On 10/28/19 11:39 AM, Dominik Csapak wrote: > but only if the ca is ours, and the cert is issued by our ca > (by checking the issuer and openssl verify) > > this way we can reduce the lifetime of the certs without having > to worry that they ran out > > Signed-off-by: Dominik Csapak > --- > PVE

[pve-devel] [PATCH manager] Fix #2476: Fix auto-ballooning QMP command

Commit 77123edbd0 (statd: refactor update_node_status) changed $target in pvestatd's auto_balloning sub into a variable: my $target = int($res->{$vmid}); but then uses it in a string as a parameter to the $log function: $log->("BALLOON $vmid to $target (%d)\n", $target - $current); This

Re: [pve-devel] [PATCH manager 1/1] renew pve-ssl.pem when it nearly expires

On November 21, 2019 1:19 pm, Thomas Lamprecht wrote: > On 10/28/19 11:39 AM, Dominik Csapak wrote: >> but only if the ca is ours, and the cert is issued by our ca >> (by checking the issuer and openssl verify) >> >> this way we can reduce the lifetime of the certs without having >> to worry that

Re: [pve-devel] [PATCH installer 1/2] Remove unused hdsize from zfs advanced options

On 11/21/19 12:48 PM, Thomas Lamprecht wrote: On 11/21/19 12:35 PM, Fabian Ebner wrote: When a zpool is created the whole disks are used, so a user cannot set a size limit in this case. are you sure?? AFAICR, this was added to ZFS so that one can leave some free space to add a swap device out

[pve-devel] applied: [PATCH manager] Fix #2476: Fix auto-ballooning QMP command

On 11/21/19 1:22 PM, Stefan Reiter wrote: > Commit 77123edbd0 (statd: refactor update_node_status) changed $target > in pvestatd's auto_balloning sub into a variable: > > my $target = int($res->{$vmid}); > > but then uses it in a string as a parameter to the $log function: > > $log->("BA

[pve-devel] [PATCH common] cli-formatter: avoid warning when trying to sort on undefined key

Example: pvesh get /nodes/{node}/qemu/{vmid}/rrddata --timeframe day If the sorting key is not defined in the dataset, e.g. when a VM was not running for some time within the given timeframe, this resulted in several ugly warnings. Signed-off-by: Christian Ebner --- src/PVE/CLIFormatter.pm

Re: [pve-devel] [PATCH cluster 4/5] Add cluster join API version check

On 11/20/19 5:43 PM, Stefan Reiter wrote: > Adds API call GET /cluster/config/apiversion to retrieve remote clusters > join-API version (0 is assumed for versions without this endpoint). > > Warn user if remote version differs, and select new fallback method only > if available. This ensures full

Re: [pve-devel] [PATCH cluster 5/5] corosync: check if IPs are configured on cluster create

On 11/20/19 5:43 PM, Stefan Reiter wrote: > Same as we do in assert_joinable, a cluster with unconfigured IPs will > fail start after creation anyway. > > Make "check_ip" a standalone sub ("check_ip_configured") and improve > error messages all around. > > Also move call to create_conf up, so if

[pve-devel] [PATCH v2 access-control 01/23] user.cfg: ensure propagate flag is 1/0 when parsing

otherwise this might end up as (arbitrary) string somewhere.. Signed-off-by: Fabian Grünbichler --- PVE/AccessControl.pm | 2 ++ 1 file changed, 2 insertions(+) diff --git a/PVE/AccessControl.pm b/PVE/AccessControl.pm index aff9137..2a4fe14 100644 --- a/PVE/AccessControl.pm +++ b/PVE/AccessCont

[pve-devel] [PATCH v2 access-control 03/23] pveum: add list commands

we already have the API paths, and they make sense to get an overview over user.cfg contents. Signed-off-by: Fabian Grünbichler --- PVE/CLI/pveum.pm | 11 +++ 1 file changed, 11 insertions(+) diff --git a/PVE/CLI/pveum.pm b/PVE/CLI/pveum.pm index c394535..537215f 100755 --- a/PVE/CLI/pv

[pve-devel] [PATCH v2 access-control 04/23] access-control: remove check_permissions/permission

they have been handled by PVE::RPCEnvironment for quite some time already, and the versions there are the complete ones that should be actually used. Signed-off-by: Fabian Grünbichler --- Notes: v1->v2: * add versioned breaks on pve-manager PVE/AccessControl.pm | 37 --

[pve-devel] [PATCH v2 access-control 07/23] refactor acl transformation code

pull it into helper sub, since we need this one more time for token ACL members. Signed-off-by: Fabian Grünbichler --- Notes: v1->v2: - rename helper and variable - also merge propagate/non-propagate loops PVE/AccessControl.pm | 74 +++- 1 fi

[pve-devel] [PATCH v2 access-control 13/23] API: add group and token info to user index

otherwise we need 1+N API calls to retrieve the full user+token picture Signed-off-by: Fabian Grünbichler --- Notes: new in v2 PVE/API2/User.pm | 21 +++-- 1 file changed, 19 insertions(+), 2 deletions(-) diff --git a/PVE/API2/User.pm b/PVE/API2/User.pm index fb985dd..0fc8

[pve-devel] [PATCH v2 access-control 14/23] API: include API tokens in ACL API endpoints

Signed-off-by: Fabian Grünbichler --- PVE/API2/ACL.pm | 30 ++ 1 file changed, 22 insertions(+), 8 deletions(-) diff --git a/PVE/API2/ACL.pm b/PVE/API2/ACL.pm index 3e42ac0..c340267 100644 --- a/PVE/API2/ACL.pm +++ b/PVE/API2/ACL.pm @@ -46,7 +46,7 @@ __PACKAGE__->regi

[pve-devel] [PATCH v2 access-control 10/23] API token: add (shadow) TokenConfig

with the format: it is just used for token value generation/deletion via the User API, token value verification will happen over pmxcfs/ipcc. Signed-off-by: Fabian Grünbichler --- Notes: new in v2 requires versioned build- and runtime-dependency on pve-cluster with priv/toke

[pve-devel] [PATCH v2 cluster 2/2] cluster: add priv/token.cfg to observed files

to allow creation of reader/writer in pve-access-control Signed-off-by: Fabian Grünbichler --- data/PVE/Cluster.pm | 1 + 1 file changed, 1 insertion(+) diff --git a/data/PVE/Cluster.pm b/data/PVE/Cluster.pm index e888ae8..1a5a634 100644 --- a/data/PVE/Cluster.pm +++ b/data/PVE/Cluster.pm @@ -6

[pve-devel] [PATCH v2 access-control 21/23] API: add group members to group index

Signed-off-by: Fabian Grünbichler --- PVE/API2/Group.pm | 7 +++ 1 file changed, 7 insertions(+) diff --git a/PVE/API2/Group.pm b/PVE/API2/Group.pm index 37f8be2..c463bd6 100644 --- a/PVE/API2/Group.pm +++ b/PVE/API2/Group.pm @@ -38,6 +38,12 @@ __PACKAGE__->register_method ({ pro

[pve-devel] [PATCH v2 access-control 08/23] API token: add REs, helpers, parsing + writing

token definitions/references in user.cfg always use the full form of the token id, consisting of: USER@REALM!TOKENID token definitions are represented by their own lines prefixed with 'token', which need to come after the corresponding user definition, but before any ACLs referencing them. parse

[pve-devel] [PATCH v2 access-control 02/23] user.cfg: sort group and pool members, role privs

makes no functional difference, but keeps the output/written config more stable. Signed-off-by: Fabian Grünbichler --- PVE/AccessControl.pm | 8 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/PVE/AccessControl.pm b/PVE/AccessControl.pm index 2a4fe14..1ba1596 100644 --- a/

[pve-devel] [PATCH v2 access-control 20/23] tests: unify config file naming

Signed-off-by: Fabian Grünbichler --- test/perm-test1.pl | 2 +- test/{user.cfg.ex1 => test1.cfg} | 0 2 files changed, 1 insertion(+), 1 deletion(-) rename test/{user.cfg.ex1 => test1.cfg} (100%) diff --git a/test/perm-test1.pl b/test/perm-test1.pl index e1bf1c7..12c95db 100755 -

[pve-devel] [PATCH v2 cluster 1/2] pmxcfs: add verify_token IPCC request

Signed-off-by: Fabian Grünbichler --- data/src/cfs-ipc-ops.h | 2 ++ data/src/server.c | 58 ++ data/src/status.c | 1 + data/PVE/Cluster.pm| 18 + 4 files changed, 79 insertions(+) diff --git a/data/src/cfs-ipc-ops.h b/data/src

[pve-devel] [PATCH v2 access-control 05/23] rpcenv: drop unused roles()

it was useful for test-cases to verify the behaviour when pools where introduced, but it is not used anywhere else in the code base and those tests can also just check on permission-level. Signed-off-by: Fabian Grünbichler --- Notes: alternatively, we can give this the same semantics w.r.t.

[pve-devel] [PATCH v2 access-control 09/23] API token: add check_token_exist API helper

the helper is modeled after the corresponding user method. the 'tokenid' option goes into PVE::AccessControl, since we need it in multiple API modules. Signed-off-by: Fabian Grünbichler --- Notes: v1->v2: - remove enabled helper (since flag was removed) - drop brackets in error mess

[pve-devel] [PATCH v2 access-control 15/23] API token: implement permission checks

non-privsep tokens will always return the roles/permissions of their associated users. privsep tokens will return unfiltered roles, but filtered permissions. Signed-off-by: Fabian Grünbichler --- Notes: v1->v2: - fix for root@pam tokens PVE/AccessControl.pm | 30 ++

[pve-devel] [PATCH v2 access-control 06/23] auth: pull username REs into variables

for reusage in API token ID format/verification Signed-off-by: Fabian Grünbichler --- PVE/Auth/Plugin.pm | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/PVE/Auth/Plugin.pm b/PVE/Auth/Plugin.pm index 5c11991..6d59b72 100755 --- a/PVE/Auth/Plugin.pm +++ b/PVE/Auth/Plugin.p

[pve-devel] [PATCH v2 access-control 16/23] api: mark some paths notoken

Signed-off-by: Fabian Grünbichler --- Notes: requires versioned dependency on libpve-common-perl requires a versioned depends on libpve-common-perl PVE/API2/AccessControl.pm | 4 1 file changed, 4 insertions(+) diff --git a/PVE/API2/AccessControl.pm b/PVE/API2/AccessControl.p

[pve-devel] [PATCH v2 common 1/1] API schema: add 'notoken' property

to mark API methods which should not be available to clients authenticated using an API token Signed-off-by: Fabian Grünbichler --- Notes: if applied, any users of this need corresponding versioned depends. src/PVE/JSONSchema.pm | 5 + 1 file changed, 5 insertions(+) diff --git a/src

[pve-devel] [PATCH v2 access-control 11/23] API token: add verification method

which checks that the user and token exist and are not expired, and then generates the string to be matched with the pmxcfs-stored token shadow config file. Signed-off-by: Fabian Grünbichler --- Notes: new in v2 requires versioned dependency on pve-cluster with PVE::Cluster::verify_

[pve-devel] [PATCH v2 access-control 19/23] API token: add tests

Signed-off-by: Fabian Grünbichler --- Notes: new in v2 test/Makefile | 1 + test/perm-test8.pl | 68 ++ test/test8.cfg | 28 +++ 3 files changed, 97 insertions(+) create mode 100644 test/perm-test8.pl create mode 100644

[pve-devel] [PATCH v2 manager 3/9] pveproxy: use new cookie extraction method

we only care about the regular cookie case for the index. Signed-off-by: Fabian Grünbichler --- Notes: versioned breaks/depends on libpve-http-perl! PVE/Service/pveproxy.pm | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/PVE/Service/pveproxy.pm b/PVE/Service/pveproxy.pm

[pve-devel] [PATCH v2 manager 5/9] www: add 'users' columns to Groups model

Signed-off-by: Fabian Grünbichler --- www/manager6/dc/GroupView.js | 6 ++ www/manager6/form/GroupSelector.js | 8 +++- 2 files changed, 13 insertions(+), 1 deletion(-) diff --git a/www/manager6/dc/GroupView.js b/www/manager6/dc/GroupView.js index dc863cc6..c40c5ba1 100644 --- a/ww

[pve-devel] [PATCH v2 access-control 23/23] pveum: add permissions sub-commands

for user and token commands, and some pretty-printing for regular text output, since the returned nested hash/dict is not very readable. Signed-off-by: Fabian Grünbichler --- PVE/CLI/pveum.pm | 66 1 file changed, 66 insertions(+) diff --git a/PV

[pve-devel] [PATCH v2 access-control 12/23] API: add API token API endpoints

and integration for user API endpoints. Signed-off-by: Fabian Grünbichler --- Notes: v1->v2: - adapted API schema somewhat - actually allow privileged users to view/modify/delete tokens of other users - remove enable flag from token schema - make API calls protected if they

[pve-devel] [PATCH v2 access-control 22/23] pveum: add 'pveum user token add/update/remove/list'

mapping 1-to-1 to the respective API paths Signed-off-by: Fabian Grünbichler --- PVE/CLI/pveum.pm | 6 ++ 1 file changed, 6 insertions(+) diff --git a/PVE/CLI/pveum.pm b/PVE/CLI/pveum.pm index 537215f..c642f6d 100755 --- a/PVE/CLI/pveum.pm +++ b/PVE/CLI/pveum.pm @@ -51,6 +51,12 @@ our $cmdd

[pve-devel] [PATCHSET v2] API Tokens

this is v2 of a patch set which aims to introduce API tokens into PVE. the basic idea is to allow users to generate API token values that - are attributed to this users - easily revokable - possibly less privileged than the user itself - allow direct API calls without round-trips to create/refresh

[pve-devel] [PATCH v2 manager 2/9] rest_handler: implement 'notoken' API endpoints

that are not available with API tokens for security reasons, such as access control related endpoints. Signed-off-by: Fabian Grünbichler --- Notes: pairs with patch in pve-common that adds this to the schema-schema. any modules setting that flag need a corresponding versioned depends o

[pve-devel] [PATCH v2 access-control 18/23] API: add 'permissions' API endpoint

and related helper, to dump permissions + propagate info for - a specific, given path - generic top-level + user.cfg-referenced paths, including pools Signed-off-by: Fabian Grünbichler --- Notes: new in v2 PVE/API2/AccessControl.pm | 56 +++ PVE/RPCEnvir

[pve-devel] [PATCH v2 http-server 1/2] allow ticket in auth header as fallback

From: Tim Marx based on idea & RFC by Tim Marx, incorporating feedback by Thomas Lamprecht. this will be extended to support API tokens in the Authorization header as well, so make it generic. Signed-off-by: Fabian Grünbichler --- Notes: semi-independent, could also leave extract_auth_cook

[pve-devel] [PATCH v2 access-control 17/23] roles()/permissions(): also return propagate flag

this information is already available, but not exposed. we need it for dumping an effective permission tree of a given user/token. Signed-off-by: Fabian Grünbichler --- Notes: new in v2 PVE/AccessControl.pm | 28 +--- PVE/RPCEnvironment.pm | 31

[pve-devel] [PATCH v2 http-server 2/2] api-server: extract, set and handle API token header

Signed-off-by: Fabian Grünbichler --- Notes: versioned breaks/depends with pve-manager and part of PMG? PVE/APIServer/AnyEvent.pm| 25 ++--- PVE/APIServer/Formatter.pm | 9 + PVE/APIServer/Formatter/Bootstrap.pm | 1 + 3 files changed, 28

[pve-devel] [PATCH v2 manager 6/9] www: add permissions button to userview

Signed-off-by: Fabian Grünbichler --- www/manager6/Makefile | 1 + www/manager6/dc/PermissionView.js | 167 ++ www/manager6/dc/UserView.js | 14 ++- 3 files changed, 181 insertions(+), 1 deletion(-) create mode 100644 www/manager6/dc/PermissionVie

[pve-devel] [PATCH v2 manager 4/9] api/tasks: attribute token tasks to user

and store token ID in separate, currently unused member. Signed-off-by: Fabian Grünbichler --- Notes: versioned depends on libpve-access-control alternatively, we could also change the fork_worker signature and encode this inside the task information on disk, but that would be

[pve-devel] [PATCH v2 manager 8/9] www: add Token to ACL

Signed-off-by: Fabian Grünbichler --- www/manager6/dc/ACLView.js | 23 ++- 1 file changed, 22 insertions(+), 1 deletion(-) diff --git a/www/manager6/dc/ACLView.js b/www/manager6/dc/ACLView.js index 1322f952..d0efe22e 100644 --- a/www/manager6/dc/ACLView.js +++ b/www/manager6/

[pve-devel] [PATCH v2 manager 7/9] www: add Token Panel + Edit Window

modeled after UserView and related code. Signed-off-by: Fabian Grünbichler --- www/manager6/Makefile | 3 + www/manager6/dc/Config.js | 8 ++ www/manager6/dc/TokenEdit.js | 125 ++ www/manager6/dc/TokenView.js | 203 ++

[pve-devel] [PATCH v2 manager 1/9] auth_handler: handle API tokens

Signed-off-by: Fabian Grünbichler --- Notes: versioned breaks/depends between pve-manager and libpve-http-server-perl! versioned depends on libpve-access-control PVE/HTTPServer.pm | 55 ++- 1 file changed, 30 insertions(+), 25 deletions(-) di

[pve-devel] [PATCH v2 manager 9/9] www: add TokenView with fixed userid

Signed-off-by: Fabian Grünbichler --- www/manager6/Workspace.js| 10 +++ www/manager6/dc/TokenEdit.js | 2 +- www/manager6/dc/TokenView.js | 122 --- 3 files changed, 108 insertions(+), 26 deletions(-) diff --git a/www/manager6/Workspace.js b/www/manager6/W

[pve-devel] [PATCH v6 00/12] Add basics for custom CPU models

Based on the RFC and following on- and off-list discussion about custom CPU models [0]. In essence, this revised patch allows a user to specify custom CPU models in /etc/pve/cpu-models.conf (section-config style [1]), where VMs using that CPU model inherit details from the definition. This removes

[pve-devel] [PATCH v6 qemu-server 06/12] Verify VM-specific CPU configs seperately

$cpu_fmt is being reused for custom CPUs as well as VM-specific CPU settings. The "pve-vm-cpu-conf" format is introduced to verify a config specifically for use as VM-specific settings. "pve-cpu-conf" is registered for use in custom CPU API calls (where no additional checks are required). Signed-

[pve-devel] [PATCH v6 qemu-server 04/12] Adapt CPUConfig to handle custom models

Turn CPUConfig into a SectionConfig with parsing/writing support for custom CPU models. IO is handled using cfs. Namespacing will be provided using "custom-" prefix for custom model names (in VM config only, cpu-models.conf will contain unprefixed names). Signed-off-by: Stefan Reiter --- Depend

[pve-devel] [PATCH v6 cluster 01/12] Add "cpu-models.conf" to observed files

Holds information about custom CPU models, section config style. Used in qemu-server. Signed-off-by: Stefan Reiter --- data/PVE/Cluster.pm | 1 + data/src/status.c | 1 + 2 files changed, 2 insertions(+) diff --git a/data/PVE/Cluster.pm b/data/PVE/Cluster.pm index 2057162..cdb80e4 100644 ---

[pve-devel] [PATCH v6 qemu-server 08/12] Rework get_cpu_options and allow custom CPU models

If a cputype is custom (check via prefix), try to load options from the custom CPU model config, and set values accordingly. While at it, extract currently hardcoded values into seperate sub and add reasonings. Since the new flag resolving outputs flags in sorted order for consistency, adapt the

[pve-devel] [PATCH v6 qemu-server 05/12] Add overrides and convenience functions to CPUConfig

Add two overrides to avoid writing redundant information to the config file. get_custom_model is used to retrieve a custom model configuration by name. Signed-off-by: Stefan Reiter --- PVE/QemuServer/CPUConfig.pm | 62 + 1 file changed, 62 insertions(+) diff

[pve-devel] [PATCH v6 qemu-server 11/12] Include "-cpu" parameter with live-migration

This is required to support custom CPU models, since the "cpu-models.conf" file is not versioned, and can be changed while a VM using a custom model is running. Changing the file in such a state can lead to a different "-cpu" argument on the receiving side. This patch fixes this by passing the ent

[pve-devel] [PATCH v6 qemu-server 03/12] Add CPUConfig file and migrate some helpers

The package will be used for custom CPU models as a SectionConfig, hence the name. For now we simply move some CPU related helper functions and declarations over from QemuServer to reduce clutter there. Exports are to avoid changing all call sites, functions have useful names on their own. Signed

[pve-devel] [PATCH v6 qemu-server 10/12] cfg2cmd: add test cases for custom CPU models

Requires a mock CPU-model config, which is given as a raw string to also test parsing capabilities. Also tests defaulting behaviour. Signed-off-by: Stefan Reiter --- test/cfg2cmd/custom-cpu-model-defaults.conf | 8 ++ .../custom-cpu-model-defaults.conf.cmd| 24 + t

[pve-devel] [PATCH v6 manager 12/12] Broadcast supported CPU flags

pvestatd will check if the KVM version has changed using kvm_user_version (which automatically clears its cache if QEMU/KVM updates), and if it has, query supported CPU flags and broadcast them as key-value pairs to the cluster. If detection fails, we clear the kv-store and set up a delay (120s),

[pve-devel] [PATCH v6 qemu-server 07/12] Add helpers to better structure CPU option handling

To avoid hardcoding even more CPU-flag related things for custom CPU models, introduce a dynamic approach to resolving flags. resolve_cpu_flags takes a list of hashes (as documented in the comment) and resolves them to a valid "-cpu" argument without duplicates. This also helps by providing a reas

[pve-devel] [PATCH v6 qemu-server 02/12] Add QEMU CPU flag querying helpers

* query_understood_cpu_flags returns all flags that QEMU/KVM knows about * query_supported_cpu_flags returns all flags that QEMU/KVM can use on this particular host. To get supported flags, a temporary VM is started with QEMU, so we can issue the "query-cpu-model-expansion" QMP command. This is

[pve-devel] [PATCH v6 qemu-server 09/12] fix #2318: allow phys-bits and host-phys-bits CPU settings

Can be specified for a particular VM or via a custom CPU model (VM takes precedence). QEMU's default limit only allows up to 1TB of RAM per VM. Increasing the physical address bits available to a VM can fix this. Signed-off-by: Stefan Reiter --- PVE/QemuServer/CPUConfig.pm | 24

Re: [pve-devel] [PATCH v2 qemu-server 2/3] Avoid collisions of unused disks when doing online migration with --targetstorage

On November 4, 2019 11:23 am, Fabian Ebner wrote: > On 10/31/19 10:19 AM, Thomas Lamprecht wrote: >> On 10/30/19 10:54 AM, Fabian Ebner wrote: >>> Doing an online migration with --targetstorage and two unused disks with the >>> same name on different storages failed, because they would collide on t

[pve-devel] [PATCH container] apply pending changes in lxc poststop hook

apply pending changes after container is stopped (via API or systemctl), and update lxc config. also affects reboots from inside the container. (but in that case we don't try to update_lxc_config again if pending changes were already applied and lxc config was updated) Signed-off-by: Oguz Bektas

[pve-devel] applied: [PATCH container] apply pending changes in lxc poststop hook

On 11/21/19 5:48 PM, Oguz Bektas wrote: > apply pending changes after container is stopped (via API or systemctl), and > update lxc config. > > also affects reboots from inside the container. (but in that case we don't try > to update_lxc_config again if pending changes were already applied and lx