Re: [pve-devel] [PATCH 24/31] PVE: move snapshot cleanup into bottom half

2020-03-06 Thread Dietmar Maurer
> for the record, this could be squashed into "[PATCH 17/31] PVE: internal > snapshot async" no biggie, but if we already go for a cleanup round.. In the first step, I only merged backup related patches (keep the rest as they was). Hoped this makes review easier ... Once that is finished, I can

[pve-devel] applied: [PATCH manager] ui: dc/options: add max_workers setting

2020-03-06 Thread Thomas Lamprecht
Signed-off-by: Thomas Lamprecht --- www/manager6/dc/OptionView.js | 6 ++ 1 file changed, 6 insertions(+) diff --git a/www/manager6/dc/OptionView.js b/www/manager6/dc/OptionView.js index 6497648c..73dbfed7 100644 --- a/www/manager6/dc/OptionView.js +++ b/www/manager6/dc/OptionView.js @@ -217

[pve-devel] applied: [PATCH manager 1/3] fix #2611: gui: use backendUnit and pretty renderer

2020-03-06 Thread Thomas Lamprecht
On 2/25/20 4:32 PM, Stefan Reiter wrote: > Slightly confusing with the units: > * Backend stores values in KiB/s > * Frontend displays MiB/s (which is a good choice when considering > current generation network speeds as opposed to huge KiB numbers IMO) > * Renderer previously showed no units, wh

Re: [pve-devel] [PATCH qemu-server] Revert "resize_vm: request new size from storage after resizing"

2020-03-06 Thread Thomas Lamprecht
On 3/4/20 10:51 AM, Fabian Ebner wrote: > This reverts commit b5490d8a98e5e7328eb4cebb0ae0b60e6d406c38. > > When resizing a volume of a running VM, a qmp block_resize command > is issued. This is non-blocking, so the size on the storage immediately > after issuing the command might still be the ol

[pve-devel] applied-series (rebased): [PATCH pve-storage 1/6] PVE/Storage/Plugin.pm: add ctime for backup files

2020-03-06 Thread Thomas Lamprecht
On 3/4/20 11:46 AM, Dietmar Maurer wrote: > Signed-off-by: Dietmar Maurer > --- > PVE/Storage/Plugin.pm | 6 ++ > 1 file changed, 6 insertions(+) > I applied this series, rebased it a bit though: * replacing the file paths in the commit subjects with a bit human friendlier and shorter t

[pve-devel] applied: [PATCH pve-manager] www/manager6/storage/ContentView.js: consider new ctime value

2020-03-06 Thread Thomas Lamprecht
On 3/4/20 11:47 AM, Dietmar Maurer wrote: > Signed-off-by: Dietmar Maurer > --- > www/manager6/storage/ContentView.js | 6 +- > 1 file changed, 5 insertions(+), 1 deletion(-) > > diff --git a/www/manager6/storage/ContentView.js > b/www/manager6/storage/ContentView.js > index ffd38fb9..001ef

[pve-devel] applied: [PATCH http-server v2 1/1] increase allowed spiceport range

2020-03-06 Thread Thomas Lamprecht
On 3/3/20 10:47 AM, Dominik Csapak wrote: > by using the new spice_port_range sub from PVE::Tools > > Signed-off-by: Dominik Csapak > --- > PVE/APIServer/AnyEvent.pm | 3 ++- > 1 file changed, 2 insertions(+), 1 deletion(-) > > diff --git a/PVE/APIServer/AnyEvent.pm b/PVE/APIServer/AnyEvent.pm

Re: [pve-devel] [PATCH 0/5] fix #2264: add virtio-rng device

2020-03-06 Thread Thomas Lamprecht
On 2/20/20 6:10 PM, Stefan Reiter wrote: > Add the ability to provice entropy to QEMU guests via an emulated hardware > RNG. > > First patch in pve-manager is independent. Rest depends on new qemu-server for > the API, I left 'onlineHelp' as a FIXME to avoid the dependecy to pve-docs. > > I look

Re: [pve-devel] [PATCH 24/31] PVE: move snapshot cleanup into bottom half

2020-03-06 Thread Thomas Lamprecht
On 3/6/20 12:30 PM, Dietmar Maurer wrote: > From: Wolfgang Bumiller > > as per: > (0ceccd858a8d) migration: qemu_savevm_state_cleanup() in cleanup > > may affect held locks and therefore change assumptions made > by that function! > > Signed-off-by: Wolfgang Bumiller > Signed-off-by: Thoma

[pve-devel] applied-series: [PATCH qemu-server 0/2] add new cpumodels

2020-03-06 Thread Thomas Lamprecht
On 3/5/20 4:15 PM, Alexandre Derumier wrote: > first patch is to add icelake cpu models (introduce in 2018) > > second patch is for qemu 4.2, new notsx variant. > Maybe can we already apply it to not forget it? applied both patches, thanks! > (and maybe add add ",hle=off,rtm=off," as cpuflags

[pve-devel] applied: [PATCH qemu-server] fix #2612: allow input-data in guest exec and make command optional

2020-03-06 Thread Thomas Lamprecht
On 2/27/20 11:47 AM, Stefan Reiter wrote: > 'input-data' can be used to pass arbitrary data to a guest when running > an agent command with 'guest-exec'. Most guest-agent implementations > treat this as STDIN to the command given by "path"/"arg", but some go as > far as relying solely on this param

Re: [pve-devel] [PATCH pve-qemu] security patches for libslirp CVE-2020-8608

2020-03-06 Thread Thomas Lamprecht
On 2/6/20 3:25 PM, Oguz Bektas wrote: > original commits and email can be found here[0] > > A out-of-bounds heap buffer access issue was found in the SLiRP > networking implementation of the QEMU emulator. It occurs in tcp_emu() > routine while emulating IRC and other protocols due to unsafe usage

Re: [pve-devel] [PATCH v2] Revision of the pvesr documentation

2020-03-06 Thread Aaron Lauterer
Looks good to me. I encountered two things, one missed insert and a misspelled you->your that I introduced in my comment on the v1. Sorry for that :/ I marked them inline Did some changes get dropped or applied in another patch? The v1 had some changes after line 66. They seem to be missing in

[pve-devel] [PATCH 31/31] PVE-Backup - proxmox backup patches for qemu

2020-03-06 Thread Dietmar Maurer
--- blockdev.c| 823 ++ hmp-commands-info.hx | 13 + hmp-commands.hx | 31 ++ include/monitor/hmp.h | 3 + monitor/hmp-cmds.c| 69 qapi/block-core.json | 91 + qapi/common.json | 13 + qapi/misc.json| 13

[pve-devel] [PATCH 30/31] PVE-Backup: add backup-dump block driver

2020-03-06 Thread Dietmar Maurer
- add backup-dump block driver block/backup-dump.c - move BackupBlockJob declaration from block/backup.c to include/block/block_int.h - block/backup.c - backup-job-create: also consider source cluster size - block/io.c - bdrv_do_drained_begin_quiesce: check for coroutine - job.c: make job_should_p

[pve-devel] [PATCH 16/31] PVE: qapi: modify spice query

2020-03-06 Thread Dietmar Maurer
From: Wolfgang Bumiller Provide the last ticket in the SpiceInfo struct optionally. Signed-off-by: Thomas Lamprecht --- qapi/ui.json| 3 +++ ui/spice-core.c | 5 + 2 files changed, 8 insertions(+) diff --git a/qapi/ui.json b/qapi/ui.json index e04525d8b4..6127990e23 100644 --- a/qapi/

[pve-devel] [PATCH 19/31] PVE: backup: modify job api

2020-03-06 Thread Dietmar Maurer
From: Wolfgang Bumiller Introduce a pause_count parameter to start a backup in paused mode. This way backups of multiple drives can be started up sequentially via the completion callback while having been started at the same point in time. Signed-off-by: Thomas Lamprecht --- block/backup.c

[pve-devel] [PATCH 13/31] PVE: [Up] qemu-img dd : add -n skip_create

2020-03-06 Thread Dietmar Maurer
From: Alexandre Derumier Signed-off-by: Thomas Lamprecht --- qemu-img.c | 23 ++- 1 file changed, 14 insertions(+), 9 deletions(-) diff --git a/qemu-img.c b/qemu-img.c index 8da1ea3951..ea3edb4f04 100644 --- a/qemu-img.c +++ b/qemu-img.c @@ -4535,7 +4535,7 @@ static int img

[pve-devel] [PATCH 18/31] PVE: block: add the zeroinit block driver filter

2020-03-06 Thread Dietmar Maurer
From: Wolfgang Bumiller Signed-off-by: Thomas Lamprecht --- block/Makefile.objs | 1 + block/zeroinit.c| 204 2 files changed, 205 insertions(+) create mode 100644 block/zeroinit.c diff --git a/block/Makefile.objs b/block/Makefile.objs index

[pve-devel] [PATCH 20/31] PVE: Add dummy -id command line parameter

2020-03-06 Thread Dietmar Maurer
From: Wolfgang Bumiller This used to be part of the qemu-side PVE authentication for VNC. Now this does nothing. Signed-off-by: Wolfgang Bumiller Signed-off-by: Thomas Lamprecht --- qemu-options.hx | 3 +++ vl.c| 8 2 files changed, 11 insertions(+) diff --git a/qemu-opt

[pve-devel] [PATCH 27/31] PVE: [Compat]: 4.0 used balloon qemu-4-0-config-size false here

2020-03-06 Thread Dietmar Maurer
From: Thomas Lamprecht The underlying issue why this change from upstream to us arised in the first place is that QEMU 4.0 was already released at the point we run into this migration issue, so we did the then obvious fallback to false for virtio-balloon-device qemu-4-0-config-size. QEMU made th

[pve-devel] [PATCH 29/31] PVE-Backup - add vma code

2020-03-06 Thread Dietmar Maurer
--- Makefile | 3 +- Makefile.objs | 1 + vma-reader.c | 857 ++ vma-writer.c | 771 + vma.c | 837 vma.h | 150 + 6 files

[pve-devel] [PATCH 23/31] PVE: savevm-async: kick AIO wait on block state write

2020-03-06 Thread Dietmar Maurer
From: Thomas Lamprecht Signed-off-by: Thomas Lamprecht --- savevm-async.c | 1 + 1 file changed, 1 insertion(+) diff --git a/savevm-async.c b/savevm-async.c index 5a20009b9a..e4bb0d24b2 100644 --- a/savevm-async.c +++ b/savevm-async.c @@ -157,6 +157,7 @@ static void coroutine_fn block_state_wr

[pve-devel] [PATCH 26/31] PVE: Acquire aio_context before calling block_job_add_bdrv

2020-03-06 Thread Dietmar Maurer
From: Stefan Reiter Otherwise backups immediately fail with 'permission denied' since _add_bdrv tries to release a lock we don't own. Signed-off-by: Stefan Reiter --- blockjob.c | 10 ++ 1 file changed, 10 insertions(+) diff --git a/blockjob.c b/blockjob.c index c6e20e2fcd..4e6074f18c

[pve-devel] [PATCH 24/31] PVE: move snapshot cleanup into bottom half

2020-03-06 Thread Dietmar Maurer
From: Wolfgang Bumiller as per: (0ceccd858a8d) migration: qemu_savevm_state_cleanup() in cleanup may affect held locks and therefore change assumptions made by that function! Signed-off-by: Wolfgang Bumiller Signed-off-by: Thomas Lamprecht --- savevm-async.c | 3 ++- 1 file changed, 2 in

[pve-devel] [PATCH 22/31] PVE: [Up+Config] file-posix: make locking optiono on create

2020-03-06 Thread Dietmar Maurer
From: Wolfgang Bumiller Otherwise creating images on nfs/cifs can be problematic. Signed-off-by: Wolfgang Bumiller Signed-off-by: Thomas Lamprecht --- block/file-posix.c | 61 +--- qapi/block-core.json | 3 ++- 2 files changed, 43 insertions(+), 21 d

[pve-devel] [PATCH 25/31] PVE: monitor: disable oob capability

2020-03-06 Thread Dietmar Maurer
From: Wolfgang Bumiller A bisect revealed that commit 8258292e18c3 ("monitor: Remove "x-oob", offer capability "oob" unconditionally") causes unexpected hangs when restoring live snapshots from some types of block devices (particularly RBD). We need to figure out what's happnening there. For now,

[pve-devel] [PATCH 14/31] PVE: virtio-balloon: improve query-balloon

2020-03-06 Thread Dietmar Maurer
From: Wolfgang Bumiller Actually provide memory information via the query-balloon command. Signed-off-by: Thomas Lamprecht --- hw/virtio/virtio-balloon.c | 33 +++-- monitor/hmp-cmds.c | 30 +- qapi/misc.json | 22

[pve-devel] [PATCH 21/31] PVE: [Config] Revert "target-i386: disable LINT0 after reset"

2020-03-06 Thread Dietmar Maurer
From: Wolfgang Bumiller This reverts commit b8eb5512fd8a115f164edbbe897cdf8884920ccb. Signed-off-by: Thomas Lamprecht --- hw/intc/apic_common.c | 9 + 1 file changed, 9 insertions(+) diff --git a/hw/intc/apic_common.c b/hw/intc/apic_common.c index 375cb6abe9..e7d479c7e9 100644 --- a/h

[pve-devel] [PATCH 11/31] PVE: [Up] qemu-img dd: add osize and read from/to stdin/stdout

2020-03-06 Thread Dietmar Maurer
From: Wolfgang Bumiller Neither convert nor dd were previously able to write to or read from a pipe. Particularly serializing an image file into a raw stream or vice versa can be useful, but using `qemu-img convert -f qcow2 -O raw foo.qcow2 /dev/stdout` in a pipe will fail trying to seek. While

[pve-devel] [PATCH 17/31] PVE: internal snapshot async

2020-03-06 Thread Dietmar Maurer
Signed-off-by: Thomas Lamprecht Signed-off-by: Dietmar Maurer --- Makefile.objs| 1 + hmp-commands-info.hx | 13 + hmp-commands.hx | 32 +++ include/migration/snapshot.h | 1 + include/monitor/hmp.h| 5 + monitor/hmp-cmds.c | 57 +++

[pve-devel] [PATCH 28/31] PVE: Allow version code in machine type

2020-03-06 Thread Dietmar Maurer
E.g. pc-i440fx-4.0+pve3 would print 'pve3' as version code while selecting pc-i440fx-4.0 as machine type. Version is made available as 'pve-version' in query-machines (same as, and only if 'is-current'). Signed-off-by: Stefan Reiter --- hw/core/machine-qmp-cmds.c | 6 ++ include/hw/boards.

[pve-devel] [PATCH 15/31] PVE: qapi: modify query machines

2020-03-06 Thread Dietmar Maurer
provide '*is-current' in MachineInfo struct Signed-off-by: Thomas Lamprecht Signed-off-by: Dietmar Maurer --- hw/core/machine-qmp-cmds.c | 6 ++ qapi/machine.json | 4 +++- 2 files changed, 9 insertions(+), 1 deletion(-) diff --git a/hw/core/machine-qmp-cmds.c b/hw/core/machine-qm

[pve-devel] [PATCH 07/31] PVE: [Config] rbd: block: rbd: disable rbd_cache_writethrough_until_flush

2020-03-06 Thread Dietmar Maurer
From: Wolfgang Bumiller Either the cache mode asks for a cache or not. There's no point in having a "temporary" cache mode. This option AFAIK was introduced as a hack for ancient virtio drivers. If anything, we should have a separate option for it. Better yet, VMs affected by the related issue sh

[pve-devel] [PATCH 05/31] PVE: [Config] smm_available = false

2020-03-06 Thread Dietmar Maurer
From: Alexandre Derumier Signed-off-by: Alexandre Derumier Signed-off-by: Thomas Lamprecht --- hw/i386/pc.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/hw/i386/pc.c b/hw/i386/pc.c index ac08e63604..4bd9ab52a0 100644 --- a/hw/i386/pc.c +++ b/hw/i386/pc.c @@ -2040,7 +2040

[pve-devel] [PATCH 06/31] PVE: [Config] glusterfs: no default logfile if daemonized

2020-03-06 Thread Dietmar Maurer
From: Wolfgang Bumiller Signed-off-by: Thomas Lamprecht --- block/gluster.c | 15 +++ 1 file changed, 11 insertions(+), 4 deletions(-) diff --git a/block/gluster.c b/block/gluster.c index 4fa4a77a47..bfb57ba098 100644 --- a/block/gluster.c +++ b/block/gluster.c @@ -42,7 +42,7 @@ #

[pve-devel] [PATCH 10/31] PVE: [Up] qemu-img: return success on info without snapshots

2020-03-06 Thread Dietmar Maurer
From: Wolfgang Bumiller Signed-off-by: Thomas Lamprecht --- qemu-img.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/qemu-img.c b/qemu-img.c index 95a24b9762..12211bed76 100644 --- a/qemu-img.c +++ b/qemu-img.c @@ -2791,7 +2791,8 @@ static int img_info(int argc, char **a

[pve-devel] [PATCH 03/31] PVE: [Config] set the CPU model to kvm64/32 instead of qemu64/32

2020-03-06 Thread Dietmar Maurer
From: Wolfgang Bumiller Signed-off-by: Thomas Lamprecht --- target/i386/cpu.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/target/i386/cpu.h b/target/i386/cpu.h index cde2a16b94..3e73104bf9 100644 --- a/target/i386/cpu.h +++ b/target/i386/cpu.h @@ -1940,9 +1940,9 @@ u

[pve-devel] [PATCH 08/31] PVE: [Up] qmp: add get_link_status

2020-03-06 Thread Dietmar Maurer
From: Wolfgang Bumiller Signed-off-by: Thomas Lamprecht --- net/net.c | 27 +++ qapi/net.json | 15 +++ qapi/qapi-schema.json | 1 + 3 files changed, 43 insertions(+) diff --git a/net/net.c b/net/net.c index 84aa6d8d00..f548202ec6 100644

[pve-devel] [PATCH 12/31] PVE: [Up] qemu-img dd: add isize parameter

2020-03-06 Thread Dietmar Maurer
From: Wolfgang Bumiller for writing small images from stdin to bigger ones In order to distinguish between an actually unexpected and an expected end of input. Signed-off-by: Wolfgang Bumiller Signed-off-by: Thomas Lamprecht --- qemu-img.c | 29 ++--- 1 file changed,

[pve-devel] [PATCH 04/31] PVE: [Config] ui/spice: default to pve certificates

2020-03-06 Thread Dietmar Maurer
From: Wolfgang Bumiller Signed-off-by: Thomas Lamprecht --- ui/spice-core.c | 15 +-- 1 file changed, 9 insertions(+), 6 deletions(-) diff --git a/ui/spice-core.c b/ui/spice-core.c index ecc2ec2c55..ca04965ead 100644 --- a/ui/spice-core.c +++ b/ui/spice-core.c @@ -668,32 +668,35 @@

[pve-devel] [PATCH 09/31] PVE: [Up] glusterfs: allow partial reads

2020-03-06 Thread Dietmar Maurer
From: Wolfgang Bumiller This should deal with qemu bug #1644754 until upstream decides which way to go. The general direction seems to be away from sector based block APIs and with that in mind, and when comparing to other network block backends (eg. nfs) treating partial reads as errors doesn't

[pve-devel] [PATCH 01/31] PVE: [Config] block/file: change locking default to off

2020-03-06 Thread Dietmar Maurer
From: Wolfgang Bumiller 'auto' only checks whether the system generally supports OFD locks but not whether the storage the file resides on supports any locking, causing issues with NFS. Signed-off-by: Wolfgang Bumiller Signed-off-by: Thomas Lamprecht --- block/file-posix.c | 4 ++-- 1 file ch

[pve-devel] [PATCH 02/31] PVE: [Config] Adjust network script path to /etc/kvm/

2020-03-06 Thread Dietmar Maurer
From: Wolfgang Bumiller Signed-off-by: Thomas Lamprecht --- include/net/net.h | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/include/net/net.h b/include/net/net.h index e175ba9677..5b9f099d21 100644 --- a/include/net/net.h +++ b/include/net/net.h @@ -208,8 +208,9 @@ vo

[pve-devel] [PATCH 00/31 qemu] PVE qemu patches rebased for qemu 4.2.0

2020-03-06 Thread Dietmar Maurer
Hi all, recent changes in qemu made it necessary to restructure our backup patches. I now use a special block driver which calls the backup_dump callback. I merged all backup related code into the last 3 patches. Alexandre Derumier (2): PVE: [Config] smm_available = false PVE: [Up] qemu-img

[pve-devel] [PATCH http-server 2/4] websocket_proxy: pull out encoding into helper sub

2020-03-06 Thread Fabian Grünbichler
to allow adding other types of frames. Signed-off-by: Fabian Grünbichler --- PVE/APIServer/AnyEvent.pm | 50 ++- 1 file changed, 28 insertions(+), 22 deletions(-) diff --git a/PVE/APIServer/AnyEvent.pm b/PVE/APIServer/AnyEvent.pm index 9f432e1..7147d81 100644

[pve-devel] [PATCH qemu-server 1/4] mtunnel: add API endpoints

2020-03-06 Thread Fabian Grünbichler
the following three endpoints are used for migration on the remote side GET /nodes/NODE/qemu/VMID/mtunnel returns identifier for this migration and ticket. both are passed to the other two endpoints to correspond that calls belong to the same migration run, and that permissions have been properly

[pve-devel] [PATCH qemu-server 2/4] migrate: add remote parameter

2020-03-06 Thread Fabian Grünbichler
Signed-off-by: Fabian Grünbichler --- PVE/API2/Qemu.pm | 29 +++-- 1 file changed, 23 insertions(+), 6 deletions(-) diff --git a/PVE/API2/Qemu.pm b/PVE/API2/Qemu.pm index 24f0dfd..a3592ed 100644 --- a/PVE/API2/Qemu.pm +++ b/PVE/API2/Qemu.pm @@ -35,6 +35,7 @@ use PVE::API2

[pve-devel] [PATCH http-server 3/4] websocket_proxy: implement ping/pong support

2020-03-06 Thread Fabian Grünbichler
needed to keep tunnel connections alive. Signed-off-by: Fabian Grünbichler --- PVE/APIServer/AnyEvent.pm | 5 + 1 file changed, 5 insertions(+) diff --git a/PVE/APIServer/AnyEvent.pm b/PVE/APIServer/AnyEvent.pm index 7147d81..bc2ec4d 100644 --- a/PVE/APIServer/AnyEvent.pm +++ b/PVE/APIServe

[pve-devel] [PoC] remote migration via websocket tunneling

2020-03-06 Thread Fabian Grünbichler
this is a rough proof of concept for - remote cluster/node configuration - generic tunneling via websocket support in our regular API server - basic (live) migration to a remote node/cluster there are plenty of not-yet-implemented parts and rough edges, but I'd appreciate input on whether we want

[pve-devel] [PATCH common] add websocket client implementation

2020-03-06 Thread Fabian Grünbichler
based on existing one in pve-client's LXC console handling code by René Jochum. it probably would make sense to generalize this further to allow code sharing with pve-http-server's server implementation. Signed-off-by: Fabian Grünbichler --- src/Makefile | 3 +- src/PVE/WebSocket.pm |

[pve-devel] [PATCH http-server 4/4] websocket_proxy: fix indentation

2020-03-06 Thread Fabian Grünbichler
Signed-off-by: Fabian Grünbichler --- PVE/APIServer/AnyEvent.pm | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/PVE/APIServer/AnyEvent.pm b/PVE/APIServer/AnyEvent.pm index bc2ec4d..8b049d4 100644 --- a/PVE/APIServer/AnyEvent.pm +++ b/PVE/APIServer/AnyEvent.pm @@ -507,9 +

[pve-devel] [PATCH cluster 2/2] add get_remote_info

2020-03-06 Thread Fabian Grünbichler
Signed-off-by: Fabian Grünbichler --- data/PVE/RemoteConfig.pm | 55 1 file changed, 55 insertions(+) diff --git a/data/PVE/RemoteConfig.pm b/data/PVE/RemoteConfig.pm index 23274de..7c395ba 100644 --- a/data/PVE/RemoteConfig.pm +++ b/data/PVE/RemoteConfig

[pve-devel] [PATCH manager] API: add node address(es) API endpoint

2020-03-06 Thread Fabian Grünbichler
Signed-off-by: Fabian Grünbichler --- PVE/API2/Nodes.pm | 70 +++ 1 file changed, 70 insertions(+) diff --git a/PVE/API2/Nodes.pm b/PVE/API2/Nodes.pm index 58497b2b..206fc423 100644 --- a/PVE/API2/Nodes.pm +++ b/PVE/API2/Nodes.pm @@ -241,6 +241,7 @@ __

[pve-devel] [PATCH qemu-server 4/4] implement PoC migration to remote cluster/node

2020-03-06 Thread Fabian Grünbichler
there's obviously lots of TODOs and FIXMEs in here, the big ones are: - better handling of storage switching - handling of network switching - implementing cleanup - actually dropping the local/source config and disks - NBD client side is kept open by Qemu, so we need to terminate the data tunnel i

[pve-devel] [PATCH cluster 1/2] remote.cfg: add new config file

2020-03-06 Thread Fabian Grünbichler
Signed-off-by: Fabian Grünbichler --- the secret 'token' could of course also be split out to a second file, or just masked when returned via the API data/PVE/Makefile | 2 +- data/src/status.c | 1 + data/PVE/Cluster.pm| 1 + data/PVE/Remo

[pve-devel] [PATCH qemu-server 3/4] nbd: listen on UNIX socket

2020-03-06 Thread Fabian Grünbichler
extracted from WIP patch by Mira Limbeck. as-is, this is a very breaking change, but it should be possible to integrate it in a backwards compatible fashion. Signed-off-by: Fabian Grünbichler --- PVE/QemuServer.pm | 7 +-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/PVE/Qem

[pve-devel] applied: [PATCH proxmox-i18n] update fr.po

2020-03-06 Thread Thomas Lamprecht
On 3/5/20 1:54 PM, Alexandre Derumier wrote: > --- > fr.po | 94 --- > 1 file changed, 51 insertions(+), 43 deletions(-) > applied, thanks! ___ pve-devel mailing list pve-devel@pve.proxmox.com h

[pve-devel] [PATCH http-server 1/4] websocket: improve masking performance

2020-03-06 Thread Fabian Grünbichler
in order to make websocket proxying feasible as general tunnel, we need to be able to transfer more than a few MB/s Signed-off-by: Fabian Grünbichler --- PVE/APIServer/AnyEvent.pm | 18 +- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/PVE/APIServer/AnyEvent.pm b/P

[pve-devel] [PATCH access-control 9/9] Domains: add sync API call

2020-03-06 Thread Dominik Csapak
this api call syncs the users and groups from LDAP/AD to the user.cfg, by default only users, but can be configured it also implements a 'prune' mode where we first delete all users/groups from the config and sync them again also add this command to pveum Signed-off-by: Dominik Csapak --- PVE/

[pve-devel] [PATCH pmg-api 1/2] use new PVE::LDAP instead of Net::LDAP directly

2020-03-06 Thread Dominik Csapak
for things like connecting/binding/etc. Signed-off-by: Dominik Csapak --- src/PMG/LDAPCache.pm | 299 ++- 1 file changed, 93 insertions(+), 206 deletions(-) diff --git a/src/PMG/LDAPCache.pm b/src/PMG/LDAPCache.pm index ca18ad6..987f8bc 100755 --- a/src/P

[pve-devel] [PATCH pmg-api 2/2] remove ldap-simple-attr

2020-03-06 Thread Dominik Csapak
which is now in pve-common Signed-off-by: Dominik Csapak --- src/PMG/LDAPConfig.pm | 13 - 1 file changed, 13 deletions(-) diff --git a/src/PMG/LDAPConfig.pm b/src/PMG/LDAPConfig.pm index 9e4fd9e..a6cd6ef 100644 --- a/src/PMG/LDAPConfig.pm +++ b/src/PMG/LDAPConfig.pm @@ -12,19 +12,6

[pve-devel] [PATCH access-control 3/9] API2/Domains.pm: fix whitespace errors

2020-03-06 Thread Dominik Csapak
Signed-off-by: Dominik Csapak --- PVE/API2/Domains.pm | 52 ++--- 1 file changed, 26 insertions(+), 26 deletions(-) diff --git a/PVE/API2/Domains.pm b/PVE/API2/Domains.pm index b5fa65e..0e8bcf3 100644 --- a/PVE/API2/Domains.pm +++ b/PVE/API2/Domains.pm @@

[pve-devel] [PATCH access-control 4/9] API2/Domains.pm: document 'type' return value

2020-03-06 Thread Dominik Csapak
this way it gets printed with 'pveum realm list' Signed-off-by: Dominik Csapak --- PVE/API2/Domains.pm | 1 + 1 file changed, 1 insertion(+) diff --git a/PVE/API2/Domains.pm b/PVE/API2/Domains.pm index 0e8bcf3..7f98e71 100644 --- a/PVE/API2/Domains.pm +++ b/PVE/API2/Domains.pm @@ -34,6 +34,7 @@

[pve-devel] [PATCH access-control 2/9] add realm commands to pveum

2020-03-06 Thread Dominik Csapak
Signed-off-by: Dominik Csapak --- PVE/CLI/pveum.pm | 9 + 1 file changed, 9 insertions(+) diff --git a/PVE/CLI/pveum.pm b/PVE/CLI/pveum.pm index 9cae774..d3721b6 100755 --- a/PVE/CLI/pveum.pm +++ b/PVE/CLI/pveum.pm @@ -10,6 +10,7 @@ use PVE::API2::Group; use PVE::API2::Role; use PVE::A

[pve-devel] [PATCH common 1/3] ldap: optionally save group name by attribute

2020-03-06 Thread Dominik Csapak
Signed-off-by: Dominik Csapak --- src/PVE/LDAP.pm | 9 +++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/src/PVE/LDAP.pm b/src/PVE/LDAP.pm index cb88977..5bf52dd 100644 --- a/src/PVE/LDAP.pm +++ b/src/PVE/LDAP.pm @@ -176,7 +176,7 @@ sub query_users { } sub query_groups {

[pve-devel] [PATCH access-control 7/9] Auth/LDAP: add get_{users, groups} subs for syncing

2020-03-06 Thread Dominik Csapak
this adds the subs which actually query the LDAP for users/groups and returns the value in format which makes it easy to insert in our parsed user.cfg when we find a user/groupname which cannot be in our config, we warn the verification error for groups, we append "-$realm" to the groupname, to l

[pve-devel] [PATCH common 2/3] ldap: add optional classes to query_users

2020-03-06 Thread Dominik Csapak
and filter by it Signed-off-by: Dominik Csapak --- src/PVE/LDAP.pm | 10 +- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/src/PVE/LDAP.pm b/src/PVE/LDAP.pm index 5bf52dd..661e3a3 100644 --- a/src/PVE/LDAP.pm +++ b/src/PVE/LDAP.pm @@ -94,7 +94,7 @@ sub auth_user_dn { }

[pve-devel] [PATCH access-control 8/9] Auth/AD: make PVE::Auth::AD a subclass of PVE::Auth::LDAP

2020-03-06 Thread Dominik Csapak
this makes it much easier to reuse the sync code from LDAP in AD. The 'authenticate_user' sub is still the same, but we now can still use the get_users and get_groups functionality of LDAP in the case of AD, the user_attr is optional in the config (would have been a breaking change) but we set it

[pve-devel] [RFC PATCH common/access-control/pmg-api] implement ldap/ad sync

2020-03-06 Thread Dominik Csapak
this series implements basic ldap/ad user/group sync via api/cli a new api call for realms called 'sync' is implemented which calls the plugins 'get_{user,group}' sub which in turn uses the realms config to get the relevant users/groups and this is then written into the user config there are some

[pve-devel] [PATCH access-control 1/9] use PVE::LDAP module instead of useing Net::LDAP directly

2020-03-06 Thread Dominik Csapak
for things like connecting/binding/etc. Signed-off-by: Dominik Csapak --- PVE/Auth/AD.pm | 44 + PVE/Auth/LDAP.pm | 64 +++- 2 files changed, 32 insertions(+), 76 deletions(-) diff --git a/PVE/Auth/AD.pm b/PVE/Auth/AD

[pve-devel] [PATCH access-control 5/9] Auth/LDAP: refactor out 'connect_and_bind'

2020-03-06 Thread Dominik Csapak
we will use this not only for authentication but also for getting users/groups from LDAP Signed-off-by: Dominik Csapak --- PVE/Auth/LDAP.pm | 18 -- 1 file changed, 16 insertions(+), 2 deletions(-) diff --git a/PVE/Auth/LDAP.pm b/PVE/Auth/LDAP.pm index 0faa40a..5eef12c 100755 --

[pve-devel] [PATCH common 3/3] add ldap-simple-attr from pmg

2020-03-06 Thread Dominik Csapak
we also need it in pve Signed-off-by: Dominik Csapak --- src/PVE/JSONSchema.pm | 13 + 1 file changed, 13 insertions(+) diff --git a/src/PVE/JSONSchema.pm b/src/PVE/JSONSchema.pm index 599dd09..fa405ac 100644 --- a/src/PVE/JSONSchema.pm +++ b/src/PVE/JSONSchema.pm @@ -265,6 +265,19

[pve-devel] [PATCH access-control 6/9] Auth/LDAP: add necessary options for syncing

2020-03-06 Thread Dominik Csapak
for syncing users/groups from ldap, we need some more options so that the users can adapt it to their LDAP setup, which are very different accross systems. sensible defaults are documented Signed-off-by: Dominik Csapak --- PVE/Auth/LDAP.pm | 58