Re: [pve-devel] [PATCH v2 qemu-server 2/2] fix clone_disk failing for nonexistent cloudinit disk

2020-10-06 Thread Mira Limbeck



On 10/5/20 5:35 PM, Thomas Lamprecht wrote:

On 28.09.20 10:36, Mira Limbeck wrote:

After migration or a rollback the cloudinit disk might not be allocated, so
volume_size_info() fails. As we override the value anyway for cloudinit
and efi disks simply move the volume_size_info() call into the 'else'
branch.

Signed-off-by: Mira Limbeck 
---
v2: changed subject

  PVE/QemuServer.pm | 4 +++-
  1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm
index 2747c66..49765b7 100644
--- a/PVE/QemuServer.pm
+++ b/PVE/QemuServer.pm
@@ -6895,10 +6895,10 @@ sub clone_disk {
$storeid = $storage if $storage;
  
  	my $dst_format = resolve_dst_disk_format($storecfg, $storeid, $volname, $format);

-   my ($size) = PVE::Storage::volume_size_info($storecfg, $drive->{file}, 
3);
  
  	print "create full clone of drive $drivename ($drive->{file})\n";

my $name = undef;
+   my $size = undef;
if (drive_is_cloudinit($drive)) {
$name = "vm-$newvmid-cloudinit";
$name .= ".$dst_format" if $dst_format ne 'raw';
@@ -6906,6 +6906,8 @@ sub clone_disk {
$size = PVE::QemuServer::Cloudinit::CLOUDINIT_DISK_SIZE;
} elsif ($drivename eq 'efidisk0') {
$size = get_efivars_size($conf);
+   } else {
+   ($size) = PVE::Storage::volume_size_info($storecfg, $drive->{file}, 
3);
}
$size /= 1024;

doesn't this logs a "use of undefined value in division" or something like that
somewhere in the non-else case?
No, in the cloudinit case we set it to a constant. In the efidisk case 
we call get_efivars_size() which dies if efivars is not a file, 
otherwise we get a size (-s). And in the else case we also die if we 
can't get the size. So size is set in every case before the first use.



$newvolid = PVE::Storage::vdisk_alloc($storecfg, $storeid, $newvmid, 
$dst_format, $name, $size);




___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel



Re: [pve-devel] [PATCH v2 qemu-server 2/2] fix clone_disk failing for nonexistent cloudinit disk

2020-10-06 Thread Mira Limbeck

On 10/6/20 10:56 AM, Mira Limbeck wrote:


On 10/5/20 5:35 PM, Thomas Lamprecht wrote:

On 28.09.20 10:36, Mira Limbeck wrote:
After migration or a rollback the cloudinit disk might not be 
allocated, so

volume_size_info() fails. As we override the value anyway for cloudinit
and efi disks simply move the volume_size_info() call into the 'else'
branch.

Signed-off-by: Mira Limbeck 
---
v2: changed subject

  PVE/QemuServer.pm | 4 +++-
  1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm
index 2747c66..49765b7 100644
--- a/PVE/QemuServer.pm
+++ b/PVE/QemuServer.pm
@@ -6895,10 +6895,10 @@ sub clone_disk {
  $storeid = $storage if $storage;
    my $dst_format = resolve_dst_disk_format($storecfg, 
$storeid, $volname, $format);
-    my ($size) = PVE::Storage::volume_size_info($storecfg, 
$drive->{file}, 3);
    print "create full clone of drive $drivename 
($drive->{file})\n";

  my $name = undef;
+    my $size = undef;
  if (drive_is_cloudinit($drive)) {
  $name = "vm-$newvmid-cloudinit";
  $name .= ".$dst_format" if $dst_format ne 'raw';
@@ -6906,6 +6906,8 @@ sub clone_disk {
  $size = PVE::QemuServer::Cloudinit::CLOUDINIT_DISK_SIZE;
  } elsif ($drivename eq 'efidisk0') {
  $size = get_efivars_size($conf);
+    } else {
+    ($size) = PVE::Storage::volume_size_info($storecfg, 
$drive->{file}, 3);

  }
  $size /= 1024;
doesn't this logs a "use of undefined value in division" or something 
like that

somewhere in the non-else case?
No, in the cloudinit case we set it to a constant. In the efidisk case 
we call get_efivars_size() which dies if efivars is not a file, 
otherwise we get a size (-s). And in the else case we also die if we 
can't get the size. So size is set in every case before the first use.
To clarify, the returned size in volume_size_info can be undefined, but 
not the other 2 cases. Would a die be a good idea in case the 
volume_size_info call returns 'undef'?


  $newvolid = PVE::Storage::vdisk_alloc($storecfg, $storeid, 
$newvmid, $dst_format, $name, $size);





___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel





___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] [PATCH pve-manager] pvestatd: stream host pressure counters

2020-10-06 Thread Alexandre Derumier
Signed-off-by: Alexandre Derumier 
---
 PVE/Service/pvestatd.pm | 4 
 1 file changed, 4 insertions(+)

diff --git a/PVE/Service/pvestatd.pm b/PVE/Service/pvestatd.pm
index 5e533ca3..238c36ae 100755
--- a/PVE/Service/pvestatd.pm
+++ b/PVE/Service/pvestatd.pm
@@ -161,12 +161,16 @@ sub update_node_status {
 );
 PVE::Cluster::broadcast_rrd("pve2-node/$nodename", $data);
 
+my $pressure = PVE::ProcFSTools::read_pressure();
+PVE::Cluster::broadcast_node_kv("pressure", encode_json($pressure));
+
 my $node_metric = {
uptime => $uptime,
cpustat => $stat,
memory => $meminfo,
blockstat => $dinfo,
nics => $netdev,
+   pressure => $pressure,
 };
 $node_metric->{cpustat}->@{qw(avg1 avg5 avg15)} = ($avg1, $avg5, $avg15);
 $node_metric->{cpustat}->{cpus} = $maxcpu;
-- 
2.20.1


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel



[pve-devel] [PATCH pve-common 0/1] ProcFSTools: add read_pressure

2020-10-06 Thread Alexandre Derumier
Hi,

I'm currently working on vm load balancing scheduler.

This patch add new pressure counters, very usefull to known
if a node is overloaded, with more granularity than loadaverage.


Alexandre Derumier (1):
  ProcFSTools: add read_pressure

 src/PVE/ProcFSTools.pm | 18 ++
 1 file changed, 18 insertions(+)

-- 
2.20.1


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel



[pve-devel] [PATCH pve-common 1/1] ProcFSTools: add read_pressure

2020-10-06 Thread Alexandre Derumier
read new /proc/pressure/(cpu,disk,io) introduced in kernel 4.20.

This give more granular informations than loadaverage.

Signed-off-by: Alexandre Derumier 
---
 src/PVE/ProcFSTools.pm | 18 ++
 1 file changed, 18 insertions(+)

diff --git a/src/PVE/ProcFSTools.pm b/src/PVE/ProcFSTools.pm
index 7cf1472..7687c13 100644
--- a/src/PVE/ProcFSTools.pm
+++ b/src/PVE/ProcFSTools.pm
@@ -132,6 +132,24 @@ sub read_loadavg {
 return wantarray ? (0, 0, 0) : 0;
 }
 
+sub read_pressure {
+
+my $res = {};
+foreach my $type (qw(cpu memory io)) {
+   if (my $fh = IO::File->new ("/proc/pressure/$type", "r")) {
+   while (defined (my $line = <$fh>)) {
+   if ($line =~ 
/^(some|full)\s+avg10\=(\d+\.\d+)\s+avg60\=(\d+\.\d+)\s+avg300\=(\d+\.\d+)\s+total\=(\d+)/)
 {
+   $res->{$type}->{$1}->{avg10} = $2;
+   $res->{$type}->{$1}->{avg60} = $3;
+   $res->{$type}->{$1}->{avg300} = $4;
+   }
+   }
+   $fh->close;
+   }
+}
+return $res;
+}
+
 my $last_proc_stat;
 
 sub read_proc_stat {
-- 
2.20.1


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel



Re: [pve-devel] [PVE-User] Proxmox Backup Server (beta)

2020-10-06 Thread Lee Lists
Hi,

I'm trying to build proxmox backup server from source,
but the build failed in compiling zstd lib bindings.

Any clues ?

Thanks,
Jurgen

   Fresh pxar v0.6.1 (/root/pxar)
   Fresh proxmox-fuse v0.1.0 (/root/proxmox-fuse)
   Fresh hyper v0.13.8
   Compiling proxmox v0.4.2 (/root/proxmox/proxmox)
   Fresh bindgen v0.49.4
 Running `rustc --crate-name proxmox --edition=2018 
/root/proxmox/proxmox/src/lib.rs --error-format=json 
--json=diagnostic-rendered-ansi,artifacts --crate-type lib 
--emit=dep-info,metadata,link -C embed-bitcode=no -C debuginfo=2 --cfg 
'feature="api-macro"' --cfg 'feature="cli"' --cfg 'feature="default"' --cfg 
'feature="futures"' --cfg 'feature="hyper"' --cfg 'feature="openssl"' --cfg 
'feature="proxmox-api-macro"' --cfg 'feature="proxmox-sortable-macro"' --cfg 
'feature="router"' --cfg 'feature="sortable-macro"' --cfg 'feature="tokio"' 
--cfg 'feature="websocket"' -C metadata=494ea91d58d02b19 -C 
extra-filename=-494ea91d58d02b19 --out-dir 
/root/proxmox-backup/target/debug/deps -C 
incremental=/root/proxmox-backup/target/debug/incremental -L 
dependency=/root/proxmox-backup/target/debug/deps --extern 
anyhow=/root/proxmox-backup/target/debug/deps/libanyhow-547643689d8f1fe1.rmeta 
--extern 
base64=/root/proxmox-backup/target/debug/deps/libbase64-75b6df1cdb5dcedb.rmeta 
--extern bytes=/root/p
 roxmox-backup/target/debug/deps/libbytes-de79ab6ad237b260.rmeta --extern 
endian_trait=/root/proxmox-backup/target/debug/deps/libendian_trait-8410c7a3f7fc6de5.rmeta
 --extern 
futures=/root/proxmox-backup/target/debug/deps/libfutures-68cbc13a6c4e5d08.rmeta
 --extern 
http=/root/proxmox-backup/target/debug/deps/libhttp-4f2085239d8db6c5.rmeta 
--extern 
hyper=/root/proxmox-backup/target/debug/deps/libhyper-33e2f11afaf2d6cd.rmeta 
--extern 
lazy_static=/root/proxmox-backup/target/debug/deps/liblazy_static-9441bed367485869.rmeta
 --extern 
libc=/root/proxmox-backup/target/debug/deps/liblibc-85afcfd6d5dd745a.rmeta 
--extern 
nix=/root/proxmox-backup/target/debug/deps/libnix-19850f768394dcc5.rmeta 
--extern 
openssl=/root/proxmox-backup/target/debug/deps/libopenssl-a45d25e9645a7846.rmeta
 --extern 
percent_encoding=/root/proxmox-backup/target/debug/deps/libpercent_encoding-00fe2006917413e4.rmeta
 --extern 
proxmox_api_macro=/root/proxmox-backup/target/debug/deps/libproxmox_api_macro-45b1df18057a8628.so
 --ex
 tern 
proxmox_sortable_macro=/root/proxmox-backup/target/debug/deps/libproxmox_sortable_macro-e0ce43c23fa4803c.so
 --extern 
regex=/root/proxmox-backup/target/debug/deps/libregex-37b4c1de7b101096.rmeta 
--extern 
rustyline=/root/proxmox-backup/target/debug/deps/librustyline-201c56bc71ec2bb7.rmeta
 --extern 
serde=/root/proxmox-backup/target/debug/deps/libserde-75724b33e89dcb58.rmeta 
--extern 
serde_derive=/root/proxmox-backup/target/debug/deps/libserde_derive-e6d2c9cdac5acf10.so
 --extern 
serde_json=/root/proxmox-backup/target/debug/deps/libserde_json-caf74f34e0a23558.rmeta
 --extern 
textwrap=/root/proxmox-backup/target/debug/deps/libtextwrap-8dccd2a72ee64e9e.rmeta
 --extern 
tokio=/root/proxmox-backup/target/debug/deps/libtokio-8c2cdd714cabf70e.rmeta 
--extern 
url=/root/proxmox-backup/target/debug/deps/liburl-0b6b3b5adf147475.rmeta`
   Compiling zstd-sys v1.4.13+zstd.1.4.3
 Running 
`/root/proxmox-backup/target/debug/build/zstd-sys-0efc8671c6ad61e7/build-script-build`
error: failed to run custom build command for `zstd-sys v1.4.13+zstd.1.4.3`

Caused by:
  process didn't exit successfully: 
`/root/proxmox-backup/target/debug/build/zstd-sys-0efc8671c6ad61e7/build-script-build`
 (exit code: 101)
...
...
...
  running: "ar" "crs" 
"/root/proxmox-backup/target/debug/build/zstd-sys-c18e259b2d671f1b/out/libzstd.a"
 
"/root/proxmox-backup/target/debug/build/zstd-sys-c18e259b2d671f1b/out/zstd/lib/common/debug.o"
 
"/root/proxmox-backup/target/debug/build/zstd-sys-c18e259b2d671f1b/out/zstd/lib/common/entropy_common.o"
 
"/root/proxmox-backup/target/debug/build/zstd-sys-c18e259b2d671f1b/out/zstd/lib/common/error_private.o"
 
"/root/proxmox-backup/target/debug/build/zstd-sys-c18e259b2d671f1b/out/zstd/lib/common/fse_decompress.o"
 
"/root/proxmox-backup/target/debug/build/zstd-sys-c18e259b2d671f1b/out/zstd/lib/common/pool.o"
 
"/root/proxmox-backup/target/debug/build/zstd-sys-c18e259b2d671f1b/out/zstd/lib/common/threading.o"
 
"/root/proxmox-backup/target/debug/build/zstd-sys-c18e259b2d671f1b/out/zstd/lib/common/xxhash.o"
 
"/root/proxmox-backup/target/debug/build/zstd-sys-c18e259b2d671f1b/out/zstd/lib/common/zstd_common.o"
 
"/root/proxmox-backup/target/debug/build/zstd-sys-c18e259b2d671f1b/out/zstd/lib/compress/f
 se_compress.o" 
"/root/proxmox-backup/target/debug/build/zstd-sys-c18e259b2d671f1b/out/zstd/lib/compress/hist.o"
 
"/root/proxmox-backup/target/debug/build/zstd-sys-c18e259b2d671f1b/out/zstd/lib/compress/huf_compress.o"
 
"/root/proxmox-backup/target/debug/build/zstd-sys-c18e259b2d671f1b/out/zstd/lib/compress/zstd_compress.o"
 
"/root/proxmox-backup/target/debug/bui

[pve-devel] [PATCH v2 qemu-server 5/7] api: add handling for new boot order format

2020-10-06 Thread Stefan Reiter
The API is updated to handle the deprecation correctly, i.e. when
updating the 'order' attribute, the old 'legacy' (default_key) values
are removed (would now be ignored anyway).

When removing a device that is in the bootorder list, it will be removed
from the aforementioned. Note that non-existing devices in the list will
not cause an error - they will simply be ignored - but it's still nice
to not have them in there.

Signed-off-by: Stefan Reiter 
---
 PVE/API2/Qemu.pm | 31 +++
 1 file changed, 31 insertions(+)

diff --git a/PVE/API2/Qemu.pm b/PVE/API2/Qemu.pm
index 0d82d3e..f1e9759 100644
--- a/PVE/API2/Qemu.pm
+++ b/PVE/API2/Qemu.pm
@@ -1191,6 +1191,12 @@ my $update_vm_api  = sub {
 
my $modified = {}; # record what $option we modify
 
+   my $bootcfg = PVE::JSONSchema::parse_property_string('pve-qm-boot', 
$conf->{boot})
+   if $conf->{boot};
+   my @bootorder = PVE::Tools::split_list($bootcfg->{order})
+   if $bootcfg && $bootcfg->{order};
+   my $bootorder_deleted = grep {$_ eq 'bootorder'} @delete;
+
foreach my $opt (@delete) {
$modified->{$opt} = 1;
$conf = PVE::QemuConfig->load_config($vmid); # update/reload
@@ -1205,6 +1211,13 @@ my $update_vm_api  = sub {
my $is_pending_val = defined($conf->{pending}->{$opt});
delete $conf->{pending}->{$opt};
 
+   # remove from bootorder if necessary
+   if (!$bootorder_deleted && @bootorder && grep {$_ eq $opt} 
@bootorder) {
+   @bootorder = grep {$_ ne $opt} @bootorder;
+   $conf->{pending}->{boot} = 
PVE::QemuServer::print_bootorder(\@bootorder);
+   $modified->{boot} = 1;
+   }
+
if ($opt =~ m/^unused/) {
my $drive = PVE::QemuServer::parse_drive($opt, $val);
PVE::QemuConfig->check_protection($conf, "can't remove 
unused disk '$drive->{file}'");
@@ -1283,6 +1296,24 @@ my $update_vm_api  = sub {
$conf->{pending}->{$opt} = $param->{$opt};
} else {
$conf->{pending}->{$opt} = $param->{$opt};
+
+   if ($opt eq 'boot') {
+   my $new_bootcfg = 
PVE::JSONSchema::parse_property_string('pve-qm-boot', $param->{$opt});
+   if ($new_bootcfg->{order}) {
+   my @devs = 
PVE::Tools::split_list($new_bootcfg->{order});
+   for my $dev (@devs) {
+   my $exists = $conf->{$dev} || 
$conf->{pending}->{$dev};
+   my $deleted = grep {$_ eq $dev} @delete;
+   die "invalid bootorder: device '$dev' does not 
exist'\n"
+   if !$exists || $deleted;
+   }
+
+   # remove legacy boot order settings if new one set
+   $conf->{pending}->{$opt} = 
PVE::QemuServer::print_bootorder(\@devs);
+   PVE::QemuConfig->add_to_pending_delete($conf, 
"bootdisk")
+   if $conf->{bootdisk};
+   }
+   }
}
PVE::QemuConfig->remove_from_pending_delete($conf, $opt);
PVE::QemuConfig->write_config($vmid, $conf);
-- 
2.20.1



___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel



[pve-devel] [PATCH v2 qemu-server 1/7] fix indentation

2020-10-06 Thread Stefan Reiter
Signed-off-by: Stefan Reiter 
---
 PVE/QemuServer.pm | 20 ++--
 1 file changed, 10 insertions(+), 10 deletions(-)

diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm
index 2747c66..bd59616 100644
--- a/PVE/QemuServer.pm
+++ b/PVE/QemuServer.pm
@@ -3468,24 +3468,24 @@ sub config_to_command {
 });
 
 for (my $i = 0; $i < $MAX_NETS; $i++) {
-next if !$conf->{"net$i"};
-my $d = parse_net($conf->{"net$i"});
-next if !$d;
+   next if !$conf->{"net$i"};
+   my $d = parse_net($conf->{"net$i"});
+   next if !$d;
 
-$use_virtio = 1 if $d->{model} eq 'virtio';
+   $use_virtio = 1 if $d->{model} eq 'virtio';
 
-if ($bootindex_hash->{n}) {
+   if ($bootindex_hash->{n}) {
$d->{bootindex} = $bootindex_hash->{n};
$bootindex_hash->{n} += 1;
-}
+   }
 
-my $netdevfull = print_netdev_full($vmid, $conf, $arch, $d, "net$i");
-push @$devices, '-netdev', $netdevfull;
+   my $netdevfull = print_netdev_full($vmid, $conf, $arch, $d, "net$i");
+   push @$devices, '-netdev', $netdevfull;
 
-my $netdevicefull = print_netdevice_full(
+   my $netdevicefull = print_netdevice_full(
$vmid, $conf, $d, "net$i", $bridges, $use_old_bios_files, $arch, 
$machine_type);
 
-push @$devices, '-device', $netdevicefull;
+   push @$devices, '-device', $netdevicefull;
 }
 
 if ($conf->{ivshmem}) {
-- 
2.20.1



___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel



[pve-devel] [PATCH v2 qemu-server 4/7] fix #3010: add 'bootorder' parameter for better control of boot devices

2020-10-06 Thread Stefan Reiter
(also fixes #3011)

Deprecates the old-style 'boot' and 'bootdisk' options by adding a new
'order=' subproperty to 'boot'.

This allows a user to specify more than one disk in the boot order,
helping with newer versions of SeaBIOS/OVMF where disks without a
bootindex won't be initialized at all (breaks soft-raid and some LVM
setups).

This also allows specifying a bootindex for USB and hostpci devices,
which was not possible before. Floppy boot support is not supported in
the new model, but I doubt that will be a problem (AFAICT we can't even
attach floppy disks to a VM?).

Default behaviour is intended to stay the same, i.e. while new VMs will
receive the new 'order' property, it will be set so the VM starts the
same as before (using get_default_bootorder).

Signed-off-by: Stefan Reiter 
---
 PVE/API2/Qemu.pm|  6 ++--
 PVE/CLI/qm.pm   |  4 +--
 PVE/QemuServer.pm   | 67 ++---
 PVE/QemuServer/Drive.pm | 21 +++--
 PVE/QemuServer/PCI.pm   |  3 +-
 PVE/QemuServer/USB.pm   | 14 ++---
 6 files changed, 64 insertions(+), 51 deletions(-)

diff --git a/PVE/API2/Qemu.pm b/PVE/API2/Qemu.pm
index 8da616a..0d82d3e 100644
--- a/PVE/API2/Qemu.pm
+++ b/PVE/API2/Qemu.pm
@@ -656,9 +656,9 @@ __PACKAGE__->register_method({
eval {
$vollist = &$create_disks($rpcenv, $authuser, $conf, $arch, 
$storecfg, $vmid, $pool, $param, $storage);
 
-   if (!$conf->{bootdisk}) {
-   my $firstdisk = 
PVE::QemuServer::Drive::resolve_first_disk($conf);
-   $conf->{bootdisk} = $firstdisk if $firstdisk;
+   if (!$conf->{boot}) {
+   my $devs = 
PVE::QemuServer::get_default_bootdevices($conf);
+   $conf->{boot} = PVE::QemuServer::print_bootorder($devs);
}
 
# auto generate uuid if user did not specify smbios1 option
diff --git a/PVE/CLI/qm.pm b/PVE/CLI/qm.pm
index 282fa86..6243b06 100755
--- a/PVE/CLI/qm.pm
+++ b/PVE/CLI/qm.pm
@@ -656,8 +656,8 @@ __PACKAGE__->register_method ({
 
# reload after disks entries have been created
$conf = PVE::QemuConfig->load_config($vmid);
-   my $firstdisk = PVE::QemuServer::Drive::resolve_first_disk($conf);
-   $conf->{bootdisk} = $firstdisk if $firstdisk;
+   my $devs = PVE::QemuServer::get_default_bootdevices($conf);
+   $conf->{boot} = PVE::QemuServer::print_bootorder($devs);
PVE::QemuConfig->write_config($vmid, $conf);
};
 
diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm
index cfac03a..8279571 100644
--- a/PVE/QemuServer.pm
+++ b/PVE/QemuServer.pm
@@ -397,15 +397,14 @@ EODESC
 },
 boot => {
optional => 1,
-   type => 'string',
-   description => "Boot on floppy (a), hard disk (c), CD-ROM (d), or 
network (n).",
-   pattern => '[acdn]{1,4}',
-   default => 'cdn',
+   type => 'string', format => 'pve-qm-boot',
+   description => "Specify guest boot order. Use with 'order=', usage with"
+. " no key or 'legacy=' is deprecated.",
 },
 bootdisk => {
optional => 1,
type => 'string', format => 'pve-qm-bootdisk',
-   description => "Enable booting from specified disk.",
+   description => "Enable booting from specified disk. Deprecated: Use 
'boot: order=foo;bar' instead.",
pattern => '(ide|sata|scsi|virtio)\d+',
 },
 smp => {
@@ -1614,8 +1613,6 @@ sub print_drive_commandline_full {
 sub print_netdevice_full {
 my ($vmid, $conf, $net, $netid, $bridges, $use_old_bios_files, $arch, 
$machine_type) = @_;
 
-my $bootorder = $conf->{boot} || $confdesc->{boot}->{default};
-
 my $device = $net->{model};
 if ($net->{model} eq 'virtio') {
  $device = 'virtio-net-pci';
@@ -3213,17 +3210,30 @@ sub config_to_command {
push @$devices, '-device', $kbd if defined($kbd);
 }
 
+my $bootorder = {};
+my $boot = parse_property_string($boot_fmt, $conf->{boot}) if 
$conf->{boot};
+if (!defined($boot) || $boot->{legacy}) {
+   $bootorder = bootorder_from_legacy($conf, $boot);
+} elsif ($boot->{order}) {
+   # start at 100 to allow user to insert devices before us with -args
+   my $i = 100;
+   for my $dev (PVE::Tools::split_list($boot->{order})) {
+   $bootorder->{$dev} = $i++;
+   }
+}
+
 # host pci device passthrough
 my ($kvm_off, $gpu_passthrough, $legacy_igd) = 
PVE::QemuServer::PCI::print_hostpci_devices(
-   $vmid, $conf, $devices, $winversion, $q35, $bridges, $arch, 
$machine_type);
+   $vmid, $conf, $devices, $winversion, $q35, $bridges, $arch, 
$machine_type, $bootorder);
 
 # usb devices
 my $usb_dev_features = {};
 $usb_dev_features->{spice_usb3} = 1 if min_version($machine_version, 4, 0);
 
 my @usbdevices = PVE::QemuServer::USB::get_usb_devices(
-$conf, $usbdesc->{format

[pve-devel] [PATCH v2 qemu-server 3/7] add new 'boot' property format and introduce legacy conversion helpers

2020-10-06 Thread Stefan Reiter
The format is unused in this commit, but will replace the current
string-based format of the 'boot' property. It is included since the
parameter of bootorder_from_legacy follows it.

Two helper methods are introduced:
* bootorder_from_legacy: Parses the legacy format into a hash closer to
what the new format represents
* get_default_bootdevices: Encapsulates the legacy default behaviour if
nothing is specified in the boot order

resolve_first_disk is simplified and gets a new $cdrom parameter to
control the behaviour of excluding CD-ROMs or instead searching for only
them.

Signed-off-by: Stefan Reiter 
---
 PVE/QemuServer.pm   | 130 
 PVE/QemuServer/Drive.pm |  11 ++--
 2 files changed, 135 insertions(+), 6 deletions(-)

diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm
index bd59616..cfac03a 100644
--- a/PVE/QemuServer.pm
+++ b/PVE/QemuServer.pm
@@ -1091,6 +1091,68 @@ for (my $i = 0; $i < $MAX_USB_DEVICES; $i++)  {
 $confdesc->{"usb$i"} = $usbdesc;
 }
 
+my $boot_fmt = {
+legacy => {
+   optional => 1,
+   default_key => 1,
+   type => 'string',
+   description => "Boot on floppy (a), hard disk (c), CD-ROM (d), or 
network (n)."
+. " Deprecated, use 'order=' instead.",
+   pattern => '[acdn]{1,4}',
+   format_description => "[acdn]{1,4}",
+
+   # note: this is also the fallback if boot: is not given at all
+   default => 'cdn',
+},
+order => {
+   optional => 1,
+   type => 'string',
+   format => 'pve-qm-bootdev-list',
+   format_description => "device[;device...]",
+   description => <{$dev};
+   return 1;
+};
+
+return $dev if $check->("net");
+return $dev if $check->("usb");
+return $dev if $check->("hostpci");
+
+return undef if $noerr;
+die "invalid boot device '$dev'\n";
+}
+
+sub print_bootorder {
+my ($devs) = @_;
+my $data = { order => join(';', @$devs) };
+return PVE::JSONSchema::print_property_string($data, $boot_fmt);
+}
+
 my $kvm_api_version = 0;
 
 sub kvm_version {
@@ -7152,6 +7214,74 @@ sub clear_reboot_request {
 return $res;
 }
 
+sub bootorder_from_legacy {
+my ($conf, $bootcfg) = @_;
+
+my $boot = $bootcfg->{legacy} || $boot_fmt->{legacy}->{default};
+my $bootindex_hash = {};
+my $i = 1;
+foreach my $o (split(//, $boot)) {
+   $bootindex_hash->{$o} = $i*100;
+   $i++;
+}
+
+my $bootorder = {};
+
+PVE::QemuConfig->foreach_volume($conf, sub {
+   my ($ds, $drive) = @_;
+
+   if (drive_is_cdrom ($drive, 1)) {
+   if ($bootindex_hash->{d}) {
+   $bootorder->{$ds} = $bootindex_hash->{d};
+   $bootindex_hash->{d} += 1;
+   }
+   } elsif ($bootindex_hash->{c}) {
+   $bootorder->{$ds} = $bootindex_hash->{c}
+   if $conf->{bootdisk} && $conf->{bootdisk} eq $ds;
+   $bootindex_hash->{c} += 1;
+   }
+});
+
+if ($bootindex_hash->{n}) {
+   for (my $i = 0; $i < $MAX_NETS; $i++) {
+   my $netname = "net$i";
+   next if !$conf->{$netname};
+   $bootorder->{$netname} = $bootindex_hash->{n};
+   $bootindex_hash->{n} += 1;
+   }
+}
+
+return $bootorder;
+}
+
+# Generate default device list for 'boot: order=' property. Matches legacy
+# default boot order, but with explicit device names. This is important, since
+# the fallback for when neither 'order' nor the old format is specified relies
+# on 'bootorder_from_legacy' above, and it would be confusing if this diverges.
+sub get_default_bootdevices {
+my ($conf) = @_;
+
+my @ret = ();
+
+# harddisk
+my $first = PVE::QemuServer::Drive::resolve_first_disk($conf, 0);
+push @ret, $first if $first;
+
+# cdrom
+$first = PVE::QemuServer::Drive::resolve_first_disk($conf, 1);
+push @ret, $first if $first;
+
+# network
+for (my $i = 0; $i < $MAX_NETS; $i++) {
+   my $netname = "net$i";
+   next if !$conf->{$netname};
+   push @ret, $netname;
+   last;
+}
+
+return \@ret;
+}
+
 # bash completion helper
 
 sub complete_backup_archives {
diff --git a/PVE/QemuServer/Drive.pm b/PVE/QemuServer/Drive.pm
index 91c33f8..b71fc93 100644
--- a/PVE/QemuServer/Drive.pm
+++ b/PVE/QemuServer/Drive.pm
@@ -584,16 +584,15 @@ sub is_volume_in_use {
 }
 
 sub resolve_first_disk {
-my $conf = shift;
+my ($conf, $cdrom) = @_;
 my @disks = valid_drive_names();
-my $firstdisk;
-foreach my $ds (reverse @disks) {
+foreach my $ds (@disks) {
next if !$conf->{$ds};
my $disk = parse_drive($ds, $conf->{$ds});
-   next if drive_is_cdrom($disk);
-   $firstdisk = $ds;
+   next if drive_is_cdrom($disk) xor $cdrom;
+   return $ds;
 }
-return $firstdisk;
+return undef;
 }
 
 1;
-- 
2.20.1



___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.

[pve-devel] [PATCH v2 manager 7/7] ui: improve boot order editor

2020-10-06 Thread Stefan Reiter
The new boot order property can express many more scenarios than the old
one. Update the editor so it can handle it.

Features a grid with all supported boot devices which can be reordered
using drag-and-drop, as well as toggled on and off with an inline
checkbox.

Support for configs still using the old format is given, with the first
write automatically updating the VM config to use the new one.

The renderer for the Options panel is updated with support for the new
format.

Note that it is very well possible to disable all boot devices, in which
case an empty 'boot: ' will be stored to the config file. I'm not sure
what that would be useful for, but there's no reason to forbid it
either, just warn the user that it's probably not what they want.

Signed-off-by: Stefan Reiter 
---

Depends on updated qemu-server for API support.

v2:
* improve GUI with two new columns
* make more schematic, putting less in initComponent
* use existing format for renderer in Options panel, instead of changing it
  (I opted to just use the existing one instead of changing it at all now,
  just updated to support the new format)
* update for new 'empty' (no bootdevs) behaviour
* update legacy conversion code to deal with 4 characters, ignoring 'a' (floppy)
  (this appears to be a never-triggered bug in the old implementation as well?)

 www/css/ext6-pve.css   |   4 +
 www/manager6/qemu/BootOrderEdit.js | 355 ++---
 www/manager6/qemu/Options.js   |  32 ++-
 3 files changed, 257 insertions(+), 134 deletions(-)

diff --git a/www/css/ext6-pve.css b/www/css/ext6-pve.css
index a91f1aaf..6430ffc4 100644
--- a/www/css/ext6-pve.css
+++ b/www/css/ext6-pve.css
@@ -583,6 +583,10 @@ table.osds td:first-of-type {
 cursor: pointer;
 }
 
+.cursor-move {
+cursor: move;
+}
+
 .x-grid-filters-filtered-column {
 font-style: italic;
 font-weight: bold;
diff --git a/www/manager6/qemu/BootOrderEdit.js 
b/www/manager6/qemu/BootOrderEdit.js
index 19d5d50a..c5726e96 100644
--- a/www/manager6/qemu/BootOrderEdit.js
+++ b/www/manager6/qemu/BootOrderEdit.js
@@ -1,150 +1,250 @@
+Ext.define('pve-boot-order-entry', {
+extend: 'Ext.data.Model',
+fields: [
+   {name: 'name', type: 'string'},
+   {name: 'enabled', type: 'bool'},
+   {name: 'desc', type: 'string'},
+]
+});
+
 Ext.define('PVE.qemu.BootOrderPanel', {
 extend: 'Proxmox.panel.InputPanel',
 alias: 'widget.pveQemuBootOrderPanel',
+
 vmconfig: {}, // store loaded vm config
+store: undefined,
 
-bootdisk: undefined,
-selection: [],
-list: [],
-comboboxes: [],
+inUpdate: false,
+controller: {
+   xclass: 'Ext.app.ViewController',
+},
 
-isBootDisk: function(value) {
+isDisk: function(value) {
return PVE.Utils.bus_match.test(value);
 },
 
-setVMConfig: function(vmconfig) {
-   var me = this;
-   me.vmconfig = vmconfig;
-   var order = me.vmconfig.boot || 'cdn';
-   me.bootdisk = me.vmconfig.bootdisk || undefined;
-
-   // get the first 3 characters
-   // ignore the rest (there should never be more than 3)
-   me.selection = order.split('').slice(0,3);
-
-   // build bootdev list
-   me.list = [];
-   Ext.Object.each(me.vmconfig, function(key, value) {
-   if (me.isBootDisk(key) &&
-   !(/media=cdrom/).test(value)) {
-   me.list.push([key, "Disk '" + key + "'"]);
-   }
-   });
-
-   me.list.push(['d', 'CD-ROM']);
-   me.list.push(['n', gettext('Network')]);
-   me.list.push(['__none__', Proxmox.Utils.noneText]);
-
-   me.recomputeList();
-
-   me.comboboxes.forEach(function(box) {
-   box.resetOriginalValue();
-   });
+isBootdev: function(dev, value) {
+   return this.isDisk(dev) ||
+   (/^net\d+/).test(dev) ||
+   (/^hostpci\d+/).test(dev) ||
+   ((/^usb\d+/).test(dev) && !(/spice/).test(value));
 },
 
-onGetValues: function(values) {
-   var me = this;
-   var order = me.selection.join('');
-   var res = { boot: order };
+setVMConfig: function(vmconfig) {
+   let me = this;
+   me.vmconfig = vmconfig;
 
-   if  (me.bootdisk && order.indexOf('c') !== -1) {
-   res.bootdisk = me.bootdisk;
-   } else {
-   res['delete'] = 'bootdisk';
+   me.store.removeAll();
+
+   let boot = PVE.Parser.parsePropertyString(me.vmconfig.boot, "legacy");
+
+   let bootorder = [];
+   if (boot.order) {
+   bootorder = boot.order.split(';').map(dev => ({name: dev, enabled: 
true}));
+   } else if (!(/^\s*$/).test(me.vmconfig.boot)) {
+   // legacy style, transform to new bootorder
+   let order = boot.legacy || 'cdn';
+   let bootdisk = me.vmconfig.bootdisk || undefined;
+
+   // get the first 4 characters (acdn)
+   // ignore the rest (there should never be more than 4)
+   let orderList = order.split(

[pve-devel] [PATCH v2 0/7] Improve boot device/order configuration

2020-10-06 Thread Stefan Reiter
The goal of the series is to modernize boot order selection by allowing the user
to specify arbitrary VM disks and devices in any order. Fixes #3010 and #3011.

First patch is a standalone cleanup. Tests are seperated into extra patches.

v2:
* split qemu-server changes into seperate patches (hence changelog only here)
* use order= property in boot: instead of standalone bootorder:
** update API and GUI to match
* fix get_default_bootdevices and bootorder_from_legacy to *actually* match the
old behaviour, as is now verified with the test
=> see GUI patch for changes there


qemu-server: Stefan Reiter (6):
  fix indentation
  cfg2cmd: add test for legacy-style bootorder
  add new 'boot' property format and introduce legacy conversion helpers
  fix #3010: add 'bootorder' parameter for better control of boot
devices
  api: add handling for new boot order format
  cfg2cmd: add tests for new boot order property

 PVE/API2/Qemu.pm   |  37 -
 PVE/CLI/qm.pm  |   4 +-
 PVE/QemuServer.pm  | 207 -
 PVE/QemuServer/Drive.pm|  32 +++-
 PVE/QemuServer/PCI.pm  |   3 +-
 PVE/QemuServer/USB.pm  |  14 +-
 test/cfg2cmd/bootorder-empty.conf  |  16 ++
 test/cfg2cmd/bootorder-empty.conf.cmd  |  38 +
 test/cfg2cmd/bootorder-legacy.conf |  17 ++
 test/cfg2cmd/bootorder-legacy.conf.cmd |  38 +
 test/cfg2cmd/bootorder.conf|  16 ++
 test/cfg2cmd/bootorder.conf.cmd|  38 +
 12 files changed, 398 insertions(+), 62 deletions(-)
 create mode 100644 test/cfg2cmd/bootorder-empty.conf
 create mode 100644 test/cfg2cmd/bootorder-empty.conf.cmd
 create mode 100644 test/cfg2cmd/bootorder-legacy.conf
 create mode 100644 test/cfg2cmd/bootorder-legacy.conf.cmd
 create mode 100644 test/cfg2cmd/bootorder.conf
 create mode 100644 test/cfg2cmd/bootorder.conf.cmd

manager: Stefan Reiter (1):
  ui: improve boot order editor with 'bootorder' support

 www/css/ext6-pve.css   |   4 +
 www/manager6/qemu/BootOrderEdit.js | 355 ++---
 www/manager6/qemu/Options.js   |  32 ++-
 3 files changed, 257 insertions(+), 134 deletions(-)

-- 
2.20.1


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel



[pve-devel] [PATCH v2 qemu-server 6/7] cfg2cmd: add tests for new boot order property

2020-10-06 Thread Stefan Reiter
Signed-off-by: Stefan Reiter 
---
 test/cfg2cmd/bootorder-empty.conf | 16 +++
 test/cfg2cmd/bootorder-empty.conf.cmd | 38 +++
 test/cfg2cmd/bootorder.conf   | 16 +++
 test/cfg2cmd/bootorder.conf.cmd   | 38 +++
 4 files changed, 108 insertions(+)
 create mode 100644 test/cfg2cmd/bootorder-empty.conf
 create mode 100644 test/cfg2cmd/bootorder-empty.conf.cmd
 create mode 100644 test/cfg2cmd/bootorder.conf
 create mode 100644 test/cfg2cmd/bootorder.conf.cmd

diff --git a/test/cfg2cmd/bootorder-empty.conf 
b/test/cfg2cmd/bootorder-empty.conf
new file mode 100644
index 000..75c6f33
--- /dev/null
+++ b/test/cfg2cmd/bootorder-empty.conf
@@ -0,0 +1,16 @@
+# TEST: Test for an empty boot parameter producing no bootindices either
+# QEMU_VERSION: 5.1
+cores: 3
+boot: 
+ide2: none,media=cdrom
+memory: 768
+name: simple
+net0: virtio=A2:C0:43:77:08:A0,bridge=vmbr0
+numa: 0
+ostype: l26
+scsi4: local:8006/vm-8006-disk-0.qcow2,discard=on,size=104858K
+smbios1: uuid=7b10d7af-b932-4c66-b2c3-3996152ec465
+sockets: 1
+virtio0: local:8006/vm-8006-disk-0.qcow2,discard=on,iothread=1,size=104858K
+virtio1: local:8006/vm-8006-disk-0.qcow2,discard=on,iothread=1,size=104858K
+vmgenid: c773c261-d800-4348-9f5d-167fadd53cf8
diff --git a/test/cfg2cmd/bootorder-empty.conf.cmd 
b/test/cfg2cmd/bootorder-empty.conf.cmd
new file mode 100644
index 000..1f2b2fb
--- /dev/null
+++ b/test/cfg2cmd/bootorder-empty.conf.cmd
@@ -0,0 +1,38 @@
+/usr/bin/kvm \
+  -id 8006 \
+  -name simple \
+  -chardev 'socket,id=qmp,path=/var/run/qemu-server/8006.qmp,server,nowait' \
+  -mon 'chardev=qmp,mode=control' \
+  -chardev 'socket,id=qmp-event,path=/var/run/qmeventd.sock,reconnect=5' \
+  -mon 'chardev=qmp-event,mode=control' \
+  -pidfile /var/run/qemu-server/8006.pid \
+  -daemonize \
+  -smbios 'type=1,uuid=7b10d7af-b932-4c66-b2c3-3996152ec465' \
+  -smp '3,sockets=1,cores=3,maxcpus=3' \
+  -nodefaults \
+  -boot 
'menu=on,strict=on,reboot-timeout=1000,splash=/usr/share/qemu-server/bootsplash.jpg'
 \
+  -vnc unix:/var/run/qemu-server/8006.vnc,password \
+  -cpu kvm64,enforce,+kvm_pv_eoi,+kvm_pv_unhalt,+lahf_lm,+sep \
+  -m 768 \
+  -object 'iothread,id=iothread-virtio0' \
+  -object 'iothread,id=iothread-virtio1' \
+  -device 'pci-bridge,id=pci.1,chassis_nr=1,bus=pci.0,addr=0x1e' \
+  -device 'pci-bridge,id=pci.2,chassis_nr=2,bus=pci.0,addr=0x1f' \
+  -device 'vmgenid,guid=c773c261-d800-4348-9f5d-167fadd53cf8' \
+  -device 'piix3-usb-uhci,id=uhci,bus=pci.0,addr=0x1.0x2' \
+  -device 'usb-tablet,id=tablet,bus=uhci.0,port=1' \
+  -device 'VGA,id=vga,bus=pci.0,addr=0x2' \
+  -device 'virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3' \
+  -iscsi 'initiator-name=iqn.1993-08.org.debian:01:aabbccddeeff' \
+  -drive 'if=none,id=drive-ide2,media=cdrom,aio=threads' \
+  -device 'ide-cd,bus=ide.1,unit=0,drive=drive-ide2,id=ide2' \
+  -device 'lsi,id=scsihw0,bus=pci.0,addr=0x5' \
+  -drive 
'file=/var/lib/vz/images/8006/vm-8006-disk-0.qcow2,if=none,id=drive-scsi4,discard=on,format=qcow2,cache=none,aio=native,detect-zeroes=unmap'
 \
+  -device 'scsi-hd,bus=scsihw0.0,scsi-id=4,drive=drive-scsi4,id=scsi4' \
+  -drive 
'file=/var/lib/vz/images/8006/vm-8006-disk-0.qcow2,if=none,id=drive-virtio0,discard=on,format=qcow2,cache=none,aio=native,detect-zeroes=unmap'
 \
+  -device 
'virtio-blk-pci,drive=drive-virtio0,id=virtio0,bus=pci.0,addr=0xa,iothread=iothread-virtio0'
 \
+  -drive 
'file=/var/lib/vz/images/8006/vm-8006-disk-0.qcow2,if=none,id=drive-virtio1,discard=on,format=qcow2,cache=none,aio=native,detect-zeroes=unmap'
 \
+  -device 
'virtio-blk-pci,drive=drive-virtio1,id=virtio1,bus=pci.0,addr=0xb,iothread=iothread-virtio1'
 \
+  -netdev 
'type=tap,id=net0,ifname=tap8006i0,script=/var/lib/qemu-server/pve-bridge,downscript=/var/lib/qemu-server/pve-bridgedown,vhost=on'
 \
+  -device 
'virtio-net-pci,mac=A2:C0:43:77:08:A0,netdev=net0,bus=pci.0,addr=0x12,id=net0' \
+  -machine 'type=pc+pve0'
diff --git a/test/cfg2cmd/bootorder.conf b/test/cfg2cmd/bootorder.conf
new file mode 100644
index 000..8de64bf
--- /dev/null
+++ b/test/cfg2cmd/bootorder.conf
@@ -0,0 +1,16 @@
+# TEST: Test for a specific bootorder given by 'boot: order=' property
+# QEMU_VERSION: 5.1
+cores: 3
+boot: order=virtio1;net0;scsi4;ide2
+ide2: none,media=cdrom
+memory: 768
+name: simple
+net0: virtio=A2:C0:43:77:08:A0,bridge=vmbr0
+numa: 0
+ostype: l26
+scsi4: local:8006/vm-8006-disk-0.qcow2,discard=on,size=104858K
+smbios1: uuid=7b10d7af-b932-4c66-b2c3-3996152ec465
+sockets: 1
+virtio0: local:8006/vm-8006-disk-0.qcow2,discard=on,iothread=1,size=104858K
+virtio1: local:8006/vm-8006-disk-0.qcow2,discard=on,iothread=1,size=104858K
+vmgenid: c773c261-d800-4348-9f5d-167fadd53cf8
diff --git a/test/cfg2cmd/bootorder.conf.cmd b/test/cfg2cmd/bootorder.conf.cmd
new file mode 100644
index 000..86cae07
--- /dev/null
+++ b/test/cfg2cmd/bootorder.conf.cmd
@@ -0,0 +1,38 @@
+/usr/bin/kvm \
+  -id 8006 \
+  -name simple \
+  -char

[pve-devel] [PATCH v2 qemu-server 2/7] cfg2cmd: add test for legacy-style bootorder

2020-10-06 Thread Stefan Reiter
Signed-off-by: Stefan Reiter 
---
 test/cfg2cmd/bootorder-legacy.conf | 17 
 test/cfg2cmd/bootorder-legacy.conf.cmd | 38 ++
 2 files changed, 55 insertions(+)
 create mode 100644 test/cfg2cmd/bootorder-legacy.conf
 create mode 100644 test/cfg2cmd/bootorder-legacy.conf.cmd

diff --git a/test/cfg2cmd/bootorder-legacy.conf 
b/test/cfg2cmd/bootorder-legacy.conf
new file mode 100644
index 000..f49b084
--- /dev/null
+++ b/test/cfg2cmd/bootorder-legacy.conf
@@ -0,0 +1,17 @@
+# TEST: Test for a specific bootorder given by legacy 'boot' value
+# QEMU_VERSION: 5.1
+cores: 3
+boot: ndca
+bootdisk: virtio1
+ide2: none,media=cdrom
+memory: 768
+name: simple
+net0: virtio=A2:C0:43:77:08:A0,bridge=vmbr0
+numa: 0
+ostype: l26
+scsi4: local:8006/vm-8006-disk-0.qcow2,discard=on,size=104858K
+smbios1: uuid=7b10d7af-b932-4c66-b2c3-3996152ec465
+sockets: 1
+virtio0: local:8006/vm-8006-disk-0.qcow2,discard=on,iothread=1,size=104858K
+virtio1: local:8006/vm-8006-disk-0.qcow2,discard=on,iothread=1,size=104858K
+vmgenid: c773c261-d800-4348-9f5d-167fadd53cf8
diff --git a/test/cfg2cmd/bootorder-legacy.conf.cmd 
b/test/cfg2cmd/bootorder-legacy.conf.cmd
new file mode 100644
index 000..f624ea2
--- /dev/null
+++ b/test/cfg2cmd/bootorder-legacy.conf.cmd
@@ -0,0 +1,38 @@
+/usr/bin/kvm \
+  -id 8006 \
+  -name simple \
+  -chardev 'socket,id=qmp,path=/var/run/qemu-server/8006.qmp,server,nowait' \
+  -mon 'chardev=qmp,mode=control' \
+  -chardev 'socket,id=qmp-event,path=/var/run/qmeventd.sock,reconnect=5' \
+  -mon 'chardev=qmp-event,mode=control' \
+  -pidfile /var/run/qemu-server/8006.pid \
+  -daemonize \
+  -smbios 'type=1,uuid=7b10d7af-b932-4c66-b2c3-3996152ec465' \
+  -smp '3,sockets=1,cores=3,maxcpus=3' \
+  -nodefaults \
+  -boot 
'menu=on,strict=on,reboot-timeout=1000,splash=/usr/share/qemu-server/bootsplash.jpg'
 \
+  -vnc unix:/var/run/qemu-server/8006.vnc,password \
+  -cpu kvm64,enforce,+kvm_pv_eoi,+kvm_pv_unhalt,+lahf_lm,+sep \
+  -m 768 \
+  -object 'iothread,id=iothread-virtio0' \
+  -object 'iothread,id=iothread-virtio1' \
+  -device 'pci-bridge,id=pci.1,chassis_nr=1,bus=pci.0,addr=0x1e' \
+  -device 'pci-bridge,id=pci.2,chassis_nr=2,bus=pci.0,addr=0x1f' \
+  -device 'vmgenid,guid=c773c261-d800-4348-9f5d-167fadd53cf8' \
+  -device 'piix3-usb-uhci,id=uhci,bus=pci.0,addr=0x1.0x2' \
+  -device 'usb-tablet,id=tablet,bus=uhci.0,port=1' \
+  -device 'VGA,id=vga,bus=pci.0,addr=0x2' \
+  -device 'virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3' \
+  -iscsi 'initiator-name=iqn.1993-08.org.debian:01:aabbccddeeff' \
+  -drive 'if=none,id=drive-ide2,media=cdrom,aio=threads' \
+  -device 'ide-cd,bus=ide.1,unit=0,drive=drive-ide2,id=ide2,bootindex=200' \
+  -device 'lsi,id=scsihw0,bus=pci.0,addr=0x5' \
+  -drive 
'file=/var/lib/vz/images/8006/vm-8006-disk-0.qcow2,if=none,id=drive-scsi4,discard=on,format=qcow2,cache=none,aio=native,detect-zeroes=unmap'
 \
+  -device 'scsi-hd,bus=scsihw0.0,scsi-id=4,drive=drive-scsi4,id=scsi4' \
+  -drive 
'file=/var/lib/vz/images/8006/vm-8006-disk-0.qcow2,if=none,id=drive-virtio0,discard=on,format=qcow2,cache=none,aio=native,detect-zeroes=unmap'
 \
+  -device 
'virtio-blk-pci,drive=drive-virtio0,id=virtio0,bus=pci.0,addr=0xa,iothread=iothread-virtio0'
 \
+  -drive 
'file=/var/lib/vz/images/8006/vm-8006-disk-0.qcow2,if=none,id=drive-virtio1,discard=on,format=qcow2,cache=none,aio=native,detect-zeroes=unmap'
 \
+  -device 
'virtio-blk-pci,drive=drive-virtio1,id=virtio1,bus=pci.0,addr=0xb,iothread=iothread-virtio1,bootindex=302'
 \
+  -netdev 
'type=tap,id=net0,ifname=tap8006i0,script=/var/lib/qemu-server/pve-bridge,downscript=/var/lib/qemu-server/pve-bridgedown,vhost=on'
 \
+  -device 
'virtio-net-pci,mac=A2:C0:43:77:08:A0,netdev=net0,bus=pci.0,addr=0x12,id=net0,bootindex=100'
 \
+  -machine 'type=pc+pve0'
-- 
2.20.1



___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel



[pve-devel] [PATCH libspice-server] update spice to 0.14.3 and add patches for CVE-2020-14355

2020-10-06 Thread Oguz Bektas
the original email is in oss-security[0], patches are from upstream
gitlab. (commits linked in the email)

removed the already applied monitor patch.

[0]: https://www.openwall.com/lists/oss-security/2020/10/06/10

Signed-off-by: Oguz Bektas 
---
 Makefile  | 10 ++---
 changelog.Debian  |  7 +++
 patches/CVE-2020-14355_1.patch| 28 
 patches/CVE-2020-14355_2.patch| 43 +++
 patches/CVE-2020-14355_3.patch| 30 +
 patches/CVE-2020-14355_4.patch| 29 +
 ...ke-sure-we-have-at-least-one-monitor.patch | 33 --
 7 files changed, 142 insertions(+), 38 deletions(-)
 create mode 100644 patches/CVE-2020-14355_1.patch
 create mode 100644 patches/CVE-2020-14355_2.patch
 create mode 100644 patches/CVE-2020-14355_3.patch
 create mode 100644 patches/CVE-2020-14355_4.patch
 delete mode 100644 patches/red-qxl-make-sure-we-have-at-least-one-monitor.patch

diff --git a/Makefile b/Makefile
index 01bed58..6a34c84 100644
--- a/Makefile
+++ b/Makefile
@@ -1,9 +1,9 @@
 SOURCE=spice
 PACKAGE=libspice-server1
 
-PKGVERSION=0.14.2
-DEBVERSION=0.14.2-4
-PVERELEASE=pve6+1
+PKGVERSION=0.14.3
+DEBVERSION=0.14.3-1
+PVERELEASE=pve6
 
 VERSION := $(DEBVERSION)~$(PVERELEASE)
 
@@ -29,7 +29,7 @@ $(DEB): $(SOURCE)_$(PKGVERSION).orig.tar.bz2 
$(SOURCE)_$(DEBVERSION).debian.tar.
tar xf $(SOURCE)_$(DEBVERSION).debian.tar.xz -C $(SOURCE)-$(PKGVERSION)
cat changelog.Debian $(PKGDIR)/debian/changelog > 
$(PKGDIR)/debian/changelog.tmp
mv $(PKGDIR)/debian/changelog.tmp $(PKGDIR)/debian/changelog
-   cd $(PKGDIR); for patch in ../patches/*.patch; do echo "applying patch 
'$$patch'" && patch -p1 < "$${patch}"; done
+   cd $(PKGDIR); for patch in ../patches/*.patch; do echo "applying patch 
'$$patch'" && patch -Np1 < "$${patch}"; done
cd ${PKGDIR}; dpkg-buildpackage -b -us -uc
lintian ${DEBS}
 
@@ -38,7 +38,7 @@ $(DEB): $(SOURCE)_$(PKGVERSION).orig.tar.bz2 
$(SOURCE)_$(DEBVERSION).debian.tar.
 download: $(SOURCE)_$(PKGVERSION).orig.tar.bz2 
$(SOURCE)_$(DEBVERSION).debian.tar.xz
 $(SOURCE)_$(PKGVERSION).orig.tar.bz2: $(SOURCE)_$(DEBVERSION).debian.tar.xz
 $(SOURCE)_$(DEBVERSION).debian.tar.xz:
-   dget http://deb.debian.org/debian/pool/main/s/spice/spice_0.14.2-4.dsc
+   dget http://deb.debian.org/debian/pool/main/s/spice/spice_0.14.3-1.dsc
 
 .PHONY: upload
 upload: ${DEBS}
diff --git a/changelog.Debian b/changelog.Debian
index 14c019c..d1b4489 100644
--- a/changelog.Debian
+++ b/changelog.Debian
@@ -1,3 +1,10 @@
+spice (0.14.3-1~pve6) pve; urgency=medium
+
+  [ Proxmox Support Team ]
+  * add patches for CVE-2020-14355 from oss-security
+
+ -- Proxmox Support Team   Tue, 06 Oct 2020 16:33:30 +0200
+
 spice (0.14.2-4~pve6+1) pve; urgency=medium
 
   [ Proxmox Support Team ]
diff --git a/patches/CVE-2020-14355_1.patch b/patches/CVE-2020-14355_1.patch
new file mode 100644
index 000..c76847c
--- /dev/null
+++ b/patches/CVE-2020-14355_1.patch
@@ -0,0 +1,28 @@
+commit 762e0abae36033ccde658fd52d3235887b60862d
+Author: Frediano Ziglio 
+Date:   Wed Apr 29 15:09:13 2020 +0100
+
+quic: Check we have some data to start decoding quic image
+
+All paths already pass some data to quic_decode_begin but for the
+test check it, it's not that expensive test.
+Checking for not 0 is enough, all other words will potentially be
+read calling more_io_words but we need one to avoid a potential
+initial buffer overflow or deferencing an invalid pointer.
+
+Signed-off-by: Frediano Ziglio 
+Acked-by: Uri Lublin 
+
+diff --git a/subprojects/spice-common/quic.c b/subprojects/spice-common/quic.c
+index e2dee0f..bc753ca 100644
+--- a/subprojects/spice-common/common/quic.c
 b/subprojects/spice-common/common/quic.c
+@@ -1136,7 +1136,7 @@ int quic_decode_begin(QuicContext *quic, uint32_t 
*io_ptr, unsigned int num_io_w
+ int channels;
+ int bpc;
+ 
+-if (!encoder_reset(encoder, io_ptr, io_ptr_end)) {
++if (!num_io_words || !encoder_reset(encoder, io_ptr, io_ptr_end)) {
+ return QUIC_ERROR;
+ }
+ 
diff --git a/patches/CVE-2020-14355_2.patch b/patches/CVE-2020-14355_2.patch
new file mode 100644
index 000..6926286
--- /dev/null
+++ b/patches/CVE-2020-14355_2.patch
@@ -0,0 +1,43 @@
+commit 404d74782c8b5e57d146c5bf3118bb41bf3378e4
+Author: Frediano Ziglio 
+Date:   Wed Apr 29 15:10:24 2020 +0100
+
+quic: Check image size in quic_decode_begin
+
+Avoid some overflow in code due to images too big or
+negative numbers.
+
+Signed-off-by: Frediano Ziglio 
+Acked-by: Uri Lublin 
+
+diff --git a/subprojects/spice-common/common/quic.c 
b/subprojects/spice-common/common/quic.c
+index bc753ca..6815316 100644
+--- a/subprojects/spice-common/common/quic.c
 b/subprojects/spice-common/common/quic.c
+@@ -56,6 +56,9 @@ typedef uint8_t BYTE;
+ #define MINwminex