This series enables importing ova/ovf from directory based storages, inclusive upload/download via the webui (ova only).
It also improves the ovf importer by parsing the ostype, nics, bootorder (and firmware from vmware exported files). We now extract the images into either a given target storage or in the import storage in the 'images' dir so accidentally left over images are discoverable by the ui/cli. This version includes a modified qemu-server patch of fabians hardening series: https://lore.proxmox.com/pve-devel/20241104104221.228730-1-f.gruenbich...@proxmox.com/ I sent the qemu-server patch from fabian again but omitted some problematic checks. I add them later with a check against the import vtype again (last patch in qemu-server) changes from v6: * rebase on master (omit already applied patches) * style/indentation fixes * omit explicit check for .ova in upload path * use assert_file_validity check for download_file_from_url * only warn on whitespace in parent image (file_size_info) to preserve backward compatibility for possible existing cases * fix cleanup on live-import (added wrong variable to cleanup list) * renamed 'import-extraction-storage' to 'import-working-storage' * mad priv check for extraction storage simpler (combine branches) * reworded some gettexts changes from v5: * removed leftover hunks in makefile * moved ova checks to correct patch * split up error messages for unexpected format * remove unnecessary untaint * reword error message * reintroduce symlink check in ova/ovf check * added sanity check for ovas after uploading/downloading * added new patch for checking import vtypes * fixed issue with files with absolute path changes from v4: * rebased on master/fabians series * added the file_size_info check for untrusted images after extracting changes from v3: * fixed dependencies in control file * removed unnecessary use statements * removed unnecessary remove helper * moved 'needs_extract' helper to qemu-server * removed import storage param from PUT call * check down/uploaded ova filename more strictly (same as listing) * improved filepath checking in ovf * forbid importing when extracted image references a base/backing file * instead of trying to manually create a proper filename, use 'alloc' to create a small (1M) file with the same format and overwrite it with renaming. this also solves the cluster locking issue * prefer using PVE::Storage functions instead of plugin methods in ova extraction code * use $vollist for cleaning up extracted images in qemu-server and add manual cleanup for the success case changes from v2: * use better 'format' values for embedded images (e.g. ova+vmdk) * use this format to decide if images should be extracted * consistent use of the 'safe character' classes when listing and parsing * also list vmdk/qcow2/raw images in content listing (this will be useful when we have a gui for the 'import-from' in the wizard/disk edit for vms) * a few gui adaptions changes from v1: * move ovf code to GuestImport * move extract/checking code to GuestImport * don't return 'image' types from import volumes * use allow 'safe' characters for filenames of ova/ovfs and inside * check for non-regular files (e.g. symlinks) after extraction * add new 'import-extraction-storage' for import * rename panel in gui for directory storages * typo fixes * and probably more, see the individual patches for details pve-storage: Dominik Csapak (11): plugin: dir: implement import content type plugin: dir: handle ova files for import ovf: improve and simplify path checking code ovf: implement parsing the ostype ovf: implement parsing out firmware type ovf: implement rudimentary boot order ovf: implement parsing nics api: allow ova upload/download plugin: enable import for nfs/btrfs/cifs/cephfs/glusterfs add 'import' content type to 'check_volume_access' plugin: file_size_info: warn on parent images with unusual path src/PVE/API2/Storage/Status.pm | 56 +++++- src/PVE/GuestImport.pm | 79 ++++++++ src/PVE/GuestImport/OVF.pm | 187 ++++++++++++++++-- src/PVE/Makefile | 1 + src/PVE/Storage.pm | 23 ++- src/PVE/Storage/BTRFSPlugin.pm | 5 + src/PVE/Storage/CIFSPlugin.pm | 6 +- src/PVE/Storage/CephFSPlugin.pm | 6 +- src/PVE/Storage/DirPlugin.pm | 52 ++++- src/PVE/Storage/GlusterfsPlugin.pm | 6 +- src/PVE/Storage/NFSPlugin.pm | 6 +- src/PVE/Storage/Plugin.pm | 16 +- .../ovf_manifests/Win10-Liz_no_default_ns.ovf | 1 + src/test/parse_volname_test.pm | 33 ++++ src/test/path_to_volume_id_test.pm | 21 ++ src/test/run_ovf_tests.pl | 14 ++ 16 files changed, 480 insertions(+), 32 deletions(-) create mode 100644 src/PVE/GuestImport.pm qemu-server: Dominik Csapak (4): use OVF from Storage api: create: implement extracting disks when needed for import-from api: create: add 'import-working-storage' parameter api: check untrusted image files for import content type Fabian Grünbichler (1): disk import: add additional safeguards for imported image files PVE/API2/Qemu.pm | 108 ++++++-- PVE/CLI/qm.pm | 4 +- PVE/QemuServer.pm | 12 + PVE/QemuServer/Helpers.pm | 5 + PVE/QemuServer/Makefile | 1 - PVE/QemuServer/OVF.pm | 242 ------------------ debian/control | 2 - test/Makefile | 5 +- test/ovf_manifests/Win10-Liz-disk1.vmdk | Bin 65536 -> 0 bytes test/ovf_manifests/Win10-Liz.ovf | 142 ---------- .../ovf_manifests/Win10-Liz_no_default_ns.ovf | 142 ---------- test/ovf_manifests/Win_2008_R2_two-disks.ovf | 145 ----------- test/ovf_manifests/disk1.vmdk | Bin 65536 -> 0 bytes test/ovf_manifests/disk2.vmdk | Bin 65536 -> 0 bytes test/run_ovf_tests.pl | 71 ----- 15 files changed, 112 insertions(+), 767 deletions(-) delete mode 100644 PVE/QemuServer/OVF.pm delete mode 100644 test/ovf_manifests/Win10-Liz-disk1.vmdk delete mode 100755 test/ovf_manifests/Win10-Liz.ovf delete mode 100755 test/ovf_manifests/Win10-Liz_no_default_ns.ovf delete mode 100755 test/ovf_manifests/Win_2008_R2_two-disks.ovf delete mode 100644 test/ovf_manifests/disk1.vmdk delete mode 100644 test/ovf_manifests/disk2.vmdk delete mode 100755 test/run_ovf_tests.pl pve-manager: Dominik Csapak (9): ui: guest import: add ova-needs-extracting warning text ui: enable import content type for relevant storages ui: enable upload/download/remove buttons for 'import' type storages ui: disable 'import' button for non importable formats ui: import: improve rendering of volume names ui: guest import: add storage selector for ova extraction storage ui: guest import: change icon/text for non-esxi import storage ui: import: show size for dir-based storages ui: import: adapt live import help text to ova www/manager6/Utils.js | 9 +++++++- www/manager6/form/ContentTypeSelector.js | 2 +- www/manager6/storage/Browser.js | 25 ++++++++++++++++----- www/manager6/storage/CephFSEdit.js | 2 +- www/manager6/storage/GlusterFsEdit.js | 2 +- www/manager6/window/GuestImport.js | 28 +++++++++++++++++++++++- www/manager6/window/UploadToStorage.js | 1 + 7 files changed, 58 insertions(+), 11 deletions(-) -- 2.39.5 _______________________________________________ pve-devel mailing list pve-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
