[pve-devel] partially-applied: [PATCH proxmox 1/6] notify: copy sendmail/forward fn's from proxmox_sys

2024-07-12 Thread Wolfgang Bumiller
Applied the first patch.
The 2nd one I skipped and is now obsolete, since I instead immediately
removed the functions since I was doing a major bump to the sys crate
anyway.


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel



Re: [pve-devel] [PATCH installer 00/14] fix #5536: implement post-(auto-)installation notification mechanism

2024-07-12 Thread Christoph Heiss
On Thu, Jul 11, 2024 at 06:49:28PM GMT, Stefan Hanreich wrote:
> Did a quick smoke test of this series by creating an ISO with an answer
> file baked in and checking the response via `nc -l`. Review is inline.

Thanks for the review & testing!

>From a quick glance all the comments make sense, I'll address them in a
v2.

>
> Consider this:
>
> Tested-By: Stefan Hanreich 
> Reviewed-By: Stefan Hanreich 
>
>
> On 7/10/24 15:27, Christoph Heiss wrote:
> > This implements a mechanism for post-installation "notifications" via a
> > POST request [0] when using the auto-installer.
> >
> > It's implemented as a separate, small utility to facilitate separation
> > of concerns and make the information gathering easier by having it
> > isolated in one place.
> >
> > Patches #1 through #10 are simply clean-ups, refactors, etc. that were
> > done along the way, which do not impact functionality in any way.
> >
> > Most interesting here will be patch #12, which adds the actual
> > implementation of the post-hook. (Bind-)mounting the installed host
> > system is done using the existing `proxmox-chroot` utility, and the HTTP
> > POST functionality can fortunately be re-used 1:1 from
> > `proxmox-fetch-answer`.
> >
> > I've also included an example of how the JSON body (pretty-printed for
> > readability) of such a post-installation request would look like below,
> > for reference.
> >
> > Tested this with both PVE and PBS ISOs, PMG did not (yet) have a
> > release with an auto-installation capable ISO. The only product-specific
> > code is the version detection in `proxmox-post-hook`, which - since it
> > works the same for PVE and PMG - be no obstacle.
> >
> > [0] https://bugzilla.proxmox.com/show_bug.cgi?id=5536
> >
> > {
> >   "debian-version": "12.5",
> >   "product-version": "pve-manager/8.2.2/9355359cd7afbae4",
> >   "kernel-version": "proxmox-kernel-6.8.4-2-pve-signed",
> >   "boot-type": "bios",
> >   "filesystem": "zfs (RAID1)",
> >   "fqdn": "host.domain",
> >   "machine-id": "f4bf9711783248b7aaffe3ccbca3e3dc",
> >   "bootdisk": [
> > {
> >   "size": 8589934592,
> >   "udev-properties": {
> > "DEVNAME": "/dev/vda", [..]
> >   }
> > },
> > {
> >   "size": 8589934592,
> >   "udev-properties": {
> > "DEVNAME": "/dev/vdb", [..]
> >   }
> > }
> >   ],
> >   "management-nic": {
> > "mac": "de:ad:f0:0d:12:34",
> > "address": "10.0.0.10/24",
> > "udev-properties": {
> >   "INTERFACE": "enp6s18", [..]
> > }
> >   },
> >   "ssh-public-host-keys": {
> > "ecdsa": "ecdsa-sha2-nistp256 [..] root@host",
> > "ed25519": "ssh-ed25519 [..] root@host",
> > "rsa": "ssh-rsa [..] root@host",
> >   }
> > }
> >
> > Christoph Heiss (14):  chroot: print full anyhow message
> >   tree-wide: fix some typos
> >   tree-wide: collect hardcoded installer runtime directory strings into
> > constant
> >   common: simplify filesystem type serializing & Display trait impl
> >   common: setup: serialize `target_hd` as string explicitly
> >   common: split out installer setup files loading functionality
> >   debian: strip unused library dependencies
> >   fetch-answer: move http-related code to gated module in
> > installer-common
> >   tree-wide: convert some more crates to use workspace dependencies
> >   auto-installer: tests: replace left/right with got/expected in output
> >   auto-installer: answer: add `posthook` section
> >   fix #5536: add post-hook utility for sending notifications after
> > auto-install
> >   fix #5536: post-hook: add some unit tests
> >   unconfigured.sh: run proxmox-post-hook after successful auto-install
> >
> >  Cargo.toml|  11 +
> >  Makefile  |   8 +-
> >  debian/control|   1 +
> >  debian/install|   1 +
> >  debian/rules  |   9 +
> >  proxmox-auto-install-assistant/Cargo.toml |  14 +-
> >  proxmox-auto-installer/Cargo.toml |  15 +-
> >  proxmox-auto-installer/src/answer.rs  |  27 +-
> >  .../src/bin/proxmox-auto-installer.rs |  15 +-
> >  proxmox-auto-installer/src/sysinfo.rs |  10 +-
> >  proxmox-auto-installer/src/utils.rs   |  15 +-
> >  proxmox-auto-installer/tests/parse-answer.rs  |  42 +-
> >  proxmox-chroot/Cargo.toml |   8 +-
> >  proxmox-chroot/src/main.rs|  19 +-
> >  proxmox-fetch-answer/Cargo.toml   |  17 +-
> >  .../src/fetch_plugins/http.rs | 100 +---
> >  .../src/fetch_plugins/partition.rs|  14 +-
> >  proxmox-installer-common/Cargo.toml   |  26 +-
> >  proxmox-installer-common/src/http.rs  |  94 
> >  proxmox-installer-common/src/lib.rs   |   5 +
> >  proxmox-installer-common/src/options.rs   | 109 ++--
> >  proxmox-installer-common/src/setup.rs | 108 +---
> >  proxmox-in

Re: [pve-devel] [PATCH installer v3 4/4] low-level: install: check for already-existing `rpool` on install

2024-07-12 Thread Aaron Lauterer

only one small nit inline

On  2024-07-11  13:57, Christoph Heiss wrote:

.. much in the same manner as the detection for LVM works.

zpools can only be renamed by importing them with a new name, so
unfortunately the import-export dance is needed.

Signed-off-by: Christoph Heiss 
---
Changes v2 -> v3:
   * skip rename prompt in auto-install mode

Changes v1 -> v2:
   * use unique pool id for renaming instead of generating random id
 ourselves
   * moved renaming functionality to own subroutine in new
 Proxmox::Sys::ZFS module

  Proxmox/Install.pm | 33 +
  Proxmox/Sys/ZFS.pm | 20 ++--
  2 files changed, 51 insertions(+), 2 deletions(-)

diff --git a/Proxmox/Install.pm b/Proxmox/Install.pm
index 7342e4b..ba699fa 100644
--- a/Proxmox/Install.pm
+++ b/Proxmox/Install.pm
@@ -16,6 +16,7 @@ use Proxmox::Install::StorageConfig;
  use Proxmox::Sys::Block qw(get_cached_disks wipe_disk 
partition_bootable_disk);
  use Proxmox::Sys::Command qw(run_command syscmd);
  use Proxmox::Sys::File qw(file_read_firstline file_write_all);
+use Proxmox::Sys::ZFS;
  use Proxmox::UI;
  
  # TODO: move somewhere better?

@@ -169,9 +170,41 @@ sub btrfs_create {
  syscmd($cmd);
  }
  
+sub zfs_ask_existing_zpool_rename {

+my ($pool_name) = @_;
+
+# At this point, no pools should be imported/active
+my $exported_pools = Proxmox::Sys::ZFS::get_exported_pools();
+
+foreach (@$exported_pools) {
+   next if $_->{name} ne $pool_name || $_->{state} ne 'ONLINE';
+   my $renamed_pool = "$_->{name}-OLD-$_->{id}";
+
+   my $response_ok = 
Proxmox::Install::Config::get_existing_storage_auto_rename();

maybe we want to name this differently to avoid confusion?
response_ok -> do_rename
or something in that regard?

but that could be done in a follow up patch as well if we want to


+   if (!$response_ok) {
+   $response_ok = Proxmox::UI::prompt(
+   "A ZFS pool named '$_->{name}' (id $_->{id}) already exists on the 
system.\n\n" .
+   "Do you want to rename the pool to '$renamed_pool' before continuing 
" .
+   "or cancel the installation?"
+   );
+   }
+
+   # Import the pool using its id, as that is unique and works even if 
there are
+   # multiple zpools with the same name.
+   if ($response_ok) {
+   Proxmox::Sys::ZFS::rename_pool($_->{id}, $renamed_pool);
+   } else {
+   warn "Canceled installation as requested by user, due to already 
existing ZFS pool '$pool_name'\n";
+   die "\n"; # causes abort without re-showing an error dialogue
+   }
+}
+}
+

[...]


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel



Re: [pve-devel] [PATCH installer v3 0/4] add check/rename for already-existing ZFS rpool

2024-07-12 Thread Aaron Lauterer
tested this series more thoroughly and checked with an existing root 
pool all 3 install options:


* TUI
* GUI
* auto

Worked well in 3 cases to rename.

One oddity seems to be if we cancel the rename in the TUI installer. it 
jumps to 100% and stays there and I then need to manually abort. Jumping 
to a "setup cancelled" box that we confirm before quitting would be 
nice. This is how the GUI installer handles it.


Otherwise, besides this and a tiny nit in patch 4, consider this:

Reviewed-By: Aaron Lauterer 
Tested-By: Aaron Lauterer 

On  2024-07-11  13:57, Christoph Heiss wrote:

Pretty straight forward overall, implements a check for an existing
`rpool` on the system and ask the user whether they would like to rename
it, much in the same way as it works for VGs already.

Without this, the installer would silently create a second (and thus
conflicting) `rpool` and cause a boot failure after the installation,
since it does not know which pool to import exactly.

v1: https://lists.proxmox.com/pipermail/pve-devel/2024-May/063874.html
v2: https://lists.proxmox.com/pipermail/pve-devel/2024-July/064619.html

Notable changes v2 -> v3:
   * make low-level option lvm_auto_rename more generic
   * skip rename prompt in auto-install mode

Notable changes v1 -> v2:
   * incorporated Aarons suggestions from v1
   * rewrote tests to use a pre-defined input instead
   * moved pool renaming to own subroutine
   * documented all new subroutines
   * split out tests into own patch

Christoph Heiss (4):
   test: add test cases for new zfs module
   low-level: install: check for already-existing `rpool` on install
   install: config: rename option lvm_auto_rename ->
 existing_storage_auto_rename
   low-level: install: automatically rename existing rpools on
 auto-installs

  Proxmox/Install.pm| 35 +++-
  Proxmox/Install/Config.pm |  6 +-
  Proxmox/Sys/ZFS.pm| 20 ++-
  proxmox-auto-installer/src/utils.rs   |  2 +-
  .../resources/parse_answer/disk_match.json|  2 +-
  .../parse_answer/disk_match_all.json  |  2 +-
  .../parse_answer/disk_match_any.json  |  2 +-
  .../tests/resources/parse_answer/minimal.json |  2 +-
  .../resources/parse_answer/nic_matching.json  |  2 +-
  .../resources/parse_answer/specific_nic.json  |  2 +-
  .../tests/resources/parse_answer/zfs.json |  2 +-
  proxmox-installer-common/src/setup.rs |  2 +-
  proxmox-tui-installer/src/setup.rs|  2 +-
  test/Makefile |  7 ++-
  test/zfs-get-pool-list.pl | 57 +++
  15 files changed, 128 insertions(+), 17 deletions(-)
  create mode 100755 test/zfs-get-pool-list.pl




___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel



[pve-devel] [PATCH proxmox-perl-rs v2 03/12] common: notify: add bindings for webhook API routes

2024-07-12 Thread Lukas Wagner
Signed-off-by: Lukas Wagner 
---
 common/src/notify.rs | 63 
 1 file changed, 63 insertions(+)

diff --git a/common/src/notify.rs b/common/src/notify.rs
index e1b006b..fe192d5 100644
--- a/common/src/notify.rs
+++ b/common/src/notify.rs
@@ -19,6 +19,9 @@ mod export {
 DeleteableSmtpProperty, SmtpConfig, SmtpConfigUpdater, SmtpMode, 
SmtpPrivateConfig,
 SmtpPrivateConfigUpdater,
 };
+use proxmox_notify::endpoints::webhook::{
+DeleteableWebhookProperty, WebhookConfig, WebhookConfigUpdater,
+};
 use proxmox_notify::matcher::{
 CalendarMatcher, DeleteableMatcherProperty, FieldMatcher, 
MatchModeOperator, MatcherConfig,
 MatcherConfigUpdater, SeverityMatcher,
@@ -393,6 +396,66 @@ mod export {
 api::smtp::delete_endpoint(&mut config, name)
 }
 
+#[export(serialize_error)]
+fn get_webhook_endpoints(
+#[try_from_ref] this: &NotificationConfig,
+) -> Result, HttpError> {
+let config = this.config.lock().unwrap();
+api::webhook::get_endpoints(&config)
+}
+
+#[export(serialize_error)]
+fn get_webhook_endpoint(
+#[try_from_ref] this: &NotificationConfig,
+id: &str,
+) -> Result {
+let config = this.config.lock().unwrap();
+api::webhook::get_endpoint(&config, id)
+}
+
+#[export(serialize_error)]
+#[allow(clippy::too_many_arguments)]
+fn add_webhook_endpoint(
+#[try_from_ref] this: &NotificationConfig,
+endpoint_config: WebhookConfig,
+) -> Result<(), HttpError> {
+let mut config = this.config.lock().unwrap();
+api::webhook::add_endpoint(
+&mut config,
+endpoint_config,
+)
+}
+
+#[export(serialize_error)]
+#[allow(clippy::too_many_arguments)]
+fn update_webhook_endpoint(
+#[try_from_ref] this: &NotificationConfig,
+name: &str,
+config_updater: WebhookConfigUpdater,
+delete: Option>,
+digest: Option<&str>,
+) -> Result<(), HttpError> {
+let mut config = this.config.lock().unwrap();
+let digest = decode_digest(digest)?;
+
+api::webhook::update_endpoint(
+&mut config,
+name,
+config_updater,
+delete.as_deref(),
+digest.as_deref(),
+)
+}
+
+#[export(serialize_error)]
+fn delete_webhook_endpoint(
+#[try_from_ref] this: &NotificationConfig,
+name: &str,
+) -> Result<(), HttpError> {
+let mut config = this.config.lock().unwrap();
+api::webhook::delete_endpoint(&mut config, name)
+}
+
 #[export(serialize_error)]
 fn get_matchers(
 #[try_from_ref] this: &NotificationConfig,
-- 
2.39.2



___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel



[pve-devel] [PATCH proxmox-perl-rs v2 04/12] common: notify: add bindings for get_targets

2024-07-12 Thread Lukas Wagner
This allows us to drop the impl of that function on the perl side.

Signed-off-by: Lukas Wagner 
---
 common/src/notify.rs | 9 +
 1 file changed, 9 insertions(+)

diff --git a/common/src/notify.rs b/common/src/notify.rs
index fe192d5..0f8a35d 100644
--- a/common/src/notify.rs
+++ b/common/src/notify.rs
@@ -27,6 +27,7 @@ mod export {
 MatcherConfigUpdater, SeverityMatcher,
 };
 use proxmox_notify::{api, Config, Notification, Severity};
+use proxmox_notify::api::Target;
 
 pub struct NotificationConfig {
 config: Mutex,
@@ -112,6 +113,14 @@ mod export {
 api::common::send(&config, ¬ification)
 }
 
+#[export(serialize_error)]
+fn get_targets(
+#[try_from_ref] this: &NotificationConfig,
+) -> Result, HttpError> {
+let config = this.config.lock().unwrap();
+api::get_targets(&config)
+}
+
 #[export(serialize_error)]
 fn test_target(
 #[try_from_ref] this: &NotificationConfig,
-- 
2.39.2



___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel



[pve-devel] [PATCH manager v2 06/12] api: notifications: use get_targets impl from proxmox-notify

2024-07-12 Thread Lukas Wagner
The get_targets API endpoint is now implemented in Rust.

Signed-off-by: Lukas Wagner 
---
 PVE/API2/Cluster/Notifications.pm | 34 +--
 1 file changed, 1 insertion(+), 33 deletions(-)

diff --git a/PVE/API2/Cluster/Notifications.pm 
b/PVE/API2/Cluster/Notifications.pm
index 68fdda2a..10b611c9 100644
--- a/PVE/API2/Cluster/Notifications.pm
+++ b/PVE/API2/Cluster/Notifications.pm
@@ -170,39 +170,7 @@ __PACKAGE__->register_method ({
my $config = PVE::Notify::read_config();
 
my $targets = eval {
-   my $result = [];
-
-   for my $target (@{$config->get_sendmail_endpoints()}) {
-   push @$result, {
-   name => $target->{name},
-   comment => $target->{comment},
-   type => 'sendmail',
-   disable => $target->{disable},
-   origin => $target->{origin},
-   };
-   }
-
-   for my $target (@{$config->get_gotify_endpoints()}) {
-   push @$result, {
-   name => $target->{name},
-   comment => $target->{comment},
-   type => 'gotify',
-   disable => $target->{disable},
-   origin => $target->{origin},
-   };
-   }
-
-   for my $target (@{$config->get_smtp_endpoints()}) {
-   push @$result, {
-   name => $target->{name},
-   comment => $target->{comment},
-   type => 'smtp',
-   disable => $target->{disable},
-   origin => $target->{origin},
-   };
-   }
-
-   $result
+   $config->get_targets();
};
 
raise_api_error($@) if $@;
-- 
2.39.2



___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel



[pve-devel] [PATCH proxmox-backup v2 10/12] ui: utils: enable webhook edit window

2024-07-12 Thread Lukas Wagner
This allows users to add/edit new webhook targets.

Signed-off-by: Lukas Wagner 
---
 www/Utils.js | 5 +
 1 file changed, 5 insertions(+)

diff --git a/www/Utils.js b/www/Utils.js
index 4853be36..b715972f 100644
--- a/www/Utils.js
+++ b/www/Utils.js
@@ -482,6 +482,11 @@ Ext.define('PBS.Utils', {
ipanel: 'pmxGotifyEditPanel',
iconCls: 'fa-bell-o',
},
+   webhook: {
+   name: 'Webhook',
+   ipanel: 'pmxWebhookEditPanel',
+   iconCls: 'fa-bell-o',
+   },
};
 },
 
-- 
2.39.2



___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel



[pve-devel] [PATCH proxmox-backup v2 09/12] api: notification: add API routes for webhook targets

2024-07-12 Thread Lukas Wagner
Copied and adapted from the Gotify ones.

Signed-off-by: Lukas Wagner 
---
 src/api2/config/notifications/mod.rs |   2 +
 src/api2/config/notifications/webhook.rs | 175 +++
 2 files changed, 177 insertions(+)
 create mode 100644 src/api2/config/notifications/webhook.rs

diff --git a/src/api2/config/notifications/mod.rs 
b/src/api2/config/notifications/mod.rs
index dfe82ed0..81ca9800 100644
--- a/src/api2/config/notifications/mod.rs
+++ b/src/api2/config/notifications/mod.rs
@@ -22,6 +22,7 @@ pub mod matchers;
 pub mod sendmail;
 pub mod smtp;
 pub mod targets;
+pub mod webhook;
 
 #[sortable]
 const SUBDIRS: SubdirMap = &sorted!([
@@ -41,6 +42,7 @@ const ENDPOINT_SUBDIRS: SubdirMap = &sorted!([
 ("gotify", &gotify::ROUTER),
 ("sendmail", &sendmail::ROUTER),
 ("smtp", &smtp::ROUTER),
+("webhook", &webhook::ROUTER),
 ]);
 
 const ENDPOINT_ROUTER: Router = Router::new()
diff --git a/src/api2/config/notifications/webhook.rs 
b/src/api2/config/notifications/webhook.rs
new file mode 100644
index ..4a040024
--- /dev/null
+++ b/src/api2/config/notifications/webhook.rs
@@ -0,0 +1,175 @@
+use anyhow::Error;
+use serde_json::Value;
+
+use proxmox_notify::endpoints::webhook::{
+DeleteableWebhookProperty, WebhookConfig, WebhookConfigUpdater,
+};
+use proxmox_notify::schema::ENTITY_NAME_SCHEMA;
+use proxmox_router::{Permission, Router, RpcEnvironment};
+use proxmox_schema::api;
+
+use pbs_api_types::{PRIV_SYS_AUDIT, PRIV_SYS_MODIFY, 
PROXMOX_CONFIG_DIGEST_SCHEMA};
+
+#[api(
+protected: true,
+input: {
+properties: {},
+},
+returns: {
+description: "List of webhook endpoints.",
+type: Array,
+items: { type: WebhookConfig },
+},
+access: {
+permission: &Permission::Privilege(&["system", "notifications"], 
PRIV_SYS_AUDIT, false),
+},
+)]
+/// List all webhook endpoints.
+pub fn list_endpoints(
+_param: Value,
+_rpcenv: &mut dyn RpcEnvironment,
+) -> Result, Error> {
+let config = pbs_config::notifications::config()?;
+
+let endpoints = proxmox_notify::api::webhook::get_endpoints(&config)?;
+
+Ok(endpoints)
+}
+
+#[api(
+protected: true,
+input: {
+properties: {
+name: {
+schema: ENTITY_NAME_SCHEMA,
+}
+},
+},
+returns: { type: WebhookConfig },
+access: {
+permission: &Permission::Privilege(&["system", "notifications"], 
PRIV_SYS_AUDIT, false),
+},
+)]
+/// Get a webhook endpoint.
+pub fn get_endpoint(name: String, rpcenv: &mut dyn RpcEnvironment) -> 
Result {
+let config = pbs_config::notifications::config()?;
+let endpoint = proxmox_notify::api::webhook::get_endpoint(&config, &name)?;
+
+rpcenv["digest"] = hex::encode(config.digest()).into();
+
+Ok(endpoint)
+}
+
+#[api(
+protected: true,
+input: {
+properties: {
+endpoint: {
+type: WebhookConfig,
+flatten: true,
+},
+},
+},
+access: {
+permission: &Permission::Privilege(&["system", "notifications"], 
PRIV_SYS_MODIFY, false),
+},
+)]
+/// Add a new webhook endpoint.
+pub fn add_endpoint(
+endpoint: WebhookConfig,
+_rpcenv: &mut dyn RpcEnvironment,
+) -> Result<(), Error> {
+let _lock = pbs_config::notifications::lock_config()?;
+let mut config = pbs_config::notifications::config()?;
+
+proxmox_notify::api::webhook::add_endpoint(&mut config, endpoint)?;
+
+pbs_config::notifications::save_config(config)?;
+Ok(())
+}
+
+#[api(
+protected: true,
+input: {
+properties: {
+name: {
+schema: ENTITY_NAME_SCHEMA,
+},
+updater: {
+type: WebhookConfigUpdater,
+flatten: true,
+},
+delete: {
+description: "List of properties to delete.",
+type: Array,
+optional: true,
+items: {
+type: DeleteableWebhookProperty,
+}
+},
+digest: {
+optional: true,
+schema: PROXMOX_CONFIG_DIGEST_SCHEMA,
+},
+},
+},
+access: {
+permission: &Permission::Privilege(&["system", "notifications"], 
PRIV_SYS_MODIFY, false),
+},
+)]
+/// Update webhook endpoint.
+pub fn update_endpoint(
+name: String,
+updater: WebhookConfigUpdater,
+delete: Option>,
+digest: Option,
+_rpcenv: &mut dyn RpcEnvironment,
+) -> Result<(), Error> {
+let _lock = pbs_config::notifications::lock_config()?;
+let mut config = pbs_config::notifications::config()?;
+let digest = digest.map(hex::decode).transpose()?;
+
+proxmox_notify::api::webhook::update_endpoint(
+&mut config,
+&name,
+updater,
+delete.as_deref(),
+digest.as_deref(),
+)?;
+
+pbs_config::notifications::save_config

[pve-devel] [PATCH manager v2 07/12] api: add routes for webhook notification endpoints

2024-07-12 Thread Lukas Wagner
These just call the API implementation via the perl-rs bindings.

Signed-off-by: Lukas Wagner 
---
 PVE/API2/Cluster/Notifications.pm | 263 +-
 1 file changed, 262 insertions(+), 1 deletion(-)

diff --git a/PVE/API2/Cluster/Notifications.pm 
b/PVE/API2/Cluster/Notifications.pm
index 10b611c9..eae2d436 100644
--- a/PVE/API2/Cluster/Notifications.pm
+++ b/PVE/API2/Cluster/Notifications.pm
@@ -108,6 +108,7 @@ __PACKAGE__->register_method ({
{ name => 'gotify' },
{ name => 'sendmail' },
{ name => 'smtp' },
+   { name => 'webhook' },
];
 
return $result;
@@ -144,7 +145,7 @@ __PACKAGE__->register_method ({
'type' => {
description => 'Type of the target.',
type  => 'string',
-   enum => [qw(sendmail gotify smtp)],
+   enum => [qw(sendmail gotify smtp webhook)],
},
'comment' => {
description => 'Comment',
@@ -1094,6 +1095,266 @@ __PACKAGE__->register_method ({
 }
 });
 
+my $webhook_properties = {
+name => {
+   description => 'The name of the endpoint.',
+   type => 'string',
+   format => 'pve-configid',
+},
+url => {
+   description => 'Server URL',
+   type => 'string',
+},
+method => {
+   description => 'HTTP method',
+   type => 'string',
+   enum => [qw(post put get)],
+},
+header => {
+   description => 'HTTP headers to set. These have to be formatted as'
+ . ' a property string in the format name=,value=',
+   type => 'array',
+   items => {
+   type => 'string',
+   },
+   optional => 1,
+},
+body => {
+   description => 'HTTP body, base64 encoded',
+   type => 'string',
+   optional => 1,
+},
+secret => {
+   description => 'Secrets to set. These have to be formatted as'
+ . ' a property string in the format name=,value=',
+   type => 'array',
+   items => {
+   type => 'string',
+   },
+   optional => 1,
+},
+comment => {
+   description => 'Comment',
+   type => 'string',
+   optional => 1,
+},
+disable => {
+   description => 'Disable this target',
+   type => 'boolean',
+   optional => 1,
+   default => 0,
+},
+};
+
+__PACKAGE__->register_method ({
+name => 'get_webhook_endpoints',
+path => 'endpoints/webhook',
+method => 'GET',
+description => 'Returns a list of all webhook endpoints',
+protected => 1,
+permissions => {
+   check => ['perm', '/mapping/notifications', ['Mapping.Modify']],
+   check => ['perm', '/mapping/notifications', ['Mapping.Audit']],
+},
+parameters => {
+   additionalProperties => 0,
+   properties => {},
+},
+returns => {
+   type => 'array',
+   items => {
+   type => 'object',
+   properties => {
+   %$webhook_properties,
+   'origin' => {
+   description => 'Show if this entry was created by a user or 
was built-in',
+   type  => 'string',
+   enum => [qw(user-created builtin modified-builtin)],
+   },
+   },
+   },
+   links => [ { rel => 'child', href => '{name}' } ],
+},
+code => sub {
+   my $config = PVE::Notify::read_config();
+   my $rpcenv = PVE::RPCEnvironment::get();
+
+   my $entities = eval {
+   $config->get_webhook_endpoints();
+   };
+   raise_api_error($@) if $@;
+
+   return $entities;
+}
+});
+
+__PACKAGE__->register_method ({
+name => 'get_webhook_endpoint',
+path => 'endpoints/webhook/{name}',
+method => 'GET',
+description => 'Return a specific webhook endpoint',
+protected => 1,
+permissions => {
+   check => ['or',
+   ['perm', '/mapping/notifications', ['Mapping.Modify']],
+   ['perm', '/mapping/notifications', ['Mapping.Audit']],
+   ],
+},
+parameters => {
+   additionalProperties => 0,
+   properties => {
+   name => {
+   type => 'string',
+   format => 'pve-configid',
+   description => 'Name of the endpoint.'
+   },
+   }
+},
+returns => {
+   type => 'object',
+   properties => {
+   %$webhook_properties,
+   digest => get_standard_option('pve-config-digest'),
+   }
+},
+code => sub {
+   my ($param) = @_;
+   my $name = extract_param($param, 'name');
+
+   my $config = PVE::Notify::read_config();
+   my $endpoint = eval {
+   $config->get_webhook_endpoint($name)
+   };
+
+   raise_api_error($@) if $@;
+   $endpoint->{digest} = $config->digest();
+
+   return $endpoint;
+}
+});
+
+__PACKAGE__->register_method ({
+name => 'create_webhook_endpoint',
+path => 'endpoints/webhook',
+

[pve-devel] [PATCH proxmox v2 01/12] notify: implement webhook targets

2024-07-12 Thread Lukas Wagner
This target type allows users to perform HTTP requests to arbitrary
third party (notification) services, for instance
ntfy.sh/Discord/Slack.

The configuration for these endpoints allows one to freely configure
the URL, HTTP Method, headers and body. The URL, header values and
body support handlebars templating to inject notification text,
metadata and secrets. Secrets are stored in the protected
configuration file (e.g. /etc/pve/priv/notification.cfg) as key value
pairs, allowing users to protect sensitive tokens/passwords.
Secrets are accessible in handlebar templating via the secrets.*
namespace, e.g. if there is a secret named 'token', a body
could contain '{{ secrets.token }}' to inject the token into the
payload.

A couple of handlebars helpers are also provided:
  - url-encoding (useful for templating in URLs)
  - escape (escape any control characters in strings)
  - json (print a property as json)

In the configuration, the body, header values and secret values
are stored in base64 encoding so that we can store any string we want.

Signed-off-by: Lukas Wagner 
---
 proxmox-notify/Cargo.toml   |   9 +-
 proxmox-notify/src/config.rs|  23 ++
 proxmox-notify/src/endpoints/mod.rs |   2 +
 proxmox-notify/src/endpoints/webhook.rs | 509 
 proxmox-notify/src/lib.rs   |  17 +
 5 files changed, 557 insertions(+), 3 deletions(-)
 create mode 100644 proxmox-notify/src/endpoints/webhook.rs

diff --git a/proxmox-notify/Cargo.toml b/proxmox-notify/Cargo.toml
index 7801814d..484aff19 100644
--- a/proxmox-notify/Cargo.toml
+++ b/proxmox-notify/Cargo.toml
@@ -9,13 +9,15 @@ exclude.workspace = true
 
 [dependencies]
 anyhow.workspace = true
-base64.workspace = true
+base64 = { workspace = true, optional = true }
 const_format.workspace = true
 handlebars = { workspace = true }
+http = { workspace = true, optional = true }
 lettre = { workspace = true, optional = true }
 log.workspace = true
 mail-parser = { workspace = true, optional = true }
 openssl.workspace = true
+percent-encoding = { workspace = true, optional = true }
 regex.workspace = true
 serde = { workspace = true, features = ["derive"] }
 serde_json.workspace = true
@@ -31,10 +33,11 @@ proxmox-time.workspace = true
 proxmox-uuid = { workspace = true, features = ["serde"] }
 
 [features]
-default = ["sendmail", "gotify", "smtp"]
+default = ["sendmail", "gotify", "smtp", "webhook"]
 mail-forwarder = ["dep:mail-parser", "dep:proxmox-sys"]
-sendmail = ["dep:proxmox-sys"]
+sendmail = ["dep:proxmox-sys", "dep:base64"]
 gotify = ["dep:proxmox-http"]
 pve-context = ["dep:proxmox-sys"]
 pbs-context = ["dep:proxmox-sys"]
 smtp = ["dep:lettre"]
+webhook = ["dep:base64", "dep:http", "dep:percent-encoding", 
"dep:proxmox-http"]
diff --git a/proxmox-notify/src/config.rs b/proxmox-notify/src/config.rs
index 789c4a7d..4d0b53f7 100644
--- a/proxmox-notify/src/config.rs
+++ b/proxmox-notify/src/config.rs
@@ -57,6 +57,17 @@ fn config_init() -> SectionConfig {
 GOTIFY_SCHEMA,
 ));
 }
+#[cfg(feature = "webhook")]
+{
+use crate::endpoints::webhook::{WebhookConfig, WEBHOOK_TYPENAME};
+
+const WEBHOOK_SCHEMA: &ObjectSchema = 
WebhookConfig::API_SCHEMA.unwrap_object_schema();
+config.register_plugin(SectionConfigPlugin::new(
+WEBHOOK_TYPENAME.to_string(),
+Some(String::from("name")),
+WEBHOOK_SCHEMA,
+));
+}
 
 const MATCHER_SCHEMA: &ObjectSchema = 
MatcherConfig::API_SCHEMA.unwrap_object_schema();
 config.register_plugin(SectionConfigPlugin::new(
@@ -110,6 +121,18 @@ fn private_config_init() -> SectionConfig {
 ));
 }
 
+#[cfg(feature = "webhook")]
+{
+use crate::endpoints::webhook::{WebhookPrivateConfig, 
WEBHOOK_TYPENAME};
+
+const WEBHOOK_SCHEMA: &ObjectSchema =
+WebhookPrivateConfig::API_SCHEMA.unwrap_object_schema();
+config.register_plugin(SectionConfigPlugin::new(
+WEBHOOK_TYPENAME.to_string(),
+Some(String::from("name")),
+WEBHOOK_SCHEMA,
+));
+}
 config
 }
 
diff --git a/proxmox-notify/src/endpoints/mod.rs 
b/proxmox-notify/src/endpoints/mod.rs
index 97f79fcc..f20bee21 100644
--- a/proxmox-notify/src/endpoints/mod.rs
+++ b/proxmox-notify/src/endpoints/mod.rs
@@ -4,5 +4,7 @@ pub mod gotify;
 pub mod sendmail;
 #[cfg(feature = "smtp")]
 pub mod smtp;
+#[cfg(feature = "webhook")]
+pub mod webhook;
 
 mod common;
diff --git a/proxmox-notify/src/endpoints/webhook.rs 
b/proxmox-notify/src/endpoints/webhook.rs
new file mode 100644
index ..7e976f6b
--- /dev/null
+++ b/proxmox-notify/src/endpoints/webhook.rs
@@ -0,0 +1,509 @@
+use handlebars::{
+Context as HandlebarsContext, Handlebars, Helper, HelperResult, Output, 
RenderContext,
+RenderError as HandlebarsRenderError,
+};
+use http::Request;
+use percent_encoding::AsciiSet;
+use proxmox_schema::property_string::PropertyString;
+use 

[pve-devel] [RFC many v2 00/12] notifications: add support for webhook endpoints

2024-07-12 Thread Lukas Wagner
Sending as an RFC because I don't want this merged yet; that being
said, the feature should be mostly finished at this point, I'd
appreciate any reviews and feedback.

This series adds support for webhook notification targets to PVE
and PBS.

A webhook is a HTTP API route provided by a third-party service that
can be used to inform the third-party about an event. In our case,
we can easily interact with various third-party notification/messaging
systems and send PVE/PBS notifications via this service.
The changes were tested against ntfy.sh, Discord and Slack.

The configuration of webhook targets allows one to configure:
  - The URL
  - The HTTP method (GET/POST/PUT)
  - HTTP Headers
  - Body

One can use handlebar templating to inject notification text and metadata
in the url, headers and body.

One challenge is the handling of sensitve tokens and other secrets.
Since the endpoint is completely generic, we cannot know in advance
whether the body/header/url contains sensitive values.
Thus we add 'secrets' which are stored in the protected config only
accessible by root (e.g. /etc/pve/priv/notifications.cfg). These
secrets are accessible in URLs/headers/body via templating:

  Url: https://example.com/{{ secrets.token }}

Secrets can only be set and updated, but never retrieved via the API.
In the UI, secrets are handled like other secret tokens/passwords.

Bumps for PVE:
  - libpve-rs-perl needs proxmox-notify bumped
  - pve-manager needs bumped proxmox-widget-toolkit and libpve-rs-perl bumped
  - proxmox-mail-forward needs proxmox-notify bumped

Bumps for PBS:
  - proxmox-backup needs proxmox-notify bumped
  - proxmox-mail-forward needs proxmox-notify bumped


Changes v1 -> v2:
  - Rebase proxmox-notify changes

proxmox:

Lukas Wagner (2):
  notify: implement webhook targets
  notify: add api for webhook targets

 proxmox-notify/Cargo.toml   |   9 +-
 proxmox-notify/src/api/mod.rs   |  20 +
 proxmox-notify/src/api/webhook.rs   | 406 +++
 proxmox-notify/src/config.rs|  23 ++
 proxmox-notify/src/endpoints/mod.rs |   2 +
 proxmox-notify/src/endpoints/webhook.rs | 509 
 proxmox-notify/src/lib.rs   |  17 +
 7 files changed, 983 insertions(+), 3 deletions(-)
 create mode 100644 proxmox-notify/src/api/webhook.rs
 create mode 100644 proxmox-notify/src/endpoints/webhook.rs


proxmox-perl-rs:

Lukas Wagner (2):
  common: notify: add bindings for webhook API routes
  common: notify: add bindings for get_targets

 common/src/notify.rs | 72 
 1 file changed, 72 insertions(+)


proxmox-widget-toolkit:

Lukas Wagner (1):
  notification: add UI for adding/updating webhook targets

 src/Makefile  |   1 +
 src/Schema.js |   5 +
 src/panel/WebhookEditPanel.js | 417 ++
 3 files changed, 423 insertions(+)
 create mode 100644 src/panel/WebhookEditPanel.js


pve-manager:

Lukas Wagner (2):
  api: notifications: use get_targets impl from proxmox-notify
  api: add routes for webhook notification endpoints

 PVE/API2/Cluster/Notifications.pm | 297 ++
 1 file changed, 263 insertions(+), 34 deletions(-)


pve-docs:

Lukas Wagner (1):
  notification: add documentation for webhook target endpoints.

 notifications.adoc | 93 ++
 1 file changed, 93 insertions(+)


proxmox-backup:

Lukas Wagner (3):
  api: notification: add API routes for webhook targets
  ui: utils: enable webhook edit window
  docs: notification: add webhook endpoint documentation

 docs/notifications.rst   | 100 +
 src/api2/config/notifications/mod.rs |   2 +
 src/api2/config/notifications/webhook.rs | 175 +++
 www/Utils.js |   5 +
 4 files changed, 282 insertions(+)
 create mode 100644 src/api2/config/notifications/webhook.rs


proxmox-mail-forward:

Lukas Wagner (1):
  bump proxmox-notify dependency

 Cargo.toml | 2 +-
 debian/control | 8 
 2 files changed, 5 insertions(+), 5 deletions(-)


Summary over all repositories:
  19 files changed, 2121 insertions(+), 42 deletions(-)

-- 
Generated by git-murpp 0.7.1


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel



[pve-devel] [PATCH proxmox-mail-forward v2 12/12] bump proxmox-notify dependency

2024-07-12 Thread Lukas Wagner
Signed-off-by: Lukas Wagner 
---
 Cargo.toml | 2 +-
 debian/control | 8 
 2 files changed, 5 insertions(+), 5 deletions(-)

diff --git a/Cargo.toml b/Cargo.toml
index f39d118..49ca079 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -20,4 +20,4 @@ nix = "0.26"
 syslog = "6.0"
 
 proxmox-sys = "0.5.3"
-proxmox-notify = {version = "0.4", features = ["mail-forwarder", 
"pve-context", "pbs-context"] }
+proxmox-notify = {version = "0.5", features = ["mail-forwarder", 
"pve-context", "pbs-context"] }
diff --git a/debian/control b/debian/control
index 7329a24..0ab74a9 100644
--- a/debian/control
+++ b/debian/control
@@ -6,10 +6,10 @@ Build-Depends: cargo:native,
librust-anyhow-1+default-dev,
librust-log-0.4+default-dev (>= 0.4.17-~~),
librust-nix-0.26+default-dev,
-   librust-proxmox-notify-0.4+default-dev,
-   librust-proxmox-notify-0.4+mail-forwarder-dev,
-   librust-proxmox-notify-0.4+pbs-context-dev,
-   librust-proxmox-notify-0.4+pve-context-dev,
+   librust-proxmox-notify-0.5+default-dev,
+   librust-proxmox-notify-0.5+mail-forwarder-dev,
+   librust-proxmox-notify-0.5+pbs-context-dev,
+   librust-proxmox-notify-0.5+pve-context-dev,
librust-proxmox-sys-0.5+default-dev (>= 0.5.3-~~),
librust-syslog-6+default-dev,
libstd-rust-dev,
-- 
2.39.2



___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel



[pve-devel] [PATCH widget-toolkit v2 05/12] notification: add UI for adding/updating webhook targets

2024-07-12 Thread Lukas Wagner
The widgets for editing the headers/secrets were adapted from
the 'Tag Edit' dialog from PVE's datacenter options.

Apart from that, the new dialog is rather standard. I've decided
to put the http method and url in a single row, mostly to
save space and also to make it analogous to how an actual http request
is structured (VERB URL, followed by headers, followed by the body).

The secrets are a mechanism to store tokens/passwords in the
protected notification config. Secrets are accessible via
templating in the URL, headers and body via {{ secrets.NAME }}.
Secrets can only be set/updated, but not retrieved/displayed.

Signed-off-by: Lukas Wagner 
---
 src/Makefile  |   1 +
 src/Schema.js |   5 +
 src/panel/WebhookEditPanel.js | 417 ++
 3 files changed, 423 insertions(+)
 create mode 100644 src/panel/WebhookEditPanel.js

diff --git a/src/Makefile b/src/Makefile
index 0478251..cfaffd7 100644
--- a/src/Makefile
+++ b/src/Makefile
@@ -78,6 +78,7 @@ JSSRC=\
panel/StatusView.js \
panel/TfaView.js\
panel/NotesView.js  \
+   panel/WebhookEditPanel.js   \
window/Edit.js  \
window/PasswordEdit.js  \
window/SafeDestroy.js   \
diff --git a/src/Schema.js b/src/Schema.js
index 42541e0..cd1c306 100644
--- a/src/Schema.js
+++ b/src/Schema.js
@@ -65,6 +65,11 @@ Ext.define('Proxmox.Schema', { // a singleton
ipanel: 'pmxGotifyEditPanel',
iconCls: 'fa-bell-o',
},
+   webhook: {
+   name: 'Webhook',
+   ipanel: 'pmxWebhookEditPanel',
+   iconCls: 'fa-bell-o',
+   },
 },
 
 // to add or change existing for product specific ones
diff --git a/src/panel/WebhookEditPanel.js b/src/panel/WebhookEditPanel.js
new file mode 100644
index 000..dfc7f3f
--- /dev/null
+++ b/src/panel/WebhookEditPanel.js
@@ -0,0 +1,417 @@
+Ext.define('Proxmox.panel.WebhookEditPanel', {
+extend: 'Proxmox.panel.InputPanel',
+xtype: 'pmxWebhookEditPanel',
+mixins: ['Proxmox.Mixin.CBind'],
+onlineHelp: 'notification_targets_webhook',
+
+type: 'webhook',
+
+columnT: [
+
+],
+
+column1: [
+   {
+   xtype: 'pmxDisplayEditField',
+   name: 'name',
+   cbind: {
+   value: '{name}',
+   editable: '{isCreate}',
+   },
+   fieldLabel: gettext('Endpoint Name'),
+   allowBlank: false,
+   },
+],
+
+column2: [
+   {
+   xtype: 'proxmoxcheckbox',
+   name: 'enable',
+   fieldLabel: gettext('Enable'),
+   allowBlank: false,
+   checked: true,
+   },
+],
+
+columnB: [
+   {
+   layout: 'hbox',
+   border: false,
+   margin: '0 0 5 0',
+   items: [
+   {
+   xtype: 'displayfield',
+   value: gettext('Method/URL:'),
+   width: 125,
+   },
+   {
+   xtype: 'proxmoxKVComboBox',
+   name: 'method',
+   //fieldLabel: gettext('Method'),
+   editable: false,
+   value: 'post',
+   comboItems: [
+   ['post', 'POST'],
+   ['put', 'PUT'],
+   ['get', 'GET'],
+   ],
+   width: 80,
+   margin: '0 5 0 0',
+   },
+   {
+   xtype: 'proxmoxtextfield',
+   //fieldLabel: gettext('URL'),
+   name: 'url',
+   allowBlank: false,
+   flex: 4,
+   },
+   ],
+   },
+   {
+   xtype: 'pmxWebhookKeyValueList',
+   name: 'header',
+   fieldLabel: gettext('Headers'),
+   maskValues: false,
+   cbind: {
+   isCreate: '{isCreate}',
+   },
+   },
+   {
+   xtype: 'textarea',
+   fieldLabel: gettext('Body'),
+   name: 'body',
+   allowBlank: true,
+   minHeight: '150',
+   fieldStyle: {
+   'font-family': 'monospace',
+   },
+   margin: '15 0 0 0',
+   },
+   {
+   xtype: 'pmxWebhookKeyValueList',
+   name: 'secret',
+   fieldLabel: gettext('Secrets'),
+   maskValues: true,
+   cbind: {
+   isCreate: '{isCreate}',
+   },
+   },
+   {
+   xtype: 'proxmoxtextfield',
+   name: 'comment',
+   fieldLabel: gettext('Comment'),
+   cbind: {
+   deleteEmpty: '{!isCreate}',
+   },
+   },
+],
+
+onSetValues: (values) => {
+   values.enable = !values.disable;
+
+   if (values.body) {
+   values.body = atob

[pve-devel] [PATCH proxmox v2 02/12] notify: add api for webhook targets

2024-07-12 Thread Lukas Wagner
All in all pretty similar to other endpoint APIs.
One thing worth noting is how secrets are handled. We never ever
return the values of previously stored secrets in get_endpoint(s)
calls, but only a list of the names of all secrets. This is needed
to build the UI, where we display all secrets that were set before in
a table.

For update calls, one is supposed to send all secrets that should be
kept and updated. If the value should be updated, the name and value
is expected, and if the current value should preseved, only the name
is sent. If a secret's name is not present in the updater, it will be
dropped. If 'secret' is present in the 'delete' array, all secrets
will be dropped, apart from those which are also set/preserved in the
same update call.

Signed-off-by: Lukas Wagner 
---
 proxmox-notify/src/api/mod.rs |  20 ++
 proxmox-notify/src/api/webhook.rs | 406 ++
 2 files changed, 426 insertions(+)
 create mode 100644 proxmox-notify/src/api/webhook.rs

diff --git a/proxmox-notify/src/api/mod.rs b/proxmox-notify/src/api/mod.rs
index a7f6261c..7f823bc7 100644
--- a/proxmox-notify/src/api/mod.rs
+++ b/proxmox-notify/src/api/mod.rs
@@ -15,6 +15,8 @@ pub mod matcher;
 pub mod sendmail;
 #[cfg(feature = "smtp")]
 pub mod smtp;
+#[cfg(feature = "webhook")]
+pub mod webhook;
 
 // We have our own, local versions of http_err and http_bail, because
 // we don't want to wrap the error in anyhow::Error. If we were to do that,
@@ -54,6 +56,9 @@ pub enum EndpointType {
 /// Gotify endpoint
 #[cfg(feature = "gotify")]
 Gotify,
+/// Webhook endpoint
+#[cfg(feature = "webhook")]
+Webhook,
 }
 
 #[api]
@@ -113,6 +118,17 @@ pub fn get_targets(config: &Config) -> Result, 
HttpError> {
 })
 }
 
+#[cfg(feature = "webhook")]
+for endpoint in webhook::get_endpoints(config)? {
+targets.push(Target {
+name: endpoint.name,
+origin: endpoint.origin.unwrap_or(Origin::UserCreated),
+endpoint_type: EndpointType::Webhook,
+disable: endpoint.disable,
+comment: endpoint.comment,
+})
+}
+
 Ok(targets)
 }
 
@@ -145,6 +161,10 @@ fn ensure_endpoint_exists(#[allow(unused)] config: 
&Config, name: &str) -> Resul
 {
 exists = exists || smtp::get_endpoint(config, name).is_ok();
 }
+#[cfg(feature = "webhook")]
+{
+exists = exists || webhook::get_endpoint(config, name).is_ok();
+}
 
 if !exists {
 http_bail!(NOT_FOUND, "endpoint '{name}' does not exist")
diff --git a/proxmox-notify/src/api/webhook.rs 
b/proxmox-notify/src/api/webhook.rs
new file mode 100644
index ..b7f17c55
--- /dev/null
+++ b/proxmox-notify/src/api/webhook.rs
@@ -0,0 +1,406 @@
+use proxmox_http_error::HttpError;
+use proxmox_schema::property_string::PropertyString;
+
+use crate::api::http_err;
+use crate::endpoints::webhook::{
+DeleteableWebhookProperty, KeyAndBase64Val, WebhookConfig, 
WebhookConfigUpdater,
+WebhookPrivateConfig, WEBHOOK_TYPENAME,
+};
+use crate::{http_bail, Config};
+
+use super::remove_private_config_entry;
+use super::set_private_config_entry;
+
+/// Get a list of all webhook endpoints.
+///
+/// The caller is responsible for any needed permission checks.
+/// Returns a list of all webhook endpoints or a `HttpError` if the config is
+/// erroneous (`500 Internal server error`).
+pub fn get_endpoints(config: &Config) -> Result, HttpError> 
{
+let mut endpoints: Vec = config
+.config
+.convert_to_typed_array(WEBHOOK_TYPENAME)
+.map_err(|e| http_err!(NOT_FOUND, "Could not fetch endpoints: {e}"))?;
+
+for endpoint in &mut endpoints {
+let priv_config: WebhookPrivateConfig = config
+.private_config
+.lookup(WEBHOOK_TYPENAME, &endpoint.name)
+.unwrap_or_default();
+
+let mut secret_names = Vec::new();
+for secret in priv_config.secret {
+secret_names.push(
+KeyAndBase64Val {
+name: secret.name.clone(),
+value: None,
+}
+.into(),
+)
+}
+
+endpoint.secret = secret_names;
+}
+
+Ok(endpoints)
+}
+
+/// Get webhook endpoint with given `name`
+///
+/// The caller is responsible for any needed permission checks.
+/// Returns the endpoint or a `HttpError` if the endpoint was not found (`404 
Not found`).
+pub fn get_endpoint(config: &Config, name: &str) -> Result {
+let mut endpoint: WebhookConfig = config
+.config
+.lookup(WEBHOOK_TYPENAME, name)
+.map_err(|_| http_err!(NOT_FOUND, "endpoint '{name}' not found"))?;
+
+let priv_config: Option = config
+.private_config
+.lookup(WEBHOOK_TYPENAME, &endpoint.name)
+.ok();
+
+let mut secret_names = Vec::new();
+if let Some(priv_config) = priv_config {
+for secret in &priv_config.secret {
+secret_names.p

[pve-devel] applied-series: [PATCH kernel 1/2] add fix for CIFS client memory leak

2024-07-12 Thread Thomas Lamprecht
On 10/07/2024 13:37, Fiona Ebner wrote:
> As reported in the community forum [0], there currently is a memory
> leak in the CIFS client code. Reproduced by running a backup with CIFS
> target storage:
> 
>> while true; do vzdump 101 --storage cifs --prune-backups keep-last=1; echo 3 
>> > /proc/sys/vm/drop_caches; done
> 
> A fix was found on the kernel mailing list tagged for stable v6.6+
> and it does solve the issue, but is not yet included in any (stable)
> kernels.
> 
> [0]: https://forum.proxmox.com/threads/147603/post-682388
> 
> Signed-off-by: Fiona Ebner 
> ---
>  ...ix-pagecache-leak-when-do-writepages.patch | 108 ++
>  1 file changed, 108 insertions(+)
>  create mode 100644 
> patches/kernel/0018-cifs-fix-pagecache-leak-when-do-writepages.patch
> 
>

applied both patches, thanks!


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel