Re: [pve-devel] [PATCH v4 many 00/11] notifications: add SMTP endpoint

[Dietmar Maurer]

> This patch series adds support for a new notification endpoint type,
> smtp. As the name suggests, this new endpoint allows PVE to talk
> to SMTP server directly, without using the system's MTA (postfix).

Isn't this totally unreliable? What if the server responds with a 
temporary error code? (An MTA retries several times).


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH v4 many 00/11] notifications: add SMTP endpoint

[Dietmar Maurer]

> On 11/8/23 16:52, Dietmar Maurer wrote:
> >> This patch series adds support for a new notification endpoint type,
> >> smtp. As the name suggests, this new endpoint allows PVE to talk
> >> to SMTP server directly, without using the system's MTA (postfix).
> > 
> > Isn't this totally unreliable? What if the server responds with a
> > temporary error code? (An MTA retries several times).
> 
> The notification system has no mechanism yet for queuing/retries,
> so yes, at the moment a SMTP endpoint would indeed be less reliable than 
> a 'sendmail' endpoint. I'm not sure though if I would call it
> 'totally unreliable'.

This kind of notification system is quite popular for (PHP) web-sites contact 
form. I have seen many over-simplified implementation overs the years,
and yes, it is totally unreliable.

That is why we always used an MTA to deliver mails...


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH v4 many 00/11] notifications: add SMTP endpoint

[Dietmar Maurer]

> > As a compromise, maybe we could just add a note to the docs
> > that discusses the reliability aspects of 'sendmail' vs 'smtp'
> > endpoints?
> > 
> 
> Sure, for now adding a general hint to the documentation that they are
> send one-shot only would be good.

Ok for me.


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH widget-toolkit] node: repos: Use won't rather than don't

[Dietmar Maurer]

Do you also plan to fix those typos in the translation files?
Else we need to re-tralstale them for all languages!

> On 24.11.2023 15:58 CET Maximiliano Sandoval  wrote:
> 
>  
> It would be preferable to use "won't" but I would rather err on the safe
> side when it comes to escapes in gettext.
> 
> Signed-off-by: Maximiliano Sandoval 
> ---
>  src/node/APTRepositories.js | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/src/node/APTRepositories.js b/src/node/APTRepositories.js
> index 4e74da2..f27e713 100644
> --- a/src/node/APTRepositories.js
> +++ b/src/node/APTRepositories.js
> @@ -471,7 +471,7 @@ Ext.define('Proxmox.node.APTRepositories', {
>  
>   if (!enterprise && !nosubscription && !test) {
>   addCritical(
> - Ext.String.format(gettext('No {0} repository is enabled, 
> you do not get any updates!'), vm.get('product')),
> + Ext.String.format(gettext('No {0} repository is enabled, 
> you would not get any updates!'), vm.get('product')),
>   );
>   } else if (errors.length > 0) {
>   // nothing extra, just avoid that we show "get updates"
> -- 
> 2.39.2
> 
> 
> 
> ___
> pve-devel mailing list
> pve-devel@lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH manager] ui: mark strings translatable

[Dietmar Maurer]

> The string `proxmox.Utils.defaultText + ' (free)'` was inlined as
> `Default (Free)` cutting translatable strings makes them harder or even
> impossible to translate in certain languages.

Well, this also duplicates the number of things to translate!

So what languages are the problem exactly? Please can you
give me an example?


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH manager] ui: mark strings translatable

[Dietmar Maurer]

To be more clear, I would use:

proxmox.Utils.defaultText + ' (' + gettext('Free') + ')'


> On 28.11.2023 15:39 CET Dietmar Maurer  wrote:
> 
>  
> > The string `proxmox.Utils.defaultText + ' (free)'` was inlined as
> > `Default (Free)` cutting translatable strings makes them harder or even
> > impossible to translate in certain languages.
> 
> Well, this also duplicates the number of things to translate!
> 
> So what languages are the problem exactly? Please can you
> give me an example?
> 
> 
> ___
> pve-devel mailing list
> pve-devel@lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH manager] ui: mark strings translatable

[Dietmar Maurer]

> I'm taking on a lot of contributions to translations and the common complaint
> I hear is that not all can be translated correctly due to such tricks (or just
> missing gettext), most translators care much more about a correct translation
> than some over-optimized ones than then break depending on context or target
> language.
> 
> Also, changing this now results in only a few new strings to be translated for
> those languages we get common contributions for, so not really much effort to
> catch up again.
> 
> > So what languages are the problem exactly? Please can you
> > give me an example?
> 
> Any right-to-left languages could be broken.

If its only right-to-left languages, we could just add another helper
to assemble lists in the correct order?


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH manager] ui: mark strings translatable

[Dietmar Maurer]

> > So what languages are the problem exactly? Please can you
> > give me an example?
> 
> Any right-to-left languages could be broken.

This is not really true. RTL languages just render string in the other 
direction, so string concatenation still works as expected.

> 
> Am 28/11/2023 um 15:46 schrieb Dietmar Maurer:
> > To be more clear, I would use:
> > 
> > proxmox.Utils.defaultText + ' (' + gettext('Free') + ')'


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH widget-toolkit] i18n: mark strings as translatable

[Dietmar Maurer]

> Do note the following examples:
> 
> #: pve-manager/www/manager6/qemu/IPConfigEdit.js:97
> #: pve-manager/www/manager6/qemu/IPConfigEdit.js:155
> msgid "DHCP"
> msgstr "بروتوكول التهيئة الآلية للمضيفين (DHCP)"
> 
> #: pve-manager/www/manager6/qemu/IPConfigEdit.js:163
> msgid "SLAAC"
> msgstr "Autoconfiguration SLAAC"
> 
> #: pve-manager/www/manager6/ceph/CephInstallWizard.js:476
> msgid "First Ceph monitor"
> msgstr "أول جهاز مراقبة"
> 
> What would be the purpose of removing the translation of the words
> themselves if they are already being translated elsewhere.

IMHO all those are terribly wrong. I would remove them immediately.


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [RFC docs 13/13] vzdump: add section about backup fleecing

[Dietmar Maurer]

Stupid question: Wouldn't It be much easier to add a simple IO-buffer
with limited capacity, implemented inside the RUST backup code?


> +WARNING: Theoretically, the fleecing image can grow to the same size as the
> +original image, e.g. if the guest re-writes a whole disk while the backup is
> +busy with another disk.
___
> pve-devel mailing list
> pve-devel@lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [RFC docs 13/13] vzdump: add section about backup fleecing

[Dietmar Maurer]

> >>Stupid question: Wouldn't It be much easier to add a simple IO-buffer
> >>with limited capacity, implemented inside the RUST backup code?
> 
> At work, we are running a backup cluster on remote location with hdd , 
> and a production cluster with super fast nvme,
> and sometimes I have really big write spikes (in GB/s), so it's
> impossible for the backup storage or network to handle it without
> increase latency or saturate link.
>  
> So with limited capacity (how much ? in memory ?), I don't think it
> solve the problem. If the buffer is full, the vm write will hang.

Ok, I can see the problem...


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH manager] ui: node: system: avoid using 'Syslog'

[Dietmar Maurer]

> The information gathered by the API call comes from the systemd
> journal. While 'Syslog' could be interpreted as a shorthand for
> "System Log", it's better to be explicit to avoid any confusion.

> - title: 'Syslog',
> + title: gettext('System Log'),

From Wikipedia: https://en.wikipedia.org/wiki/Syslog

> In computing, syslog /ˈsɪslɒɡ/ is a standard for message logging.

IMHO, it is a commonly used technical term, and that is why we
do not even translate it. Every reasonable admin known what is meant!

With your change:

-   title: 'Syslog',
+   title: gettext('System Log'),

we now need to translate "System Log" to >20 different languages,
and I am quite sure those translation makes it less clear what
it actually refers to.


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH manager] ui: node: system: avoid using 'Syslog'

[Dietmar Maurer]

I can also imaging using "Events" instead of "Syslog".

-   title: 'Syslog',
+   title: gettext('Events'),

IMHO this is easier to translate.


> With your change:
> 
> - title: 'Syslog',
> + title: gettext('System Log'),
> 
> we now need to translate "System Log" to >20 different languages,
> and I am quite sure those translation makes it less clear what
> it actually refers to.


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH manager] ui: node: system: avoid using 'Syslog'

[Dietmar Maurer]

> while I don't mind (at all!) that that part of the UI/API is labelled syslog 
> (I don't think it's hard to understand that it gives you the system logs of 
> that node, and "syslog" is a bit like "Kleenex" in that regard ;)) - I do 
> have to disagree here ;) journald does a lot more than just collecting 
> syslog(3) messages - it also collects stdout/stderr of systemd-managed 
> services (unless configured in a non-default fashion), messages explicitly 
> sent to it over its own interface (like those by systemd ;)), and messages 
> from the kernel (like dmesg), including the audit subsystem..

IMHO, those are just Implementation details of systemd init..


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH storage 1/1] storage/plugins: pass scfg to parse_volname

[Dietmar Maurer]

> On 29.2.2024 16:09 CET Roland Kammerer via pve-devel 
>  wrote:
> All in all, yes, this is specific for our use case, otherwise
> parse_volname would already have that additional parameter as all the
> other plugin functions, but I don't see where this would hurt existing
> code, and it certainly helps us to enable reassigning disks to VMs.
> Passing in a param all other functions already get access to also does
> not sound too terrible to me.
> 
> If there are further questions please feel free to ask.

Are you aware that parse_volname() is sometimes called for
all volumes, i.e inside volume_is_base_and_used().

Would that be fast enough? IMHO its a bad idea to make a network query for each 
volume there...


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH proxmox] proxmox-acme: add api-types feature

[Dietmar Maurer]

Because AccountData is exposed via our API (currently as type Object).

Signed-off-by: Dietmar Maurer 
---
 proxmox-acme/Cargo.toml | 3 +++
 proxmox-acme/src/account.rs | 7 +++
 proxmox-acme/src/eab.rs | 5 +
 3 files changed, 15 insertions(+)

diff --git a/proxmox-acme/Cargo.toml b/proxmox-acme/Cargo.toml
index 8f8f6e1..857c61d 100644
--- a/proxmox-acme/Cargo.toml
+++ b/proxmox-acme/Cargo.toml
@@ -19,6 +19,8 @@ openssl.workspace = true
 # For the client
 native-tls = { workspace = true, optional = true }
 
+proxmox-schema = { workspace = true, optional = true, features = [ 
"api-macro", "api-types" ] }
+
 [dependencies.ureq]
 optional = true
 version = "2.4"
@@ -27,6 +29,7 @@ features = [ "native-tls", "gzip" ]
 
 [features]
 default = []
+api-types = [ "dep:proxmox-schema" ]
 client = ["ureq", "native-tls"]
 
 [dev-dependencies]
diff --git a/proxmox-acme/src/account.rs b/proxmox-acme/src/account.rs
index 9f3af26..e244c09 100644
--- a/proxmox-acme/src/account.rs
+++ b/proxmox-acme/src/account.rs
@@ -279,6 +279,7 @@ impl CertificateRevocation<'_> {
 }
 
 /// Status of an ACME account.
+#[cfg_attr(feature="api-types", proxmox_schema::api())]
 #[derive(Clone, Copy, Eq, PartialEq, Deserialize, Serialize)]
 #[serde(rename_all = "camelCase")]
 pub enum AccountStatus {
@@ -309,6 +310,12 @@ impl AccountStatus {
 }
 }
 
+#[cfg_attr(feature="api-types", proxmox_schema::api(
+properties: {
+extra: { type: Object, properties: {}, additional_properties: true },
+contact: { type: Array, items: { type: String, description: "Contact 
Info." }}
+}
+))]
 /// ACME Account data. This is the part of the account returned from and 
possibly sent to the ACME
 /// provider. Some fields may be uptdated by the user via a request to the 
account location, others
 /// may not be changed.
diff --git a/proxmox-acme/src/eab.rs b/proxmox-acme/src/eab.rs
index a4c0642..f006a3f 100644
--- a/proxmox-acme/src/eab.rs
+++ b/proxmox-acme/src/eab.rs
@@ -14,11 +14,16 @@ struct Protected {
 kid: String,
 }
 
+#[cfg_attr(feature="api-types", proxmox_schema::api())]
+/// External Account Bindings
 #[derive(Debug, Serialize, Deserialize, Clone)]
 #[serde(rename_all = "camelCase")]
 pub struct ExternalAccountBinding {
+/// JOSE Header (see RFC 7515)
 protected: String,
+/// Payload
 payload: String,
+/// HMAC signature
 signature: String,
 }
 
-- 
2.39.2


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH proxmox] proxmox-acme: derive PartialEq for API types

[Dietmar Maurer]

Signed-off-by: Dietmar Maurer 
---
 proxmox-acme/src/account.rs | 2 +-
 proxmox-acme/src/eab.rs | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/proxmox-acme/src/account.rs b/proxmox-acme/src/account.rs
index e244c09..7f00143 100644
--- a/proxmox-acme/src/account.rs
+++ b/proxmox-acme/src/account.rs
@@ -319,7 +319,7 @@ impl AccountStatus {
 /// ACME Account data. This is the part of the account returned from and 
possibly sent to the ACME
 /// provider. Some fields may be uptdated by the user via a request to the 
account location, others
 /// may not be changed.
-#[derive(Clone, Deserialize, Serialize)]
+#[derive(Clone, PartialEq, Deserialize, Serialize)]
 #[serde(rename_all = "camelCase")]
 pub struct AccountData {
 /// The current account status.
diff --git a/proxmox-acme/src/eab.rs b/proxmox-acme/src/eab.rs
index f006a3f..9d044f3 100644
--- a/proxmox-acme/src/eab.rs
+++ b/proxmox-acme/src/eab.rs
@@ -16,7 +16,7 @@ struct Protected {
 
 #[cfg_attr(feature="api-types", proxmox_schema::api())]
 /// External Account Bindings
-#[derive(Debug, Serialize, Deserialize, Clone)]
+#[derive(Debug, Serialize, Deserialize, Clone, PartialEq)]
 #[serde(rename_all = "camelCase")]
 pub struct ExternalAccountBinding {
 /// JOSE Header (see RFC 7515)
-- 
2.39.2


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH qemu-server v2 4/4] parse vm config: remove "\s*" from multi-line comment regex

[Dietmar Maurer]

> On 02/24/2022 3:49 PM Stefan Sterz  wrote:
> 
>  
> To be consistent with PBS's implementation of multi-line comments
> remove "\s*" here too. Since the regex isn't lazy .* matches
> everything \s* would anyway.

But the old regex trimm spaces from the end, so this is quite different!


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH proxmox-openid-rs] fix Open ID with Azure as provider

[Dietmar Maurer]

>  let response = if let Method::POST = request.method {
> -req.send(&*request.body)
> +let bytes = request.body.as_slice();
> +req.send_bytes(bytes)

Does this have the side effect of changing the transfer encoding? If so, it is 
worth to add an inline comment.


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH pve-docs] added Memory Encryption documentation

[Dietmar Maurer]

Live migration works?

> +Limitations:
> +
> +* Memory usage on host is always wrong and around 82% Usage
> +* Snapshots do not work
> +* edk2-OVMF required
> +* Recommendable: VirtIO RNG for more entropy (VMs sometimes will not


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] vncpropxy question

[Dietmar Maurer]

in qemu-server, I wonder why we set $ENV{LC_PVE_TICKET} conditionally? Does not 
make any sense to me, because it make all other connection failing...


diff --git a/PVE/API2/Qemu.pm b/PVE/API2/Qemu.pm
index 99b426e..c6a3ac1 100644
--- a/PVE/API2/Qemu.pm
+++ b/PVE/API2/Qemu.pm
@@ -2102,7 +2102,7 @@ __PACKAGE__->register_method({
 
} else {
 
-   $ENV{LC_PVE_TICKET} = $password if $websocket; # set ticket 
with "qm vncproxy"
+   $ENV{LC_PVE_TICKET} = $password; 
 
$cmd = [@$remcmd, "/usr/sbin/qm", 'vncproxy', $vmid];


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] vncpropxy question

[Dietmar Maurer]

> addendum:
> 
> 'it doesn't do anything here' is not completely correct
> for 'regular' vm displays it just does not set the ticket which
> breaks the connection

I think this ("break the connection") is important, because otherwise it would 
allow unecrypted VNC traffic over the network. I guess we do not want that.

But qemu now supports more VNC auth types, so maybe we can allow TLS encrypted 
VNC from outside, and unecrypted VNC for local proxy code.

I will take a look at that when I am back from vacation...


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] qemu 7.0 : fleecing backup (aka: local temp write cache"

[Dietmar Maurer]

> This is really a blocker for me,I can't use pbs because I'm using nvme 
> is production, and a 7200k hdd backup in a remote 200km site with 5ms 
> latency.

Why don't you use a local(fast) PBS instance, then sync to the slow remote?


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] qemu 7.0 : fleecing backup (aka: local temp write cache"

[Dietmar Maurer]

> Can I use a small local fast PBS instance without need to keep the full 
> datastore chunks ?
> 
> I have 300TB nvme in production, I don't want to buy 300TB nvme for backup.

no, sorry.

Do you want to use Ceph as temp backup storage, or simply an additional (node 
local) nvme?


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] appllied: [PATCH] read firstline: only map ENOENT to undef, raise error otherwise

[Dietmar Maurer]

> One could argue that the case for not existent should return undef,
> while an empty file should return an empty string, but for that we
> might want to check all use-sites first.

AFAIR I use this function many times assuming that it does not throw errors in 
case of empty files. That is quite convenient.


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH pve-ceph] fix compatibility with CPUs not supporting SSE 4.1 instructions

[Dietmar Maurer]

> One of our users ran into issues with running Ceph on older CPU
> architectures [1]. This is apparently due to a bug in gcc-12 that
> leads to SSE 4.1 instructions always being executed rather than
> dynamically dispatching functions using those instructions.

Cant we fix the GCC bug instead?


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH pve-xtermjs] termproxy: allow to use unix sockets for auth requests

[Dietmar Maurer]

Remove ureq, because it does not support unix sockets.

Signed-off-by: Dietmar Maurer 
---
 termproxy/Cargo.toml  |  2 +-
 termproxy/src/cli.rs  | 29 +
 termproxy/src/main.rs | 59 +--
 3 files changed, 71 insertions(+), 19 deletions(-)

diff --git a/termproxy/Cargo.toml b/termproxy/Cargo.toml
index a49d6b0..f6d8814 100644
--- a/termproxy/Cargo.toml
+++ b/termproxy/Cargo.toml
@@ -22,4 +22,4 @@ nix = "0.26.1"
 pico-args = "0.4"
 proxmox-io = "1"
 proxmox-lang = "1.1"
-ureq = { version = "2.4", default-features = false, features = [ "gzip" ] }
+form_urlencoded = "1.1"
diff --git a/termproxy/src/cli.rs b/termproxy/src/cli.rs
index cc44655..409cbe1 100644
--- a/termproxy/src/cli.rs
+++ b/termproxy/src/cli.rs
@@ -4,14 +4,15 @@ use std::os::fd::RawFd;
 use anyhow::{bail, Result};
 
 const CMD_HELP: &str = "\
-Usage: proxmox-termproxy [OPTIONS] --path   -- 
...
+Usage: proxmox-termproxy  [OPTIONS] --path  -- 
...
 
 Arguments:
  Port or file descriptor to listen for TCP connections
   ...   The command to run connected via a proxied PTY
 
 Options:
-  --authportPort to relay auth-request, default 85
+  --authportPort or unix socket to relay auth-request,
+  default is port 85
   --port-as-fdUse  as file descriptor.
   --pathACL object path to test  on.
   --permPermission to test.
@@ -40,14 +41,20 @@ impl PortOrFd {
 }
 }
 
+#[derive(Debug)]
+pub enum DaemonAddress {
+Port(u16),
+UnixSocket(String),
+}
+
 #[derive(Debug)]
 pub struct Options {
 /// The actual command to run proxied in a pseudo terminal.
 pub terminal_command: Vec,
 /// The port or FD that termproxy will listen on for an incoming conection
 pub listen_port: PortOrFd,
-/// The port of the local privileged daemon that authentication is relayed 
to. Defaults to `85`
-pub api_daemon_port: u16,
+/// The port (or unix socket path) of the local privileged daemon that 
authentication is relayed to. Defaults to port `85`
+pub api_daemon_port: DaemonAddress,
 /// The ACL object path the 'acl_permission' is checked on
 pub acl_path: String,
 /// The ACL permission that the ticket, read from the stream, is required 
to have on 'acl_path'
@@ -81,7 +88,19 @@ impl Options {
 let options = Self {
 terminal_command: terminal_command.unwrap(), // checked above
 listen_port: PortOrFd::from_cli(args.free_from_str()?, 
args.contains("--port-as-fd"))?,
-api_daemon_port: 
args.opt_value_from_str("--authport")?.unwrap_or(85),
+api_daemon_port: {
+let authport: Option = 
args.opt_value_from_str("--authport")?;
+match authport {
+Some(authport) => {
+if let Ok(port) = authport.parse::() {
+DaemonAddress::Port(port)
+} else {
+DaemonAddress::UnixSocket(authport)
+}
+}
+None => DaemonAddress::Port(85),
+}
+},
 acl_path: args.value_from_str("--path")?,
 acl_permission: args.opt_value_from_str("--perm")?,
 };
diff --git a/termproxy/src/main.rs b/termproxy/src/main.rs
index c674993..ca4f01b 100644
--- a/termproxy/src/main.rs
+++ b/termproxy/src/main.rs
@@ -1,11 +1,13 @@
 use std::cmp::min;
 use std::collections::HashMap;
 use std::ffi::OsString;
-use std::io::{ErrorKind, Write};
+use std::io::{ErrorKind, Read, Write};
 use std::os::unix::io::{AsRawFd, FromRawFd};
+use std::os::unix::net::UnixStream;
 use std::os::unix::process::CommandExt;
 use std::process::Command;
 use std::time::{Duration, Instant};
+//use std::io::prelude::*;
 
 use anyhow::{bail, format_err, Result};
 use mio::net::{TcpListener, TcpStream};
@@ -145,6 +147,39 @@ fn read_ticket_line(
 }
 }
 
+fn simple_auth_request(mut stream: S, params: &[(&str, 
&str)]) -> Result<()> {
+let mut form = form_urlencoded::Serializer::new(String::new());
+
+for (name, value) in params {
+form.append_pair(name, value);
+}
+let body = form.finish();
+let raw_body = body.as_bytes();
+
+let mut raw_request = String::new();
+raw_request.push_str("POST /api2/json/access/ticket HTTP/1.1\r\n");
+raw_request.push_str("Connection: close\r\n");
+raw_request.push_str("User-Agent: termproxy/1.0\r\n");
+raw_request.push_str("Content-Type: 
application/x-www-form-urlencoded;charset=UTF-8\r\n");
+raw_request.push_str(&format!("Content-Length: {}\r\n", raw_body.len()));
+raw_request.pu

[pve-devel] [PATCH pve-xtermjs v2] termproxy: allow to use unix sockets for auth requests

[Dietmar Maurer]

Remove ureq, because it does not support unix sockets.

Signed-off-by: Dietmar Maurer 
---

Changes since v1:

 - use extra --authsocket cli option
 - use single format!() instead of multiple push_str()
 - cleanup variable names

 termproxy/Cargo.toml  |  2 +-
 termproxy/src/cli.rs  | 26 +++---
 termproxy/src/main.rs | 63 ++-
 3 files changed, 73 insertions(+), 18 deletions(-)

diff --git a/termproxy/Cargo.toml b/termproxy/Cargo.toml
index a49d6b0..f6d8814 100644
--- a/termproxy/Cargo.toml
+++ b/termproxy/Cargo.toml
@@ -22,4 +22,4 @@ nix = "0.26.1"
 pico-args = "0.4"
 proxmox-io = "1"
 proxmox-lang = "1.1"
-ureq = { version = "2.4", default-features = false, features = [ "gzip" ] }
+form_urlencoded = "1.1"
diff --git a/termproxy/src/cli.rs b/termproxy/src/cli.rs
index cc44655..02049f5 100644
--- a/termproxy/src/cli.rs
+++ b/termproxy/src/cli.rs
@@ -4,7 +4,7 @@ use std::os::fd::RawFd;
 use anyhow::{bail, Result};
 
 const CMD_HELP: &str = "\
-Usage: proxmox-termproxy [OPTIONS] --path   -- 
...
+Usage: proxmox-termproxy  [OPTIONS] --path  -- 
...
 
 Arguments:
  Port or file descriptor to listen for TCP connections
@@ -12,6 +12,7 @@ Arguments:
 
 Options:
   --authportPort to relay auth-request, default 85
+  --authsocketUnix socket to relay auth-request (conflicts 
with --authport)
   --port-as-fdUse  as file descriptor.
   --pathACL object path to test  on.
   --permPermission to test.
@@ -40,14 +41,20 @@ impl PortOrFd {
 }
 }
 
+#[derive(Debug)]
+pub enum DaemonAddress {
+Port(u16),
+UnixSocket(String),
+}
+
 #[derive(Debug)]
 pub struct Options {
 /// The actual command to run proxied in a pseudo terminal.
 pub terminal_command: Vec,
 /// The port or FD that termproxy will listen on for an incoming conection
 pub listen_port: PortOrFd,
-/// The port of the local privileged daemon that authentication is relayed 
to. Defaults to `85`
-pub api_daemon_port: u16,
+/// The port (or unix socket path) of the local privileged daemon that 
authentication is relayed to. Defaults to port `85`
+pub api_daemon_address: DaemonAddress,
 /// The ACL object path the 'acl_permission' is checked on
 pub acl_path: String,
 /// The ACL permission that the ticket, read from the stream, is required 
to have on 'acl_path'
@@ -81,7 +88,18 @@ impl Options {
 let options = Self {
 terminal_command: terminal_command.unwrap(), // checked above
 listen_port: PortOrFd::from_cli(args.free_from_str()?, 
args.contains("--port-as-fd"))?,
-api_daemon_port: 
args.opt_value_from_str("--authport")?.unwrap_or(85),
+api_daemon_address: {
+let authport: Option = 
args.opt_value_from_str("--authport")?;
+let authsocket: Option = 
args.opt_value_from_str("--authsocket")?;
+match (authport, authsocket) {
+(Some(authport), None) => DaemonAddress::Port(authport),
+(None, Some(authsocket)) => 
DaemonAddress::UnixSocket(authsocket),
+(None, None) => DaemonAddress::Port(85),
+(Some(_), Some(_)) => {
+bail!("conflicting options: --authport and 
--authsocket are mutually exclusive.")
+}
+}
+},
 acl_path: args.value_from_str("--path")?,
 acl_permission: args.opt_value_from_str("--perm")?,
 };
diff --git a/termproxy/src/main.rs b/termproxy/src/main.rs
index c674993..54e6bf9 100644
--- a/termproxy/src/main.rs
+++ b/termproxy/src/main.rs
@@ -1,11 +1,13 @@
 use std::cmp::min;
 use std::collections::HashMap;
 use std::ffi::OsString;
-use std::io::{ErrorKind, Write};
+use std::io::{ErrorKind, Read, Write};
 use std::os::unix::io::{AsRawFd, FromRawFd};
+use std::os::unix::net::UnixStream;
 use std::os::unix::process::CommandExt;
 use std::process::Command;
 use std::time::{Duration, Instant};
+//use std::io::prelude::*;
 
 use anyhow::{bail, format_err, Result};
 use mio::net::{TcpListener, TcpStream};
@@ -145,6 +147,43 @@ fn read_ticket_line(
 }
 }
 
+fn simple_auth_request(mut stream: S, params: &[(&str, 
&str)]) -> Result<()> {
+let mut form = form_urlencoded::Serializer::new(String::new());
+
+for (name, value) in params {
+form.append_pair(name, value);
+}
+let body = form.finish();
+let raw_body = body.as_bytes();
+
+let raw_request = format!(
+concat!(
+"POST /api2/json/access/ticket HTTP/1.1\r\n",
+"Connection: close\r\n",
+"User-Agent: termproxy/1.0\r\n"

[pve-devel] [PATCH pve-xtermjs] termproxy: fix the command line help text

[Dietmar Maurer]

The  need to be the first argument.

Signed-off-by: Dietmar Maurer 
---
 termproxy/src/cli.rs | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/termproxy/src/cli.rs b/termproxy/src/cli.rs
index cc44655..adfd830 100644
--- a/termproxy/src/cli.rs
+++ b/termproxy/src/cli.rs
@@ -4,7 +4,7 @@ use std::os::fd::RawFd;
 use anyhow::{bail, Result};
 
 const CMD_HELP: &str = "\
-Usage: proxmox-termproxy [OPTIONS] --path   -- 
...
+Usage: proxmox-termproxy  [OPTIONS] --path  -- 
...
 
 Arguments:
  Port or file descriptor to listen for TCP connections
-- 
2.39.2


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH pve-xtermjs v3] termproxy: allow to use unix sockets for auth requests

[Dietmar Maurer]

Remove ureq, because it does not support unix sockets.

Signed-off-by: Dietmar Maurer 
---

Changes sinve v2:
  split out the command line help text change into patch:
  [PATCH pve-xtermjs] termproxy: fix the command line help text

Changes since v1:

 - use extra --authsocket cli option
 - use single format!() instead of multiple push_str()
 - cleanup variable names


 termproxy/Cargo.toml  |  2 +-
 termproxy/src/cli.rs  | 24 ++---
 termproxy/src/main.rs | 63 ++-
 3 files changed, 72 insertions(+), 17 deletions(-)

diff --git a/termproxy/Cargo.toml b/termproxy/Cargo.toml
index a49d6b0..f6d8814 100644
--- a/termproxy/Cargo.toml
+++ b/termproxy/Cargo.toml
@@ -22,4 +22,4 @@ nix = "0.26.1"
 pico-args = "0.4"
 proxmox-io = "1"
 proxmox-lang = "1.1"
-ureq = { version = "2.4", default-features = false, features = [ "gzip" ] }
+form_urlencoded = "1.1"
diff --git a/termproxy/src/cli.rs b/termproxy/src/cli.rs
index adfd830..02049f5 100644
--- a/termproxy/src/cli.rs
+++ b/termproxy/src/cli.rs
@@ -12,6 +12,7 @@ Arguments:
 
 Options:
   --authportPort to relay auth-request, default 85
+  --authsocketUnix socket to relay auth-request (conflicts 
with --authport)
   --port-as-fdUse  as file descriptor.
   --pathACL object path to test  on.
   --permPermission to test.
@@ -40,14 +41,20 @@ impl PortOrFd {
 }
 }
 
+#[derive(Debug)]
+pub enum DaemonAddress {
+Port(u16),
+UnixSocket(String),
+}
+
 #[derive(Debug)]
 pub struct Options {
 /// The actual command to run proxied in a pseudo terminal.
 pub terminal_command: Vec,
 /// The port or FD that termproxy will listen on for an incoming conection
 pub listen_port: PortOrFd,
-/// The port of the local privileged daemon that authentication is relayed 
to. Defaults to `85`
-pub api_daemon_port: u16,
+/// The port (or unix socket path) of the local privileged daemon that 
authentication is relayed to. Defaults to port `85`
+pub api_daemon_address: DaemonAddress,
 /// The ACL object path the 'acl_permission' is checked on
 pub acl_path: String,
 /// The ACL permission that the ticket, read from the stream, is required 
to have on 'acl_path'
@@ -81,7 +88,18 @@ impl Options {
 let options = Self {
 terminal_command: terminal_command.unwrap(), // checked above
 listen_port: PortOrFd::from_cli(args.free_from_str()?, 
args.contains("--port-as-fd"))?,
-api_daemon_port: 
args.opt_value_from_str("--authport")?.unwrap_or(85),
+api_daemon_address: {
+let authport: Option = 
args.opt_value_from_str("--authport")?;
+let authsocket: Option = 
args.opt_value_from_str("--authsocket")?;
+match (authport, authsocket) {
+(Some(authport), None) => DaemonAddress::Port(authport),
+(None, Some(authsocket)) => 
DaemonAddress::UnixSocket(authsocket),
+(None, None) => DaemonAddress::Port(85),
+(Some(_), Some(_)) => {
+bail!("conflicting options: --authport and 
--authsocket are mutually exclusive.")
+}
+}
+},
 acl_path: args.value_from_str("--path")?,
 acl_permission: args.opt_value_from_str("--perm")?,
 };
diff --git a/termproxy/src/main.rs b/termproxy/src/main.rs
index c674993..54e6bf9 100644
--- a/termproxy/src/main.rs
+++ b/termproxy/src/main.rs
@@ -1,11 +1,13 @@
 use std::cmp::min;
 use std::collections::HashMap;
 use std::ffi::OsString;
-use std::io::{ErrorKind, Write};
+use std::io::{ErrorKind, Read, Write};
 use std::os::unix::io::{AsRawFd, FromRawFd};
+use std::os::unix::net::UnixStream;
 use std::os::unix::process::CommandExt;
 use std::process::Command;
 use std::time::{Duration, Instant};
+//use std::io::prelude::*;
 
 use anyhow::{bail, format_err, Result};
 use mio::net::{TcpListener, TcpStream};
@@ -145,6 +147,43 @@ fn read_ticket_line(
 }
 }
 
+fn simple_auth_request(mut stream: S, params: &[(&str, 
&str)]) -> Result<()> {
+let mut form = form_urlencoded::Serializer::new(String::new());
+
+for (name, value) in params {
+form.append_pair(name, value);
+}
+let body = form.finish();
+let raw_body = body.as_bytes();
+
+let raw_request = format!(
+concat!(
+"POST /api2/json/access/ticket HTTP/1.1\r\n",
+"Connection: close\r\n",
+"User-Agent: termproxy/1.0\r\n",
+"Content-Type: 
application/x-www-form-urlencoded;charset=UTF-8\r\n",
+"Content-Length: {}\r\n",
+"\r\n&

Re: [pve-devel] [RFC qemu/storage/qemu-server/container/manager 00/23] backup provider API

[Dietmar Maurer]

> Would adding support for offloading incremental difference detection
> to the underlying storage be feasible with the API updates? The QEMU
> bitmap strategy works for all storage devices but is far from
> optimal.

Sorry, but why do you think this is far from optimal?


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [RFC qemu/storage/qemu-server/container/manager 00/23] backup provider API

[Dietmar Maurer]

> Today, I believe the client is reading the data and pushing it to
> PBS. In the case of CEPH, wouldn't this involve sourcing data from
> multiple nodes and then sending it to PBS? Wouldn't it be more
> efficient for PBS to read it directly from storage? In the case of
> centralized storage, we'd like to eliminate the client load
> completely, having PBS ingest increment differences directly from
> storage without passing through the client.

But Ceph is not a central storage. Instead, data is distributed among the 
nodes, so you always need to send some data over the network.
There is no way to "read it directly from storage".


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [RFC qemu/storage/qemu-server/container/manager 00/23] backup provider API

[Dietmar Maurer]

> The biggest issue we see reported related to QEMU bitmaps is
> persistence. The lack of durability results in unpredictable backup
> behavior at scale. If a host, rack, or data center loses power, you're
> in for a full backup cycle. Even if several VMs are powered off for
> some reason, it can be a nuisance. Several storage solutions can
> generate the incremental difference bitmaps from durable sources,
> eliminating the issue.

Several storage solutions provides internal snapshots, but none of them has an 
API to access the dirty bitmap (please correct me if I am wrong). Or what 
storage solution do you talk about exactly?

Storing the dirty bitmap persistently would be relatively easy, but so far we 
found no way to make sure the bitmap is always up-to-date. 
We support shared storages, so multiple nodes can access and modify the data 
without updating the dirty bitmap, which would lead to corrupt backup images...


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [RFC qemu/storage/qemu-server/container/manager 00/23] backup provider API

[Dietmar Maurer]

> In hyper-converged deployments, the node performing the backup is sourcing 
> ((nodes-1)/(nodes))*bytes) of backup data (i.e., ingress traffic) and then 
> sending 1*bytes to PBS (i.e., egress traffic). If PBS were to pull the data 
> from the nodes directly, the maximum load on any one host would be 
> (1/nodes)*bytes of egress traffic only... that's a considerable improvement!

I guess it would be possible to write a tool like proxmox-backup-client that 
pull ceph backups directly from PBS. Or extend the backup protokoll allowing 
direct storage access. But this is a considerable amount of development, and 
needs much more configuration/setup than the current approach. But patches are 
always welcome...

Also, it is not clear to me how we can implement a "backup provider API" if we 
add such optimizations?

And yes, network traffic would be reduced. But IMHO it is easier to add a 
dedicated network card for the backup server (if the network is the limiting 
factor). With this setup, the maximum load on the ceph network is 
(1/nodes)*bytess of egress traffic only. The backup traffic is on the dedicated 
backup net.


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] Post mail to the list

[Dietmar Maurer]

> Hello, i wan't to make a patch for proxmox to implements DRBD, in a 
> different way that LINSTOR do. I want to discuss about its usefulness 
> and implementation with the community.

I think you should discuss that with the DRBD people (LINSTOR).
But I am not sure they are on this list.


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] rust build questions

[Dietmar Maurer]

> currently, I have tried to build "proxmox" rust
> 
> https://git.proxmox.com/?p=proxmox.git;a=shortlog
> with a simple
> "make deb" + installed needed build depend.
> 
> but I have this error
> 
> 
> "
> test src/tools/websocket.rs - tools::websocket::create_frame (line 161)
> ... ok
> test src/tools/websocket.rs - tools::websocket::create_frame (line 148)
> ... ok
> 
> failures:
> 
>  src/api/api_type_macros.rs - api_string_type (line 5) stdout 
> error[E0432]: unresolved import `proxmox::api::api`
>  --> src/api/api_type_macros.rs:6:5
>   |
> 4 | use proxmox::api::api;
>   | ^ no `api` in `api`

Sorry, this was a bug in the proxmox crate. Please update an test again:

commit fa3b5374ed61da3c40a1fc58070d6a16c877c3af (HEAD -> master, origin/master, 
origin/HEAD)
Author: Dietmar Maurer 
Date:   Wed May 5 08:03:35 2021 +0200

api_type_macros.rs: disable failing doctest (set it to ignore instead of 
no_run)


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] cfs-locked 'authkey' operation: pve cluster filesystem not online

[Dietmar Maurer]

Hi Julien,


> Hello to all.
> 
> I have the plan to implement the SSO authentication feature with the SAML 
> protocol.
> However, I have an error that prevents me from validating the authentication 
> process.
> It is about the locks.
> The first step is to store the request_saml_id. If I try to create a file by 
> your libraries, I get an 500 error with msg:
> error during cfs-locked \'file-request_tmp\' operation: pve cluster 
> filesystem not online /etc/pve/priv/lock.

Your cluster fs is not working (pmxcfs). See you run on a broken installation.

> https://github.com/jbsky/proxmox-saml2-auth/commit/d75dc621aae719c8fdd251859af9641cda0e526b
> Ok, I can make a temp workaround.
> 
> 2nd step :
> When I try to create a ticket with the function create_ticket in package 
> PVE::API2::AccessControl;
> I've got this error :
> authentication failure; rhost=127.0.0.1 user=admin@DOM msg=error during 
> cfs-locked 'authkey' operation: pve cluster filesystem not online 
> /etc/pve/priv/lock

Again, the pmxcfs is not online.

> src : 
> https://github.com/jbsky/proxmox-saml2-auth/commit/93b02727d2e172968c14c4ce3a7c27e8d5c0feb0
> 
> I have really bad luck with these locks!
> Can you help me to understand the prerequisites to make the lock work?

You need a working PVE installation before doing any API calls...


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH] [PATCH pve-http-server] SSO feature: login with SAMLv2

[Dietmar Maurer]

I am trying to test your code, so I need a SAML Identity provider. What is
the best OSS implementation for that?

I tried lemonldap-ng, but there example configuration is a nightmare and
I was unable to get that running. Is there anything else I can use to test?. 

- Dietmar


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH] [PATCH pve-access-control] SSO feature: login with SAMLv2

[Dietmar Maurer]

I wonder why you want to store temporary data in /etc/pve/tmp/saml. Wouldn't it 
we good enough
to store that on the local file system?

> On 05/27/2021 11:55 PM Julien BLAIS  wrote:
> 
>  
> Added a new endpoint usable by api2/html/access/saml?realm=$DOM
>   which allows to initiate a redirection to an IdP.
>   During initialization, the /etc/pve/tmp/saml file is filled with the format 
> REALM:SAML_REQUEST_ID:TIME
> Modification of the endpoint /access/ticket to support SAMLResponse.
>   The information is extracted from the SAMLResponse variable in order to 
> check
>   if the SAML_REQUEST_ID exists in /etc/pve/tmp/saml, we extract from this 
> file the REALM used to initiate the SSO connection.


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH] [PATCH pve-access-control] SSO feature: login with SAMLv2

[Dietmar Maurer]

Unfortunately, your code depends on code not packaged for Debian. Any idea 
how to replace that (cpanm Net::SAML2)?

Or better, is there a 'rust' implementaion for SAML2? If so, we could make perl 
bindings
for that and reuse the code with Proxmox Backup Server.

Other ideas?

> diff --git a/src/PVE/Auth/SAML.pm b/src/PVE/Auth/SAML.pm
> new file mode 100644
> index 000..4653cb7
> --- /dev/null
> +++ b/src/PVE/Auth/SAML.pm
> @@ -0,0 +1,248 @@
> +# Instructions for installation :
> +# apt-get install libxml2 make gcc libssl-dev libperl-dev git cpanminus
> +# cpanm Net::SAML2
> +# ln -s /usr/local/share/perl/5.28.1/Net/SAML2 
> /usr/share/perl/5.28.1/Net/SAML2
> +# ln -s /usr/local/share/perl/5.28.1/Net/SAML2 /usr/share/perl5/Net/SAML2


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] RE : [PATCH] [PATCH pve-access-control] SSO feature:login with SAMLv2

[Dietmar Maurer]

> > I wonder why you want to store temporary data in /etc/pve/tmp/saml. 
> > Wouldn't it we good enough
> > to store that on the local file system?
> On the one hand, I enjoyed reusing your work.
> On the other hand, I think it is more secure to put this kind of data in 
> /etc/pve/tmp/saml than in /tmp/saml/

No, I consider this less secure. Especially if the write is triggered by 
unauthenticated request...

> Then, yes, it is possible to store it on /tmp/saml for example, it is 
> variable data. Nothing is fixed, you are free to do what you want.
> 
> > Unfortunately, your code depends on code not packaged for Debian. Any idea
> > how to replace that (cpanm Net::SAML2)?
> 
> Since I'm not a perl specialist, I took what seemed to me the most standard 
> in this language. Have you considered cloning this repos available on 
> GitHub(https://github.com/perl-net-saml2/perl-Net-SAML2)?

This module has way to much dependencies (Moose). I do not want to use that.

> > Or better, is there a 'rust' implementaion for SAML2? If so, we could make 
> > perl bindings
> > for that and reuse the code with Proxmox Backup Server.
> 
> Do you have a specific project or library in mind?
> 
> Unfortunately, I don't have any knowledge about rust and I'll have a hard 
> time accompanying you on this topic. However, it seems that there are 
> projects on github in opensource, for example 
> https://github.com/njaremko/samael.

All those SAML libs are basically unmaintained, without any large user base.

> I'll tell you again,nothing is fixed, you are free to do what you want.

I also wonder why SAML? Would it be an option to use OpenId connect instead?


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] RE : RE : [PATCH] [PATCH pve-access-control] SSO feature:login with SAMLv2

[Dietmar Maurer]

> On 06/02/2021 12:16 PM wb  wrote:
> 
> 
> > I also wonder why SAML? Would it be an option to use OpenId connect instead?
> As I was able to use SAML, I know the functional part and therefore, if I 
> used SAML, it is only by ease.
> 
> Switch to OpenID, why not. The time I set up a functional POC.
> 
> On the other hand, I would like to know your constraints.

Sorry, what do you want to know exactly?

> Do you still want to use Rust? 

Yes. But I am still searching for usable crates:

openidconnect: https://github.com/ramosbugs/openidconnect-rs

Seems promising, but I have not done any testing so far...

> If yes, I am curious to know how to bind perl to Rust? Do you have an example?

https://git.proxmox.com/?p=perlmod.git;a=summary

Hope the inline docs and examples are good enough to start...

> I noticed from our exchange :
> During an API call, if the user is not authenticated, do not pass in private 
> and privileged the writing on /tmp/.

yes, unprivileged users should not be able to write anything.


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH pve-access-control 4/4] implement OpenID autocreate user feature

[Dietmar Maurer]

---
 src/PVE/API2/OpenId.pm | 33 ++---
 1 file changed, 30 insertions(+), 3 deletions(-)

diff --git a/src/PVE/API2/OpenId.pm b/src/PVE/API2/OpenId.pm
index db9f9eb..3814895 100644
--- a/src/PVE/API2/OpenId.pm
+++ b/src/PVE/API2/OpenId.pm
@@ -9,9 +9,10 @@ use PVE::RS::OpenId;
 use PVE::Exception qw(raise raise_perm_exc raise_param_exc);
 use PVE::SafeSyslog;
 use PVE::RPCEnvironment;
-use PVE::Cluster qw(cfs_read_file);
+use PVE::Cluster qw(cfs_read_file cfs_write_file);
 use PVE::AccessControl;
 use PVE::JSONSchema qw(get_standard_option);
+use PVE::Auth::Plugin;
 
 use PVE::RESTHandler;
 
@@ -182,8 +183,34 @@ __PACKAGE__->register_method ({
 
my $username = "${unique_name}\@${realm}";
 
-   # test if user exists and is enabled
-   $rpcenv->check_user_enabled($username);
+   # first, check if $username respects our naming conventions
+   PVE::Auth::Plugin::verify_username($username);
+
+   if ($config->{'autocreate'} && 
!$rpcenv->check_user_exist($username, 1)) {
+   PVE::AccessControl::lock_user_config(sub {
+   my $usercfg = cfs_read_file("user.cfg");
+
+   die "user '$username' already exists\n" if 
$usercfg->{users}->{$username};
+
+   my $entry = { enable => 1 };
+   if (defined(my $email = $info->{'email'})) {
+   $entry->{email} = $email;
+   }
+   if (defined(my $given_name = $info->{'given_name'})) {
+   $entry->{firstname} = $given_name;
+   }
+   if (defined(my $family_name = $info->{'family_name'})) {
+   $entry->{lastname} = $family_name;
+   }
+
+   $usercfg->{users}->{$username} = $entry;
+
+   cfs_write_file("user.cfg", $usercfg);
+   }, "autocreate openid user failed");
+   } else {
+   # test if user exists and is enabled
+   $rpcenv->check_user_enabled($username);
+   }
 
my $ticket = PVE::AccessControl::assemble_ticket($username);
my $csrftoken = 
PVE::AccessControl::assemble_csrf_prevention_token($username);
-- 
2.30.2


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH pve-manager] ui: implement OpenId login

[Dietmar Maurer]

---
 PVE/HTTPServer.pm  |   4 +-
 www/manager6/Utils.js  |   8 +++
 www/manager6/window/LoginWindow.js | 105 -
 3 files changed, 114 insertions(+), 3 deletions(-)

diff --git a/PVE/HTTPServer.pm b/PVE/HTTPServer.pm
index 636b562b..dabdf7f3 100755
--- a/PVE/HTTPServer.pm
+++ b/PVE/HTTPServer.pm
@@ -68,7 +68,9 @@ sub auth_handler {
 
 # explicitly allow some calls without auth
 if (($rel_uri eq '/access/domains' && $method eq 'GET') ||
-   ($rel_uri eq '/access/ticket' && ($method eq 'GET' || $method eq 
'POST'))) {
+   ($rel_uri eq '/access/ticket' && ($method eq 'GET' || $method eq 
'POST')) ||
+   ($rel_uri eq '/access/openid/login' &&  $method eq 'POST') ||
+   ($rel_uri eq '/access/openid/auth-url' &&  $method eq 'POST')) {
$require_auth = 0;
 }
 
diff --git a/www/manager6/Utils.js b/www/manager6/Utils.js
index 3415c9eb..c2d139f9 100644
--- a/www/manager6/Utils.js
+++ b/www/manager6/Utils.js
@@ -1742,6 +1742,14 @@ Ext.define('PVE.Utils', {
 
return true;
 },
+
+openid_login_param: function() {
+   let param = Ext.Object.fromQueryString(window.location.search);
+   if (param.state !== undefined && param.code !== undefined) {
+   return param;
+   }
+   return undefined;
+},
 },
 
 singleton: true,
diff --git a/www/manager6/window/LoginWindow.js 
b/www/manager6/window/LoginWindow.js
index 72078080..5d3d06b8 100644
--- a/www/manager6/window/LoginWindow.js
+++ b/www/manager6/window/LoginWindow.js
@@ -2,6 +2,21 @@
 Ext.define('PVE.window.LoginWindow', {
 extend: 'Ext.window.Window',
 
+viewModel: {
+   data: {
+   openid: false,
+   },
+   formulas: {
+   button_text: function(get) {
+   if (get("openid") === true) {
+   return gettext("Login (OpenID redirect)");
+   } else {
+   return gettext("Login");
+   }
+   },
+   },
+},
+
 controller: {
 
xclass: 'Ext.app.ViewController',
@@ -18,6 +33,33 @@ Ext.define('PVE.window.LoginWindow', {
return;
}
 
+   let redirect_url = location.origin;
+   let params = form.getValues();
+
+   if (this.getViewModel().data.openid === true) {
+   let realm = params.realm;
+   Proxmox.Utils.API2Request({
+   url: '/api2/extjs/access/openid/auth-url',
+   params: {
+   realm: realm,
+   "redirect-url": redirect_url,
+   },
+   method: 'POST',
+   success: function(resp, opts) {
+   window.location = resp.result.data;
+   },
+   failure: function(resp, opts) {
+   Proxmox.Utils.authClear();
+   form.unmask();
+   Ext.MessageBox.alert(
+   gettext('Error'),
+   gettext('OpenId redirect failed. Please try 
againError: ' + resp.htmlStatus),
+   );
+   },
+   });
+   return;
+   }
+
view.el.mask(gettext('Please wait...'), 'x-mask-loading');
 
// set or clear username
@@ -162,11 +204,21 @@ Ext.define('PVE.window.LoginWindow', {
window.location.reload();
},
},
-'button[reference=loginButton]': {
+   'field[name=realm]': {
+   change: function(f, value) {
+   let record = f.store.getById(value);
+   if (record === undefined) return;
+   let data = record.data;
+   this.getViewModel().set("openid", data.type === "openid");
+   },
+   },
+   'button[reference=loginButton]': {
click: 'onLogon',
 },
'#': {
show: function() {
+   var me = this;
+
var sp = Ext.state.Manager.getProvider();
var checkboxField = this.lookupReference('saveunField');
var unField = this.lookupReference('usernameField');
@@ -180,6 +232,42 @@ Ext.define('PVE.window.LoginWindow', {
var pwField = this.lookupReference('passwordField');
pwField.focus();
}
+
+   let param = PVE.Utils.openid_login_param();
+   if (param !== undefined) {
+   Proxmox.Utils.authClear();
+
+   let loginForm = this.lookupReference('loginForm');
+   loginForm.mask(gettext('OpenID login - please 
wait...'), 'x-mask-loading');
+
+   let redirect_url = location.origin;
+
+   Proxmox.Utils.API2Request({
+   url: '/api2/extjs/

[pve-devel] [PATCH pve-access-control 2/4] depend on libpve-rs-perl

[Dietmar Maurer]

---
 debian/control | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/debian/control b/debian/control
index 81a32bd..3ef748b 100644
--- a/debian/control
+++ b/debian/control
@@ -10,6 +10,7 @@ Build-Depends: debhelper (>= 12~),
lintian,
perl,
libpve-cluster-perl,
+  libpve-rs-perl,
pve-cluster (>= 6.1-4),
pve-doc-generator (>= 5.3-3),
 Standards-Version: 4.5.1
@@ -27,6 +28,7 @@ Depends: libauthen-pam-perl,
  libnet-ssleay-perl,
  libpve-common-perl (>= 6.0-18),
  libpve-cluster-perl,
+libpve-rs-perl,
  libpve-u2f-server-perl (>= 1.0-2),
  libuuid-perl,
  pve-cluster (>= 6.1-4),
-- 
2.30.2


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH pve-access-control 1/4] add OpenId configuration

[Dietmar Maurer]

---
 src/PVE/AccessControl.pm |  2 ++
 src/PVE/Auth/Makefile|  3 +-
 src/PVE/Auth/OpenId.pm   | 67 
 3 files changed, 71 insertions(+), 1 deletion(-)
 create mode 100755 src/PVE/Auth/OpenId.pm

diff --git a/src/PVE/AccessControl.pm b/src/PVE/AccessControl.pm
index 2569a35..8efb89d 100644
--- a/src/PVE/AccessControl.pm
+++ b/src/PVE/AccessControl.pm
@@ -24,6 +24,7 @@ use PVE::Auth::AD;
 use PVE::Auth::LDAP;
 use PVE::Auth::PVE;
 use PVE::Auth::PAM;
+use PVE::Auth::OpenId;
 
 # load and initialize all plugins
 
@@ -31,6 +32,7 @@ PVE::Auth::AD->register();
 PVE::Auth::LDAP->register();
 PVE::Auth::PVE->register();
 PVE::Auth::PAM->register();
+PVE::Auth::OpenId->register();
 PVE::Auth::Plugin->init();
 
 # $authdir must be writable by root only!
diff --git a/src/PVE/Auth/Makefile b/src/PVE/Auth/Makefile
index 58ae362..be7bde3 100644
--- a/src/PVE/Auth/Makefile
+++ b/src/PVE/Auth/Makefile
@@ -4,7 +4,8 @@ AUTH_SOURCES=   \
PVE.pm  \
PAM.pm  \
AD.pm   \
-   LDAP.pm
+   LDAP.pm \
+   OpenId.pm
 
 .PHONY: install
 install:
diff --git a/src/PVE/Auth/OpenId.pm b/src/PVE/Auth/OpenId.pm
new file mode 100755
index 000..8f35575
--- /dev/null
+++ b/src/PVE/Auth/OpenId.pm
@@ -0,0 +1,67 @@
+package PVE::Auth::OpenId;
+
+use strict;
+use warnings;
+
+use PVE::Tools;
+use PVE::Auth::Plugin;
+use PVE::Cluster qw(cfs_register_file cfs_read_file cfs_write_file 
cfs_lock_file);
+
+use base qw(PVE::Auth::Plugin);
+
+sub type {
+return 'openid';
+}
+
+sub properties {
+return {
+   "issuer-url" => {
+   description => "OpenID Issuer Url",
+   type => 'string',
+   maxLength => 256,
+   },
+   "client-id" => {
+description => "OpenID Client ID",
+type => 'string',
+   maxLength => 256,
+   },
+   "client-key" => {
+   description => "OpenID Client Key",
+   type => 'string',
+   optional => 1,
+   maxLength => 256,
+   },
+   autocreate => {
+  description => "Automatically create users if they do not exist.",
+  optional => 1,
+  type => 'boolean',
+  default => 0,
+   },
+   "user-attr" => {
+  type => 'string',
+  enum => ['subject', 'username', 'email'],
+  optional => 1,
+   },
+   };
+}
+
+sub options {
+return {
+   "issuer-url" => {},
+"client-id" => {},
+"client-key" => { optional => 1 },
+autocreate => { optional => 1 },
+"user-attr" => { optional => 1, fixed => 1 },
+default => { optional => 1 },
+comment => { optional => 1 },
+};
+}
+
+sub authenticate_user {
+my ($class, $config, $realm, $username, $password) = @_;
+
+die "OpenID realm does not allow password verification.\n";
+}
+
+
+1;
-- 
2.30.2


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH pve-access-control 3/4] api: implement openid API

[Dietmar Maurer]

This moves compute_api_permission() into RPCEnvironment.pm.
---
 src/PVE/API2/AccessControl.pm |  60 ++
 src/PVE/API2/Makefile |   3 +-
 src/PVE/API2/OpenId.pm| 214 ++
 src/PVE/RPCEnvironment.pm |  49 
 4 files changed, 273 insertions(+), 53 deletions(-)
 create mode 100644 src/PVE/API2/OpenId.pm

diff --git a/src/PVE/API2/AccessControl.pm b/src/PVE/API2/AccessControl.pm
index a77694b..6dec66c 100644
--- a/src/PVE/API2/AccessControl.pm
+++ b/src/PVE/API2/AccessControl.pm
@@ -19,9 +19,9 @@ use PVE::API2::User;
 use PVE::API2::Group;
 use PVE::API2::Role;
 use PVE::API2::ACL;
+use PVE::API2::OpenId;
 use PVE::Auth::Plugin;
 use PVE::OTP;
-use PVE::Tools;
 
 my $u2f_available = 0;
 eval {
@@ -56,6 +56,11 @@ __PACKAGE__->register_method ({
 path => 'domains',
 });
 
+__PACKAGE__->register_method ({
+subclass => "PVE::API2::OpenId",
+path => 'openid',
+});
+
 __PACKAGE__->register_method ({
 name => 'index',
 path => '',
@@ -165,55 +170,6 @@ my $create_ticket = sub {
 };
 };
 
-my $compute_api_permission = sub {
-my ($rpcenv, $authuser) = @_;
-
-my $usercfg = $rpcenv->{user_cfg};
-
-my $res = {};
-my $priv_re_map = {
-   vms => qr/VM\.|Permissions\.Modify/,
-   access => qr/(User|Group)\.|Permissions\.Modify/,
-   storage => qr/Datastore\.|Permissions\.Modify/,
-   nodes => qr/Sys\.|Permissions\.Modify/,
-   sdn => qr/SDN\.|Permissions\.Modify/,
-   dc => qr/Sys\.Audit|SDN\./,
-};
-map { $res->{$_} = {} } keys %$priv_re_map;
-
-my $required_paths = ['/', '/nodes', '/access/groups', '/vms', '/storage', 
'/sdn'];
-
-my $checked_paths = {};
-foreach my $path (@$required_paths, keys %{$usercfg->{acl}}) {
-   next if $checked_paths->{$path};
-   $checked_paths->{$path} = 1;
-
-   my $path_perm = $rpcenv->permissions($authuser, $path);
-
-   my $toplevel = ($path =~ /^\/(\w+)/) ? $1 : 'dc';
-   if ($toplevel eq 'pool') {
-   foreach my $priv (keys %$path_perm) {
-   if ($priv =~ m/^VM\./) {
-   $res->{vms}->{$priv} = 1;
-   } elsif ($priv =~ m/^Datastore\./) {
-   $res->{storage}->{$priv} = 1;
-   } elsif ($priv eq 'Permissions.Modify') {
-   $res->{storage}->{$priv} = 1;
-   $res->{vms}->{$priv} = 1;
-   }
-   }
-   } else {
-   my $priv_regex = $priv_re_map->{$toplevel} // next;
-   foreach my $priv (keys %$path_perm) {
-   next if $priv !~ m/^($priv_regex)/;
-   $res->{$toplevel}->{$priv} = 1;
-   }
-   }
-}
-
-return $res;
-};
-
 __PACKAGE__->register_method ({
 name => 'get_ticket',
 path => 'ticket',
@@ -314,7 +270,7 @@ __PACKAGE__->register_method ({
die PVE::Exception->new("authentication failure\n", code => 401);
}
 
-   $res->{cap} = &$compute_api_permission($rpcenv, $username)
+   $res->{cap} = $rpcenv->compute_api_permission($username)
if !defined($res->{NeedTFA});
 
my $clinfo = PVE::Cluster::get_clinfo();
@@ -659,7 +615,7 @@ __PACKAGE__->register_method({
 
return {
ticket => PVE::AccessControl::assemble_ticket($authuser),
-   cap => &$compute_api_permission($rpcenv, $authuser),
+   cap => $rpcenv->compute_api_permission($authuser),
}
 }});
 
diff --git a/src/PVE/API2/Makefile b/src/PVE/API2/Makefile
index 1bf8c05..4e49037 100644
--- a/src/PVE/API2/Makefile
+++ b/src/PVE/API2/Makefile
@@ -5,7 +5,8 @@ API2_SOURCES=   \
ACL.pm  \
Role.pm \
Group.pm\
-   User.pm
+   User.pm \
+   OpenId.pm
 
 .PHONY: install
 install:
diff --git a/src/PVE/API2/OpenId.pm b/src/PVE/API2/OpenId.pm
new file mode 100644
index 000..db9f9eb
--- /dev/null
+++ b/src/PVE/API2/OpenId.pm
@@ -0,0 +1,214 @@
+package PVE::API2::OpenId;
+
+use strict;
+use warnings;
+
+use PVE::Tools qw(extract_param);
+use PVE::RS::OpenId;
+
+use PVE::Exception qw(raise raise_perm_exc raise_param_exc);
+use PVE::SafeSyslog;
+use PVE::RPCEnvironment;
+use PVE::Cluster qw(cfs_read_file);
+use PVE::AccessControl;
+use PVE::JSONSchema qw(get_standard_option);
+
+use PVE::RESTHandler;
+
+use base qw(PVE::RESTHandler);
+
+my $openid_state_path = "/var/lib/pve-manager";
+
+my $lookup_openid_auth = sub {
+my ($realm, $redirect_url) = @_;
+
+my $cfg = cfs_read_file('domains.cfg');
+my $ids = $cfg->{ids};
+
+die "authentication domain '$realm' does not exist\n" if !$ids->{$realm};
+
+my $config = $ids->{$realm};
+die "wrong realm type ($config->{type} != openid)" if $config->{type} ne 
"openid";
+
+my $openid_config = {
+   issuer_url => $config->{'issuer-url'},
+   client_id => $config->{'client-id'},
+   client_key => $config->{'client-key'},
+  

[pve-devel] [PATCH pve-access-control v2 1/5] check_user_enabled: also check if user is expired

[Dietmar Maurer]

---
 src/PVE/AccessControl.pm | 16 +++-
 1 file changed, 7 insertions(+), 9 deletions(-)

diff --git a/src/PVE/AccessControl.pm b/src/PVE/AccessControl.pm
index 2569a35..8628678 100644
--- a/src/PVE/AccessControl.pm
+++ b/src/PVE/AccessControl.pm
@@ -428,12 +428,10 @@ sub verify_token {
 check_user_enabled($usercfg, $username);
 check_token_exist($usercfg, $username, $token);
 
-my $ctime = time();
-
 my $user = $usercfg->{users}->{$username};
-die "account expired\n" if $user->{expire} && ($user->{expire} < $ctime);
-
 my $token_info = $user->{tokens}->{$token};
+
+my $ctime = time();
 die "token expired\n" if $token_info->{expire} && ($token_info->{expire} < 
$ctime);
 
 die "invalid token value!\n" if !PVE::Cluster::verify_token($tokenid, 
$value);
@@ -579,6 +577,11 @@ sub check_user_enabled {
 
 die "user '$username' is disabled\n" if !$noerr;
 
+my $ctime = time();
+my $expire = $usercfg->{users}->{$username}->{expire};
+
+die "account expired\n" if $expire && ($expire < $ctime);
+
 return undef;
 }
 
@@ -629,11 +632,6 @@ sub authenticate_user {
 
 check_user_enabled($usercfg, $username);
 
-my $ctime = time();
-my $expire = $usercfg->{users}->{$username}->{expire};
-
-die "account expired\n" if $expire && ($expire < $ctime);
-
 my $domain_cfg = cfs_read_file('domains.cfg');
 
 my $cfg = $domain_cfg->{ids}->{$realm};
-- 
2.30.2


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH pve-access-control v2 3/5] depend on libpve-rs-perl

[Dietmar Maurer]

---
 debian/control | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/debian/control b/debian/control
index 81a32bd..3ef748b 100644
--- a/debian/control
+++ b/debian/control
@@ -10,6 +10,7 @@ Build-Depends: debhelper (>= 12~),
lintian,
perl,
libpve-cluster-perl,
+  libpve-rs-perl,
pve-cluster (>= 6.1-4),
pve-doc-generator (>= 5.3-3),
 Standards-Version: 4.5.1
@@ -27,6 +28,7 @@ Depends: libauthen-pam-perl,
  libnet-ssleay-perl,
  libpve-common-perl (>= 6.0-18),
  libpve-cluster-perl,
+libpve-rs-perl,
  libpve-u2f-server-perl (>= 1.0-2),
  libuuid-perl,
  pve-cluster (>= 6.1-4),
-- 
2.30.2


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH pve-access-control v2 0/5] add OpenId realms

[Dietmar Maurer]

Changes in v2:
- also check if user is expired (in check_user_enabled)
- always die with newline
- rename "user-attr" to "username-claim"


Dietmar Maurer (5):
  check_user_enabled: also check if user is expired
  add OpenId configuration
  depend on libpve-rs-perl
  api: implement openid API
  implement OpenID autocreate user feature

 debian/control|   2 +
 src/PVE/API2/AccessControl.pm |  60 ++---
 src/PVE/API2/Makefile |   3 +-
 src/PVE/API2/OpenId.pm| 238 ++
 src/PVE/AccessControl.pm  |  18 +--
 src/PVE/Auth/Makefile |   3 +-
 src/PVE/Auth/OpenId.pm|  68 ++
 src/PVE/RPCEnvironment.pm |  49 +++
 8 files changed, 378 insertions(+), 63 deletions(-)
 create mode 100644 src/PVE/API2/OpenId.pm
 create mode 100755 src/PVE/Auth/OpenId.pm

-- 
2.30.2


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH pve-access-control v2 2/5] add OpenId configuration

[Dietmar Maurer]

---
 src/PVE/AccessControl.pm |  2 ++
 src/PVE/Auth/Makefile|  3 +-
 src/PVE/Auth/OpenId.pm   | 68 
 3 files changed, 72 insertions(+), 1 deletion(-)
 create mode 100755 src/PVE/Auth/OpenId.pm

diff --git a/src/PVE/AccessControl.pm b/src/PVE/AccessControl.pm
index 8628678..3d8d01c 100644
--- a/src/PVE/AccessControl.pm
+++ b/src/PVE/AccessControl.pm
@@ -24,6 +24,7 @@ use PVE::Auth::AD;
 use PVE::Auth::LDAP;
 use PVE::Auth::PVE;
 use PVE::Auth::PAM;
+use PVE::Auth::OpenId;
 
 # load and initialize all plugins
 
@@ -31,6 +32,7 @@ PVE::Auth::AD->register();
 PVE::Auth::LDAP->register();
 PVE::Auth::PVE->register();
 PVE::Auth::PAM->register();
+PVE::Auth::OpenId->register();
 PVE::Auth::Plugin->init();
 
 # $authdir must be writable by root only!
diff --git a/src/PVE/Auth/Makefile b/src/PVE/Auth/Makefile
index 58ae362..be7bde3 100644
--- a/src/PVE/Auth/Makefile
+++ b/src/PVE/Auth/Makefile
@@ -4,7 +4,8 @@ AUTH_SOURCES=   \
PVE.pm  \
PAM.pm  \
AD.pm   \
-   LDAP.pm
+   LDAP.pm \
+   OpenId.pm
 
 .PHONY: install
 install:
diff --git a/src/PVE/Auth/OpenId.pm b/src/PVE/Auth/OpenId.pm
new file mode 100755
index 000..515d2f4
--- /dev/null
+++ b/src/PVE/Auth/OpenId.pm
@@ -0,0 +1,68 @@
+package PVE::Auth::OpenId;
+
+use strict;
+use warnings;
+
+use PVE::Tools;
+use PVE::Auth::Plugin;
+use PVE::Cluster qw(cfs_register_file cfs_read_file cfs_write_file 
cfs_lock_file);
+
+use base qw(PVE::Auth::Plugin);
+
+sub type {
+return 'openid';
+}
+
+sub properties {
+return {
+   "issuer-url" => {
+   description => "OpenID Issuer Url",
+   type => 'string',
+   maxLength => 256,
+   },
+   "client-id" => {
+description => "OpenID Client ID",
+type => 'string',
+   maxLength => 256,
+   },
+   "client-key" => {
+   description => "OpenID Client Key",
+   type => 'string',
+   optional => 1,
+   maxLength => 256,
+   },
+   autocreate => {
+  description => "Automatically create users if they do not exist.",
+  optional => 1,
+  type => 'boolean',
+  default => 0,
+   },
+   "username-claim" => {
+  description => "OpenID claim used to generate the unique username.",
+  type => 'string',
+  enum => ['subject', 'username', 'email'],
+  optional => 1,
+   },
+   };
+}
+
+sub options {
+return {
+   "issuer-url" => {},
+"client-id" => {},
+"client-key" => { optional => 1 },
+autocreate => { optional => 1 },
+"username-claim" => { optional => 1, fixed => 1 },
+default => { optional => 1 },
+comment => { optional => 1 },
+};
+}
+
+sub authenticate_user {
+my ($class, $config, $realm, $username, $password) = @_;
+
+die "OpenID realm does not allow password verification.\n";
+}
+
+
+1;
-- 
2.30.2


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH pve-access-control v2 5/5] implement OpenID autocreate user feature

[Dietmar Maurer]

---
 src/PVE/API2/OpenId.pm | 35 +++
 1 file changed, 31 insertions(+), 4 deletions(-)

diff --git a/src/PVE/API2/OpenId.pm b/src/PVE/API2/OpenId.pm
index d0b29fc..8384729 100644
--- a/src/PVE/API2/OpenId.pm
+++ b/src/PVE/API2/OpenId.pm
@@ -9,9 +9,10 @@ use PVE::RS::OpenId;
 use PVE::Exception qw(raise raise_perm_exc raise_param_exc);
 use PVE::SafeSyslog;
 use PVE::RPCEnvironment;
-use PVE::Cluster qw(cfs_read_file);
+use PVE::Cluster qw(cfs_read_file cfs_write_file);
 use PVE::AccessControl;
 use PVE::JSONSchema qw(get_standard_option);
+use PVE::Auth::Plugin;
 
 use PVE::RESTHandler;
 
@@ -161,7 +162,7 @@ __PACKAGE__->register_method ({
die "missing openid claim 'sub'\n" if !defined($subject);
 
my $unique_name = $subject; # default
-   if (defined(my $user_attr = $config->{'user-attr'})) {
+   if (defined(my $user_attr = $config->{'username-claim'})) {
if ($user_attr eq 'subject') {
$unique_name = $subject;
} elsif ($user_attr eq 'username') {
@@ -179,8 +180,34 @@ __PACKAGE__->register_method ({
 
my $username = "${unique_name}\@${realm}";
 
-   # test if user exists and is enabled
-   $rpcenv->check_user_enabled($username);
+   # first, check if $username respects our naming conventions
+   PVE::Auth::Plugin::verify_username($username);
+
+   if ($config->{'autocreate'} && 
!$rpcenv->check_user_exist($username, 1)) {
+   PVE::AccessControl::lock_user_config(sub {
+   my $usercfg = cfs_read_file("user.cfg");
+
+   die "user '$username' already exists\n" if 
$usercfg->{users}->{$username};
+
+   my $entry = { enable => 1 };
+   if (defined(my $email = $info->{'email'})) {
+   $entry->{email} = $email;
+   }
+   if (defined(my $given_name = $info->{'given_name'})) {
+   $entry->{firstname} = $given_name;
+   }
+   if (defined(my $family_name = $info->{'family_name'})) {
+   $entry->{lastname} = $family_name;
+   }
+
+   $usercfg->{users}->{$username} = $entry;
+
+   cfs_write_file("user.cfg", $usercfg);
+   }, "autocreate openid user failed");
+   } else {
+   # test if user exists and is enabled
+   $rpcenv->check_user_enabled($username);
+   }
 
my $ticket = PVE::AccessControl::assemble_ticket($username);
my $csrftoken = 
PVE::AccessControl::assemble_csrf_prevention_token($username);
-- 
2.30.2


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH pve-access-control v2 4/5] api: implement openid API

[Dietmar Maurer]

This moves compute_api_permission() into RPCEnvironment.pm.
---
 src/PVE/API2/AccessControl.pm |  60 ++
 src/PVE/API2/Makefile |   3 +-
 src/PVE/API2/OpenId.pm| 211 ++
 src/PVE/RPCEnvironment.pm |  49 
 4 files changed, 270 insertions(+), 53 deletions(-)
 create mode 100644 src/PVE/API2/OpenId.pm

diff --git a/src/PVE/API2/AccessControl.pm b/src/PVE/API2/AccessControl.pm
index a77694b..6dec66c 100644
--- a/src/PVE/API2/AccessControl.pm
+++ b/src/PVE/API2/AccessControl.pm
@@ -19,9 +19,9 @@ use PVE::API2::User;
 use PVE::API2::Group;
 use PVE::API2::Role;
 use PVE::API2::ACL;
+use PVE::API2::OpenId;
 use PVE::Auth::Plugin;
 use PVE::OTP;
-use PVE::Tools;
 
 my $u2f_available = 0;
 eval {
@@ -56,6 +56,11 @@ __PACKAGE__->register_method ({
 path => 'domains',
 });
 
+__PACKAGE__->register_method ({
+subclass => "PVE::API2::OpenId",
+path => 'openid',
+});
+
 __PACKAGE__->register_method ({
 name => 'index',
 path => '',
@@ -165,55 +170,6 @@ my $create_ticket = sub {
 };
 };
 
-my $compute_api_permission = sub {
-my ($rpcenv, $authuser) = @_;
-
-my $usercfg = $rpcenv->{user_cfg};
-
-my $res = {};
-my $priv_re_map = {
-   vms => qr/VM\.|Permissions\.Modify/,
-   access => qr/(User|Group)\.|Permissions\.Modify/,
-   storage => qr/Datastore\.|Permissions\.Modify/,
-   nodes => qr/Sys\.|Permissions\.Modify/,
-   sdn => qr/SDN\.|Permissions\.Modify/,
-   dc => qr/Sys\.Audit|SDN\./,
-};
-map { $res->{$_} = {} } keys %$priv_re_map;
-
-my $required_paths = ['/', '/nodes', '/access/groups', '/vms', '/storage', 
'/sdn'];
-
-my $checked_paths = {};
-foreach my $path (@$required_paths, keys %{$usercfg->{acl}}) {
-   next if $checked_paths->{$path};
-   $checked_paths->{$path} = 1;
-
-   my $path_perm = $rpcenv->permissions($authuser, $path);
-
-   my $toplevel = ($path =~ /^\/(\w+)/) ? $1 : 'dc';
-   if ($toplevel eq 'pool') {
-   foreach my $priv (keys %$path_perm) {
-   if ($priv =~ m/^VM\./) {
-   $res->{vms}->{$priv} = 1;
-   } elsif ($priv =~ m/^Datastore\./) {
-   $res->{storage}->{$priv} = 1;
-   } elsif ($priv eq 'Permissions.Modify') {
-   $res->{storage}->{$priv} = 1;
-   $res->{vms}->{$priv} = 1;
-   }
-   }
-   } else {
-   my $priv_regex = $priv_re_map->{$toplevel} // next;
-   foreach my $priv (keys %$path_perm) {
-   next if $priv !~ m/^($priv_regex)/;
-   $res->{$toplevel}->{$priv} = 1;
-   }
-   }
-}
-
-return $res;
-};
-
 __PACKAGE__->register_method ({
 name => 'get_ticket',
 path => 'ticket',
@@ -314,7 +270,7 @@ __PACKAGE__->register_method ({
die PVE::Exception->new("authentication failure\n", code => 401);
}
 
-   $res->{cap} = &$compute_api_permission($rpcenv, $username)
+   $res->{cap} = $rpcenv->compute_api_permission($username)
if !defined($res->{NeedTFA});
 
my $clinfo = PVE::Cluster::get_clinfo();
@@ -659,7 +615,7 @@ __PACKAGE__->register_method({
 
return {
ticket => PVE::AccessControl::assemble_ticket($authuser),
-   cap => &$compute_api_permission($rpcenv, $authuser),
+   cap => $rpcenv->compute_api_permission($authuser),
}
 }});
 
diff --git a/src/PVE/API2/Makefile b/src/PVE/API2/Makefile
index 1bf8c05..4e49037 100644
--- a/src/PVE/API2/Makefile
+++ b/src/PVE/API2/Makefile
@@ -5,7 +5,8 @@ API2_SOURCES=   \
ACL.pm  \
Role.pm \
Group.pm\
-   User.pm
+   User.pm \
+   OpenId.pm
 
 .PHONY: install
 install:
diff --git a/src/PVE/API2/OpenId.pm b/src/PVE/API2/OpenId.pm
new file mode 100644
index 000..d0b29fc
--- /dev/null
+++ b/src/PVE/API2/OpenId.pm
@@ -0,0 +1,211 @@
+package PVE::API2::OpenId;
+
+use strict;
+use warnings;
+
+use PVE::Tools qw(extract_param);
+use PVE::RS::OpenId;
+
+use PVE::Exception qw(raise raise_perm_exc raise_param_exc);
+use PVE::SafeSyslog;
+use PVE::RPCEnvironment;
+use PVE::Cluster qw(cfs_read_file);
+use PVE::AccessControl;
+use PVE::JSONSchema qw(get_standard_option);
+
+use PVE::RESTHandler;
+
+use base qw(PVE::RESTHandler);
+
+my $openid_state_path = "/var/lib/pve-manager";
+
+my $lookup_openid_auth = sub {
+my ($realm, $redirect_url) = @_;
+
+my $cfg = cfs_read_file('domains.cfg');
+my $ids = $cfg->{ids};
+
+die "authentication domain '$realm' does not exist\n" if !$ids->{$realm};
+
+my $config = $ids->{$realm};
+die "wrong realm type ($config->{type} != openid)\n" if $config->{type} ne 
"openid";
+
+my $openid_config = {
+   issuer_url => $config->{'issuer-url'},
+   client_id => $config->{'client-id'},
+   client_key => $config->{'client-key'},
+

[pve-devel] [PATCH pve-docs] add OpenId Connect docu

[Dietmar Maurer]

---
 pveum.adoc | 88 +-
 1 file changed, 87 insertions(+), 1 deletion(-)

diff --git a/pveum.adoc b/pveum.adoc
index a1adbaa..9329583 100644
--- a/pveum.adoc
+++ b/pveum.adoc
@@ -29,7 +29,7 @@ endif::manvolnum[]
 
 Proxmox VE supports multiple authentication sources, e.g. Linux PAM,
 an integrated Proxmox VE authentication server, LDAP, Microsoft Active
-Directory.
+Directory and OpenId Connect.
 
 By using the role based user- and permission management for all
 objects (VMs, storages, nodes, etc.) granular access can be defined.
@@ -194,6 +194,92 @@ Microsoft Active Directory::
 A server and authentication domain need to be specified. Like with LDAP, an
 optional fallback server, port, and SSL encryption can be configured.
 
+OpenId Connect::
+
+OpenID Connect allows clients to verify the identity of the user based
+on the authentication performed by an external authorization
+server.
+
+
+[[pveum_openid]]
+OpenId Connect
+~~
+
+The main OpenID Connect configuration options are:
+
+* `issuer-url`: This is the Url to the authorization server. Proxmox
+uses the OpenID Connect Discovery protocol to automatiocally configure
+further details.
++
+While it is possible to use unencrypted `http://` Urls, we strongly recommend 
to
+use encrypted `https://` connections.
+
+* `client-id`:  OpenID Client ID.
+
+* `client-key`: Optional OpenID Client Key.
+
+* `autocreate`: Automatically create users if they do not exist. While
+authentification is done at the OpenID server, all users still need an
+entry in the {pve} user configuration. You can either add them
+manually, or use the `autocreate` option to automatically add new
+users.
+
+* `username-claim`: OpenID claim used to generate the unique username
+  (`subject`, `username` or `email`).
+
+Username mapping
+
+
+The Openid Connect specification defines a single unique attribute
+('claim' in OpenId terms) named `subject`. By default, we use the
+value of this attribute to generate {pve} usernames, by simple adding
+`@` and the realm name: `${subject}@${realm}`.
+
+Unfortunately, most OpenID server use random strings for `subject`, like
+`DGH76OKH34BNG3245SB`, so a typical username would look like
+`DGH76OKH34BNG3245SB@yourrealm`. While unique, it is really hard for
+humans to remember such random strings, making it quite impossible to
+associate real users with that.
+
+The `username-claim` setting allows you to use other attributes for
+the username mapping. Setting it to `username` is preferred, if the
+OpenId Connect server provides that attribute and guarrantee its
+uniqueness.
+
+Another option is to use `email`, which also yields to human readable
+usernames. Again, only use this setting if the server guarrantees the
+uniqueness of this attribute.
+
+Examples
+
+
+Here is an example to create an OpenId realm using Google. You need to
+replace `--client-id` and `--client-key` with the values
+from your Google OpenId settings.
+
+
+pveum realm add myrealm1 --type openid --issuer-url  
https://accounts.google.com --client-id  --client-key  --username-claim 
email
+
+
+Above setup uses `--username-claim email`, so the usernames at the
+{pve} side looks like `example.u...@google.com@myrealm1`.
+
+KeyCloak (https://www.keycloak.org/) is a popular Open Source Identity
+and Access Management supporting OpenId Connect. In the following
+example, you need to replace the `--issuer-url` and `--client-id` with
+your setting:
+
+
+pveum realm add myrealm2 --type openid --issuer-url  
https://your.server:8080/auth/realms/your-realm --client-id XXX 
--username-claim username
+
+
+Using `--username-claim username` yields to simple usernames on the
+{pve} side, like `example.user@myrealm2`.
+
+WARNING: You need to make sure that the user is not allowed to edit
+the username setting himself (on the Keycloak server).
+
+
 [[pveum_ldap_sync]]
 Syncing LDAP-based realms
 ~
-- 
2.30.2


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] Flow for i18n

[Dietmar Maurer]

Isn't it easy enough to edit a po file?

I am not particularly keen to setup and maintain another tool/service.

> On 08/18/2021 12:46 PM Claudio Ferreira  wrote:
> 
>  
> Hi for all
> 
> My name is Claudio and I did some translations for many open source
> projects, and I want to help in ProxMox for Brazilian Portuguese (pt-BR)
> language. I saw your docs for translators and tried to understand your flow
> for i18n. It is simple using gettext, but without a tool for optimization,
> if I saw correctly.
> 
> In some project translations, like Debian Handbook, they use Weblate[1], a
> web translation system developed by a Debian Developer, just to help open
> source projects to have an easy system to maintain your translations, so I
> think that ProxMox could use it too.
> 
> Weblate have integration with git, so you can integrate it with your
> repository. You maintain the original po/pot file(s) and publish in git.
> Import to weblate (via git), generate lang files or import the existing
> files, and translation volunteers enter in weblate and fastly translate
> your respective files. After, reimport (as a PR) in your git.
> 
> What do you think? Is interesting for Promox project use a tool to turn
> more easy maintain i18n flow?
> 
> Regards,
> Claudio Ferreira
> ___
> pve-devel mailing list
> pve-devel@lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] building pve-docs - Can't locate PVE/RESTHandler.pm

[Dietmar Maurer]

> I don't know where to find the PVE::RESTHandler module.

Any (working) PVE installation should have that installed...


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [proxmox-backup-qemu] update dependencies to latest proxmox-backup git version

[Dietmar Maurer]

Signed-off-by: Dietmar Maurer 
---
 Cargo.toml  | 15 +++
 src/backup.rs   | 11 +++
 src/commands.rs | 15 +--
 src/lib.rs  | 11 ++-
 src/restore.rs  | 13 ++---
 src/shared_cache.rs |  2 +-
 src/upload_queue.rs |  6 --
 7 files changed, 48 insertions(+), 25 deletions(-)

diff --git a/Cargo.toml b/Cargo.toml
index 5dbe483..d9a7de1 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -27,10 +27,17 @@ lazy_static = "1.4"
 libc = "0.2"
 once_cell = "1.5"
 openssl = "0.10"
-proxmox = { version = "0.11.5", features = [ "sortable-macro", "api-macro" ] }
-#proxmox = { path = "../proxmox/proxmox" }
-proxmox-backup = { git = "git://git.proxmox.com/git/proxmox-backup.git", 
commit = "74a4f9efc95794acb3ea1e8d72ef9ead8f78e250" }
-#proxmox-backup = { path = "../proxmox-backup" }
+proxmox = { version = "0.15.0", features = [ "sortable-macro"] }
+proxmox-schema = { version = "1", features = [ "api-macro" ] }
+proxmox-lang = "1"
+
+pbs-runtime = { git = "git://git.proxmox.com/git/proxmox-backup.git", rev = 
"2bc1250c28de0ecf595b9933af9f900cbee52fdc" }
+pbs-api-types = { git = "git://git.proxmox.com/git/proxmox-backup.git", rev = 
"2bc1250c28de0ecf595b9933af9f900cbee52fdc" }
+pbs-tools = { git = "git://git.proxmox.com/git/proxmox-backup.git", rev = 
"2bc1250c28de0ecf595b9933af9f900cbee52fdc" }
+pbs-config = { git = "git://git.proxmox.com/git/proxmox-backup.git", rev = 
"2bc1250c28de0ecf595b9933af9f900cbee52fdc" }
+pbs-datastore = { git = "git://git.proxmox.com/git/proxmox-backup.git", rev = 
"2bc1250c28de0ecf595b9933af9f900cbee52fdc" }
+pbs-client = { git = "git://git.proxmox.com/git/proxmox-backup.git" , rev = 
"2bc1250c28de0ecf595b9933af9f900cbee52fdc" }
+
 serde_json = "1.0"
 tokio = { version = "1.6", features = [ "fs", "io-util", "macros", "net", 
"rt-multi-thread", "signal", "time" ] }
 tokio-stream = "0.1.1"
diff --git a/src/backup.rs b/src/backup.rs
index 8f64979..186730f 100644
--- a/src/backup.rs
+++ b/src/backup.rs
@@ -7,12 +7,15 @@ use std::os::raw::c_int;
 use futures::future::{Future, Either, FutureExt};
 use tokio::runtime::Runtime;
 
-use proxmox_backup::tools::runtime::get_runtime_with_builder;
-use proxmox_backup::backup::{CryptConfig, CryptMode, BackupDir, 
BackupManifest, KeyConfig, load_and_decrypt_key, rsa_encrypt_key_config};
-use proxmox_backup::client::{HttpClient, HttpClientOptions, BackupWriter};
-
 use proxmox::tools::fs::file_get_contents;
 
+use pbs_runtime::get_runtime_with_builder;
+use pbs_api_types::CryptMode;
+use pbs_tools::crypt_config::CryptConfig;
+use pbs_config::key_config::{KeyConfig, load_and_decrypt_key, 
rsa_encrypt_key_config};
+use pbs_datastore::{BackupDir, BackupManifest,};
+use pbs_client::{HttpClient, HttpClientOptions, BackupWriter};
+
 use super::BackupSetup;
 use crate::capi_types::*;
 use crate::registry::Registry;
diff --git a/src/commands.rs b/src/commands.rs
index 5fdf318..c2a1abb 100644
--- a/src/commands.rs
+++ b/src/commands.rs
@@ -6,12 +6,16 @@ use std::os::raw::c_int;
 use futures::future::{Future, TryFutureExt};
 use serde_json::json;
 
-use proxmox_backup::backup::*;
-use proxmox_backup::client::*;
+use pbs_api_types::CryptMode;
+use pbs_tools::crypt_config::CryptConfig;
+use pbs_datastore::index::IndexFile;
+use pbs_datastore::data_blob::DataChunkBuilder;
+use pbs_datastore::manifest::{BackupManifest, MANIFEST_BLOB_NAME, 
ENCRYPTED_KEY_BLOB_NAME};
+use pbs_client::{BackupWriter, H2Client, UploadOptions};
 
 use crate::registry::Registry;
-use crate::capi_types::*;
-use crate::upload_queue::*;
+use crate::capi_types::DataPointer;
+use crate::upload_queue::{ChunkUploadInfo, UploadQueueSender, 
UploadResultReceiver,  create_upload_queue};
 
 use lazy_static::lazy_static;
 
@@ -160,7 +164,7 @@ pub(crate) fn check_last_encryption_key(
 let fingerprint_guard = PREVIOUS_KEY_FINGERPRINT.lock().unwrap();
 match (*fingerprint_guard, config)  {
 (Some(last_fingerprint), Some(current_config)) => {
-current_config.fingerprint().bytes() == &last_fingerprint
+current_config.fingerprint() == last_fingerprint
 || crypt_config_digest(current_config) == last_fingerprint
 },
 (None, None) => true,
@@ -462,7 +466,6 @@ pub(crate) async fn finish_backup(
 Some(current_config) => {
 let fp = current_config
 .fingerprint()
-.bytes()
 .to_owned();
 Some(fp)
 },
diff --git a/src/lib.rs b/src/lib.rs
index aa167f7..436a592 100644
--- a/src/lib.rs
++

Re: [pve-devel] Groups for OpenID Connect?

[Dietmar Maurer]

> However, is there any support for groups in OpenID Connect, or a similar 
> concept?

In OpenID, it is possible to request "scopes" from the server, which can then 
send additional data (claims). 

But I am unsure if and how people use those system to manage groups. So what 
kind of OpenID server do you use, and how does it store the group information?


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] Groups for OpenID Connect?

[Dietmar Maurer]

> This endpoint here would be Google Workspace (i.e. Google's OIDC provider).
> 
> Currently, in the Proxmox LDAP sync - it translates Google Groups (in the 
> Google Workspace domain) into LDAP groups, which is what we want.
> 
> I'm not too familiar with the OIDC - I do know that Google Workspace has it's 
> own APIs to lookup group membership:

OIDC does not provide any snyc protocol, so this kind of thing is impossible.


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] auto ballooning && ksm thresholds both at 80%

[Dietmar Maurer]

> Currently both are at 80%,
> that mean that ballooning is vm reducing memory fast, and ksm don't
> have time to run.
> 
> as ballooning is a lot more intrusive than ksm, I wonder if it couldn't
> be set to something like 90%.

That sounds reasonable to me, but can you see that theoretical effect when you 
run your test?


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] Enabling mips/mipsel support in Hypervisor

[Dietmar Maurer]

> So how can I get it to build also mips/mipsel qemu-system image?
> That to me did not really tell me how it's disabled but what would be 
> needed so I can build the support?

We do not support MIPS, so you need to find out yourself what is required.


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] applied: [PATCH] ui: Close #2908: fixed password validator for adding PBS as storage

[Dietmar Maurer]

applied, thanks!


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] Proxmox, Qemu and COLO feature...

[Dietmar Maurer]

> Could you guys tell me how the status of qemu COLO feature regarding
> Proxmox VE?
> Is there any plan for this?

No plans for now ...


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] More than 10 interfaces in lxc containers

[Dietmar Maurer]

> For me, I have that need too for a firewall container.

Why does your firewall need more the 10 interface?

> Would you please consider raising the limit? 

No, unless someone can explain why that is required ;-)


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] More than 10 interfaces in lxc containers

[Dietmar Maurer]

> > For me, I have that need too for a firewall container.
> 
> Why does your firewall need more the 10 interface?

Sigh. too early in the morning... I wanted to ask:

Why does your firewall need more than 10 interfaces?

Normally, a firewall uses one interface per zone, and more
than 10 zones are quite uncommon?

> > Would you please consider raising the limit? 
> 
> No, unless someone can explain why that is required ;-)
> 
> 
> ___
> pve-devel mailing list
> pve-devel@lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] More than 10 interfaces in lxc containers

[Dietmar Maurer]

> If it would be possible to provide a 'trunk' openvswitch interface to 
> the CT, then from within the CT vlan devices could be setup from the 
> trunk, but in the end that will still create 10+ interfaces in the 
> container itself.

Cant you simply use a single network interface, then configure the vlans
inside the firewall?

IMHO, using one interface for each VLAN is the wrong approach. I am sure
next time people will ask for 4095 interfaces ...


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] More than 10 interfaces in lxc containers

[Dietmar Maurer]

> If I don't put a tag on the device, it seems to behave like a trunk. So, 
> that would solve my problem. _If_ the hosts where openvswitch enabled.

I am unable to see why you need openvswitch for that? This also works with
standard linux network.


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] More than 10 interfaces in lxc containers

[Dietmar Maurer]

> On 08/24/2020 12:54 PM Stephan Leemburg  wrote:
> 
>  
> On 24-08-2020 06:53, Dietmar Maurer wrote:
> >> If I don't put a tag on the device, it seems to behave like a trunk. So,
> >> that would solve my problem. _If_ the hosts where openvswitch enabled.
> > I am unable to see why you need openvswitch for that? This also works with
> > standard linux network.
> 
> Hi Dietmar,
> 
> Oh, that is new for me.
> 
> So, I can have a vlan aware traditional bridge in the firewall that 
> receives tagged frames and at the same time have the clients on the 
> specific 'vlans' receive non-tagged frames for their respective pvid?
> 
> How can this be configured in Proxmox?

You do not not any special config on the pve host if you do all VLAN related
stuff inside the VM.


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH common] get_options: allow optional arguments "arg_params" if no ambiguity

[Dietmar Maurer]

> If we run out of passed arguments from the user but still had defined
> "arg_params" (those params which went after the command in fixed
> order without option -- dashes) we always errored out with "not
> enough arguments". But, there are situations where the remaining
> arg_params are all marked as optional in the schema, so we do not
> need to error out in that case.

Ok for me, but

> A prime (future) use case is "pvesm prune-backups". Currently the
> usage is:
> > pvesm prune-backups storeid --prune-backups keep-last=1,keep-...

I would like to have a different CLI for that:

pvesm prune-backups storeid --keep-last 2 --keep-weekly 5

any opinions?


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] corosync bug: cluster break after 1 node clean shutdown

[Dietmar Maurer]

> do you think it could be possible to add an extra optionnal layer of security 
> check, not related to corosync ?

I would try to find the bug instead.

> I'm still afraid of this corosync bug since years, and still don't use HA. 
> (or I have tried to enable it 2months ago,and this give me a disaster 
> yesterday..)
> 
> Something like an extra heartbeat between nodes daemons, and check if we also 
> have quorum with theses heartbeats ?

Was this even related to corosync? What exactly caused the reboot?


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [RFC proxmox-backup] avoid chrono dependency

[Dietmar Maurer]

depend on proxmox 0.3.7
---
 Cargo.toml |   3 +-
 examples/download-speed.rs |   4 +-
 examples/upload-speed.rs   |   2 +-
 src/api2/admin/datastore.rs|  28 ++---
 src/api2/backup.rs |   2 +-
 src/api2/node/time.rs  |  10 +-
 src/api2/reader.rs |   5 +-
 src/backup/backup_info.rs  |  56 --
 src/backup/catalog.rs  |  11 +-
 src/backup/crypt_config.rs |   5 +-
 src/backup/datastore.rs|   3 +-
 src/backup/fixed_index.rs  |  15 +--
 src/backup/key_derivation.rs   |  17 ++-
 src/backup/manifest.rs |   2 +-
 src/backup/prune.rs|  29 +++--
 src/bin/proxmox-backup-client.rs   |  58 +-
 src/bin/proxmox_backup_client/benchmark.rs |   3 +-
 src/bin/proxmox_backup_client/key.rs   |   5 +-
 src/client/backup_reader.rs|   5 +-
 src/client/backup_writer.rs|   5 +-
 src/client/http_client.rs  |   5 +-
 src/client/pull.rs |   2 +-
 src/pxar/tools.rs  |  10 +-
 src/server/upid.rs |   3 +-
 src/server/worker_task.rs  |  11 +-
 src/tools/file_logger.rs   |   7 +-
 src/tools/format.rs|  10 +-
 src/tools/systemd.rs   |   1 -
 src/tools/systemd/time.rs  |   5 +-
 src/tools/systemd/tm_editor.rs | 119 -
 30 files changed, 146 insertions(+), 295 deletions(-)
 delete mode 100644 src/tools/systemd/tm_editor.rs

diff --git a/Cargo.toml b/Cargo.toml
index 0625ffb..17d9bb8 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -18,7 +18,6 @@ apt-pkg-native = "0.3.1" # custom patched version
 base64 = "0.12"
 bitflags = "1.2.1"
 bytes = "0.5"
-chrono = "0.4" # Date and time library for Rust
 crc32fast = "1"
 endian_trait = { version = "0.6", features = ["arrays"] }
 anyhow = "1.0"
@@ -39,7 +38,7 @@ pam-sys = "0.5"
 percent-encoding = "2.1"
 pin-utils = "0.1.0"
 pathpatterns = "0.1.2"
-proxmox = { version = "0.3.5", features = [ "sortable-macro", "api-macro", 
"websocket" ] }
+proxmox = { version = "0.3.7", features = [ "sortable-macro", "api-macro", 
"websocket" ] }
 #proxmox = { git = "ssh://gitoli...@proxdev.maurer-it.com/rust/proxmox", 
version = "0.1.2", features = [ "sortable-macro", "api-macro" ] }
 #proxmox = { path = "../proxmox/proxmox", features = [ "sortable-macro", 
"api-macro", "websocket" ] }
 proxmox-fuse = "0.1.0"
diff --git a/examples/download-speed.rs b/examples/download-speed.rs
index 694c55d..0ec6cb6 100644
--- a/examples/download-speed.rs
+++ b/examples/download-speed.rs
@@ -2,8 +2,6 @@ use std::io::Write;
 
 use anyhow::{Error};
 
-use chrono::{DateTime, Utc};
-
 use proxmox_backup::api2::types::Userid;
 use proxmox_backup::client::{HttpClient, HttpClientOptions, BackupReader};
 
@@ -36,7 +34,7 @@ async fn run() -> Result<(), Error> {
 
 let client = HttpClient::new(host, username, options)?;
 
-let backup_time = "2019-06-28T10:49:48Z".parse::>()?;
+let backup_time = 
proxmox::tools::time::parse_rfc3339("2019-06-28T10:49:48Z")?;
 
 let client = BackupReader::start(client, None, "store2", "host", "elsa", 
backup_time, true)
 .await?;
diff --git a/examples/upload-speed.rs b/examples/upload-speed.rs
index d67e9da..6be29b0 100644
--- a/examples/upload-speed.rs
+++ b/examples/upload-speed.rs
@@ -16,7 +16,7 @@ async fn upload_speed() -> Result {
 
 let client = HttpClient::new(host, username, options)?;
 
-let backup_time = chrono::Utc::now();
+let backup_time = proxmox::tools::time::time()?;
 
 let client = BackupWriter::start(client, None, datastore, "host", 
"speedtest", backup_time, false, true).await?;
 
diff --git a/src/api2/admin/datastore.rs b/src/api2/admin/datastore.rs
index be2796d..acfc265 100644
--- a/src/api2/admin/datastore.rs
+++ b/src/api2/admin/datastore.rs
@@ -172,7 +172,7 @@ fn list_groups(
 let result_item = GroupListItem {
 backup_type: group.backup_type().to_string(),
 backup_id: group.backup_id().to_string(),
-last_backup: info.backup_dir.backup_time().timestamp(),
+last_backup: info.backup_dir.backup_time(),
 backup_count: list.len() as u64,
 files: info.files.clone(),
 owner: Some(owner),
@@ -230,7 +230,7 @@ pub fn list_snapshot_files(
 
 let datastore = DataStore::lookup_datastore(&store)?;
 
-let snapshot = BackupDir::new(backup_type, backup_id, backup_time)?;
+let snapshot = BackupDir::new(backup_type, backup_id, backup_time);
 
 let allowed = (user_privs & (PRIV_DATASTORE_AUDIT | PRIV_DATASTORE_READ)) 
!= 0;
 if !allowed { check_backup_owner(&datastore, snapshot.group(), &userid)?; }
@@ -280,7 +280,7 

Re: [pve-devel] corosync bug: cluster break after 1 node clean shutdown

[Dietmar Maurer]

> I wonder if something like pacemaker sbd could be implemented in proxmox as 
> extra layer of protection ?

AFAIK Thomas already has patches to implement active fencing.

But IMHO this will not solve the corosync problems..


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH proxmox-backup] avoid chrono dependency, depend on proxmox 0.3.8

[Dietmar Maurer]

- remove chrono dependency

- depend on proxmox 0.3.8

- remove epoch_now, epoch_now_u64 and epoch_now_f64

- remove tm_editor (moved to proxmox crate)

- use new helpers from proxmox 0.3.8
  * epoch_i64 and epoch_f64
  * parse_rfc3339
  * epoch_to_rfc3339_utc
  * strftime_local

- BackupDir changes:
  * store epoch and rfc3339 string instead of DateTime
  * backup_time_to_string now return a Result
  * remove unnecessary TryFrom<(BackupGroup, i64)> for BackupDir

- DynamicIndexHeader: change ctime to i64

- FixedIndexHeader: change ctime to i64
---
 Cargo.toml |   3 +-
 examples/download-speed.rs |   4 +-
 examples/upload-speed.rs   |   2 +-
 src/api2/admin/datastore.rs|  12 +--
 src/api2/node/rrd.rs   |   3 +-
 src/api2/node/time.rs  |  10 +-
 src/api2/reader.rs |   3 +-
 src/api2/status.rs |   3 +-
 src/auth_helpers.rs|   5 +-
 src/backup/backup_info.rs  |  77 +++--
 src/backup/catalog.rs  |   9 +-
 src/backup/crypt_config.rs |   5 +-
 src/backup/datastore.rs|   3 +-
 src/backup/dynamic_index.rs|  14 +--
 src/backup/fixed_index.rs  |  29 +++--
 src/backup/key_derivation.rs   |  17 ++-
 src/backup/manifest.rs |   2 +-
 src/backup/prune.rs|  61 +--
 src/bin/proxmox-backup-client.rs   |  62 +--
 src/bin/proxmox-backup-proxy.rs|  41 +++
 src/bin/proxmox_backup_client/benchmark.rs |   3 +-
 src/bin/proxmox_backup_client/key.rs   |   5 +-
 src/client/backup_reader.rs|   5 +-
 src/client/backup_writer.rs|   5 +-
 src/client/http_client.rs  |   5 +-
 src/config/jobstate.rs |   5 +-
 src/pxar/tools.rs  |   8 +-
 src/rrd/cache.rs   |   3 +-
 src/server/upid.rs |   3 +-
 src/server/worker_task.rs  |  10 +-
 src/tools.rs   |  14 ---
 src/tools/file_logger.rs   |   6 +-
 src/tools/format.rs|   8 +-
 src/tools/systemd.rs   |   1 -
 src/tools/systemd/time.rs  |   5 +-
 src/tools/systemd/tm_editor.rs | 119 -
 src/tools/ticket.rs|   8 +-
 37 files changed, 198 insertions(+), 380 deletions(-)
 delete mode 100644 src/tools/systemd/tm_editor.rs

diff --git a/Cargo.toml b/Cargo.toml
index 0625ffb..9d24242 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -18,7 +18,6 @@ apt-pkg-native = "0.3.1" # custom patched version
 base64 = "0.12"
 bitflags = "1.2.1"
 bytes = "0.5"
-chrono = "0.4" # Date and time library for Rust
 crc32fast = "1"
 endian_trait = { version = "0.6", features = ["arrays"] }
 anyhow = "1.0"
@@ -39,7 +38,7 @@ pam-sys = "0.5"
 percent-encoding = "2.1"
 pin-utils = "0.1.0"
 pathpatterns = "0.1.2"
-proxmox = { version = "0.3.5", features = [ "sortable-macro", "api-macro", 
"websocket" ] }
+proxmox = { version = "0.3.8", features = [ "sortable-macro", "api-macro", 
"websocket" ] }
 #proxmox = { git = "ssh://gitoli...@proxdev.maurer-it.com/rust/proxmox", 
version = "0.1.2", features = [ "sortable-macro", "api-macro" ] }
 #proxmox = { path = "../proxmox/proxmox", features = [ "sortable-macro", 
"api-macro", "websocket" ] }
 proxmox-fuse = "0.1.0"
diff --git a/examples/download-speed.rs b/examples/download-speed.rs
index 694c55d..0ec6cb6 100644
--- a/examples/download-speed.rs
+++ b/examples/download-speed.rs
@@ -2,8 +2,6 @@ use std::io::Write;
 
 use anyhow::{Error};
 
-use chrono::{DateTime, Utc};
-
 use proxmox_backup::api2::types::Userid;
 use proxmox_backup::client::{HttpClient, HttpClientOptions, BackupReader};
 
@@ -36,7 +34,7 @@ async fn run() -> Result<(), Error> {
 
 let client = HttpClient::new(host, username, options)?;
 
-let backup_time = "2019-06-28T10:49:48Z".parse::>()?;
+let backup_time = 
proxmox::tools::time::parse_rfc3339("2019-06-28T10:49:48Z")?;
 
 let client = BackupReader::start(client, None, "store2", "host", "elsa", 
backup_time, true)
 .await?;
diff --git a/examples/upload-speed.rs b/examples/upload-speed.rs
index d67e9da..896e31e 100644
--- a/examples/upload-speed.rs
+++ b/examples/upload-speed.rs
@@ -16,7 +16,7 @@ async fn upload_speed() -> Result {
 
 let client = HttpClient::new(host, username, options)?;
 
-let backup_time = chrono::Utc::now();
+let backup_time = proxmox::tools::time::epoch_i64();
 
 let client = BackupWriter::start(client, None, datastore, "host", 
"speedtest", backup_time, false, true).await?;
 
diff --git a/src/api2/admin/datastore.rs b/src/api2/admin/datastore.rs
index be2796d..af3af0a 100644
--- a/src/api2/admin/datastore

[pve-devel] applied: [PATCH widget-toolkit] fix autoscrolling on browser zoom

[Dietmar Maurer]

applied


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] applied: [PATCH pve-common 1/1] ProcFSTools: add read_pressure

[Dietmar Maurer]

applied


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH pve-manager] pvestatd: stream host pressure counters

[Dietmar Maurer]

I wonder if we want to store that information with RRD instead?

> On 10/06/2020 1:58 PM Alexandre Derumier  wrote:
> 
>  
> Signed-off-by: Alexandre Derumier 
> ---
>  PVE/Service/pvestatd.pm | 4 
>  1 file changed, 4 insertions(+)
> 
> diff --git a/PVE/Service/pvestatd.pm b/PVE/Service/pvestatd.pm
> index 5e533ca3..238c36ae 100755
> --- a/PVE/Service/pvestatd.pm
> +++ b/PVE/Service/pvestatd.pm
> @@ -161,12 +161,16 @@ sub update_node_status {
>  );
>  PVE::Cluster::broadcast_rrd("pve2-node/$nodename", $data);
>  
> +my $pressure = PVE::ProcFSTools::read_pressure();
> +PVE::Cluster::broadcast_node_kv("pressure", encode_json($pressure));
> +


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH pve-common 1/1] ProcFSTools: add read_pressure

[Dietmar Maurer]

> I have notice that it's possible to get pressure info for each vm/ct
> through cgroups
> 
> /sys/fs/cgroup/unified/qemu.slice/.scope/cpu.pressure
> /sys/fs/cgroup/unified/lxc//cpu.pressure
> 
> 
> Maybe it could be great to have some new rrd graphs for each vm/ct ?
> They are very useful counters to known a specific vm/ct is overloaded

I have no idea how reliable this is, because we do not use cgroups v2. But yes,
I think this would be useful.


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH pve-common 1/1] ProcFSTools: add read_pressure

[Dietmar Maurer]

> BTW, I'm currently playing with reading the rrd files, and I have notice than 
> lower precision is 1minute.
> as pvestatd send values around each 10s, is this 1minute precision an average 
> of 6x10s values send by pvestatd ?

Yes (we also store the MAX)

> I'm currently working on a poc of vm balancing, but I would like to have 
> something like 15min of 10s precision (90 samples of 10s).

Why do you need 10s resulution? Isn't 1min good enough?

> So currently I'm getting stats each 10s manually with 
> PVE::API2Tools::extract_vm_stats like the ressource api.
> (This use PVE::Cluster::rrd_dump , but I don't understand the ipcc_. code. 
> does it only return current streamed values?
> then after the rrdcached daemon is writing to rrd file the average values 
> each minute ?)
> 
> I don't known if we could have rrd files with 15min of 10s precision ? (don't 
> known the write load impact on disks)

We use the following RRD conf, step is 60 seconds (see 
pve-cluster/src/status.c):

static const char *rrd_def_node[] = {
"DS:loadavg:GAUGE:120:0:U",
"DS:maxcpu:GAUGE:120:0:U",
"DS:cpu:GAUGE:120:0:U",
"DS:iowait:GAUGE:120:0:U",
"DS:memtotal:GAUGE:120:0:U",
"DS:memused:GAUGE:120:0:U",
"DS:swaptotal:GAUGE:120:0:U",
"DS:swapused:GAUGE:120:0:U",
"DS:roottotal:GAUGE:120:0:U",
"DS:rootused:GAUGE:120:0:U",
"DS:netin:DERIVE:120:0:U",
"DS:netout:DERIVE:120:0:U",

"RRA:AVERAGE:0.5:1:70", // 1 min avg - one hour
"RRA:AVERAGE:0.5:30:70", // 30 min avg - one day
"RRA:AVERAGE:0.5:180:70", // 3 hour avg - one week
"RRA:AVERAGE:0.5:720:70", // 12 hour avg - one month
"RRA:AVERAGE:0.5:10080:70", // 7 day avg - ony year

"RRA:MAX:0.5:1:70", // 1 min max - one hour
"RRA:MAX:0.5:30:70", // 30 min max - one day
"RRA:MAX:0.5:180:70",  // 3 hour max - one week
"RRA:MAX:0.5:720:70", // 12 hour max - one month
"RRA:MAX:0.5:10080:70", // 7 day max - ony year
NULL,
};

Also See: man rrdcreate

So no, you do not get 10s precission from RRD.


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH proxmox-widget-toolkit] correctly encode upid paramenter (use encodeURIComponent)

[Dietmar Maurer]

---
 src/window/TaskViewer.js | 8 
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/src/window/TaskViewer.js b/src/window/TaskViewer.js
index 0333472..31e1ebc 100644
--- a/src/window/TaskViewer.js
+++ b/src/window/TaskViewer.js
@@ -14,7 +14,7 @@ Ext.define('Proxmox.window.TaskProgress', {
let task = Proxmox.Utils.parse_task_upid(me.upid);
 
let statstore = Ext.create('Proxmox.data.ObjectStore', {
-url: "/api2/json/nodes/" + task.node + "/tasks/" + me.upid + 
"/status",
+url: "/api2/json/nodes/" + task.node + "/tasks/" + 
encodeURIComponent(me.upid) + "/status",
interval: 1000,
rows: {
status: { defaultValue: 'unknown' },
@@ -190,7 +190,7 @@ Ext.define('Proxmox.window.TaskViewer', {
};
 
let statstore = Ext.create('Proxmox.data.ObjectStore', {
-url: "/api2/json/nodes/" + task.node + "/tasks/" + me.upid + 
"/status",
+url: "/api2/json/nodes/" + task.node + "/tasks/" + 
encodeURIComponent(me.upid) + "/status",
interval: 1000,
rows: rows,
});
@@ -199,7 +199,7 @@ Ext.define('Proxmox.window.TaskViewer', {
 
let stop_task = function() {
Proxmox.Utils.API2Request({
-   url: "/nodes/" + task.node + "/tasks/" + me.upid,
+   url: "/nodes/" + task.node + "/tasks/" + 
encodeURIComponent(me.upid),
waitMsgTarget: me,
method: 'DELETE',
failure: function(response, opts) {
@@ -233,7 +233,7 @@ Ext.define('Proxmox.window.TaskViewer', {
title: gettext('Output'),
tbar: [stop_btn2],
border: false,
-   url: "/api2/extjs/nodes/" + task.node + "/tasks/" + me.upid + 
"/log",
+   url: "/api2/extjs/nodes/" + task.node + "/tasks/" + 
encodeURIComponent(me.upid) + "/log",
});
 
me.mon(statstore, 'load', function() {
-- 
2.20.1


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH pve-manager] pvestatd: stream host pressure counters

[Dietmar Maurer]

> I would like also to add some improvement for vm memory/cpu stats.
> 
> for cpu, currently, we only monitor the qemu process cpu usage, but
> with virtio-net + vhost-net,
> we are missing vhost-* process cpu usage.  (For vms will a lot of
> traffic, this is really signifiant).
> 
> I would like to add something like "hostcpu" counter, will real metric
> (reading the vm cgroup cpu.stat)

make sense.

> Also, for memory, when ballooning is enabled, we don't have anymore
> the real memory usage on the host.
> For example, a windows vm with 16g mem is filling memory with zeroes
> at boot, so from the host all the 16G memory is used, but in windows
> guest we only see something like 2G.
> 
> so, I would like to add something like "hostmemory" counter
> 
> 
> what do you think about this ?

Sounds reasonable. We just need to find a way to add new values
in a compatible way.


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [RFC PATCH] fix #3106: correctly queue incoming connections

[Dietmar Maurer]

---

 based, on Domink's patch, but with the following changes:

 - factor out code into separate function accept_connections()
 - no select with shutdown future (no needed)
 - remove sender2.send_timeout() - not sure why this was there?
 - restict number of spawned tasks

 Seems to work, but I get many handshake errors when connetion
 with the GUI:

 > https handshake failed - the handshake failed: unexpected EOF

 This is because of pve status ping (Thomas will fix that in pve)

 But I am not sure why I get the following?
 
 > https handshakeX failed - the handshake failed: error:14094416:SSL 
 > routines:ssl3_read_bytes:sslv3 alert certificate 
 > unknown:../ssl/record/rec_layer_s3.c:1544:SSL alert number 46


 src/bin/proxmox-backup-proxy.rs | 81 ++---
 1 file changed, 64 insertions(+), 17 deletions(-)

diff --git a/src/bin/proxmox-backup-proxy.rs b/src/bin/proxmox-backup-proxy.rs
index 78ea4d53..1f0c16b4 100644
--- a/src/bin/proxmox-backup-proxy.rs
+++ b/src/bin/proxmox-backup-proxy.rs
@@ -1,4 +1,4 @@
-use std::sync::{Arc};
+use std::sync::{Arc, atomic::{AtomicUsize, Ordering}};
 use std::path::{Path, PathBuf};
 use std::os::unix::io::AsRawFd;
 
@@ -116,25 +116,12 @@ async fn run() -> Result<(), Error> {
 let server = daemon::create_daemon(
 ([0,0,0,0,0,0,0,0], 8007).into(),
 |listener, ready| {
-let connections = 
proxmox_backup::tools::async_io::StaticIncoming::from(listener)
-.map_err(Error::from)
-.try_filter_map(move |(sock, _addr)| {
-let acceptor = Arc::clone(&acceptor);
-async move {
-sock.set_nodelay(true).unwrap();
-
-let _ = set_tcp_keepalive(sock.as_raw_fd(), 
PROXMOX_BACKUP_TCP_KEEPALIVE_TIME);
 
-Ok(tokio_openssl::accept(&acceptor, sock)
-.await
-.ok() // handshake errors aren't be fatal, so 
return None to filter
-)
-}
-});
-let connections = 
proxmox_backup::tools::async_io::HyperAccept(connections);
+let connections = accept_connections(listener, acceptor);
+let connections = hyper::server::accept::from_stream(connections);
 
 Ok(ready
-.and_then(|_| hyper::Server::builder(connections)
+   .and_then(|_| hyper::Server::builder(connections)
 .serve(rest_server)
 .with_graceful_shutdown(server::shutdown_future())
 .map_err(Error::from)
@@ -170,6 +157,66 @@ async fn run() -> Result<(), Error> {
 Ok(())
 }
 
+fn accept_connections(
+mut listener: tokio::net::TcpListener,
+acceptor: Arc,
+) -> 
tokio::sync::mpsc::Receiver,
 Error>> {
+
+let (sender, receiver) = tokio::sync::mpsc::channel(100);
+
+let accept_counter = Arc::new(AtomicUsize::new(0));
+
+const MAX_PENDING_ACCEPTS: usize = 100;
+
+tokio::spawn(async move {
+loop {
+match listener.accept().await {
+Err(err) => {
+eprintln!("error accepting tcp connection: {}", err);
+}
+Ok((sock, _addr)) =>  {
+sock.set_nodelay(true).unwrap();
+let _ = set_tcp_keepalive(sock.as_raw_fd(), 
PROXMOX_BACKUP_TCP_KEEPALIVE_TIME);
+let acceptor = Arc::clone(&acceptor);
+let mut sender = sender.clone();
+
+if accept_counter.load(Ordering::SeqCst) > 
MAX_PENDING_ACCEPTS {
+eprintln!("connection rejected - to many open 
connections");
+continue;
+}
+accept_counter.fetch_add(1, Ordering::SeqCst);
+
+let accept_counter = accept_counter.clone();
+tokio::spawn(async move {
+let accept_future = tokio::time::timeout(
+Duration::new(10, 0), 
tokio_openssl::accept(&acceptor, sock));
+
+let result = accept_future.await;
+
+match result {
+Ok(Ok(connection)) => {
+if let Err(_) = 
sender.send(Ok(connection)).await {
+eprintln!("detect closed connection 
channel");
+}
+}
+Ok(Err(err)) => {
+eprintln!("https handshakeX failed - {}", err);
+}
+Err(_) => {
+eprintln!("https handshake timeout");
+}
+}
+
+accept_counter.fetch_sub(1, Ordering::SeqCst);
+});
+}
+}
+  

Re: [pve-devel] [RFC PATCH] fix #3106: correctly queue incoming connections

[Dietmar Maurer]

> > -let connections = 
> > proxmox_backup::tools::async_io::HyperAccept(connections);
> > +let connections = accept_connections(listener, acceptor);
> > +let connections = 
> > hyper::server::accept::from_stream(connections);
> 
> If we move the `from_stream` into the function below...

I have tried to do that for 2 hours, then gave up
So please tell m e how to make that work!

> >  
> >  Ok(ready
> > -.and_then(|_| hyper::Server::builder(connections)
> > +   .and_then(|_| hyper::Server::builder(connections)
> >  .serve(rest_server)
> >  .with_graceful_shutdown(server::shutdown_future())
> >  .map_err(Error::from)
> > @@ -170,6 +157,66 @@ async fn run() -> Result<(), Error> {
> >  Ok(())
> >  }
> >  
> > +fn accept_connections(
> > +mut listener: tokio::net::TcpListener,
> > +acceptor: Arc,
> > +) -> 
> > tokio::sync::mpsc::Receiver,
> >  Error>> {
> 
> ... then this could probably be shortened to
> 
> ) -> impl Accept {
> 
> shortens the line by 80 ;-)
> 
> > +
> > +let (sender, receiver) = tokio::sync::mpsc::channel(100);
> > +
> > +let accept_counter = Arc::new(AtomicUsize::new(0));
> > +
> > +const MAX_PENDING_ACCEPTS: usize = 100;
> > +
> > +tokio::spawn(async move {
> > +loop {
> > +match listener.accept().await {
> > +Err(err) => {
> > +eprintln!("error accepting tcp connection: {}", err);
> > +}
> > +Ok((sock, _addr)) =>  {
> > +sock.set_nodelay(true).unwrap();
> > +let _ = set_tcp_keepalive(sock.as_raw_fd(), 
> > PROXMOX_BACKUP_TCP_KEEPALIVE_TIME);
> > +let acceptor = Arc::clone(&acceptor);
> > +let mut sender = sender.clone();
> > +
> > +if accept_counter.load(Ordering::SeqCst) > 
> > MAX_PENDING_ACCEPTS {
> > +eprintln!("connection rejected - to many open 
> > connections");
> > +continue;
> > +}
> > +accept_counter.fetch_add(1, Ordering::SeqCst);
> 
> We should think about making a counter guard for this sort of thing,
> because from this point onward we're not allowed to use `?` anywhere,
> which is quite annoying.

yes

> 
> > +
> > +let accept_counter = accept_counter.clone();
> > +tokio::spawn(async move {
> > +let accept_future = tokio::time::timeout(
> > +Duration::new(10, 0), 
> > tokio_openssl::accept(&acceptor, sock));
> > +
> > +let result = accept_future.await;
> > +
> > +match result {
> > +Ok(Ok(connection)) => {
> > +if let Err(_) = 
> > sender.send(Ok(connection)).await {
> > +eprintln!("detect closed connection 
> > channel");
> > +}
> > +}
> > +Ok(Err(err)) => {
> > +eprintln!("https handshakeX failed - {}", 
> > err);
> > +}
> > +Err(_) => {
> > +eprintln!("https handshake timeout");
> > +}
> > +}
> 
> which is why I'd rather thave the part above in its own `async fn`
> followed by the `fetch_sub` below, followed by the `eprintln!()`s.
> 
> > +
> > +accept_counter.fetch_sub(1, Ordering::SeqCst);
> > +});
> > +}
> > +}
> > +}
> > +});
> > +
> > +receiver
> > +}
> > +
> >  fn start_stat_generator() {
> >  let abort_future = server::shutdown_future();
> >  let future = Box::pin(run_stat_generator());
> > -- 
> > 2.20.1


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [RFC PATCH] fix #3106: correctly queue incoming connections

[Dietmar Maurer]

> On Tue, Nov 03, 2020 at 02:25:21PM +0100, Dietmar Maurer wrote:
> > > > -let connections = 
> > > > proxmox_backup::tools::async_io::HyperAccept(connections);
> > > > +let connections = accept_connections(listener, acceptor);
> > > > +let connections = 
> > > > hyper::server::accept::from_stream(connections);
> > > 
> > > If we move the `from_stream` into the function below...
> > 
> > I have tried to do that for 2 hours, then gave up
> > So please tell m e how to make that work!
> 
> nvm, the required feature is still unstable, we'd need to
> 
> -> impl Accept
> 
> but trait bounds on associated types in impl returns is unstable...
> Link: https://github.com/rust-lang/rust/issues/52662
> 
> :(

Well, I call this a serious compiler bug ...

Anyways, my workaround is a single line of code.


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [RFC PATCH] fix #3106: correctly queue incoming connections

[Dietmar Maurer]

> > +Ok((sock, _addr)) =>  {
> > +sock.set_nodelay(true).unwrap();
> > +let _ = set_tcp_keepalive(sock.as_raw_fd(), 
> > PROXMOX_BACKUP_TCP_KEEPALIVE_TIME);
> > +let acceptor = Arc::clone(&acceptor);
> > +let mut sender = sender.clone();
> > +
> > +if accept_counter.load(Ordering::SeqCst) > 
> > MAX_PENDING_ACCEPTS {
> > +eprintln!("connection rejected - to many open 
> > connections");
> > +continue;
> > +}
> > +accept_counter.fetch_add(1, Ordering::SeqCst);
> 
> We should think about making a counter guard for this sort of thing,
> because from this point onward we're not allowed to use `?` anywhere,
> which is quite annoying.

I wonder if we can simply use an Arc for that? The Arc already has
an atomic counter with deref on drop!

And we can query that counter with Arc::strong_count

What do you think?


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [RFC PATCH] fix #3106: correctly queue incoming connections

[Dietmar Maurer]

Answering myself, this works as expected.

I now simply use Arc::new(()) to count references.

> On 11/03/2020 6:45 PM Dietmar Maurer  wrote:
> 
>  
> > > +Ok((sock, _addr)) =>  {
> > > +sock.set_nodelay(true).unwrap();
> > > +let _ = set_tcp_keepalive(sock.as_raw_fd(), 
> > > PROXMOX_BACKUP_TCP_KEEPALIVE_TIME);
> > > +let acceptor = Arc::clone(&acceptor);
> > > +let mut sender = sender.clone();
> > > +
> > > +if accept_counter.load(Ordering::SeqCst) > 
> > > MAX_PENDING_ACCEPTS {
> > > +eprintln!("connection rejected - to many open 
> > > connections");
> > > +continue;
> > > +}
> > > +accept_counter.fetch_add(1, Ordering::SeqCst);
> > 
> > We should think about making a counter guard for this sort of thing,
> > because from this point onward we're not allowed to use `?` anywhere,
> > which is quite annoying.
> 
> I wonder if we can simply use an Arc for that? The Arc already has
> an atomic counter with deref on drop!
> 
> And we can query that counter with Arc::strong_count
> 
> What do you think?


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] applied: [PATCH docs] installation: don't mention ext3 as an option anymore

[Dietmar Maurer]

applied


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] Improve container backup speed dramatically (factor 100-1000)

[Dietmar Maurer]

> Container backup is very slow compared to VM backup. I have a 500 GB 
> container (sftp server) with minimal changing files, but even the incremental 
> bakcups take 2 hours with heavy disk activity. Almost nothing is transfered 
> to the backup server. It seems that it it reads the whole container 
> everytime, without any optimization. Before I did backup with zfs send it 
> there it took only a couple of seconds or minutes for every didfferencal 
> backup. 

Yes, that is how the current variable sized chunking algorithm works.
  
> See discussion here: 
> https://forum.proxmox.com/threads/no-differantial-container-backup-with-big-containers.75676/#post-338868
>  
> 
>  
>  
> PBS is not storage agnostic but uses underlying snapshot feature according to 
> the documentation: For container, the underlying snapshot feature of the file 
> system ARE used, it already uses ZFS feature. 
> https://pve.proxmox.com/wiki/Backup_and_Restore 
>  

Yes, we use the snapshot feature. But the backup code is totally storage 
agnostic.

> For zfs file systems the set of changed file between snapshots can easy be 
> displayed with "zfs diff", so PBS should use this feature to speed up large 
> container backups dramatically.

"zfs diff" does not provide the information needed for our deduplication 
algorithm, so we cannot use that. But if you have ideas how to make that work, 
please shared them here.


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] applied: [pbs-devel] [PATCH proxmox-backup v2 1/1] client: add 'snapshot notes show/update' command

[Dietmar Maurer]

applied


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH v3 0/3] Add proxmox-backup-qemu version to VM status output

[Dietmar Maurer]

> proxmox-backup-qemu is missing a not-pushed version bump commit, otherwise 
> I'd have applied this.

sorry, just pushed the missing commit.


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] applied: [PATCH v3 proxmox-backup-qemu 2/3] return version via rust fn instead of header define

[Dietmar Maurer]

applied (and built a new proxmox-backup-qemu package)


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH v3 manager] ad #3140: allow interface suffix in dns entries

[Dietmar Maurer]

What kind of format is that? RFC2373 does not mention it. Please can
you give me a hint?


> On 11/25/2020 11:36 AM Wolfgang Bumiller  wrote:
> 
>  
> Signed-off-by: Wolfgang Bumiller 
> ---
> changes to v2:
> * use `for of` loop in verify_ip64_address_list
> 
>  www/manager6/Toolkit.js | 17 ++---
>  www/manager6/Utils.js   | 27 +++
>  www/manager6/lxc/DNS.js |  2 +-
>  3 files changed, 30 insertions(+), 16 deletions(-)
> 
> diff --git a/www/manager6/Toolkit.js b/www/manager6/Toolkit.js
> index 55b127c5..0569d649 100644
> --- a/www/manager6/Toolkit.js
> +++ b/www/manager6/Toolkit.js
> @@ -9,21 +9,8 @@ Ext.apply(Ext.form.field.VTypes, {
>   return 
> (/^(now|\d{4}-\d{1,2}-\d{1,2}(T\d{1,2}:\d{1,2}:\d{1,2})?)$/).test(v);
>  },
>  QemuStartDateText: gettext('Format') + ': "now" or "2006-06-17T16:01:21" 
> or "2006-06-17"',
> -IP64AddressList: function(v) {
> - var list = v.split(/[\ \,\;]+/);
> - var i;
> - for (i = 0; i < list.length; i++) {
> - if (list[i] == '') {
> - continue;
> - }
> -
> - if (!Proxmox.Utils.IP64_match.test(list[i])) {
> - return false;
> - }
> - }
> -
> - return true;
> -},
> +IP64AddressList: v => PVE.Utils.verify_ip64_address_list(v, false),
> +IP64AddressWithSuffixList: v => PVE.Utils.verify_ip64_address_list(v, 
> true),
>  IP64AddressListText: gettext('Example') + ': 192.168.1.1,192.168.1.2',
>  IP64AddressListMask: /[A-Fa-f0-9\,\:\.\;\ ]/
>  });
> diff --git a/www/manager6/Utils.js b/www/manager6/Utils.js
> index 6d2f7a04..6e6498a2 100644
> --- a/www/manager6/Utils.js
> +++ b/www/manager6/Utils.js
> @@ -1612,6 +1612,33 @@ Ext.define('PVE.Utils', { utilities: {
>   "Host": 4,
>   "_default_": 5, // includes custom models
>  },
> +
> +verify_ip64_address_list: function(value, with_suffix) {
> + for (let addr of value.split(/[ ,;]+/)) {
> + if (addr === '') {
> + continue;
> + }
> +
> + if (with_suffix) {
> + let parts = addr.split('%');
> + addr = parts[0];
> +
> + if (parts.length > 2) {
> + return false;
> + }
> +
> + if (parts.length > 1 && !addr.startsWith('fe80:')) {
> + return false;
> + }
> + }
> +
> + if (!Proxmox.Utils.IP64_match.test(addr)) {
> + return false;
> + }
> + }
> +
> + return true;
> +},
>  },
>  
>  singleton: true,
> diff --git a/www/manager6/lxc/DNS.js b/www/manager6/lxc/DNS.js
> index a15db5a9..b1e03920 100644
> --- a/www/manager6/lxc/DNS.js
> +++ b/www/manager6/lxc/DNS.js
> @@ -41,7 +41,7 @@ Ext.define('PVE.lxc.DNSInputPanel', {
>   {
>   xtype: 'proxmoxtextfield',
>   fieldLabel: gettext('DNS servers'),
> - vtype: 'IP64AddressList',
> + vtype: 'IP64AddressWithSuffixList',
>   allowBlank: true,
>   emptyText: gettext('use host settings'),
>   name: 'nameserver',
> -- 
> 2.20.1
> 
> 
> 
> ___
> pve-devel mailing list
> pve-devel@lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH v3 manager] ad #3140: allow interface suffix in dns entries

[Dietmar Maurer]

Answering myself, it is defined in RFC4007.

But "man resolv.conf" say address must be RFC2373 ?


> On 11/25/2020 12:08 PM Dietmar Maurer  wrote:
> 
>  
> What kind of format is that? RFC2373 does not mention it. Please can
> you give me a hint?
>


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH v3 manager] ad #3140: allow interface suffix in dns entries

[Dietmar Maurer]

Thanks for the info. But what encoding does that text use? I cannot find that 
in RFC4007 (they only
talk about strings and text).

> > Answering myself, it is defined in RFC4007.
> > 
> > But "man resolv.conf" say address must be RFC2373 ?
> 
> It'll still work. It's a very common notation for link local addresses,
> since with multiple interfaces you have multiple routes with the exact
> same prefix (including length). One `fe80::/64 dev ` entry for
> every ipv6 enabled interface, so things such as `ping fe80::1` don't
> know what to do, so you need to either add `-I IFACE` or use
> `fe80::1%IFACE`.
> 
> `getaddrinfo()` also generally supports it, so any application using
> it properly will work fine with this.
> (eg. `ssh foo@fe80::1%eth0` works)


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH v3 manager] ad #3140: allow interface suffix in dns entries

[Dietmar Maurer]

> What text do you mean exactly? The interface name?
> Arbitrary null-terminated byte string...

Ok

> (Yes I can name an interface "---" or 💩 (poop-emoji)...,
> neither of which our iface schema in JSONSchema.pm would allow...)

great.


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH pve-zsync 0/1] Allow pve-zsync jobs to share dest

[Dietmar Maurer]

> It has been 5 months since my patch has been applied, however the version
> for pve-zsync has not been incremented and this patch is not in the version
> presented by the repo. What needs to be done?

Ok, just bumped the version and created a new package. So this will be part
of the next release (soon) ...


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH pve-manager]: api2:/cluster return intiger format for VMID

[Dietmar Maurer]

>   for (my $i = 100; $i < 1; $i++) {
> - return $i if !defined($idlist->{$i});
> + return int($i) if !defined($idlist->{$i});

IMO, this does not solve the problem, because $i is already and int.


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH pve-manager]: api2:/cluster return intiger format for VMID

[Dietmar Maurer]

> On 12/01/2020 10:41 AM Dietmar Maurer  wrote:
> 
>  
> > for (my $i = 100; $i < 1; $i++) {
> > -   return $i if !defined($idlist->{$i});
> > +   return int($i) if !defined($idlist->{$i});
> 
> IMO, this does not solve the problem, because $i is already and int.

or does $idlist->{$i} converts $i to string??


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] Compiling PBS on arm64

[Dietmar Maurer]

> On 12/06/2020 8:41 PM Kamil Trzciński  wrote:
> 
>  
> I'm slightly progressing, but I stumbled across some `debcargo` problem. It
> appears that
> Proxmox uses their own fork of `debcargo`, which is needed in order to
> build crates
> without the usage of crates.io. 

I guess you can simply adopt/remove the .cargo/config file we ship with our
packages. Cargo will use normal repos then.


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] Adding new data storage

[Dietmar Maurer]

> I would like to add new data storage. This storage would resemble ZFS 
> over iSCSI but will use different API to access storage.

I am curios, what API exactly?


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH v2 ha-manager] ha-simulator: add xauth dependency

[Dietmar Maurer]

This is true for anything. X11 forwarding simply works that way. So I am quite 
unsure if we should add xauth here...

Or is this a common practice (I am unaware of)?

> 
> When installing the ha-simulator on a PVE node to start it via ssh with
> x11 forwarding, the xauth package helps to avoid `Unable to init server:
> Could not connect: Connection refused` errors.


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel