Re: [pve-devel] ZFS tool for asynchronous replica

2015-02-16 Thread Pablo Ruiz
on linux (source) and nas4free/freebsd (destination). Sent from my iPhone > On 16 Feb 2015, at 08:57, Wolfgang Link wrote: > > Do you use this on Linux or on BSD/SunOS? > >> On 02/13/2015 01:44 PM, Pablo Ruiz wrote: >> I am using this same one at production on a f

Re: [pve-devel] [PATCH] ZFSPoolPlugin: Added the ability to use nested ZVOLs

2015-02-13 Thread Pablo Ruiz
ith iscsi export should not in the storage.cfg. > Am 12.02.2015 21:25 schrieb "Pablo Ruiz" : > >> Hi, >> >> IMHO, I see no reason to not default for the most common case (ie. >> auto-importing) if there's a way to override it, and such a way is >> som

Re: [pve-devel] ZFS tool for asynchronous replica

2015-02-13 Thread Pablo Ruiz
I am using this same one at production on a few machines w/o an issue. Also around google you will find a port over bash instead of ksh (which in fact requires changing no more than 10 lines).. Sometimes when a software has no recent releases, does not mean it is not maintained, but that it requir

Re: [pve-devel] [PATCH] ZFSPoolPlugin: Added the ability to use nested ZVOLs

2015-02-12 Thread Pablo Ruiz
Hi, IMHO, I see no reason to not default for the most common case (ie. auto-importing) if there's a way to override it, and such a way is some-what documented.. ;) On Thu, Feb 12, 2015 at 8:35 PM, Adrian Costin wrote: > > AFAIK having a setting to control wether auto-import of pool is desirable

Re: [pve-devel] [PATCH] ZFSPoolPlugin: Added the ability to use nested ZVOLs

2015-02-12 Thread Pablo Ruiz
Hi, AFAIK having a setting to control wether auto-import of pool is desirable would be a plus. As in some situations the import/export of the pool is controlled by any other means, and an accidental pool of the pool may be a destructive action (ie. when the pool maybe from a shared medium like isc

Re: [pve-devel] ZFS storage

2014-08-02 Thread Pablo Ruiz
Well, in this case ZFS is not running on the host, but on an external system (a two node redhat cluster running zfs on linux, with attached shared storage disks, to be exact). On Sat, Aug 2, 2014 at 7:20 PM, Michael Rasmussen wrote: > On Sat, 2 Aug 2014 16:57:48 + > Dietmar Maurer wrote:

Re: [pve-devel] ZFS storage

2014-08-02 Thread Pablo Ruiz
Implementing a simple provider which would simple call an script was my initial attempt at.. and I even sent to the list a first initial set of patches, which then as requested by list members I evolved into the more radical approach you've seen now. :? I'll try to refactor the code back into an s

Re: [pve-devel] ZFS storage

2014-07-31 Thread Pablo Ruiz
n off.. I would be glad to hear from the community or from the ZFSPlugin maintainer/original developer in order to merge this feature(s) somehow. ;) Regards Pablo On Wed, Jul 30, 2014 at 3:37 AM, Michael Rasmussen wrote: > On Wed, 30 Jul 2014 02:51:30 +0200 > Pablo Ruiz wrote: >

Re: [pve-devel] ZFS storage

2014-07-29 Thread Pablo Ruiz
There are also a few patches I sent some time ago with an overhaul of ZFS LUN handling code, which are awaiting some comments from proxmox community.. I have created my own testing deb packages and I could share the repo with anyone interested on testing. Sent from my iPhone On 29 Jul 2014, a

Re: [pve-devel] zfs plugin: creating 2 disks

2014-05-03 Thread Pablo Ruiz
Oh, In such a case, his issue would (hopefully) be fixed once this patch reaches pve-testing.. ;) On Sun, May 4, 2014 at 2:09 AM, Michael Rasmussen wrote: > On Sun, 4 May 2014 01:56:36 +0200 > Pablo Ruiz wrote: > > > Yeah. > > > > If Adrian is already using

Re: [pve-devel] zfs plugin: creating 2 disks

2014-05-03 Thread Pablo Ruiz
Yeah. If Adrian is already using this patch, and the problem persists, I could take a look at it tomorrow or maybe by monday. So, just le me know. Regards Pablo On Sun, May 4, 2014 at 12:27 AM, Michael Rasmussen wrote: > On Sat, 3 May 2014 21:46:21 +0200 > Pablo Ruiz wrote: > >

Re: [pve-devel] zfs plugin: creating 2 disks

2014-05-03 Thread Pablo Ruiz
I sent a patch a couple of months ago allowing support for nested pools. I think it was merged into testing. On Sat, May 3, 2014 at 9:23 PM, Michael Rasmussen wrote: > On Sat, 3 May 2014 21:58:43 +0300 > Adrian Costin wrote: > > > Small bug when creating a second disk for the same VM on the sa

Re: [pve-devel] using a vlan on host and guest

2014-04-24 Thread Pablo Ruiz
Setup vmbr0.102 as host interface instead of bond0.102. Sent from my iPhone > On 24 Apr 2014, at 21:59, Stefan Priebe wrote: > > Hi, > > I've the following config on my host: > auto bond0 > iface bond0 inet manual >slaves eth0 eth1 eth2 >bond_mode 802.3ad > > auto vmbr0 > ifac

[pve-devel] [PATCH] Added missing 'import-lu' command to refactored ZFSPlugin.

2014-04-13 Thread Pablo Ruiz García
Signed-off-by: Pablo Ruiz García --- PVE/Storage/ZFSPlugin.pm |3 +++ 1 files changed, 3 insertions(+), 0 deletions(-) diff --git a/PVE/Storage/ZFSPlugin.pm b/PVE/Storage/ZFSPlugin.pm index ac8eb0a..d310a8c 100644 --- a/PVE/Storage/ZFSPlugin.pm +++ b/PVE/Storage/ZFSPlugin.pm @@ -41,6 +41,9

[pve-devel] [PATCH 3/5] Do not expose every value from storage.cfg as PMXCFG_*, and instead expose those few 'usefull' values like portal, target, pool, etc. Everything else should be handled by helpe

2014-04-12 Thread Pablo Ruiz García
Signed-off-by: Pablo Ruiz García --- PVE/Storage/ZFSPlugin.pm | 16 +++- 1 files changed, 3 insertions(+), 13 deletions(-) diff --git a/PVE/Storage/ZFSPlugin.pm b/PVE/Storage/ZFSPlugin.pm index 80fc0ea..ac8eb0a 100644 --- a/PVE/Storage/ZFSPlugin.pm +++ b/PVE/Storage/ZFSPlugin.pm

[pve-devel] [PATCH 4/5] Added existing LunCmd code rewritten as independent scripts

2014-04-12 Thread Pablo Ruiz García
Signed-off-by: Pablo Ruiz García --- zfs-helpers/Common.pm | 285 zfs-helpers/comstar | 114 ++ zfs-helpers/iet | 488 zfs-helpers/istgt | 593 + 4 files

[pve-devel] [PATCH 1/5] This patch refactors ZFS's LUN management code, removing the existing LunCmds implementations, in favor of a single lun management infraestructure which just invokes a 'managem

2014-04-12 Thread Pablo Ruiz García
return the given ZVOL/LUN's number. This actual commands are modelled after the ones invoked by Comstar's LunCmd perl module, which even if it maybe a bit too coupled to Comstar's/Solaris' way of doing things. It seems to be flexible enough as to be usefull for other implementati

[pve-devel] [PATCH 5/5] Implemented --remote option to zfs helper scripts, allowing local vs remote execution.

2014-04-12 Thread Pablo Ruiz García
Signed-off-by: Pablo Ruiz García --- zfs-helpers/comstar | 12 ++-- zfs-helpers/iet | 20 zfs-helpers/istgt | 36 +--- 3 files changed, 47 insertions(+), 21 deletions(-) diff --git a/zfs-helpers/comstar b/zfs-helpers

[pve-devel] New Feature: ZFS 'generic' support for LUN management. (Take 3 ; ])

2014-04-12 Thread Pablo Ruiz García
Hello, This is a new version of my ZFS's LUN management code refactor, this time completely removing 'embedded' LunCmd drivers, and adding them as independent perl scripts to pve-storage repo. ZFSPlugin functionality has been tested and it works fine with our one zfs-helper, also, I've tried to

[pve-devel] [PATCH 2/5] Remove now deprecated LunCmd drivers.

2014-04-12 Thread Pablo Ruiz García
Signed-off-by: Pablo Ruiz García --- PVE/Storage/LunCmd/Comstar.pm | 102 --- PVE/Storage/LunCmd/Iet.pm | 478 - PVE/Storage/LunCmd/Istgt.pm | 580 - PVE/Storage/LunCmd/Makefile |5 - PVE/Storage/Makefile

Re: [pve-devel] New Feature: ZFS 'generic' support for LUN management.

2014-03-20 Thread Pablo Ruiz
Mar 2014, at 17:51, Michael Rasmussen wrote: > > On Thu, 20 Mar 2014 16:32:44 +0100 > Pablo Ruiz wrote: > >> >> Any preferences on this subject? >> > If your scripts should be able to execute on non-proxmox servers then > the answer is obvious. >

Re: [pve-devel] New Feature: ZFS 'generic' support for LUN management.

2014-03-20 Thread Pablo Ruiz
Another question regarding this lun scripts.. I will be adding them, as requested, into a new dir /scripts at pve-storage repo. However, the actual lun helpers (ie LunCmd/*) make use of (among others) PVE::Tools package's functions, and I am facing two options: 1) Import such packages into each o

Re: [pve-devel] [PATCH 1/2] This patch refactors ZFS's LUN management code, removing the existing LunCmds implementations, in favor of a single lun management infraestructure which just invokes a 'man

2014-03-20 Thread Pablo Ruiz
ettings (which looks a bit of a kludge) and only expose a selected subset as PMXVARs. Regards Pablo On Wed, Mar 19, 2014 at 10:42 AM, Pablo Ruiz wrote: > Hi, > > While using command line arguments may seem the obvious approach, in the > end this is a more fragile mechanism to pass d

Re: [pve-devel] [PATCH 1/2] This patch refactors ZFS's LUN management code, removing the existing LunCmds implementations, in favor of a single lun management infraestructure which just invokes a 'man

2014-03-20 Thread Pablo Ruiz
Humm, thats a good point and it's perfectly workable.. ;) I've reviewed the current code and I will be adding 'pool' and 'target' as PMXVAR, which seems like the most obvious values needed by lunhelper. And thus removing all previouslly exposed PMXCFG variables. On Thu, Mar 20, 2014 at 2:09 PM,

Re: [pve-devel] [PATCH 1/2] This patch refactors ZFS's LUN management code, removing the existing LunCmds implementations, in favor of a single lun management infraestructure which just invokes a 'man

2014-03-20 Thread Pablo Ruiz
It is needed by the ZFS code in order to invoke zfs volume creation with an specific blocksize. ZFS plugin handles interaction with zfs, while lun-helper handles exposing/sharing such ZFS volume as an iSCSI LUN. (Which makes sense, as ZFS volume creation, etc. it's allways common acros all ZFS impl

Re: [pve-devel] [PATCH 1/2] This patch refactors ZFS's LUN management code, removing the existing LunCmds implementations, in favor of a single lun management infraestructure which just invokes a 'man

2014-03-20 Thread Pablo Ruiz
Well I was thinking on passing some discriminator value which can be used by the script to difference in case of multiple callers using different zpools. (on my own test cluster I have setup a couple of storages/zpools). However, specifying the pool would be a valid discriminator too, alas it wi

Re: [pve-devel] [PATCH 1/2] This patch refactors ZFS's LUN management code, removing the existing LunCmds implementations, in favor of a single lun management infraestructure which just invokes a 'man

2014-03-20 Thread Pablo Ruiz
Humm, that makes a lot of sense. ;) I will pass the storage's name/id as anotjer PMXVAR, and remove all PMXCFG variables, this way we will produce a much shorter command string. On Mar 20, 2014 7:30 AM, "Dietmar Maurer" wrote: > > > Just curious - how long are the command lines this patch genera

Re: [pve-devel] [PATCH 1/2] This patch refactors ZFS's LUN management code, removing the existing LunCmds implementations, in favor of a single lun management infraestructure which just invokes a 'man

2014-03-19 Thread Pablo Ruiz
Hi, While using command line arguments may seem the obvious approach, in the end this is a more fragile mechanism to pass data to helper script, as backward/forward compatbility would be harder to achieve once new 'data' is to be passed. Command line arguments need to respect ordering, or define

[pve-devel] Refactor: Remove ZFS Plugin's LunCmd drivers in favor of a single helper-based 'generic' support for LUN management.

2014-03-18 Thread Pablo Ruiz García
Hello, This is a followup at my privous attempt at providing generic support of LUN management to ZFS Plugin by using an independent helper script/binary. This patch series refactors current ZFS Plugin, removing support for perl-based LunCmd drivers, and instead provides a generic-way of invokin

[pve-devel] [PATCH 2/2] Remove now deprecated LunCmd drivers.

2014-03-18 Thread Pablo Ruiz García
Signed-off-by: Pablo Ruiz García --- PVE/Storage/LunCmd/Comstar.pm | 102 --- PVE/Storage/LunCmd/Iet.pm | 478 - PVE/Storage/LunCmd/Istgt.pm | 580 - PVE/Storage/LunCmd/Makefile |5 - PVE/Storage/Makefile

[pve-devel] [PATCH 1/2] This patch refactors ZFS's LUN management code, removing the existing LunCmds implementations, in favor of a single lun management infraestructure which just invokes a 'managem

2014-03-18 Thread Pablo Ruiz García
return the given ZVOL/LUN's number. This actual commands are modelled after the ones invoked by Comstar's LunCmd perl module, which even if it maybe a bit too coupled to Comstar's/Solaris' way of doing things. It seems to be flexible enough as to be usefull for other implementati

Re: [pve-devel] New Feature: ZFS 'generic' support for LUN management.

2014-03-18 Thread Pablo Ruiz
ng to tests the new code. Volunters? 2) Where should I include such scripts? Maybe some sort of contrib repository? Or placing them at my own github account would be enough? Regards Pablo On Mon, Mar 3, 2014 at 2:09 PM, Pablo Ruiz wrote: > Daniel, That's exctly the idea. ;) >

Re: [pve-devel] ZFS Storage Patches

2014-03-12 Thread Pablo Ruiz
Hi Chris, I am working on a refactor of ZFS Plugin which will decouple specific LUN implementations from the driver, by providing a single interface for LUN implementation by means of invoking an external script/program/binary. As such, I will try to review your patches and incorporate "what is no

Re: [pve-devel] New Feature: ZFS 'generic' support for LUN management.

2014-03-03 Thread Pablo Ruiz
Daniel, That's exctly the idea. ;) I'll be a bit busy this week attending some conferences, etc. But I will work on a revised patch the next week so it can be reviewed by any interested peers on this same list. Regards Pablo On Sun, Mar 2, 2014 at 9:08 PM, Daniel Hunsaker wrote: > > I might ha

Re: [pve-devel] New Feature: ZFS 'generic' support for LUN management.

2014-02-27 Thread Pablo Ruiz
Yeah! If I understand it correctly, what you mean is removing all LunCmd implementations and just invoke a remote script (defined at storage.cfg) from the ZFS plugin directly. That would be great and would still allow each implementor of a ZFS backend to freely implement their own LUN management l

Re: [pve-devel] New Feature: ZFS 'generic' support for LUN management.

2014-02-27 Thread Pablo Ruiz
Hi Dietmar, What I am using is a customized redhat cluster setup with ZFS on linux, plus some resource agent & helper scripts (available here: https://github.com/pruiz/zfs-cluster) which allow management of NFS/iSCSI exports, along with raking care of mounting/unmounting (export/import in ZFS term

Re: [pve-devel] New Feature: ZFS 'generic' support for LUN management.

2014-02-26 Thread Pablo Ruiz
Hi Dietmar, I am going to deep into dynamic loading of lun plugins and wil be seinding a new version of the code ASAP. However, would you consider merging the second patch ([PATCH 2/2] Improve parsing of zfs volumes (ZVOLs) in order to avoid) which indeed is not related to the lun plugins? Regard

Re: [pve-devel] New Feature: ZFS 'generic' support for LUN management.

2014-02-26 Thread Pablo Ruiz
This is an interesting idea, however in such a case, my main issue would be validating new store.cfg parameters each new plugin would need/add. As json schema validation needs to be done before store.cfg' contents are loaded and thus we would have not loaded the plugins either. Any ideas? On Wed

Re: [pve-devel] New Feature: ZFS 'generic' support for LUN management.

2014-02-24 Thread Pablo Ruiz
This is just an additional plugin under LunCmd which allows developing lun plugins without depending on proxmox's release schedle. And which canbe used in cases where the plugin to be developed is going to be too specific to be usefull for others. The way I see it, ending up with a lot of specific

[pve-devel] New Feature: ZFS 'generic' support for LUN management.

2014-02-22 Thread Pablo Ruiz
Our iSCSI/ZFS infraestructure is somewhat specific to our environment, and I felt like adding a driver just for us was of no use for proxmox folks, nor for the community at large, so I've opted for an alternative way by introducing a 'generic' LUN management driver which just invokes an independent

[pve-devel] [PATCH 1/2] Added a new 'generic' LUN management driver to ZFS Plugin. This patch introduces a 'generic' LUN management driver which just invokes an independent 'management helper' (which

2014-02-22 Thread Pablo Ruiz García
From: Pablo Ruiz Garcia This way, the specifics of the lun management can be developed independently of proxmox, and we can avoid interfering with proxmox's release schedule. The 'protocol' which proxmox uses to communicate with the helper is simply based on exposing a set

[pve-devel] [PATCH 2/2] Improve parsing of zfs volumes (ZVOLs) in order to avoid filtering of zvols nested under another ZFS dataset.

2014-02-22 Thread Pablo Ruiz García
From: Pablo Ruiz Garcia The actual code would only accept zvols like: POOL/vm-123-disk-1. However, using POOL/DataSet/vm-123-disk-1 allows setting specific proparties at POOL/DataSet level (like compression, etc.) which would be inherited by any zvol created under such DataSet. This allows more

[pve-devel] New Feature: ZFS 'generic' support for LUN management

2014-02-22 Thread Pablo Ruiz García
Hello, Our iSCSI/ZFS infraestructure is somewhat specific to our environment, and I felt like adding a driver just for us was of no use for proxmox folks, nor for the community at large, so I've opted for an alternative way by introducing a 'generic' LUN management driver which just invokes an ind

Re: [pve-devel] PATCH: Add support for bridges with more than one physical link.

2014-02-12 Thread Pablo Ruiz
/optimizations as of today.. ;)) On Wed, Feb 12, 2014 at 9:55 AM, Pablo Ruiz wrote: > That's what MSTP/PVSTP+ is supposed to avoid. (And infact, it does so in > our environment).. however, it requires switches with such capability. > > > On Wed, Feb 12, 2014 at 9:53 AM, An

Re: [pve-devel] PATCH: Add support for bridges with more than one physical link.

2014-02-12 Thread Pablo Ruiz
> It is the same as looping a cable between two ports on a switch that does > not have edge-safeguard functionality. > > Just my 2c. > > > > > On Wed, Feb 12, 2014 at 6:28 PM, Pablo Ruiz wrote: > >> Hi, >> >> In our proxmox cluster, each node has two b

[pve-devel] [PATCH 1/3] Add support for bridges with more than one physical link (ie. two bonds each connected to a different switch using MSTP/PVSTP to load balance vlans across links)

2014-02-11 Thread Pablo Ruiz Garcia
Signed-off-by: Pablo Ruiz Garcia --- data/PVE/Network.pm | 72 --- 1 files changed, 34 insertions(+), 38 deletions(-) diff --git a/data/PVE/Network.pm b/data/PVE/Network.pm index 9ad34f1..d5550a3 100644 --- a/data/PVE/Network.pm +++ b/data/PVE

[pve-devel] [PATCH 2/3] Handle cases where the vlan bridge has already been setup previouslly.

2014-02-11 Thread Pablo Ruiz Garcia
Signed-off-by: Pablo Ruiz Garcia --- data/PVE/Network.pm |3 +++ 1 files changed, 3 insertions(+), 0 deletions(-) diff --git a/data/PVE/Network.pm b/data/PVE/Network.pm index d5550a3..96cf20b 100644 --- a/data/PVE/Network.pm +++ b/data/PVE/Network.pm @@ -142,6 +142,9 @@ sub

[pve-devel] [PATCH 3/3] Delayed vlan interface creation until all checks are done.

2014-02-11 Thread Pablo Ruiz Garcia
Signed-off-by: Pablo Ruiz Garcia --- data/PVE/Network.pm | 20 +++- 1 files changed, 11 insertions(+), 9 deletions(-) diff --git a/data/PVE/Network.pm b/data/PVE/Network.pm index 96cf20b..2fbb715 100644 --- a/data/PVE/Network.pm +++ b/data/PVE/Network.pm @@ -166,6 +166,14

Re: [pve-devel] PATCH: Add support for bridges with more than one physical link.

2014-02-11 Thread Pablo Ruiz
Hi Dietmar, Here goes an v2 with you requested changes. Just tested it on briefly by rebooting nodes and migrating a couple of VMs from/to the nodes with this new changes applied. diff --git a/data/PVE/Network.pm b/data/PVE/Network.pm index 9ad34f1..2fbb715 100644 --- a/data/PVE/Network.pm +++ b/

Re: [pve-devel] PATCH: Add support for bridges with more than one physical link.

2014-02-11 Thread Pablo Ruiz
Sure ;) diff --git a/data/PVE/Network.pm b/data/PVE/Network.pm index 9ad34f1..96cf20b 100644 --- a/data/PVE/Network.pm +++ b/data/PVE/Network.pm @@ -122,36 +122,10 @@ sub copy_bridge_config { } } -sub activate_bridge_vlan { -my ($bridge, $tag_param) = @_; - -die "bridge '$bridge' is

Re: [pve-devel] PATCH: Add support for bridges with more than one physical link.

2014-02-11 Thread Pablo Ruiz
Btw, I've included a link to the first commit only. Full branch with feature can be found at: https://github.com/pruiz/pve-common/tree/bridge-multislave On Wed, Feb 12, 2014 at 6:28 AM, Pablo Ruiz wrote: > Hi, > > In our proxmox cluster, each node has two bond interfaces,

[pve-devel] PATCH: Add support for bridges with more than one physical link.

2014-02-11 Thread Pablo Ruiz
Hi, In our proxmox cluster, each node has two bond interfaces, and each bond interface connects to and independent switch. This allows us to enable MSTP/PVSTP+ and thus load share traffic on different vlans across switches. +==+