[pve-devel] Integration of FreeNAS iSCSI target initiator in Proxmox Enterprise repo
Hello, I was wondering if there are any plans to integrate the Proxmox FreeNAS iSCSI initiator patches developped by the TheGrandWazoo and available in github repository here : https://github.com/TheGrandWazoo/freenas-proxmox into the main FreeNAS Enterprise repo. I am not a huge fan of "custom patches" and will clearly avoid touching source scripts from Proxmox… That being said the patches are looking nice and I am not sure what would be the reason not to upstream this into Proxmox business repo. Can the Proxmox Team discuss this a bit since FreeNAS is widely used, stable and a very nice solution to implement as an iSCSI target node. Benefiting from the Proxmox team's integration efforts would mean that a FreeNAS / TrueNAS could be used a target solution with long term support. Relates to this thread : https://forum.proxmox.com/threads/guide-setup-zfs-over-iscsi-with-pve-5x-and-freenas-11.54611/ Thanks for your answer. Sincerely yours. --- ToDoo - osnet.eu - DynFi.com 6 rue Montmartre - 75001 Paris b...@todoo.biz web: https://www.dynfi.com PGP_ID: 0x1BA3C2FD ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] Integration of FreeNAS iSCSI target initiator in Proxmox Enterprise repo
> Le 3 juin 2020 à 12:03, Michael Rasmussen a écrit : > > Signé partie PGP > On Wed, 3 Jun 2020 11:34:35 +0200 > Andreas Steinel wrote: > >> >> If I remember correctly, the problem with the ZFS-over-iSCSI stuff is >> that the backend provider in FreeNAS/FreeBSD has changed numerous >> times and at least one version did not support online >> reconfiguration. That may be solved right now, but I haven't look at >> the plugin for a long time. It just works for Debian as Mario >> (@fireon) described in [2]. >> > > A few years ago the FreeNAS API was fluctuating between every new minor > release (eg, 9.3 -> 9.4) and was as such called 'not production ready' > by IXSystems and therefore labelled beta, experimental etc (I know that > no product developed by Alfabet (google) never has transitioned from > beta to production state ;-) This means that maintaining such a plugin > is very time consuming and since this is bade in peoples free time > there likely will be the same resources required to do that. A lot of time has passed since version 9 of FreeBSD / FreeNAS. Six years to be precise. iSCSI on FreeBSD is used by very large corporation (Gandi not to name them https://www.gandi.net/en <https://www.gandi.net/en> is hosting all it's VM using FreeBSD + Linux on the initiator side) with an excellent level of stability. Solution derivated from Illumos are either overpriced, closed source and scarcely used or abandoned (if not all at the same time). I think that It might be time to re-considering FreeBSD as a potential stable solution to host iSCSI targets. Considering the very important efforts FreeBSD has made to implement ZFS and stick to the OpenZFS standards. > On the > other hands those plugin already present in Proxmox uses production > stamped solutions which rarely/ever makes breaking changes is more > likely to be maintainable in peoples free time. As a side note: The > current iscsi initiator (ctld) in FreeBSD still has an upper limit of > 1024 LUNs per target so therefore qualifies as an plugin alternative to > proxmox > > #define MAX_LUNS 1024 [0] > > [0] https://github.com/freebsd/freebsd/blob/master/usr.sbin/ctld/ctld.h This is not true. The limit that you are mentioning here has been overridden by a tunable parameter. https://www.freebsd.org/cgi/man.cgi?query=ctl&sektion=4 <https://www.freebsd.org/cgi/man.cgi?query=ctl&sektion=4> > TUNABLE VARIABLES > <https://www.freebsd.org/cgi/man.cgi?query=ctl&sektion=4#end> > The following variables are available as loader(8) > <https://www.freebsd.org/cgi/man.cgi?query=loader&sektion=8&apropos=0&manpath=FreeBSD+12.1-RELEASE+and+Ports> > tunables: > > kern.cam.ctl.max_luns >Specifies the maximum number of LUNs we support, must be a power >of 2. The default value is 1024. It turns out that this patch has been developed by one of my fellow developer couple of years ago to bypass the limitations of 1024 LUNs that you mentioned. So It should qualify to be selected as an enterprise grade solution to host iSCSI target. The fact that some developers are patching Proxmox code to allow *BSD / FreeNAS to perform is a sign that there might have a need for such tool. So beside this max luns problem, what else seems to be causing problem with the BSD target ? Sincerely yours. > > -- > Hilsen/Regards > Michael Rasmussen > > Get my public GnuPG keys: > michael rasmussen cc > https://pgp.key-server.io/pks/lookup?search=0xD3C9A00E > mir datanom net > https://pgp.key-server.io/pks/lookup?search=0xE501F51C > mir miras org > https://pgp.key-server.io/pks/lookup?search=0xE3E80917 > ------ > /usr/games/fortune -es says: > Youth doesn't excuse everything. > -- Dr. Janice Lester (in Kirk's body), "Turnabout > Intruder", stardate 5928.5. > > --- ToDoo - osnet.eu - DynFi.com 6 rue Montmartre - 75001 Paris b...@todoo.biz web: https://www.dynfi.com PGP_ID: 0x1BA3C2FD ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] Integration of FreeNAS iSCSI target initiator in Proxmox Enterprise repo
> Le 3 juin 2020 à 18:06, Michael Rasmussen a écrit : > > Signé partie PGP > On Wed, 3 Jun 2020 16:54:17 +0200 > "b...@todoo.biz" wrote: > >> >> A lot of time has passed since version 9 of FreeBSD / FreeNAS. Six >> years to be precise. >> > The version numbers was for explanation purposes only. > BTW. When FreeNAS soon will vanish an be consumed be TrueNAS what > happens then? Will it be a closed and commercial product only? > >> iSCSI on FreeBSD is used by very large corporation (Gandi not to >> name them https://www.gandi.net/en <https://www.gandi.net/en> is >> hosting all it's VM using FreeBSD + Linux on the initiator side) with >> an excellent level of stability. >> > I have nothing against FreeBSD on the contrary I have very much fun > using FreeBSD myself (For one usecase I am addicted to pfSense). Only > shortcoming in Proxmox realm in the past was the very non enterprise > iSCSI implementation (istgt) which was replaced by cltd, a very fine > iSCSI implementation, but limited to 1024 LUNs per target in the past > but according to your links has since increased. (By the time I was > looking into to this I had numerous discussions with the developer > about this issue and I promised to inform me personally when he removed > the limitation - I haven't heard from him) > >> Solution derivated from Illumos are either overpriced, closed source >> and scarcely used or abandoned (if not all at the same time). >> > This is where you I misinformed ;-) Illumos is truly FOSS an lives > well. Commercially with SmartOS, but not closed source since all > patches are sent upstream. The non-commercial branch of Illumos also > lives well as OmniosCE and are well maintained by Andy and Tobias to > mention a few (I have a small part as well ;-). I think you are > thinking of Solaris as in Oracle Solaris which is definitely on the > usual way to desintegration as all other stuff Larry gets his hands on. > > You should also remember that all ZFS development, apart from the black > caves near Redwood City, is done in OpenZFS and all current > implementations apart from Oracle uses the same source tree. > >> I think that It might be time to re-considering FreeBSD as a >> potential stable solution to host iSCSI targets. Considering the very >> important efforts FreeBSD has made to implement ZFS and stick to the >> OpenZFS standards. >> > This has never been my attitude. FreeBSD is rock solid an has been for > decades. > >> >> This is not true. >> >> The limit that you are mentioning here has been overridden by a >> tunable parameter. >> https://www.freebsd.org/cgi/man.cgi?query=ctl&sektion=4 >> <https://www.freebsd.org/cgi/man.cgi?query=ctl&sektion=4> >> >> It turns out that this patch has been developed by one of my fellow >> developer couple of years ago to bypass the limitations of 1024 LUNs >> that you mentioned. >> > Do you have any personal experiences with raising the number of LUNs? > Eg. increase it by a factor of 10? What Manu, (FreeBSD kernel developer part of my team) has expertise with is having passed the limit of 1024. He is the one which has patched this in order to put this in a tunable variable. What is your upper limit for this ? I am putting him in copy. > >> So It should qualify to be selected as an enterprise grade solution >> to host iSCSI target. >> >> The fact that some developers are patching Proxmox code to allow *BSD >> / FreeNAS to perform is a sign that there might have a need for such >> tool. >> >> So beside this max luns problem, what else seems to be causing >> problem with the BSD target ? >> > The LUN limitation was my only objection when I stopped to develop the > storage plugin. It should be rather easy to write such a plugin using > the istgt code an replace the iSCSI part with cltd. For a motivated > developer this should not take more than a weekends work :-) I'll try to see what can be done… But I am more involved in firewalling hardcore dev than Proxmox at this stage ! I have a large project with Proxmox which might be popping up, if this is the case, I'll try so sponsor this ;-) > > -- > Hilsen/Regards > Michael Rasmussen > > Get my public GnuPG keys: > michael rasmussen cc > https://pgp.key-server.io/pks/lookup?search=0xD3C9A00E > mir datanom net > https://pgp.key-server.io/pks/lookup?search=0xE501F51C > mir miras org > https://pgp.key-server.io/pks/lookup?search=0xE3E80917 > -- > /usr/games/fortune -es says: > Quid me anxius sum? > > [ What? Me, worry? ] > > --- ToDoo - osnet.eu - DynFi.com 6 rue Montmartre - 75001 Paris b...@todoo.biz web: https://www.dynfi.com PGP_ID: 0x1BA3C2FD ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] Integration of FreeNAS iSCSI target initiator in Proxmox Enterprise repo
Le 3 juin 2020 à 18:25, Michael Rasmussen a écrit : > > Signé partie PGP > On Wed, 3 Jun 2020 18:16:11 +0200 > "b...@todoo.biz" wrote: > >> >> What is your upper limit for this ? >> > I prefer a world without limitations;-) > BTW. Some years ago I asked phk (Poul Henning Kamp) about this and > according to him there was no kernel reason behind the limitation and > to the best of his knowledge the number of LUNs, within reasons, should > be nearly endless. Of course the number of CPU cores and the amount of > memory should be equipped accordingly. > >> I have a large project with Proxmox which might be popping up, if >> this is the case, I'll try so sponsor this ;-) > > I am convinced that the Proxmox team will welcome such a plugin since > supported more storage solution can only benefit Proxmox as a product. I didn't understand in the first place what was the real limitation of the FreeNAS plugin provided by TheGrandWazoo (beside not being in line with Proxmox coding practices). Because maybe in this case the Proxmox Core team could take a little of their precious time and modify the submitted patches to "style" them accordingly. Then we could do the testing part of the job… > > -- > Hilsen/Regards > Michael Rasmussen > > Get my public GnuPG keys: > michael rasmussen cc > https://pgp.key-server.io/pks/lookup?search=0xD3C9A00E > mir datanom net > https://pgp.key-server.io/pks/lookup?search=0xE501F51C > mir miras org > https://pgp.key-server.io/pks/lookup?search=0xE3E80917 > -- > /usr/games/fortune -es says: > Delta: We never make the same mistake three times. -- David Letterman > > --- ToDoo - osnet.eu - DynFi.com 6 rue Montmartre - 75001 Paris b...@todoo.biz web: https://www.dynfi.com PGP_ID: 0x1BA3C2FD ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
