Thanks for the suggestion, and having a CI bot running on oslo.privsep would be a good thing once the basic code works on Windows - I would have expected it to be already running on all oslo projects (that the hyper-v hypervisor does/might depend on) tbh.
But that's a clumsy way to actually develop something. I _know_ privsep won't work currently on windows (imports pwd/grp for starters), and having to add print statements + submit + push-to-gerrit + wait for a CI bot + dig through CI bot logs + repeat is a pretty terrible workflow ;-) (I already have very little empathy for anyone choosing to run Windows just to provide yet-another-x86-hypervisor when far easier alternatives are available, and I won't donate a disproportionate amount of my time to a well-funded company who has historically actively undermined the intellectual commons when _I_ have far more rewarding alternatives available. I'm afraid this has to be easy for me or I'm just not going to be able to sustain the effort required :-/ ) - Gus On Thu, 26 Nov 2015 at 11:56 Alessandro Pilotti < apilo...@cloudbasesolutions.com> wrote: > Hi Angus, > > First thanks for your concern on code portability! It still happens that > we have to ask to revert patches on Oslo projects due to some Linux > specific code that we discover only when the actual Oslo modules are used > by Nova, Neutron, Cinder or other projects. Typically a running Windows CI > suddenly turns red on every patch and that’s how we get to find out. We’re > working on adding many more projects under Windows CI tests, so at some > point all the relevant Oslo ones will be covered and we’ll be able to > prevent those situations before the code gets merged. > > What about preparing some basic integration tests for oslo.privsep that we > could add to our Windows CI infrastructure? This would give you peace of > mind during development on Linux without needing to test manually all your > patches on Windows, knowing that the Windows CI will give an error if the > patch contains non portable code (or code that doesn’t behave as expected). > > Alessandro > > > > From: Angus Lees <g...@inodes.org> > Reply-To: "OpenStack Development Mailing List (not for usage questions)" < > openstack-dev@lists.openstack.org> > Date: Thursday 26 November 2015 at 01:56 > To: Claudiu Belu <cb...@cloudbasesolutions.com>, "OpenStack Development > Mailing List (not for usage questions)" <openstack-dev@lists.openstack.org > > > Subject: Re: [openstack-dev] [hyper-v] oslo.privsep vs Windows > > So I spent a day yesterday trying to get to the point where I could just > run a no-change "tox" successfully on windows. Unfortunately I gave up > when I realised I still had several days of downloading+building things > ahead of me and clearly I was doing it the hard way :( > > Could you point me to the "dev environment how-to" doc for hyper-v work? > I'm thinking of something like > http://docs.openstack.org/developer/nova/development.environment.html#setup > with > simple cut+paste instructions for the totally-windows-clueless like me ;) > Or perhaps a pre-prepared disk image, if the Microsoft license allows such > a thing. In particular, the details on > https://wiki.openstack.org/wiki/Hyper-V look out of date (Grizzly, and > several docs.openstack.org links are broken), and the links to things > like https://cloudbase.it/hyper-v-nova-compute-installer-unattended-setup/ > look > like deployment rather than development environments (?). > > In particular, I think(?) I should be careful *not* to install too much of > a cygwin-style environment, since I think(?) that might no longer be > representative of the environment in which hyper-v is expected to operate. > If I'm wrong, and cygwin/msys is ok, then it looks like there are several > free-software-on-windows "distributions" that would make life a whole lot > simpler (by frankly being less like Windows). Some guidance from people > who understand the issues here would be appreciated. > > - Gus > > On Tue, 24 Nov 2015 at 22:01 Claudiu Belu <cb...@cloudbasesolutions.com> > wrote: > >> Hello, >> >> Thanks Dims for raising the concern and Angus for reaching out. :) >> >> Most of the time, python development on Windows is not too far off from >> Linux. But the two systems are quite different, which imply different >> modules (fcntl, pwd, grp modules do not exist in Windows) or different >> implementations of some modules (multiprocessing uses Popen instead of >> os.fork, os module is quite different) or some socket options and signals >> are different in Windows. >> >> 1.a. As I've said earlier, some modules do not exist in Windows. All, or >> at least most standard modules document the fact that they are strictly for >> Linux. [1][2][3] >> b. At the very least, running the unit tests in a Windows environment can >> at least detect simple problems (e.g. imports). Secondly, there is a >> Hyper-V / Windows CI running on some of the OpenStack projects (nova, >> neutron, networking_hyperv, cinder) that can be checked before merging. >> >> 2. This is a bit more complicated question. Well, for functions, you >> could have separate modules for Linux specific functions and Windows >> specific functions. This has been done before: [4] As for object-oriented >> implementations, I'd suggest having the system-specific calls be done in >> private methods, which will be overriden by Windows / Linux subclasses with >> their specific implementations. We've done something like this before, but >> solutions were pretty much straight-forward; it might not be as simple for >> oslo_privsep, since it is very Linux-specific. >> >> 3. Typically, the OpenStack services on Hyper-V / Windows are run with >> users that have enough privileges to do their job. For example, the >> nova-compute service is run with a user that has Hyper-V Admin privileges >> and is not necessarily in the "Administrator" user group. We haven't used >> rootwrap in our usecases, it is disabled by default, plus, oslo.rootwrap >> imports pwd, which doesn't exist in Windows. >> >> [1] https://docs.python.org/2/library/fcntl.html >> [2] https://docs.python.org/2/library/pwd.html >> [3] https://docs.python.org/2/library/grp.html >> [4] >> https://github.com/openstack/neutron/blob/master/neutron/agent/common/utils.py >> [5] >> https://github.com/openstack/oslo.rootwrap/blob/master/oslo_rootwrap/wrapper.py#L19 >> >> If you have any further questions, feel free to ask. :) >> >> Best regards, >> Claudiu Belu >> >> >> ------------------------------ >> *From:* Angus Lees [g...@inodes.org] >> *Sent:* Tuesday, November 24, 2015 6:18 AM >> *To:* OpenStack Development Mailing List (not for usage questions); >> Claudiu Belu >> *Subject:* [hyper-v] oslo.privsep vs Windows >> >> Dims has just raised[1] the excellent concern that oslo.privsep will need >> to at least survive on Windows, because hyper-v. I have no real experience >> coding on windows (I wrote a windows C program once, but I only ever ran it >> under wine ;) and certainly none within an OpenStack/python context: >> >> 1) How can I test whatever I'm working on to see if I have mistakenly >> introduced something Linux-specific? Surely this is a challenge common >> across every project in the nova/oslo/hyper-v stack. >> >> 2) What predicate should I use to guard the inevitable Linux-specific or >> Windows-specific code branches? >> >> and I guess: >> 3) What does a typical OpenStack/hyper-v install even look like? Do we >> run rootwrap with some sudo-like-thing, or just run everything as the >> superuser? >> What _should_ oslo.privsep do for this environment? >> >> [1] https://review.openstack.org/#/c/244984 >> <https://review.openstack.org/#/c/244984/1> >> >> - Gus >> > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
