Hi Angus, First thanks for your concern on code portability! It still happens that we have to ask to revert patches on Oslo projects due to some Linux specific code that we discover only when the actual Oslo modules are used by Nova, Neutron, Cinder or other projects. Typically a running Windows CI suddenly turns red on every patch and that’s how we get to find out. We’re working on adding many more projects under Windows CI tests, so at some point all the relevant Oslo ones will be covered and we’ll be able to prevent those situations before the code gets merged.
What about preparing some basic integration tests for oslo.privsep that we could add to our Windows CI infrastructure? This would give you peace of mind during development on Linux without needing to test manually all your patches on Windows, knowing that the Windows CI will give an error if the patch contains non portable code (or code that doesn’t behave as expected). Alessandro From: Angus Lees <g...@inodes.org<mailto:g...@inodes.org>> Reply-To: "OpenStack Development Mailing List (not for usage questions)" <openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>> Date: Thursday 26 November 2015 at 01:56 To: Claudiu Belu <cb...@cloudbasesolutions.com<mailto:cb...@cloudbasesolutions.com>>, "OpenStack Development Mailing List (not for usage questions)" <openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>> Subject: Re: [openstack-dev] [hyper-v] oslo.privsep vs Windows So I spent a day yesterday trying to get to the point where I could just run a no-change "tox" successfully on windows. Unfortunately I gave up when I realised I still had several days of downloading+building things ahead of me and clearly I was doing it the hard way :( Could you point me to the "dev environment how-to" doc for hyper-v work? I'm thinking of something like http://docs.openstack.org/developer/nova/development.environment.html#setup with simple cut+paste instructions for the totally-windows-clueless like me ;) Or perhaps a pre-prepared disk image, if the Microsoft license allows such a thing. In particular, the details on https://wiki.openstack.org/wiki/Hyper-V look out of date (Grizzly, and several docs.openstack.org<http://docs.openstack.org> links are broken), and the links to things like https://cloudbase.it/hyper-v-nova-compute-installer-unattended-setup/ look like deployment rather than development environments (?). In particular, I think(?) I should be careful *not* to install too much of a cygwin-style environment, since I think(?) that might no longer be representative of the environment in which hyper-v is expected to operate. If I'm wrong, and cygwin/msys is ok, then it looks like there are several free-software-on-windows "distributions" that would make life a whole lot simpler (by frankly being less like Windows). Some guidance from people who understand the issues here would be appreciated. - Gus On Tue, 24 Nov 2015 at 22:01 Claudiu Belu <cb...@cloudbasesolutions.com<mailto:cb...@cloudbasesolutions.com>> wrote: Hello, Thanks Dims for raising the concern and Angus for reaching out. :) Most of the time, python development on Windows is not too far off from Linux. But the two systems are quite different, which imply different modules (fcntl, pwd, grp modules do not exist in Windows) or different implementations of some modules (multiprocessing uses Popen instead of os.fork, os module is quite different) or some socket options and signals are different in Windows. 1.a. As I've said earlier, some modules do not exist in Windows. All, or at least most standard modules document the fact that they are strictly for Linux. [1][2][3] b. At the very least, running the unit tests in a Windows environment can at least detect simple problems (e.g. imports). Secondly, there is a Hyper-V / Windows CI running on some of the OpenStack projects (nova, neutron, networking_hyperv, cinder) that can be checked before merging. 2. This is a bit more complicated question. Well, for functions, you could have separate modules for Linux specific functions and Windows specific functions. This has been done before: [4] As for object-oriented implementations, I'd suggest having the system-specific calls be done in private methods, which will be overriden by Windows / Linux subclasses with their specific implementations. We've done something like this before, but solutions were pretty much straight-forward; it might not be as simple for oslo_privsep, since it is very Linux-specific. 3. Typically, the OpenStack services on Hyper-V / Windows are run with users that have enough privileges to do their job. For example, the nova-compute service is run with a user that has Hyper-V Admin privileges and is not necessarily in the "Administrator" user group. We haven't used rootwrap in our usecases, it is disabled by default, plus, oslo.rootwrap imports pwd, which doesn't exist in Windows. [1] https://docs.python.org/2/library/fcntl.html [2] https://docs.python.org/2/library/pwd.html [3] https://docs.python.org/2/library/grp.html [4] https://github.com/openstack/neutron/blob/master/neutron/agent/common/utils.py [5] https://github.com/openstack/oslo.rootwrap/blob/master/oslo_rootwrap/wrapper.py#L19 If you have any further questions, feel free to ask. :) Best regards, Claudiu Belu ________________________________ From: Angus Lees [g...@inodes.org<mailto:g...@inodes.org>] Sent: Tuesday, November 24, 2015 6:18 AM To: OpenStack Development Mailing List (not for usage questions); Claudiu Belu Subject: [hyper-v] oslo.privsep vs Windows Dims has just raised[1] the excellent concern that oslo.privsep will need to at least survive on Windows, because hyper-v. I have no real experience coding on windows (I wrote a windows C program once, but I only ever ran it under wine ;) and certainly none within an OpenStack/python context: 1) How can I test whatever I'm working on to see if I have mistakenly introduced something Linux-specific? Surely this is a challenge common across every project in the nova/oslo/hyper-v stack. 2) What predicate should I use to guard the inevitable Linux-specific or Windows-specific code branches? and I guess: 3) What does a typical OpenStack/hyper-v install even look like? Do we run rootwrap with some sudo-like-thing, or just run everything as the superuser? What _should_ oslo.privsep do for this environment? [1] https://review.openstack.org/#/c/244984<https://review.openstack.org/#/c/244984/1> - Gus
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
