So seriously - how do you set up a dev environment for hyper-v? It doesn't have to be written up all nice and pretty, just a 30s outline in an email would be really useful :-/
- Gus On Thu, 26 Nov 2015 at 13:13 Alessandro Pilotti < apilo...@cloudbasesolutions.com> wrote: > Angus, > > "I'm afraid this has to be easy for me or I'm just not going to be able to > sustain the effort required :-/ )" > > What a tragedy, I’m soooo sorry that life is so terribly bitter and > requires some effort to support people with views different from yours! :-) > > Just kidding, we’re here to help, but we obviously won’t do your work. > Given that Oslo code needs to be portable, I was just trying to give you > some alternative that doesn’t require you to do useless manual work during > development. > > 1) Unit tests are not the answer, as hopefully they mock out most > significant underlying OS features. Sure, you will catch the basic issues, > like importing a module that doesn’t exist on Windows, but you cannot rely > on them as a proof of portability. > 2) Integration testing on the other side provide by definition a proof of > reliability on the target system (in the limits of the tests coverage at > least) and CI testing needs eventually to be there on Windows as well for > oslo.<insert_any_name_here>. This is obvious not a way to develop, but a > “guard”, against issues. In other words, regardless of your opinion on > Windows, the Windows CI needs to be there, why not adding it now for this > project? > > Getting to development practices, when designing features, hopefully > portability gets evaluated _before_ starting coding, so you should not need > too many manual runs on every platform before being confident enough to > send a patch to Gerrit and wait for CI results. And for that you don’t need > tox or anything fancy, just a Python environment and some very basic OS > skills. > > When in doubt, an option is to drop by on #openstack-hyper-v and ask. But > take care, you might find open minded people, make sure you're at ease with > this. ;-) > > Alessandro > > > From: Angus Lees <g...@inodes.org> > Reply-To: "OpenStack Development Mailing List (not for usage questions)" < > openstack-dev@lists.openstack.org> > Date: Thursday 26 November 2015 at 03:23 > To: "OpenStack Development Mailing List (not for usage questions)" < > openstack-dev@lists.openstack.org>, Claudiu Belu < > cb...@cloudbasesolutions.com> > > Subject: Re: [openstack-dev] [hyper-v] oslo.privsep vs Windows > > Thanks for the suggestion, and having a CI bot running on oslo.privsep > would be a good thing once the basic code works on Windows - I would have > expected it to be already running on all oslo projects (that the hyper-v > hypervisor does/might depend on) tbh. > > But that's a clumsy way to actually develop something. I _know_ privsep > won't work currently on windows (imports pwd/grp for starters), and having > to add print statements + submit + push-to-gerrit + wait for a CI bot + dig > through CI bot logs + repeat is a pretty terrible workflow ;-) > > (I already have very little empathy for anyone choosing to run Windows > just to provide yet-another-x86-hypervisor when far easier alternatives are > available, and I won't donate a disproportionate amount of my time to a > well-funded company who has historically actively undermined the > intellectual commons when _I_ have far more rewarding alternatives > available. I'm afraid this has to be easy for me or I'm just not going to > be able to sustain the effort required :-/ ) > > > - Gus > > On Thu, 26 Nov 2015 at 11:56 Alessandro Pilotti < > apilo...@cloudbasesolutions.com> wrote: > >> Hi Angus, >> >> First thanks for your concern on code portability! It still happens that >> we have to ask to revert patches on Oslo projects due to some Linux >> specific code that we discover only when the actual Oslo modules are used >> by Nova, Neutron, Cinder or other projects. Typically a running Windows CI >> suddenly turns red on every patch and that’s how we get to find out. We’re >> working on adding many more projects under Windows CI tests, so at some >> point all the relevant Oslo ones will be covered and we’ll be able to >> prevent those situations before the code gets merged. >> >> What about preparing some basic integration tests for oslo.privsep that >> we could add to our Windows CI infrastructure? This would give you peace of >> mind during development on Linux without needing to test manually all your >> patches on Windows, knowing that the Windows CI will give an error if the >> patch contains non portable code (or code that doesn’t behave as expected). >> >> Alessandro >> >> >> >> From: Angus Lees <g...@inodes.org> >> Reply-To: "OpenStack Development Mailing List (not for usage questions)" >> <openstack-dev@lists.openstack.org> >> Date: Thursday 26 November 2015 at 01:56 >> To: Claudiu Belu <cb...@cloudbasesolutions.com>, "OpenStack Development >> Mailing List (not for usage questions)" < >> openstack-dev@lists.openstack.org> >> Subject: Re: [openstack-dev] [hyper-v] oslo.privsep vs Windows >> >> So I spent a day yesterday trying to get to the point where I could just >> run a no-change "tox" successfully on windows. Unfortunately I gave up >> when I realised I still had several days of downloading+building things >> ahead of me and clearly I was doing it the hard way :( >> >> Could you point me to the "dev environment how-to" doc for hyper-v work? >> I'm thinking of something like >> http://docs.openstack.org/developer/nova/development.environment.html#setup >> with >> simple cut+paste instructions for the totally-windows-clueless like me ;) >> Or perhaps a pre-prepared disk image, if the Microsoft license allows such >> a thing. In particular, the details on >> https://wiki.openstack.org/wiki/Hyper-V look out of date (Grizzly, and >> several docs.openstack.org links are broken), and the links to things >> like >> https://cloudbase.it/hyper-v-nova-compute-installer-unattended-setup/ look >> like deployment rather than development environments (?). >> >> In particular, I think(?) I should be careful *not* to install too much >> of a cygwin-style environment, since I think(?) that might no longer be >> representative of the environment in which hyper-v is expected to operate. >> If I'm wrong, and cygwin/msys is ok, then it looks like there are several >> free-software-on-windows "distributions" that would make life a whole lot >> simpler (by frankly being less like Windows). Some guidance from people >> who understand the issues here would be appreciated. >> >> - Gus >> >> On Tue, 24 Nov 2015 at 22:01 Claudiu Belu <cb...@cloudbasesolutions.com> >> wrote: >> >>> Hello, >>> >>> Thanks Dims for raising the concern and Angus for reaching out. :) >>> >>> Most of the time, python development on Windows is not too far off from >>> Linux. But the two systems are quite different, which imply different >>> modules (fcntl, pwd, grp modules do not exist in Windows) or different >>> implementations of some modules (multiprocessing uses Popen instead of >>> os.fork, os module is quite different) or some socket options and signals >>> are different in Windows. >>> >>> 1.a. As I've said earlier, some modules do not exist in Windows. All, or >>> at least most standard modules document the fact that they are strictly for >>> Linux. [1][2][3] >>> b. At the very least, running the unit tests in a Windows environment >>> can at least detect simple problems (e.g. imports). Secondly, there is a >>> Hyper-V / Windows CI running on some of the OpenStack projects (nova, >>> neutron, networking_hyperv, cinder) that can be checked before merging. >>> >>> 2. This is a bit more complicated question. Well, for functions, you >>> could have separate modules for Linux specific functions and Windows >>> specific functions. This has been done before: [4] As for object-oriented >>> implementations, I'd suggest having the system-specific calls be done in >>> private methods, which will be overriden by Windows / Linux subclasses with >>> their specific implementations. We've done something like this before, but >>> solutions were pretty much straight-forward; it might not be as simple for >>> oslo_privsep, since it is very Linux-specific. >>> >>> 3. Typically, the OpenStack services on Hyper-V / Windows are run with >>> users that have enough privileges to do their job. For example, the >>> nova-compute service is run with a user that has Hyper-V Admin privileges >>> and is not necessarily in the "Administrator" user group. We haven't used >>> rootwrap in our usecases, it is disabled by default, plus, oslo.rootwrap >>> imports pwd, which doesn't exist in Windows. >>> >>> [1] https://docs.python.org/2/library/fcntl.html >>> [2] https://docs.python.org/2/library/pwd.html >>> [3] https://docs.python.org/2/library/grp.html >>> [4] >>> https://github.com/openstack/neutron/blob/master/neutron/agent/common/utils.py >>> [5] >>> https://github.com/openstack/oslo.rootwrap/blob/master/oslo_rootwrap/wrapper.py#L19 >>> >>> If you have any further questions, feel free to ask. :) >>> >>> Best regards, >>> Claudiu Belu >>> >>> >>> ------------------------------ >>> *From:* Angus Lees [g...@inodes.org] >>> *Sent:* Tuesday, November 24, 2015 6:18 AM >>> *To:* OpenStack Development Mailing List (not for usage questions); >>> Claudiu Belu >>> *Subject:* [hyper-v] oslo.privsep vs Windows >>> >>> Dims has just raised[1] the excellent concern that oslo.privsep will >>> need to at least survive on Windows, because hyper-v. I have no real >>> experience coding on windows (I wrote a windows C program once, but I only >>> ever ran it under wine ;) and certainly none within an OpenStack/python >>> context: >>> >>> 1) How can I test whatever I'm working on to see if I have mistakenly >>> introduced something Linux-specific? Surely this is a challenge common >>> across every project in the nova/oslo/hyper-v stack. >>> >>> 2) What predicate should I use to guard the inevitable Linux-specific or >>> Windows-specific code branches? >>> >>> and I guess: >>> 3) What does a typical OpenStack/hyper-v install even look like? Do we >>> run rootwrap with some sudo-like-thing, or just run everything as the >>> superuser? >>> What _should_ oslo.privsep do for this environment? >>> >>> [1] https://review.openstack.org/#/c/244984 >>> <https://review.openstack.org/#/c/244984/1> >>> >>> - Gus >>> >> __________________________________________________________________________ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: >> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
