Thanks Monty, I started reviews on these patches as well. Also, I wasn't able to run tox, as I keep getting a timeout when it is trying to download the lxml==2.3 package (other packages seem to download fine). Perhaps it is just a temporary thing, I will try again in a bit.
Dan On Wed, Jan 18, 2012 at 5:15 PM, Monty Taylor <mord...@inaugust.com> wrote: > Ok. python-quantumclient is in gerrit, and I've just submitted the > change to quantum for review for removing it. > > https://review.openstack.org/3164 > https://review.openstack.org/3165 > > python-quantumclient is the new poster child for tox integration with > jenkins - it's using all of the new hotness, so it tests against py26 > and 27. We'll be moving the other projects to this as soon as E3 is cut > (I think ttx would kill me if I made that kind of change this week) > > That said - the above patches make tox work for quantum as well (but > should not break non-tox builds) > > Monty > > On 01/18/2012 08:55 AM, Dan Wendlandt wrote: >> >> >> On Tue, Jan 17, 2012 at 1:52 PM, Monty Taylor <mord...@inaugust.com >> <mailto:mord...@inaugust.com>> wrote: >> >> > >> > I've got a python-quantumclient repo made and pushed to >> > https://github.com/emonty/python-quantumclient >> > >> > >> > Huh, this is a bit different than I was expecting.... I would have >> > expected that the 'quantum' cli utility would be part of the client >> > repo. It looks like your patch keeps this in the server code, and >> makes >> > the server code depend on the client code. I was under the impression >> > that with nova the 'nova' utility was actually packaged with the >> client >> > code, or is that not the case? I'm sure you have a reason for doing >> > this, but I'd be curious to know what it was :) >> >> >> It is part of the client repo - it's just done via setup.py. (The old >> bin/quantum was simply a wrapper which called quantum.client.cli.main() >> ... setuptools console_scripts do that for us. >> >> >> whoops, should have dug deeper. thanks. >> >> dan >> >> >> >> >> > >> > I moved a good portion of quantum.common in to >> python-quantumclient >> > because both quantum and quantum.client need it, but since >> quantum will >> > depend on quantum.client, it means that it'll be available to both >> > things. I don't think this is a great solution long term, but >> it kinda >> > of physically works at the moment. Some of quantum.common can >> get moved >> > to openstack.common - the main thing that'll be odd long term >> will be >> > Exceptions ... but I'm going to suggest we try this for a >> moment, see >> > how much we hate it and then try to solve that. :) >> > >> > >> > Ok, that makes sense. >> > >> > >> > >> > I've also got python-quantumclient set up initially to use >> tox... so >> > just grabbing the repo and running "tox" should work and should do >> > virtualenv based tests in both python2.6 and python2.7. >> > >> > >> > Looks like both client and server now can run "tox". cool.. >> > >> > >> > >> > I've got a patch up for quantum: >> > >> > https://review.openstack.org/3084 >> > >> > Which makes quantum a namespace package so that things can be >> installed >> > over top of things (which then allows the python-quantumclient >> repo >> > to work) >> > >> > After that, I've got this: >> > >> > https://github.com/emonty/quantum/tree/remove-client >> > >> > Which removes quantum.client and quantum.common as well as making >> > Which I can submit for review >> > >> > >> > Sure, submit for review if you think its ready. Probably better >> to get >> > detailed comments on gerrit than on the ML. >> >> I shall - we'll need python-quantumclient to exist first. I'll get >> python-quantumclient up so that we've got a good way to review the >> quantum patch. >> >> > >> > I think we should chat about how plugins all fit in to this >> picture, >> > >> > >> > In general, the client lib accesses only the API (which should be >> plugin >> > agnostic). There are definitely API extensions that are plugin >> specific >> > though. Is that what you are referring to, or something else? >> >> I context switched there - not related to client split and not relevant >> for now. :) >> >> > because I think we're on the verge of having some really >> kickass stuff >> > going on here. >> > >> > >> > indeed. thanks again for your help with this! >> > >> >> My pleasure! Glad I can be helpful! >> >> Monty >> >> > >> > Anyway - poke at stuff, play around with it - tell me if you >> think I'm >> > nuts or not. >> > >> > Monty >> > >> > On 01/12/2012 08:08 AM, Dan Wendlandt wrote: >> > > >> > > >> > > On Tue, Jan 10, 2012 at 5:32 PM, Monty Taylor >> > <mord...@inaugust.com <mailto:mord...@inaugust.com> >> <mailto:mord...@inaugust.com <mailto:mord...@inaugust.com>> >> > > <mailto:mord...@inaugust.com <mailto:mord...@inaugust.com> >> <mailto:mord...@inaugust.com <mailto:mord...@inaugust.com>>>> wrote: >> > > >> > > So, while looking at this, I discovered the file you >> guys have, >> > > version.py. I've been working on a standalone library, >> vcsversion >> > > (https://github.com/emonty/vcsversion) which does some >> similar >> > things >> > > and which encapsulates some of the logic that's current >> in the >> > > build_tarball.sh script. I'd love to (doesn't need to happen >> > to today) >> > > port any missing functionality that's in version.py into >> > vcsversion and >> > > get quantum et al using it. >> > > >> > > >> > > great. the more common code we can rip out of quantum >> proper and >> > put in >> > > a shared lib the better in my opinion. >> > > >> > > dan >> > > >> > > >> > > >> > > Great minds think alike, right? :) >> > > >> > > Monty >> > > >> > > On 01/10/2012 02:48 PM, Dan Wendlandt wrote: >> > > > Ok, based on the meeting today we are going ahead with >> this. >> > > > >> > > > Blueprint created and assigned to >> > > > monty: >> > > >> > >> https://blueprints.launchpad.net/quantum/+spec/separate-client-repo >> > > > >> > > > Monty, I assume that you will setup the >> infrastructure, then >> > at some >> > > > point need to hand off to a quantum dev (or not). Please >> > ping the >> > > list >> > > > when you get to that point. >> > > > >> > > > And as cdub mentioned at the end of the meeting, the >> sooner >> > we can do >> > > > this, the better in terms of distros that will need to >> > change their >> > > > packaging in time for essex. >> > > > >> > > > thanks for driving this. >> > > > >> > > > Dan >> > > > >> > > > >> > > > On Tue, Jan 3, 2012 at 1:00 PM, Monty Taylor >> > <mord...@inaugust.com <mailto:mord...@inaugust.com> >> <mailto:mord...@inaugust.com <mailto:mord...@inaugust.com>> >> > > <mailto:mord...@inaugust.com >> <mailto:mord...@inaugust.com> <mailto:mord...@inaugust.com >> <mailto:mord...@inaugust.com>>> >> > > > <mailto:mord...@inaugust.com >> <mailto:mord...@inaugust.com> <mailto:mord...@inaugust.com >> <mailto:mord...@inaugust.com>> >> > <mailto:mord...@inaugust.com <mailto:mord...@inaugust.com> >> <mailto:mord...@inaugust.com <mailto:mord...@inaugust.com>>>>> wrote: >> > > > >> > > > ++ >> > > > >> > > > I think as a parallel thing, some of the things >> that are in >> > > > quantum.common _might_ want to live in >> openstack.common. >> > > > >> > > > On 01/03/2012 12:55 PM, Brad Hall wrote: >> > > > > I still vote for #2. I don't think there will >> be a ton of >> > > > duplication so we're probably ok in that regard. >> Also, >> > it sounds >> > > > like following suit will make life easier for the ci >> > folks. I >> > > think >> > > > #3 will be painful (both in initial work and >> maintenance). >> > > > > >> > > > > If we do #2 then we could put quantum-client in >> > > tools/pip-requires >> > > > and there wouldn't be much of any additional >> burden on the >> > > developer >> > > > unless they were also going to work on the client >> -- in >> > which case >> > > > pull a separate repo. >> > > > > >> > > > > Thanks, >> > > > > Brad >> > > > > >> > > > > On Jan 3, 2012, at 12:47 AM, Dan Wendlandt wrote: >> > > > > >> > > > >> Hi folks, >> > > > >> >> > > > >> tl;dr we need to decide on splitting client >> repo out >> > of main >> > > > repo, per monty's comments. Because this is what >> all other >> > > > openstack projects are doing, bias is to move in that >> > > direction, but >> > > > benefits are not clear. If we do split, my >> suggestion is to >> > > create >> > > > two repos (client/server), not three repos >> > (client/server/common) >> > > > >> >> > > > >> longer version: >> > > > >> >> > > > >> At the 11/29 netstack meeting, one of the topics >> > discussed was >> > > > how to handle monty's suggestion that we split out the >> > client into >> > > > another repo. >> > > > >> >> > > > >> From the 11/29 meeting, here's some background: >> > > > >> >> > > > >> 22:12:59 <mtaylor> danwent: >> > > > >> the problem is that we've got projects that >> need to >> > depend on >> > > > this stuff (horizon being a good example) >> > > > >> >> > > > >> 22:13:17 <mtaylor> >> > > > >> so we need to be able to reference the actual >> thing >> > that's >> > > > needed, wihtout pulling in the entire server >> > > > >> >> > > > >> 22:13:31 <danwent> mtaylor: >> > > > >> definitely understand that.. we want to be able to >> > package >> > > > things separately, definitely >> > > > >> >> > > > >> 22:13:40 <cdub> >> > > > >> sounds more like packagin then repo mgmt >> > > > >> >> > > > >> 22:13:51 <danwent> >> > > > >> I think the question is why packaging needs to be >> > tied to repo >> > > > so tightly. >> > > > >> >> > > > >> 22:13:58 <cdub> >> > > > >> e.g. simple to build multpile packages from a >> single >> > repo >> > > > >> >> > > > >> 22:13:59 <danwent> >> > > > >> i definitely understand that you have a lot of >> > experience >> > > here… :) >> > > > >> >> > > > >> 22:14:17 <danwent> cdub: >> > > > >> yes, in fact that is what we do already. >> > > > >> >> > > > >> 22:14:23 <cdub> >> > > > >> right >> > > > >> >> > > > >> 22:14:25 <mtaylor> >> > > > >> it's for a few reasons >> > > > >> >> > > > >> 22:14:42 <mtaylor> >> > > > >> pip requires git lines can't reference sub >> directories >> > > > >> >> > > > >> 22:15:01 <mtaylor> >> > > > >> so you wind up with venv installs that require >> > actual releases >> > > > to pypi to be consumable >> > > > >> >> > > > >> 22:15:36 <mtaylor> >> > > > >> additionally, once you start having submodules >> in tree, >> > > standard >> > > > tooling winds up being difficult to apply - knowing to >> > > interact with >> > > > a setup.py in trunk is straight forward >> > > > >> >> > > > >> 22:15:50 <mtaylor> >> > > > >> knowing that there are 4 setup.py files spread >> > throughout the >> > > > tree is less obviouos >> > > > >> >> > > > >> 22:16:05 <bhall> >> > > > >> as of yesterday all setup.py's are in the root :) >> > > > >> >> > > > >> 22:16:24 <danwent> >> > > > >> ok, so the trade-off here is if we want the pip >> > files of other >> > > > projects to automatically pull "trunk", we need >> separate >> > repos? >> > > > >> >> > > > >> 22:16:41 <mtaylor> >> > > > >> yes >> > > > >> >> > > > >> 22:17:21 <mtaylor> >> > > > >> also, it's the model we're using for the other >> > projects, >> > > so in >> > > > keeping with a consistent project-wide approach, >> I'd kind of >> > > want to >> > > > hear a really good reason that it can't work >> > > > >> >> > > > >> 22:17:28 <bhall> >> > > > >> how many repos are we talking about? client, >> common, >> > server, >> > > > plugins? >> > > > >> >> > > > >> 22:18:16 <danwent> >> > > > >> I'm also worried about trying to keep quantum >> simple >> > > enough that >> > > > its easy for new people to join and hack on the >> project >> > > > >> >> > > > >> 22:18:25 <mtaylor> >> > > > >> I don't think, as of yet, that we need >> separate ones for >> > > plugins >> > > > >> >> > > > >> 22:18:55 <mtaylor> >> > > > >> the most important one in my brain in client - >> as we >> > have a >> > > > python-*client project either in existence or >> coming in to >> > > existence >> > > > for every openstack project >> > > > >> >> > > > >> 22:19:08 <bhall> >> > > > >> could we go with two repos? client, >> > common/server/plugins >> > > > >> >> > > > >> 22:19:11 <mtaylor> >> > > > >> the common one is just because that's how you've >> > organized >> > > your >> > > > code - and i believe client depends on it, yeah/ >> > > > >> >> > > > >> 22:19:18 <mtaylor> >> > > > >> does client depend on common? >> > > > >> >> > > > >> 22:19:25 <danwent> >> > > > >> I believe that was the original goal >> > > > >> >> > > > >> 22:19:50 <danwent> >> > > > >> but perhaps we should revisit that… I'm not >> sure how >> > much is >> > > > currently shared. >> > > > >> >> > > > >> 22:19:58 <bhall> >> > > > >> I think it does currently >> > > > >> >> > > > >> 22:20:06 <bhall> >> > > > >> yeah, I don't think there is a lot of code that it >> > depends on >> > > > >> >> > > > >> 22:20:17 <danwent> mtaylor: >> > > > >> so the main goal here is pip install, which >> are mainly >> > > targetted >> > > > at developers right? >> > > > >> >> > > > >> 22:21:05 <danwent> >> > > > >> (I assume non-developers will get everything via >> > packages, >> > > which >> > > > should have dependencies described in terms of >> packages) >> > > > >> >> > > > >> 22:21:43 <mtaylor> danwent: >> > > > >> yes >> > > > >> >> > > > >> 22:22:12 <danwent> >> > > > >> ok, thanks for the explanations… I think I >> > understand this >> > > > better now, but still need to noodle on whether there >> > are ways we >> > > > can both be happy :) >> > > > >> >> > > > >> 22:22:18 <mtaylor> danwent: >> > > > >> there is no thought that people will be doing >> production >> > > > releases from pip :) >> > > > >> >> > > > >> 22:22:23 <mtaylor> danwent: awesome >> > > > >> >> > > > >> An here's some more follow-up from the 12/6 >> meeting: >> > > > >> >> > > > >> 22:18:55 <danwent> >> > > > >> first, as discussed last week with mtaylor, we >> need >> > to figure >> > > > out our strategy for whether we are going to split >> > quantum into >> > > > multiple repos. >> > > > >> >> > > > >> 22:19:49 <salv> >> > > > >> before deciding on a strategy let's decide whether >> > we want to >> > > > split the repos or not! :) >> > > > >> >> > > > >> 22:20:02 <mtaylor> danwent: >> > > > >> re that: we've got openstack.common going now - so >> > perhaps >> > > > quantum.common can live there? >> > > > >> >> > > > >> 22:20:03 <danwent> >> > > > >> we can have the detailed discussion on the ML, >> but the >> > > > high-level options seem to be (1) keep as is, all one >> > repo (2) >> > > split >> > > > into two repos, client and server, but potentially >> > duplicate the >> > > > little bits of shared code and (3) split into three >> > repos, client, >> > > > server, and common >> > > > >> >> > > > >> 22:20:09 <salv> >> > > > >> Splitting in which way exactly? client and server >> > components? >> > > > >> >> > > > >> 22:20:30 <danwent> mtaylor: >> > > > >> I was thinking something very similar myself, >> if we >> > decide >> > > to go >> > > > with #2 >> > > > >> >> > > > >> 22:20:57 <danwent> salv: >> > > > >> up for discussion >> > > > >> >> > > > >> 22:21:14 <danwent> #TODO: #danwent, start ML >> thread for >> > > splitting >> > > > repos, include #mtaylor as well >> > > > >> 22:21:17 <bhall> >> > > > >> my vote is for #2 (client and "the rest") >> > > > >> >> > > > >> 22:21:19 <mtaylor> #2 with some elements in >> > openstack.common >> > > > would be the thing that matches the other projects >> more >> > closely >> > > > >> 22:21:55 <danwent> if anyone else wants to comment >> > now, go >> > > ahead, >> > > > otherwise i'll forward this content to start a >> thead on >> > the ML >> > > > >> >> > > > >> It seems like we have three primary options >> (feel free to >> > > suggest >> > > > others). >> > > > >> >> > > > >> #1: do nothing, keep every in single repo >> > > > >> #2: split into two repos: client + server, but >> > potentially >> > > > duplicate some of the shared code currently in >> "common" >> > > > >> #3: split into three repos, client, server and >> common.. >> > > > >> >> > > > >> A couple thoughts: >> > > > >> >> > > > >> I'm still not totally clear on the value of >> splitting the >> > > repos. >> > > > Multiple repos are not required for anyone installing >> > > openstack via >> > > > packages, as we can easily generate separate >> client/server >> > > packages >> > > > when using a single repo (in fact, we already >> do).. My >> > > understanding >> > > > is that splitting the repos aims to help developers in >> > the case >> > > > where a project like horizon specifies a dependency on >> > quantum by >> > > > adding a direct reference to the quantum git repo in >> > pip-requires. >> > > > So one thing I'd like to better understand is the >> cost of >> > > > specifying a quantum git repo that includes client + >> > server, as >> > > > opposed to one that just includes the client. If >> its just a >> > > matter >> > > > of pulling down some more bytes, that doesn't seem >> that >> > > painful (the >> > > > quantum source is tiny anyway). Is there some greater >> > benefit to >> > > > splitting repos that I am missing? >> > > > >> >> > > > >> I think the cost of splitting repos is non-trivial, >> > as it makes >> > > > it harder for developers to get things running from >> > source tree, >> > > > (again, I'm assuming this is mainly a developer >> issue, >> > as users >> > > > should mainly install from packages).. It may >> also mean >> > we need to >> > > > duplicate some common code, though the need for >> that may go >> > > down if >> > > > we can get both client + server code bases to have >> > common code in >> > > > openstack-common. >> > > > >> >> > > > >> Another important consideration is that it >> seems that all >> > > of the >> > > > other openstack projects are moving to a model >> with the >> > client >> > > in a >> > > > separate repo from the service itself. There would >> > definitely a >> > > > cost both in terms of developer confusion and >> pain to the >> > > > infrastructure team to Quantum being an oddball in >> that >> > > department. >> > > > This alone may be reason enough to follow suit, >> but I'd >> > love if >> > > > there was a clear cut reason other than "everyone >> else is >> > > doing it". >> > > > >> >> > > > >> If we do decide to split repos, my bias would >> be for >> > #2, which >> > > > avoid the even more significant complexity of having >> > three repos. >> > > > While some code in common may need to be >> duplicated, it may >> > > be the >> > > > case that such code is duplicated because it really >> > belongs in >> > > > something like openstack-common, which could be a >> > dependency of >> > > > both. A lof of the code in quantum.common seems to be >> > of that >> > > > variety (I suspect much of this code is actually only >> > used by the >> > > > server at this point). The two chunks of common code >> > used by the >> > > > main client library are: >> > > > >> >> > > > >> from quantum.common import exceptions >> > > > >> from quantum.common.serializer import Serializer >> > > > >> >> > > > >> Serializer seems to be pretty much all stock >> code that >> > > should be >> > > > in openstack-common. >> > > > >> >> > > > >> The quantum client only uses a few of the >> Exceptions >> > classes in >> > > > common/exeptions.py. A few of these may need to be >> > duplicated in >> > > > both the client/server code. Though if we make >> the changes >> > > > suggested by Aaron to use only standard HTTP error >> > codes, the >> > > > applicability of this code to the client decreases to >> > the point of >> > > > almost no overlap, I suspect. >> > > > >> >> > > > >> Please chime in with your thoughts. Right now I'm >> > not sure who >> > > > would work on this, so if you think you have some >> > cycles, please >> > > > speak up. >> > > > >> >> > > > >> Dan >> > > > >> >> > > > >> >> > > > >> -- >> > > > >> ~~~~~~~~~~~~~~~~~~~~~~~~~~~ >> > > > >> Dan Wendlandt >> > > > >> Nicira Networks: www.nicira.com >> <http://www.nicira.com> >> > <http://www.nicira.com> <http://www.nicira.com> >> > > <http://www.nicira.com> >> > > > >> twitter: danwendlandt >> > > > >> ~~~~~~~~~~~~~~~~~~~~~~~~~~~ >> > > > >> >> > > > >> -- >> > > > >> Mailing list: https://launchpad.net/~netstack >> > > > >> Post to : netstack@lists.launchpad.net >> <mailto:netstack@lists.launchpad.net> >> > <mailto:netstack@lists.launchpad.net >> <mailto:netstack@lists.launchpad.net>> >> > > <mailto:netstack@lists.launchpad.net >> <mailto:netstack@lists.launchpad.net> >> > <mailto:netstack@lists.launchpad.net >> <mailto:netstack@lists.launchpad.net>>> >> > > > <mailto:netstack@lists.launchpad.net >> <mailto:netstack@lists.launchpad.net> >> > <mailto:netstack@lists.launchpad.net >> <mailto:netstack@lists.launchpad.net>> >> > > <mailto:netstack@lists.launchpad.net >> <mailto:netstack@lists.launchpad.net> >> > <mailto:netstack@lists.launchpad.net >> <mailto:netstack@lists.launchpad.net>>>> >> > > > >> Unsubscribe : https://launchpad.net/~netstack >> > > > >> More help : https://help.launchpad.net/ListHelp >> > > > > >> > > > > >> > > > >> > > > >> > > > >> > > > >> > > > -- >> > > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~ >> > > > Dan Wendlandt >> > > > Nicira Networks: www.nicira.com >> <http://www.nicira.com> <http://www.nicira.com> >> > <http://www.nicira.com> >> > > <http://www.nicira.com> >> > > > twitter: danwendlandt >> > > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~ >> > > > >> > > >> > > >> > > >> > > >> > > -- >> > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~ >> > > Dan Wendlandt >> > > Nicira Networks: www.nicira.com <http://www.nicira.com> >> <http://www.nicira.com> >> > <http://www.nicira.com> >> > > twitter: danwendlandt >> > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~ >> > > >> > >> > >> > >> > >> > -- >> > ~~~~~~~~~~~~~~~~~~~~~~~~~~~ >> > Dan Wendlandt >> > Nicira Networks: www.nicira.com <http://www.nicira.com> >> <http://www.nicira.com> >> > twitter: danwendlandt >> > ~~~~~~~~~~~~~~~~~~~~~~~~~~~ >> > >> >> >> >> >> -- >> ~~~~~~~~~~~~~~~~~~~~~~~~~~~ >> Dan Wendlandt >> Nicira Networks: www.nicira.com <http://www.nicira.com> >> twitter: danwendlandt >> ~~~~~~~~~~~~~~~~~~~~~~~~~~~ >> -- ~~~~~~~~~~~~~~~~~~~~~~~~~~~ Dan Wendlandt Nicira Networks: www.nicira.com twitter: danwendlandt ~~~~~~~~~~~~~~~~~~~~~~~~~~~ -- Mailing list: https://launchpad.net/~netstack Post to : netstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~netstack More help : https://help.launchpad.net/ListHelp
