I still vote for #2. I don't think there will be a ton of duplication so we're probably ok in that regard. Also, it sounds like following suit will make life easier for the ci folks. I think #3 will be painful (both in initial work and maintenance).
If we do #2 then we could put quantum-client in tools/pip-requires and there wouldn't be much of any additional burden on the developer unless they were also going to work on the client -- in which case pull a separate repo. Thanks, Brad On Jan 3, 2012, at 12:47 AM, Dan Wendlandt wrote: > Hi folks, > > tl;dr we need to decide on splitting client repo out of main repo, per > monty's comments. Because this is what all other openstack projects are > doing, bias is to move in that direction, but benefits are not clear. If we > do split, my suggestion is to create two repos (client/server), not three > repos (client/server/common) > > longer version: > > At the 11/29 netstack meeting, one of the topics discussed was how to handle > monty's suggestion that we split out the client into another repo. > > From the 11/29 meeting, here's some background: > > 22:12:59 <mtaylor> danwent: > the problem is that we've got projects that need to depend on this stuff > (horizon being a good example) > > 22:13:17 <mtaylor> > so we need to be able to reference the actual thing that's needed, wihtout > pulling in the entire server > > 22:13:31 <danwent> mtaylor: > definitely understand that.. we want to be able to package things > separately, definitely > > 22:13:40 <cdub> > sounds more like packagin then repo mgmt > > 22:13:51 <danwent> > I think the question is why packaging needs to be tied to repo so tightly. > > 22:13:58 <cdub> > e.g. simple to build multpile packages from a single repo > > 22:13:59 <danwent> > i definitely understand that you have a lot of experience here… :) > > 22:14:17 <danwent> cdub: > yes, in fact that is what we do already. > > 22:14:23 <cdub> > right > > 22:14:25 <mtaylor> > it's for a few reasons > > 22:14:42 <mtaylor> > pip requires git lines can't reference sub directories > > 22:15:01 <mtaylor> > so you wind up with venv installs that require actual releases to pypi to be > consumable > > 22:15:36 <mtaylor> > additionally, once you start having submodules in tree, standard tooling > winds up being difficult to apply - knowing to interact with a setup.py in > trunk is straight forward > > 22:15:50 <mtaylor> > knowing that there are 4 setup.py files spread throughout the tree is less > obviouos > > 22:16:05 <bhall> > as of yesterday all setup.py's are in the root :) > > 22:16:24 <danwent> > ok, so the trade-off here is if we want the pip files of other projects to > automatically pull "trunk", we need separate repos? > > 22:16:41 <mtaylor> > yes > > 22:17:21 <mtaylor> > also, it's the model we're using for the other projects, so in keeping with > a consistent project-wide approach, I'd kind of want to hear a really good > reason that it can't work > > 22:17:28 <bhall> > how many repos are we talking about? client, common, server, plugins? > > 22:18:16 <danwent> > I'm also worried about trying to keep quantum simple enough that its easy > for new people to join and hack on the project > > 22:18:25 <mtaylor> > I don't think, as of yet, that we need separate ones for plugins > > 22:18:55 <mtaylor> > the most important one in my brain in client - as we have a python-*client > project either in existence or coming in to existence for every openstack > project > > 22:19:08 <bhall> > could we go with two repos? client, common/server/plugins > > 22:19:11 <mtaylor> > the common one is just because that's how you've organized your code - and i > believe client depends on it, yeah/ > > 22:19:18 <mtaylor> > does client depend on common? > > 22:19:25 <danwent> > I believe that was the original goal > > 22:19:50 <danwent> > but perhaps we should revisit that… I'm not sure how much is currently > shared. > > 22:19:58 <bhall> > I think it does currently > > 22:20:06 <bhall> > yeah, I don't think there is a lot of code that it depends on > > 22:20:17 <danwent> mtaylor: > so the main goal here is pip install, which are mainly targetted at > developers right? > > 22:21:05 <danwent> > (I assume non-developers will get everything via packages, which should have > dependencies described in terms of packages) > > 22:21:43 <mtaylor> danwent: > yes > > 22:22:12 <danwent> > ok, thanks for the explanations… I think I understand this better now, but > still need to noodle on whether there are ways we can both be happy :) > > 22:22:18 <mtaylor> danwent: > there is no thought that people will be doing production releases from pip :) > > 22:22:23 <mtaylor> danwent: awesome > > An here's some more follow-up from the 12/6 meeting: > > 22:18:55 <danwent> > first, as discussed last week with mtaylor, we need to figure out our > strategy for whether we are going to split quantum into multiple repos. > > 22:19:49 <salv> > before deciding on a strategy let's decide whether we want to split the > repos or not! :) > > 22:20:02 <mtaylor> danwent: > re that: we've got openstack.common going now - so perhaps quantum.common > can live there? > > 22:20:03 <danwent> > we can have the detailed discussion on the ML, but the high-level options > seem to be (1) keep as is, all one repo (2) split into two repos, client and > server, but potentially duplicate the little bits of shared code and (3) > split into three repos, client, server, and common > > 22:20:09 <salv> > Splitting in which way exactly? client and server components? > > 22:20:30 <danwent> mtaylor: > I was thinking something very similar myself, if we decide to go with #2 > > 22:20:57 <danwent> salv: > up for discussion > > 22:21:14 <danwent> #TODO: #danwent, start ML thread for splitting repos, > include #mtaylor as well > 22:21:17 <bhall> > my vote is for #2 (client and "the rest") > > 22:21:19 <mtaylor> #2 with some elements in openstack.common would be the > thing that matches the other projects more closely > 22:21:55 <danwent> if anyone else wants to comment now, go ahead, otherwise > i'll forward this content to start a thead on the ML > > It seems like we have three primary options (feel free to suggest others). > > #1: do nothing, keep every in single repo > #2: split into two repos: client + server, but potentially duplicate some of > the shared code currently in "common" > #3: split into three repos, client, server and common. > > A couple thoughts: > > I'm still not totally clear on the value of splitting the repos. Multiple > repos are not required for anyone installing openstack via packages, as we > can easily generate separate client/server packages when using a single repo > (in fact, we already do). My understanding is that splitting the repos aims > to help developers in the case where a project like horizon specifies a > dependency on quantum by adding a direct reference to the quantum git repo in > pip-requires. So one thing I'd like to better understand is the cost of > specifying a quantum git repo that includes client + server, as opposed to > one that just includes the client. If its just a matter of pulling down some > more bytes, that doesn't seem that painful (the quantum source is tiny > anyway). Is there some greater benefit to splitting repos that I am missing? > > I think the cost of splitting repos is non-trivial, as it makes it harder for > developers to get things running from source tree, (again, I'm assuming this > is mainly a developer issue, as users should mainly install from packages). > It may also mean we need to duplicate some common code, though the need for > that may go down if we can get both client + server code bases to have common > code in openstack-common. > > Another important consideration is that it seems that all of the other > openstack projects are moving to a model with the client in a separate repo > from the service itself. There would definitely a cost both in terms of > developer confusion and pain to the infrastructure team to Quantum being an > oddball in that department. This alone may be reason enough to follow suit, > but I'd love if there was a clear cut reason other than "everyone else is > doing it". > > If we do decide to split repos, my bias would be for #2, which avoid the even > more significant complexity of having three repos. While some code in common > may need to be duplicated, it may be the case that such code is duplicated > because it really belongs in something like openstack-common, which could be > a dependency of both. A lof of the code in quantum.common seems to be of > that variety (I suspect much of this code is actually only used by the server > at this point). The two chunks of common code used by the main client > library are: > > from quantum.common import exceptions > from quantum.common.serializer import Serializer > > Serializer seems to be pretty much all stock code that should be in > openstack-common. > > The quantum client only uses a few of the Exceptions classes in > common/exeptions.py. A few of these may need to be duplicated in both the > client/server code. Though if we make the changes suggested by Aaron to use > only standard HTTP error codes, the applicability of this code to the client > decreases to the point of almost no overlap, I suspect. > > Please chime in with your thoughts. Right now I'm not sure who would work on > this, so if you think you have some cycles, please speak up. > > Dan > > > -- > ~~~~~~~~~~~~~~~~~~~~~~~~~~~ > Dan Wendlandt > Nicira Networks: www.nicira.com > twitter: danwendlandt > ~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > -- > Mailing list: https://launchpad.net/~netstack > Post to : netstack@lists.launchpad.net > Unsubscribe : https://launchpad.net/~netstack > More help : https://help.launchpad.net/ListHelp -- Mailing list: https://launchpad.net/~netstack Post to : netstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~netstack More help : https://help.launchpad.net/ListHelp
