Hi folks, tl;dr we need to decide on splitting client repo out of main repo, per monty's comments. Because this is what all other openstack projects are doing, bias is to move in that direction, but benefits are not clear. If we do split, my suggestion is to create two repos (client/server), not three repos (client/server/common)
longer version: At the 11/29 netstack meeting, one of the topics discussed was how to handle monty's suggestion that we split out the client into another repo. >From the 11/29 meeting, here's some background: 22:12:59 <mtaylor> danwent: the problem is that we've got projects that need to depend on this stuff (horizon being a good example)22:13:17 <mtaylor> so we need to be able to reference the actual thing that's needed, wihtout pulling in the entire server22:13:31 <danwent> mtaylor: definitely understand that.. we want to be able to package things separately, definitely22:13:40 <cdub> sounds more like packagin then repo mgmt22:13:51 <danwent> I think the question is why packaging needs to be tied to repo so tightly.22:13:58 <cdub> e.g. simple to build multpile packages from a single repo22:13:59 <danwent> i definitely understand that you have a lot of experience here… :)22:14:17 <danwent> cdub: yes, in fact that is what we do already.22:14:23 <cdub> right22:14:25 <mtaylor> it's for a few reasons22:14:42 <mtaylor> pip requires git lines can't reference sub directories22:15:01 <mtaylor> so you wind up with venv installs that require actual releases to pypi to be consumable22:15:36 <mtaylor> additionally, once you start having submodules in tree, standard tooling winds up being difficult to apply - knowing to interact with a setup.py in trunk is straight forward22:15:50 <mtaylor> knowing that there are 4 setup.py files spread throughout the tree is less obviouos22:16:05 <bhall> as of yesterday all setup.py's are in the root :)22:16:24 <danwent> ok, so the trade-off here is if we want the pip files of other projects to automatically pull "trunk", we need separate repos?22:16:41 <mtaylor> yes22:17:21 <mtaylor> also, it's the model we're using for the other projects, so in keeping with a consistent project-wide approach, I'd kind of want to hear a really good reason that it can't work22:17:28 <bhall> how many repos are we talking about? client, common, server, plugins?22:18:16 <danwent> I'm also worried about trying to keep quantum simple enough that its easy for new people to join and hack on the project22:18:25 <mtaylor> I don't think, as of yet, that we need separate ones for plugins22:18:55 <mtaylor> the most important one in my brain in client - as we have a python-*client project either in existence or coming in to existence for every openstack project22:19:08 <bhall> could we go with two repos? client, common/server/plugins22:19:11 <mtaylor> the common one is just because that's how you've organized your code - and i believe client depends on it, yeah/22:19:18 <mtaylor> does client depend on common?22:19:25 <danwent> I believe that was the original goal22:19:50 <danwent> but perhaps we should revisit that… I'm not sure how much is currently shared.22:19:58 <bhall> I think it does currently22:20:06 <bhall> yeah, I don't think there is a lot of code that it depends on22:20:17 <danwent> mtaylor: so the main goal here is pip install, which are mainly targetted at developers right?22:21:05 <danwent> (I assume non-developers will get everything via packages, which should have dependencies described in terms of packages)22:21:43 <mtaylor> danwent: yes22:22:12 <danwent> ok, thanks for the explanations… I think I understand this better now, but still need to noodle on whether there are ways we can both be happy :)22:22:18 <mtaylor> danwent: there is no thought that people will be doing production releases from pip :)22:22:23 <mtaylor> danwent: awesome An here's some more follow-up from the 12/6 meeting: 22:18:55 <danwent> first, as discussed last week with mtaylor, we need to figure out our strategy for whether we are going to split quantum into multiple repos.22:19:49 <salv> before deciding on a strategy let's decide whether we want to split the repos or not! :)22:20:02 <mtaylor> danwent: re that: we've got openstack.common going now - so perhaps quantum.common can live there?22:20:03 <danwent> we can have the detailed discussion on the ML, but the high-level options seem to be (1) keep as is, all one repo (2) split into two repos, client and server, but potentially duplicate the little bits of shared code and (3) split into three repos, client, server, and common22:20:09 <salv> Splitting in which way exactly? client and server components?22:20:30 <danwent> mtaylor: I was thinking something very similar myself, if we decide to go with #222:20:57 <danwent> salv: up for discussion22:21:14 <danwent> #TODO: #danwent, start ML thread for splitting repos, include #mtaylor as well22:21:17 <bhall> my vote is for #2 (client and "the rest")22:21:19 <mtaylor> #2 with some elements in openstack.common would be the thing that matches the other projects more closely22:21:55 <danwent> if anyone else wants to comment now, go ahead, otherwise i'll forward this content to start a thead on the ML It seems like we have three primary options (feel free to suggest others). #1: do nothing, keep every in single repo #2: split into two repos: client + server, but potentially duplicate some of the shared code currently in "common" #3: split into three repos, client, server and common. A couple thoughts: I'm still not totally clear on the value of splitting the repos. Multiple repos are not required for anyone installing openstack via packages, as we can easily generate separate client/server packages when using a single repo (in fact, we already do). My understanding is that splitting the repos aims to help developers in the case where a project like horizon specifies a dependency on quantum by adding a direct reference to the quantum git repo in pip-requires. So one thing I'd like to better understand is the cost of specifying a quantum git repo that includes client + server, as opposed to one that just includes the client. If its just a matter of pulling down some more bytes, that doesn't seem that painful (the quantum source is tiny anyway). Is there some greater benefit to splitting repos that I am missing? I think the cost of splitting repos is non-trivial, as it makes it harder for developers to get things running from source tree, (again, I'm assuming this is mainly a developer issue, as users should mainly install from packages). It may also mean we need to duplicate some common code, though the need for that may go down if we can get both client + server code bases to have common code in openstack-common. Another important consideration is that it seems that all of the other openstack projects are moving to a model with the client in a separate repo from the service itself. There would definitely a cost both in terms of developer confusion and pain to the infrastructure team to Quantum being an oddball in that department. This alone may be reason enough to follow suit, but I'd love if there was a clear cut reason other than "everyone else is doing it". If we do decide to split repos, my bias would be for #2, which avoid the even more significant complexity of having three repos. While some code in common may need to be duplicated, it may be the case that such code is duplicated because it really belongs in something like openstack-common, which could be a dependency of both. A lof of the code in quantum.common seems to be of that variety (I suspect much of this code is actually only used by the server at this point). The two chunks of common code used by the main client library are: from quantum.common import exceptions from quantum.common.serializer import Serializer Serializer seems to be pretty much all stock code that should be in openstack-common. The quantum client only uses a few of the Exceptions classes in common/exeptions.py. A few of these may need to be duplicated in both the client/server code. Though if we make the changes suggested by Aaron to use only standard HTTP error codes, the applicability of this code to the client decreases to the point of almost no overlap, I suspect. Please chime in with your thoughts. Right now I'm not sure who would work on this, so if you think you have some cycles, please speak up. Dan -- ~~~~~~~~~~~~~~~~~~~~~~~~~~~ Dan Wendlandt Nicira Networks: www.nicira.com twitter: danwendlandt ~~~~~~~~~~~~~~~~~~~~~~~~~~~
-- Mailing list: https://launchpad.net/~netstack Post to : netstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~netstack More help : https://help.launchpad.net/ListHelp
