I'd suggest to reduce number of Murano repositories for several reasons: * All other OpenStack projects have a single repo per project. While this point might look like something not worth mentioning, it's really important: - unified project structure simplifies life for new developers. once they get familiar with one project, they can expect something similar from another project - unified project structure simplifies life for deployers. similar project structure simplifies packaging and deployment automation
* Another important reason is to simplify gated testing. Just take a look at Solum layout [1], they have everything needed (contrib, functionaltests) to run dvsm job in a single repo. One simple job definition [2] allows to install Solum in DevStack and run Tempest tests for Solum. * As a side-effect, this approach will improve integrity of project components. Having murano-common in the same repo with other components will help to catch integration issues earlier. In an ideal world there will be only the following repos: - murano (api, common, conductor, docs, repository, tests) - python-muranoclient - murano-dashboard - murano-agent - puppet-murano (optional, but nice to have) [1] https://github.com/stackforge/solum [2] https://github.com/openstack-infra/config/blob/master/modules/openstack_project/files/jenkins_job_builder/config/solum.yaml Thanks, Ruslan On Thu, Feb 6, 2014 at 3:14 PM, Serg Melikyan <smelik...@mirantis.com>wrote: > Hi, Alexander, > > In general I am completely agree with Clint and Robert, and as one of > contributors of Murano I don't see any practical reasons for repositories > reorganization. And regarding of your proposal I have a few thoughts that I > would like to share below: > > >This enourmous amount of repositories adds too much infrustructural > complexity > Creating a new repository is a quick, easy and completely automated > procedure that requires only simple commit to Zuul configuration. All > infrastructure related to repositories is handled by Openstack CI and > supported by Openstack Infra Team, and actually don't require anything from > project development team. About what infrastructure complexity you are > talking about? > > >I actually think keeping them separate is a great way to make sure you > have ongoing API stability. (c) Clint > I would like to share a little statistic gathered by Stan Lagun > a little time ago regarding repositories count in different PaaS solution. > If you are concerned about large number of repositories used by Murano, you > will be quite amused: > > - https://github.com/heroku - 275 > - https://github.com/cloudfoundry - 132 > - https://github.com/openshift - 49 > - https://github.com/CloudifySource - 46 > > >First of all, I would suggest to have a single reposository for all the > three main components of Murano: main murano API (the contents of the > present), workflow execution engine (currently murano-conductor; also it > was suggested to rename the component itself to murano-engine for more > consistent naming) and metadata repository (currently murano-repository). > > *murano-api* and *murano-repository* have many things in common, they are > both present HTTP API to the user, and I hope would be rewritten to common > framework (Pecan?). But *murano-conductor* have only one thing in common > with other two components: code shared under *murano-common*. That > repository may be eventually eliminated by moving to Oslo (as it should be > done). > > >Also, it has been suggested to move our agents (both windows and unified > python) into the main repository as well - just to put them into a separate > subfolder. I don't see any reasons why they should be separated from core > Murano: I don't believe we are going to have any third-party > implementations of our "Unified agent" proposals, while this possibility > was the main reason for separatinng them. > > Main reason for murano-agent to have separate repository was not a > possibility to have another implementation, but that all sources that > should be able to be built as package, have tests and can be uploaded to > PyPI (or any other gate job) should be placed in different repository. > OpenStack CI have several rules regarding how repositories should be > organized to support running different gate jobs. For example, to run tests > *tox.ini* is need to be present in root directory, to build package > *setup.py* should be present in root directory. So we could not simply > move them to separate directories in main repository and have same > capabilities as in separate repository. > > >Next, deployment scripts and auto-generated docs: are there reasons why > they should be in their own repositories, instead of "docs" and > "tools/deployment" folders of the primary repo? I would prefer the latter: > docs and deployment scripts have no meaning without the sources which they > document/deploy - so it is better to have them consistent. > We have *developers documentation* alongside with all sources: > murano-conductor<https://github.com/stackforge/murano-conductor/tree/master/doc/source>, > murano-api<https://github.com/stackforge/murano-api/tree/master/doc/source> > and > so on. It is true that we have not so much documentation there, and not > much code is documented to add auto-generated documentation. Documentation > that is found in *murano-docs* repository actually is a docbook > documentation, that is presented in book manner, and follows documentation > patterns found in core projects itself: > openstack-manuals<https://github.com/openstack/openstack-manuals/tree/master/doc> > . > > *murano-deployment* contains scripts and other artefacts related to > deployment, but not necessary to source code. This repository don't use > much of CI capabilities, but raise it is logical place where we can place > different thing related to deployment: various scripts, specs, patches and > so on. Also with separate repository we can not to spam our deployment > engineers with software engineers related commits. > > > > On Tue, Jan 21, 2014 at 11:55 PM, Alexander Tivelkov < > ativel...@mirantis.com> wrote: > >> Hi folks, >> >> As we are moving towards incubation application, I took a closer look at >> what is going on with our repositories. >> An here is what I found. We currently have 11 repositories at stackforge: >> >> - murano-api >> - murano-conductor >> - murano-repository >> - murano-dashboard >> - murano-common >> - python-muranoclient >> - murano-metadataclient >> - murano-agent >> - murano-docs >> - murano-tests >> - murano-deployment >> >> This enourmous amount of repositories adds too much infrustructural >> complexity, and maintaining the changes in in consistent and reliable >> manner becomes a really tricky tasks. We often have changes which require >> modifing two or more repositories - and thus we have to make several >> changesets in gerrit, targeting different repositories. Quite often the >> dependencies between these changesets are not obvious, the patches get >> reviewed and approved on wrong order (yes, this also questions the quality >> of the code review, but that is a different topic), which causes in >> inconsostent state of the repositories. >> >> Well, anyway, this has to be changed in some way or another. >> I suggest to initiate the discussions on how to do all this. >> >> Below you may find my position. This is not final in any meaning, just a >> proposal. Please, feel free to discuss :) >> >> First of all, I would suggest to have a single reposository for all the >> three main components of Murano: main murano API (the contents of the >> present), workflow execution engine (currently murano-conductor; also it >> was suggested to rename the component itself to murano-engine for more >> consistent naming) and metadata repository (currently murano-repository). >> These should remain as independent modules, being able to run as >> different daemons, but stored within a single repository (similar to how >> heat has heat-api, heat-cfn and heat-engine under the same hood). The name >> of this repository is tentative: I think none of the existing match, so I >> would suggest to create a new repo (simple "murano" seems to fit the best), >> and then relocate all the content from other 3 repos and remove them >> aftwerwards. >> >> When the api, the repository and the engine are merged into a single >> repo, there will be no sense in having murano-common repo for storing their >> common classes: instead, there should be a "common" package inside the main >> murano repository. >> >> Also, it has been suggested to move our agents (both windows and unified >> python) into the main repository as well - just to put them into a separate >> subfolder. I don't see any reasons why they should be separated from core >> Murano: I don't believe we are going to have any third-party >> implementations of our "Unified agent" proposals, while this possibility >> was the main reason for separatinng them. >> >> Next, deployment scripts and auto-generated docs: are there reasons why >> they should be in their own repositories, instead of "docs" and >> "tools/deployment" folders of the primary repo? I would prefer the latter: >> docs and deployment scripts have no meaning without the sources which they >> document/deploy - so it is better to have them consistent. >> >> >> Then, the python bindings: "There can be only one" (c). Yes, the metadata >> API and the main murano API are different indeed, however there is no >> reason in having two repositories for their clients: let's have a single >> repo, containing two packages inside. Are there any technical reasons >> preventing us from doing that? >> CLI should be common as well - I think there should be a single >> command-line utility ("murano" should be the name), allowing to query both >> APIs. This CLI will eventually evolve into the developer's utility: it will >> get commands to package, sign and submit application packages. >> >> Openstack Dashboard plugin - aka Murano-dashboard - should remain in a >> separated repo, I have no objections here :) >> >> murano-tests may reamin independent as well - however, this repository is >> not likely to be transferred when we go to incubation: incubated projects >> should have tempest test in their repositories, shoudn't they? Our our test >> may remain on stackforge - this is irrelevant. >> >> And finally, we need some place to store sources of our metadata objects: >> the definition of core murano library, as well as example services, with >> all their stuff - metadata and ui definitions, heat templates, scripts >> etc. Here I propose to create a new repo, specially dedicated for this >> purpose. If we succeed in building the ecosystem for application developers >> and publishers, this will be the repo in which they should contribute, >> while the core murano repo's will remain relativele stable. >> >> >> So, this brings us to the following list of repos: >> >> - *murano* - main services, common, agents docs, deployments scripts >> - *python-muranoclient* - python bindings and CLI >> - *murano-dashboard* - OS Dashboard plugin >> - *murano-apps* - new repo for metadata, including core library and >> example apps. >> - *murano-tests* - existing test-repo, not going to be transferred >> when incubated. >> >> >> This leaves us with just 4 repositories (plus one additional which will >> remain on stackforge), with clear separation of concerns. >> >> There may be technical issues in doing this mergement (we do not want to >> loose revision history, do we?), but they should be solvable (I'll write to >> infra asking on what is possible and what is not), but in general this is >> the direction in which we should be moving, as it seems to me. >> >> -- >> Regards, >> Alexander Tivelkov >> >> _______________________________________________ >> OpenStack-dev mailing list >> OpenStack-dev@lists.openstack.org >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> > > > -- > Serg Melikyan, Senior Software Engineer at Mirantis, Inc. > http://mirantis.com | smelik...@mirantis.com > > +7 (495) 640-4904, 0261 > +7 (903) 156-0836 > > _______________________________________________ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > >
_______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
