Is it possible to generate a MANIFEST.in file using setuptools-git? Would that solve simplicity and efficiency?
Sent from my iPhone On Dec 4, 2012, at 9:07 PM, "Thierry Carrez" <thie...@openstack.org> wrote: > Sascha Peilicke wrote: >> Currently, the majority of OpenStack components make use of the >> Python module "setuptools_git" in order to install additional >> configuration files. This is basically the same functionality that >> the MANIFEST.in file (setuptools/distribute) provides, but >> automatic. > > Note: This is a development topic, it should (have) be(en) posted to > openstack-dev to reach the appropriate audience. Please follow-up there. > >> However, we recently discovered that this approach has issues from >> a packaging perspective. We weren't getting all the data/config >> files that the python package usually gets even though we were >> running the same commands: >> >> $ python setup.py build >> >> followed by: >> >> $ python setup.py install --skip-build >> >> We are building RPM packages from release tarballs (such as [1]), >> which of course don't include the .git directory. Therefore the >> setuptools_git approach can't do its magic, thus our package builds >> get wrong results. Having OpenStack components rely on >> setuptools_git at build time means we have to distribute the whole >> git repository along with the source code tarball. Of course this >> makes no sense, since it would increase the size of release >> tarballs dramatically and won't get shipped in distributions >> anyway.Therefore, we (and potentially other distribution vendors) >> would have to track these files manually in our RPM spec files. >> Some reviews have already been opened on the topic (admittedly >> before we discovered the real issue). Given the different outcome >> of each review it seems that not everybody is aware that >> setuptools_git is used or of what it does. >> >> https://review.openstack.org/#/c/17122/ (ceilometer) - this one >> got accepted before we knew what was going on >> >> https://review.openstack.org/#/c/17347/ (cinder) - abandoned until >> the situation is clarified >> >> https://review.openstack.org/#/c/17355/ (nova) - rejected >> >> So the better solution would be to stop using setuptools_git and >> just include all the data/config files that are meant to be >> distributed in the MANIFEST.in file. This is what every Python >> developer should know about and has the benefit of increased >> transparency about what gets installed and what not. We created a >> blueprint to track this [2]. >> >> Thoughts? > > A bit of history here: > > We used to rely on MANIFEST.in to list all files, but people routinely > forgot to add new files to it. Apparently "every Python developer" > doesn't know (or care) about this. The end result was that we > discovered very late (sometimes after the release itself) that we > built incomplete tarballs. As a quick search[1] shows, I have > personally filed 27 bugs so far on the subject, so it's not a corner case. > > [1] http://bit.ly/TDim7U > > Relying on setuptools_git instead allows to avoid that issue > altogether. The projects that adopted it became a non-issue. The > projects that didn't adopt it yet are still a problem. I was about to > push setuptools_git support to projects that don't use it yet. > > In summary, I would hate it if we went back to the previous situation. > I'm not personally attached to setuptools_git, but any proposed > replacement solution should keep its simplicity. > > -- > Thierry Carrez (ttx) > Release Manager, OpenStack > > _______________________________________________ > Mailing list: https://launchpad.net/~openstack > Post to : openstack@lists.launchpad.net > Unsubscribe : https://launchpad.net/~openstack > More help : https://help.launchpad.net/ListHelp _______________________________________________ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
