git-dpm vs gbp-pq: new upstream and patch refresh (long)
tox has a new upstream so I decided to take the opportunity to A/B git-dpm and gbp-pq on a more complicated, but probably common task, simply stated:: upgrade to the new upstream, refresh the patches, handling any conflicts, and regenerate a source package for testing. TL;DR: You can make things work with both workflows, but both are very picky about the order in which you do things. Documentation needs to be better too. I've formed my opinion on a way forward, which I'll describe in a follow up message. == git-dpm == Since tox is still maintained under svn, I wanted to jump start maintenance under git-dpm. For this experiment, I don't care about detailed history. There's no `git-dpm import-dscs --debsnap` so I wrote my own: $ git clone git://git.debian.org/users/barry/import-dscs.git will give you import-dscs.py which will effectively use snapshots to get all the uploaded versions and call `git-dpm import-dsc --ptc` in a loop (with tagging). Then it calls `git-dpm prepare` and the repo should be ready to go. Of course, my script could be doing stupid things. WARNING: import-dscs.py requires an unreleased bug fix to python3-apt, so you'll need to grab the python-apt git repo, build, and install the latest head, otherwise the snapshots will not be applied in the correct order. http://tinyurl.com/mjoms6h (You'll also need packages like pristine-tar, git-dpm, git-buildpackage, and devscripts.) In any case, now I do the following: $ git init toxdpm $ cd toxdpm $ .../import-dscs.py tox Right, so now its time to switch to the new upstream. git-dpm doesn't have a --uscan option, so: $ uscan $ git-dpm import-new-upstream --rebase ../tox_1.7.2.orig.tar.gz There will be merge conflicts, so you need to resolve those manually, calling `git add ` and `git rebase --continue` after each resolution. You may not know exactly how to resolve each conflict, but I don't think the details matter. Once everything's resolved: $ git-dpm update-patches leaves you on the master branch. Now *before* you try to build the source package, you must add a new d/changelog entry, otherwise git-buildpackage will build the old version, and the quilt stack won't apply cleanly. So: $ dch -v1.7.2-1 $ git-buildpackage --git-ignore-new --git-export=WC -S -us -uc (or commit and just `git-buildpackage -S -us -uc`) That gets you a source package, but the binary package FTBFS because one additional test cannot be run during the build process (there's a DEP-8 test for full coverage). Now though, you *must* commit or stash the d/changelog change. Here's where things get a little frustrating. What I need to do is patch a file to add a line that skips a test. I'd like to append that to the 0003- patch but it's not the top of the quilt stack (there's an 0004- patch). But this doesn't seem possible, and `git-dpm apply-patch` doesn't seem to do this. Besides, it's very inconvenient to name the full d/p/*.patch file you want to apply. What I really want is the equivalent of `quilt push`. The best it seems you can do is: $ git-dpm checkout-patched $ git-dpm update-patches The file is patched, but now I have an d/p/0005- file instead of a modified 0003- patch file. Sigh. git-buildpackage && sbuild && adt-run and everything passes. We're done. == gbp pq == gbp-pq seems *much* more finicky about how to get a new upstream with updated patches. If you do not do things in exactly the right order, your new upstream merge will be totally broken. Or, at least they were for me. I had a ton of trial-and-error before I got this incantation correct. git-buildpackage has the bootstrapping step built-in: $ gbp import-dscs --debsnap tox $ gbp pq import $ gbp pq export This renames the d/patches files, and you *must* do this before you try to update to a new upstream your you'll be screwed. If you do it out of order, `gbp pq import` and `gbp pq rebase` will either do nothing, do the wrong thing (e.g. delete d/patches with no renames), or fail (you'll get patch application conflict errors with no way left to resolve them). $ git add debian/patches/ $ git stat On branch master Changes to be committed: (use "git reset HEAD ..." to unstage) renamed:debian/patches/intersphinx_mapping.patch -> debian/patches/0001-intersphinx_mapping.patch renamed:debian/patches/hack-requires.patch -> debian/patches/0002-hack-requires.patch renamed:debian/patches/build-time-tests.patch -> debian/patches/0003-build-time-tests.patch renamed:debian/patches/de-google-ify.patch -> debian/patches/0004-de-google-ify.patch modified: debian/patches/series $ git commit -a -m"patch import/export" Now get the new upstream. $ gbp import-orig --uscan $ dch -v1.7.2-1 "New upstream release." $ git commit -a -m"New upstream release." Now we should be able to rebase the patches onto 1.7.2: $ gbp pq rebase (I largely resolve the merge conflicts in the same way as with git-dpm.) $ gbp pq export $ git
Re: git-dpm vs gbp-pq: new upstream and patch refresh (long)
On Thursday, September 04, 2014 15:40:42 Barry Warsaw wrote: > That gets you a source package, but the binary package FTBFS because one > additional test cannot be run during the build process (there's a DEP-8 test > for full coverage). Now though, you *must* commit or stash the d/changelog > change. > > Here's where things get a little frustrating. What I need to do is patch a > file to add a line that skips a test. I'd like to append that to the 0003- > patch but it's not the top of the quilt stack (there's an 0004- patch). But > this doesn't seem possible, and `git-dpm apply-patch` doesn't seem to do > this. Besides, it's very inconvenient to name the full d/p/*.patch file you > want to apply. What I really want is the equivalent of `quilt push`. > > The best it seems you can do is: > > $ git-dpm checkout-patched > > $ git-dpm update-patches > > The file is patched, but now I have an d/p/0005- file instead of a modified > 0003- patch file. Sigh. I did already run into this with pkg-clamav. See the patch in https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=754052 for one way to do it. Scott K signature.asc Description: This is a digitally signed message part.
Re: git-dpm vs gbp-pq: new upstream and patch refresh (long)
On Thursday, September 04, 2014 16:05:53 Scott Kitterman wrote: > On Thursday, September 04, 2014 15:40:42 Barry Warsaw wrote: > > That gets you a source package, but the binary package FTBFS because one > > additional test cannot be run during the build process (there's a DEP-8 > > test for full coverage). Now though, you *must* commit or stash the > > d/changelog change. > > > > Here's where things get a little frustrating. What I need to do is patch > > a > > file to add a line that skips a test. I'd like to append that to the > > 0003- > > patch but it's not the top of the quilt stack (there's an 0004- patch). > > But this doesn't seem possible, and `git-dpm apply-patch` doesn't seem to > > do this. Besides, it's very inconvenient to name the full d/p/*.patch > > file you want to apply. What I really want is the equivalent of `quilt > > push`. > > > > The best it seems you can do is: > > > > $ git-dpm checkout-patched > > > > $ git-dpm update-patches > > > > The file is patched, but now I have an d/p/0005- file instead of a > > modified > > 0003- patch file. Sigh. > > I did already run into this with pkg-clamav. See the patch in > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=754052 for one way to do > it. > > Scott K Actually, nevermind. That's not the problem you were trying to solve, although you could remove the patch as described and then apply the updated patch at the end of the series. Scott K signature.asc Description: This is a digitally signed message part.
Re: git-dpm vs gbp-pq: new upstream and patch refresh (long)
On Sep 04, 2014, at 04:36 PM, Scott Kitterman wrote: >Actually, nevermind. That's not the problem you were trying to solve, >although you could remove the patch as described and then apply the updated >patch at the end of the series. Yeah, though sometimes for legitimate reasons you can't reorder patches. It vaguely feels like with git-dpm since the patch branch is never pushed, you could "uncommit" (`git reset --hard HEAD^`) and stash each commit until you got to the one you wanted to amend, then unstash and recommit back up the stack. E.g. just like quilt push/pop. -Barry signature.asc Description: PGP signature
Recommendation: adopt git-dpm
I've done enough experimentation to feel confident in my opinion that the team should adopt git-dpm as its git packaging regime. Note that this is just my personal opinion. I look forward to feedback from other team members and interested parties, either for or against my recommendation. I've only considered git-dpm and gbp-pq since the other technologies (e.g. dgit + git-debcherry) don't seem to be viable right now in unstable. git-debcherry in particular does not seem to be part of the released gitpkg package (although it's in upstream's git repo). Both git-dpm and gbp-pq have discomforts, and I would consider contributing to either project to help smooth out some edges. Despite those, git-dpm seems like the best workflow, the easiest to describe and comprehend, and the more robust of the regimes. IMHO, git-dpm's downsides are: its tag names (I prefer them with slashes), some minor missing features (import-dscs command and --uscan option), and the fact that it's written in shell. I'm fairly comfortable with shell hacking, but take a look at git-dpm and tell me that doesn't just scream for a more comprehensible re-implementation in Python 3. :) Even with those complaints, git-dpm feels like the better tool for team package management in git. The problems are minor and probably easily fixable. I look forward to other opinions. Cheers, -Barry signature.asc Description: PGP signature
RE:git-dpm vs gbp-pq: new upstream and patch refresh (long)
> > The file is patched, but now I have an d/p/0005- file instead of a > > modified > > 0003- patch file. Sigh. In this case you can use git rebase -i master edit the commit to merge 0003- and 0005- Cheers Frederic -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/a2a20ec3b8560d408356cac2fc148e53b1f17...@sun-dag3.synchrotron-soleil.fr
Re: git-dpm vs gbp-pq: new upstream and patch refresh (long)
On Thu, 04 Sep 2014, Barry Warsaw wrote: > On Sep 04, 2014, at 04:36 PM, Scott Kitterman wrote: > > >Actually, nevermind. That's not the problem you were trying to solve, > >although you could remove the patch as described and then apply the updated > >patch at the end of the series. > > Yeah, though sometimes for legitimate reasons you can't reorder patches. It > vaguely feels like with git-dpm since the patch branch is never pushed, you > could "uncommit" (`git reset --hard HEAD^`) and stash each commit until you > got to the one you wanted to amend, then unstash and recommit back up the > stack. E.g. just like quilt push/pop. As others have mentionned, you should use "git rebase -i ". This is what you want to use on your patch-queue branch to modifiy individual commits, reorder them, or drop them. Cheers, -- Raphaël Hertzog ◈ Debian Developer Discover the Debian Administrator's Handbook: → http://debian-handbook.info/get/ -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20140904222559.ga7...@x230-buxy.home.ouaza.com
Re: Recommendation: adopt git-dpm
On Thu, 04 Sep 2014, Barry Warsaw wrote: > Even with those complaints, git-dpm feels like the better tool for team > package management in git. The problems are minor and probably easily > fixable. >From my point of view, since you're anyway using features of git-buildpackage, it would be better to improve git-buildpackage... I like how git-dpm can keep patches applied on the packaging branch and porting the required shell to "gbp pq" should not be too complicated. It would also be nice if we could fix "gbp pq" to not rename quilt patches. Cheers, -- Raphaël Hertzog ◈ Debian Developer Discover the Debian Administrator's Handbook: → http://debian-handbook.info/get/ -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20140904223231.gb7...@x230-buxy.home.ouaza.com
Re: git-dpm vs gbp-pq: new upstream and patch refresh (long)
On Sep 05, 2014, at 12:25 AM, Raphael Hertzog wrote: >As others have mentionned, you should use "git rebase -i ". This is >what you want to use on your patch-queue branch to modifiy individual >commits, reorder them, or drop them. Brilliant. For git-dpm then this would be: $ git-dpm checkout-patched $ git rebase -i upstream # *not* master! $ git-dpm update-patches Joy! Cheers, -Barry signature.asc Description: PGP signature
Re: Recommendation: adopt git-dpm
On Sep 05, 2014, at 12:32 AM, Raphael Hertzog wrote: >From my point of view, since you're anyway using features of >git-buildpackage, it would be better to improve git-buildpackage... >I like how git-dpm can keep patches applied on the packaging >branch and porting the required shell to "gbp pq" should not be >too complicated. It would also be nice if we could fix "gbp pq" to not >rename quilt patches. Maybe. What I thankfully left out of my description was the dozen gbp-pq directories I had to trash when I tried to do something that *seemed* to make sense but ended up just borking my directory. Sometimes I could chuck the changes and try again, but more often than not, I just rmdir'd and started over. Even after getting it right (and updating my draft), I still don't feel like gbp-pq is very resilient. I wrote the git-dpm bits after just one or two such experiments. Either way, I think we need better focused task-oriented documentation! Cheers, -Barry -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20140904184310.0c4a5...@anarchist.wooz.org
