Re: Python 3 Statsmodels & Pandas
Hi, On Mon, Sep 18, 2017 at 05:48:48PM +1000, Stuart Prescott wrote: > On Sunday, 17 September 2017 22:14:18 AEST Diane Trout wrote: > > I just did it that way because it was the least disruptive change I > > could make that would let me build and test the package. > > Sure, that's entirely sensible. > > > In my experience I'm much more likely to to notice a build time test > > failure than one from the CI system. > > Absolutely agreed. I'm thinking that this is a rather exceptional situation > because of the circular dependency and the fall-out that we regularly see > from > that. > > > What do other people think? If there are autopkgtests, should you still > > let dh_auto_test run tests? > > (I know I'm not the 'other people' from whom you solicit replies, I was just > perhaps unclear in what I was musing on before so I want to be more clear now) > > Like you, I would *normally* run all tests in both places: buildd tests catch > severe problems right now; ci.d.n reruns the tests each time new versions of > dependencies are uploaded too and later breakage is caught. > > Perhaps this is not a normal situation, however. Either one of "circular > dependencies" or "packages that often FTBFS on one or more architectures" > would be unusual enough to justify doing something different. All I am > wondering (from my position of ignorance!) if in this case, perhaps the tests > that cause the circular dependency can be disabled or xfailed, with the > remaining tests run as normal. If my poor opinion counts: For the moment we should run those tests in the build process than can be easily be run. Everything else should probably be sorted out later (in autopkgtest or another later upload if somebody has a clue how we can solve the circular depenendecies). We somehow need to get some working spatstats to continue with other packages. Thanks to all who contributed Andreas. -- http://fam-tille.de
Re: Python 3 Statsmodels & Pandas
On 21/09/17 15:22, Andreas Tille wrote: We somehow need to get some working spatstats to continue with other packages. I second Andreas' opinion. In principle, we'd want all tests to run at build and integration times and we might achieve that at some point during the Buster cycle. Right now, this situation with pandas and statsmodels is slowing work on other scientific packages down significantly. Cheers, Ghis
Re: Python 3 Statsmodels & Pandas
> If my poor opinion counts: For the moment we should run those tests > in > the build process than can be easily be run. Everything else should > probably be sorted out later (in autopkgtest or another later upload > if > somebody has a clue how we can solve the circular depenendecies). > > We somehow need to get some working spatstats to continue with other > packages. > Status: [X] Pandas builds with nocheck, nodoc [X] Statsmodels builds with Python 3 using above pandas [X] Pandas tests pass with statsmodels for Python 2 & 3 installed. [ ] Pandas builds docs with statsmodels installed My most recent build error was about pandoc not being available. Unfortunately the tests take a long time enough that I can write this email before I know if adding pandoc fixed the problem. dh_auto_tests run Python 2.7, 3.5, 3.6 tests, and then autopkgtests runs them again. I posted the larger fixes to pandas I've done to the appropriate bugs #875807 python3-pandas FTBFS: 3 timezone unit tests fail #875805 python3-pandas: Please break circular dependency There's a few more minor patches on my laptop that I haven't attached to a bug for pandas. * Updating standards version * using debhelper 10 * switching sphinx doc build to use python3 * and deleting a few more build files in dh_clean target. I made larger changes to statsmodels, by using pybuild instead of the previous multiple targets in debian/rules. All of those changes are currently on alioth in detrout-python3. When all these tests pass, shall I add myself to uploaders and release? or does someone else want to review first? Diane
Re: Python 3 Statsmodels & Pandas
On Thu, 2017-09-21 at 17:56 -0400, Yaroslav Halchenko wrote: > If you could allow to review would be great. > Thanks for all the work. > I was btw also trying to build with the patch you shared yesterday Once I have all the changes for pandas would you like me to put them on a branch on alioth? Or should I send them via format-patch somewhere? Also it looked to me like you were changing debian/changelog with each change? (Some people just use the git log and then apply all the updates to d/changelog in one go with git buildpackage. Diane
Re: Python 3 Statsmodels & Pandas
If you could allow to review would be great. Thanks for all the work. I was btw also trying to build with the patch you shared yesterday On September 21, 2017 5:48:58 PM EDT, Diane Trout wrote: > >> If my poor opinion counts: For the moment we should run those tests >> in >> the build process than can be easily be run. Everything else should >> probably be sorted out later (in autopkgtest or another later upload >> if >> somebody has a clue how we can solve the circular depenendecies). >> >> We somehow need to get some working spatstats to continue with other >> packages. >> > >Status: > >[X] Pandas builds with nocheck, nodoc >[X] Statsmodels builds with Python 3 using above pandas >[X] Pandas tests pass with statsmodels for Python 2 & 3 installed. >[ ] Pandas builds docs with statsmodels installed > >My most recent build error was about pandoc not being available. > >Unfortunately the tests take a long time enough that I can write this >email before I know if adding pandoc fixed the problem. > >dh_auto_tests run Python 2.7, 3.5, 3.6 tests, and then autopkgtests >runs them again. > >I posted the larger fixes to pandas I've done to the appropriate bugs > >#875807 python3-pandas FTBFS: 3 timezone unit tests fail >#875805 python3-pandas: Please break circular dependency > >There's a few more minor patches on my laptop that I haven't attached >to a bug for pandas. > >* Updating standards version >* using debhelper 10 >* switching sphinx doc build to use python3 >* and deleting a few more build files in dh_clean target. > >I made larger changes to statsmodels, by using pybuild instead of the >previous multiple targets in debian/rules. > >All of those changes are currently on alioth in detrout-python3. > >When all these tests pass, shall I add myself to uploaders and release? >or does someone else want to review first? > >Diane -- Sent from a phone which beats iPhone.
Re: Python 3 Statsmodels & Pandas
On Thu, 21 Sep 2017, Diane Trout wrote: > On Thu, 2017-09-21 at 17:56 -0400, Yaroslav Halchenko wrote: > > If you could allow to review would be great. > > Thanks for all the work. > > I was btw also trying to build with the patch you shared yesterday > Once I have all the changes for pandas would you like me to put them on > a branch on alioth? Or should I send them via format-patch somewhere? I could work with either. I will upload now -2 (sorry if we duplicated some efforts) with following changes (diff for changelog): diff -Nru pandas-0.20.3/debian/changelog pandas-0.20.3/debian/changelog --- pandas-0.20.3/debian/changelog 2017-07-10 20:00:59.0 -0400 +++ pandas-0.20.3/debian/changelog 2017-09-21 16:11:29.0 -0400 @@ -1,3 +1,14 @@ +pandas (0.20.3-2) unstable; urgency=medium + + * debian/control +- boosted policy to 4.0.0 (I think we should be ok) +- drop statsmodels from build-depends to altogether avoid the circular + build-depends (Closes: #875805) + * Diane Trout: +- Add dateutil-2.6.1-fixed-ambiguous-tz-dst-be.patch (Closes: #875807) + + -- Yaroslav Halchenko Thu, 21 Sep 2017 16:11:29 -0400 I could also add you to pkg-exppsy team under which we still have the "active" vcs for pandas. > Also it looked to me like you were changing debian/changelog with each > change? (Some people just use the git log and then apply all the > updates to d/changelog in one go with git buildpackage. I prefer indeed reflecting each change in a changelog with a commit and using debcommit to commit it ;) -- Yaroslav O. Halchenko Center for Open Neuroscience http://centerforopenneuroscience.org Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755 Phone: +1 (603) 646-9834 Fax: +1 (603) 646-1419 WWW: http://www.linkedin.com/in/yarik
Re: Python 3 Statsmodels & Pandas
I managed to merge the more important doc changes. I have a patch to switch doc building to using Python 3 components, as there's a goal of removing the Python 2 components at some point. The there's the patch for the nodoc/nocheck build profile, as well as adding a bunch of other dependencies for doc building. After more testing I'll try posting them. But it probably wont be until tomorrow. diff -Nru pandas-0.20.3/debian/changelog pandas- > 0.20.3/debian/changelog > --- pandas-0.20.3/debian/changelog 2017-07-10 20:00:59.0 > -0400 > +++ pandas-0.20.3/debian/changelog 2017-09-21 16:11:29.0 > -0400 > @@ -1,3 +1,14 @@ > +pandas (0.20.3-2) unstable; urgency=medium > + > + * debian/control > +- boosted policy to 4.0.0 (I think we should be ok) > +- drop statsmodels from build-depends to altogether avoid the > circular While I was looking at the policy updating checklist in 4.0.1 they renamed XS-Testsuite to Testsuite. I think that's the only change I noticed. The version of debhelper and debian/compat is also pretty old. > > I could also add you to pkg-exppsy team under which we still have the > "active" > vcs for pandas. Thank you that sounds useful. Diane > >
Re: Python 3 Statsmodels & Pandas
Hi Yaroslav, On Thu, Sep 21, 2017 at 10:24:53PM -0400, Yaroslav Halchenko wrote: > On Thu, 21 Sep 2017, Diane Trout wrote: > > > On Thu, 2017-09-21 at 17:56 -0400, Yaroslav Halchenko wrote: > > > If you could allow to review would be great. > > > Thanks for all the work. > > > I was btw also trying to build with the patch you shared yesterday > > > Once I have all the changes for pandas would you like me to put them on > > a branch on alioth? Or should I send them via format-patch somewhere? > > I could work with either. I'd consider pushing a branch is the typical approach a team should work with Git. > I will upload now -2 (sorry if we duplicated some efforts) with > following changes (diff for changelog): > > diff -Nru pandas-0.20.3/debian/changelog pandas-0.20.3/debian/changelog > --- pandas-0.20.3/debian/changelog 2017-07-10 20:00:59.0 -0400 > +++ pandas-0.20.3/debian/changelog 2017-09-21 16:11:29.0 -0400 > @@ -1,3 +1,14 @@ > +pandas (0.20.3-2) unstable; urgency=medium > + > + * debian/control > +- boosted policy to 4.0.0 (I think we should be ok) Current is 4.1.0 (despite lintian claims 4.0.0) > I could also add you to pkg-exppsy team under which we still have the "active" > vcs for pandas. I would prefer to move pandas to Debian Science or Debian Python. I fail to see the specific use in NeuroDebian field. Kind regards Andreas. -- http://fam-tille.de
