Re: dpkg armhf patch acceptance status?

[Riku Voipio]

On Fri, Feb 18, 2011 at 04:27:52AM +, Luke Kenneth Casson Leighton wrote:
>  precisely.  this is another, (clearer or at least different) way of
> stating what i've been advocating.  by having such a delta-maintaining
> tool, complex sets of deltas can be maintained indefinitely, or in
> fact completely reworked.

We dont *want* to maintain deltas. If we start maintaining deltas, we are no
longer Debian, we are a fork. Lets end this discussion now.


-- 
To UNSUBSCRIBE, email to debian-dpkg-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20110218095330.ga25...@afflict.kos.to

Re: Bug#594179: dpkg armhf patch acceptance status?

[Guillem Jover]

[ CCing Matthias, as I'd like your opinion on my proposed solution
  involving some Debian gcc changes. ]

Hi!

On Thu, 2011-02-17 at 12:27:30 +0100, Loïc Minier wrote:
>  Trying to kick the dust a bit as having the triplet "in the air" is
>  kind of an unhappy situation for armhf   :-)

I think it's fair to assume several of you have already peeked into the
notes I circulated on IRC the other day, :) I'm reincluding them here
although slightly redacted and extended, for the benefit of the rest
of the people. I'll still reply to specific points you rised below.


GNU triplets


* gcc upstream claims GNU triplets do not fully encode ABI:
  - Thus cannot be used properly to map deb arch ←→ GNU triplet.
  - Thus cannot be used for multiarch paths.
  - Thus cannot be used for default cross-toolchains sharing GNU triplet.
  - Thus we'd need yet another ABI encoding and mapping for multiarch
and dpkg arches, instead of being able to just use GNU triplets.
The one being currently proposed can be found at
.

* But the GNU triplets have always encoded the ABI, it's even
  extremely explicit in some cases, for example with the arm EABI,
  executable formats (elf, aout, ecoff, etc) or stuff like
  powerpc-*-eabispe* or powerpc-*-eabialtivec* (supported by gcc
  upstream).

* And we already use GNU triplets for powerpcspe (powerpc-linux-gnuspe)
  and lpia (i386-linux-gnulp) arches, and these do not need gcc upstream
  support as are handled transparently by the -gnu* patterns, and any
  differing ABI options are selected by the Debian gcc packaging.

* There's other gcc options which can change the ABI of the default
  compiler besides the --with-float one, like --with-long-double-128,
  etc, and still they are not a problem mainly for two reasons:

  1) In case they do not need to coexist, they can be handled like
 other transitions, like it was done in Debian, by renaming
 shared library packages or with versioned symbols and handling
 both ABIs at the same time.

  2) No one seems to have needed to create coexisting default toolchains
 with diverging ABI options at the same time until now, or they have
 done so privately.

* I think it's important to be able to consistently support co-installing
  different default cross-compilers for different ABIs which would
  otherwise share the same triplet. And while we could avoid the problem
  for multiarch libraries co-installations by using the multiarch tuples,
  we would not be able to avoid it for the cross-compilers.

* The assumption that each GNU triplet denotes a different ABI is so
  entrenched in the GNU build system, that we have things like the
  following all over the place to properly support cross-building:

  ,---
  ifeq ($(DEB_BUILD_GNU_TYPE), $(DEB_HOST_GNU_TYPE))
confflags += --build $(DEB_HOST_GNU_TYPE)
  else
confflags += --build $(DEB_BUILD_GNU_TYPE) --host $(DEB_HOST_GNU_TYPE)
  endif
  `---

  This would not hold true any longer if we didn't use a unique triplet
  per arch, and would thus break in quite annoying ways. For further
  details why that's needed check the autotools-dev section
  “Calling GNU configure properly” in its README.Debian.

* Ignoring the bi-directional 1:1 mapping would be a PITA, as it implies
  not being able to automatically bootstrap dpkg on a new port or
  dpkg package builds on foreign distros. The former is annoying but not
  done frequently, the latter is going to be a problem for each non-dpkg
  based distribution, as they'll have to bootstrap dpkg on each arch
  where dpkg is built anew. It ends up being a matter of off-loading the
  knowledge of the arch and build system from the dpkg/gcc combo to the
  porters/maintainers, which seems rather unappealing.

* Other packaging systems seem to not have made quite the same assumption
  about the GNU triplet as dpkg/Debian does. Gentoo's portage at least
  relies on the GNU triplet being unique per different arch as itseems to
  be using it on the path somehow. rpm and conary for example use uname
  which is not enough by itself, and quite unsatisfactory as it's lacking
  quite a bit of the ABI information, which probably has not presented as
  a problem for them as they do not have the amount of supported arches
  as Debian does, neither do they seem to have the same amount of
  mixes/convinations we do. So the other approaches seem to be actually
  inferior.

* The remaining problem at least for multiarch is the use of more
  specialized cpu names for the i386 triplets, i486-linux-gnu on Debian,
  which might change depending on the base instruction set to support,
  for example i686-linux-gnu on Ubuntu.

  Due to #611741, it already crossed my mind (but removed it from the
  list of solutions before sending the reply as at the time it seemed
  slighly preposterous just to fix the divergence with Ubuntu) that we
  should switch back our i386 arch to use i386 as the base cpu name

Cross-upgrading packages with multiarch packages

[Raphael Hertzog]

Hello,

I'd like to seek feedback about what kind of upgrades dpkg should support
with multi-arch packages.

dpkg treats foo:native and foo:all like the same package and it's thus
possible to upgrade foo_1.0_all to foo_2.0_ and vice-versa.

However if you have installed foo_1.0_, you can't upgrade that
package to foo_2.0_all (and vice-versa). Same goes for foo_1.0_ to
foo_1.0_ (provided they are not multi-arch: same, otherwise they
could coexist). You have to remove the conflicting package first and
reinstall afterwards.

In a similar vein, if you have several instances of the same package
(Multi-Arch: same, say foo_1.0_ and foo_1.0_), you can't
install foo_2.0_all because it's in conflict with foo_1.0_.


Is this behaviour what we want?

Cheers,
-- 
Raphaël Hertzog ◈ Debian Developer

Follow my Debian News ▶ http://RaphaelHertzog.com (English)
  ▶ http://RaphaelHertzog.fr (Français)


-- 
To UNSUBSCRIBE, email to debian-dpkg-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20110218113118.ga7...@rivendell.home.ouaza.com

Re: Bug#594179: dpkg armhf patch acceptance status?

[Matthias Klose]

On 18.02.2011 11:13, Guillem Jover wrote:

[ CCing Matthias, as I'd like your opinion on my proposed solution
   involving some Debian gcc changes. ]


The armhf patch for gcc looks ok, however I would like to see this better 
addressed in Linaro and/or upstream.



  Yes but x86 goes to the other extreme, with separate triplets for
  compatible ABIs ({i386,i486,i586,i686}-linux-gnu); the fact is the
  triplet is not a good ABI specifier.


Sure, but as mentioned above, I'm now convinced the correct solution is
to switch back to i386-linux-gnu.


No, this doesn't work. the triplet currently is set to i586-linux-gnu. Switching 
back to to i386 would loose the sync primitives and a not working gomp.  There 
is too much relying on >= i586 in many configure's of other packages.  Not only 
for the good, as the switch in Ubuntu to i686 did show, because many configure 
files assume sse with i686-linux-gnu.


  Matthias


--
To UNSUBSCRIBE, email to debian-dpkg-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/4d5e665b.1060...@debian.org

dpkg fails on low-memory-hardware

[Martin Klaiber]

Hello everybody, this is my first post to this list.

Some time ago I wanted to upgrade an old laptop (Toshiba T4900CT,
from 1994) from Debian Etch to Debian Lenny. Unfortunately dpkg
(resp. apt-get or aptitude) was running out of memory. The maximum
amount of RAM for this laptop is limited to 40 MB.

I use this laptop for daily work for many years now. The limited
memory is no problem for this purpose. All programs I need, run fine
with this amount of RAM (jed, latex, tin, mutt, fbi, fbgs, links).

The only problem seems to be dpkg or associated programs, like
aptitude oder apt-get. Strangely enough, the laptop has 256 MB of
swap-space which dpkg (aptitude, apt-get) seem to ignore. Other
programs use swap, so it should be no general problem.

I know this laptop is old hardware and modern hardware is cheap. But
I got used to this machine so much and I would really love to keep it
alive. Plus I would like to continue using Debian. I tried NetBSD on
it. It works fine but I prefer Debian.

So could someone tell me please, how I could use dpkg with max. 40 MB
of RAM or how I could tell dpkg to use swap-space?

TIA, Martin


-- 
To UNSUBSCRIBE, email to debian-dpkg-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/20110218132419.ga3...@martinkl.dialup.fu-berlin.de

Re: dpkg fails on low-memory-hardware

[Raphael Hertzog]

Hi,

On Fri, 18 Feb 2011, Martin Klaiber wrote:
> Hello everybody, this is my first post to this list.
> 
> Some time ago I wanted to upgrade an old laptop (Toshiba T4900CT,
> from 1994) from Debian Etch to Debian Lenny. Unfortunately dpkg
> (resp. apt-get or aptitude) was running out of memory. The maximum
> amount of RAM for this laptop is limited to 40 MB.

dpkg uses an amount of RAM proportional to the number of packages
installed, but apt-get/aptitude use an amount of RAM proportional to the
number of available packages.

And the number of available packages has grown as usual. So it's not
unexpected if apt-get uses all the memory.

Are you sure dpkg alone does not work?

> The only problem seems to be dpkg or associated programs, like
> aptitude oder apt-get. Strangely enough, the laptop has 256 MB of
> swap-space which dpkg (aptitude, apt-get) seem to ignore. Other
> programs use swap, so it should be no general problem.

How do you know that they are not using swap ?

I don't know of any reason why dpkg/apt-get would not use swap.

Cheers,
-- 
Raphaël Hertzog ◈ Debian Developer

Follow my Debian News ▶ http://RaphaelHertzog.com (English)
  ▶ http://RaphaelHertzog.fr (Français)


-- 
To UNSUBSCRIBE, email to debian-dpkg-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20110218172611.gb31...@rivendell.home.ouaza.com

Re: Bug#594179: dpkg armhf patch acceptance status?

[Steve Langasek]

On Fri, Feb 18, 2011 at 01:30:19PM +0100, Matthias Klose wrote:
> On 18.02.2011 11:13, Guillem Jover wrote:
> >[ CCing Matthias, as I'd like your opinion on my proposed solution
> >   involving some Debian gcc changes. ]

> The armhf patch for gcc looks ok, however I would like to see this
> better addressed in Linaro and/or upstream.

I'm not sure how Linaro could better address this, short of persuading
upstream to allocate a separate triplet for armhf - which has been
explicitly refused on the upstream mailing list.  Do you have something else
in mind here?

-- 
Steve Langasek   Give me a lever long enough and a Free OS
Debian Developer   to set it on, and I can move the world.
Ubuntu Developerhttp://www.debian.org/
slanga...@ubuntu.com vor...@debian.org


signature.asc
Description: Digital signature

Re: Cross-upgrading packages with multiarch packages

[Steve Langasek]

On Fri, Feb 18, 2011 at 12:31:18PM +0100, Raphael Hertzog wrote:

> I'd like to seek feedback about what kind of upgrades dpkg should support
> with multi-arch packages.

> dpkg treats foo:native and foo:all like the same package and it's thus
> possible to upgrade foo_1.0_all to foo_2.0_ and vice-versa.

> However if you have installed foo_1.0_, you can't upgrade that
> package to foo_2.0_all (and vice-versa). Same goes for foo_1.0_ to
> foo_1.0_ (provided they are not multi-arch: same, otherwise they
> could coexist). You have to remove the conflicting package first and
> reinstall afterwards.

> In a similar vein, if you have several instances of the same package
> (Multi-Arch: same, say foo_1.0_ and foo_1.0_), you can't
> install foo_2.0_all because it's in conflict with foo_1.0_.

> Is this behaviour what we want?

I think that actually sounds entirely reasonable.  If you have
foo_1.0_native, foo_1.0_foreign installed, and you try to install
foo_2.0_all, which package is it an upgrade of?  I don't think it makes
sense from dpkg's POV to be a simultaneous upgrade of *both* packages.  So
it's an upgrade of foo_1.0_native, and should be handled the same way as you
would handle an attempt to upgrade to foo_2.0_native without also upgrading
to foo_2.0_foreign.

Higher-level package managers should detect this and do the needful. 
foo_1.0_foreign should be removed as part of the upgrade, solving the
conflict and allowing foo_2.0_all to be configured.

So yeah, that seems ok to me, but I guess you're not convinced or you
wouldn't have asked. :)  What other way do you see this working?  Should
dpkg auto-remove multiarch packages when an upgrade to _all is requested?
That seems very inconsistent with dpkg's "do what I say" approach, IMHO.

Cheers,
-- 
Steve Langasek   Give me a lever long enough and a Free OS
Debian Developer   to set it on, and I can move the world.
Ubuntu Developerhttp://www.debian.org/
slanga...@ubuntu.com vor...@debian.org


signature.asc
Description: Digital signature

Re: Cross-upgrading packages with multiarch packages

[Raphael Hertzog]

On Fri, 18 Feb 2011, Steve Langasek wrote:
> So yeah, that seems ok to me, but I guess you're not convinced or you
> wouldn't have asked. :)  What other way do you see this working?  Should
> dpkg auto-remove multiarch packages when an upgrade to _all is requested?
> That seems very inconsistent with dpkg's "do what I say" approach, IMHO.

Well, from the implementer point of view, this behaviour is what works
with the least modification to dpkg. So I'm happy if we can agree that
this is correct.

That said when I wrote my non-regression tests, I wrote them in a way
that checks that you can upgrade a M-A: foreign foo_1.0_ to
foo_2.0_all (and vice-versa). Because that's what seemed most logical from
a user point of view.

Upgrading multiple M-A: same to a single foo_2.0_all should definitely be
forbidden though. But the opposite foo_1.0_all -> M-A: same
foo_2.0_ ?

So I wanted to know how other people feel on this issue.

Cheers,
-- 
Raphaël Hertzog ◈ Debian Developer

Follow my Debian News ▶ http://RaphaelHertzog.com (English)
  ▶ http://RaphaelHertzog.fr (Français)


-- 
To UNSUBSCRIBE, email to debian-dpkg-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20110218193705.ga13...@rivendell.home.ouaza.com

Re: Cross-upgrading packages with multiarch packages

[Steve Langasek]

On Fri, Feb 18, 2011 at 08:37:05PM +0100, Raphael Hertzog wrote:

> Upgrading multiple M-A: same to a single foo_2.0_all should definitely be
> forbidden though. But the opposite foo_1.0_all -> M-A: same
> foo_2.0_ ?

I think the applicable principle here is KISS.  You could allow dpkg to
upgrade foo_1.0_all to foo_2.0_foreign, but *nobody would understand what
dpkg was doing*, and why this case was different than the others. :)

-- 
Steve Langasek   Give me a lever long enough and a Free OS
Debian Developer   to set it on, and I can move the world.
Ubuntu Developerhttp://www.debian.org/
slanga...@ubuntu.com vor...@debian.org


signature.asc
Description: Digital signature

Re: Bug#594179: dpkg armhf patch acceptance status?

[Steve Langasek]

Hi Guillem,

Thanks for letting us know your thoughts.

On Fri, Feb 18, 2011 at 11:13:11AM +0100, Guillem Jover wrote:
> * The assumption that each GNU triplet denotes a different ABI is so
>   entrenched in the GNU build system, that we have things like the
>   following all over the place to properly support cross-building:

>   ,---
>   ifeq ($(DEB_BUILD_GNU_TYPE), $(DEB_HOST_GNU_TYPE))
> confflags += --build $(DEB_HOST_GNU_TYPE)
>   else
> confflags += --build $(DEB_BUILD_GNU_TYPE) --host $(DEB_HOST_GNU_TYPE)
>   endif
>   `---

But then we also have biarch and triarch toolchains in the archive, where
the host triplet doesn't change at all and instead gcc multilib is used to
designate what ABI you want.

The upstream argument seems to be that armel/armhf isn't "cross-compiling",
it's merely ABI selection within an architecture.  This sort of "cross"
compiling doesn't yield binaries that can't be run on the build machine, and
therefore most of the cross-compilation logic in autotools doesn't apply.

(Never mind that upstream does not apply this reasoning consistently; the
fact that there's precedent means it's going to be harder to argue the point
with them.)

> * The remaining problem at least for multiarch is the use of more
>   specialized cpu names for the i386 triplets, i486-linux-gnu on Debian,
>   which might change depending on the base instruction set to support,
>   for example i686-linux-gnu on Ubuntu.

>   Due to #611741, it already crossed my mind (but removed it from the
>   list of solutions before sending the reply as at the time it seemed
>   slighly preposterous just to fix the divergence with Ubuntu) that we
>   should switch back our i386 arch to use i386 as the base cpu name for
>   the triplet (i386-linux-gnu) in the same way we use it in other
>   arches, like on arm where we use arm-linux-gnu instead of something
>   like armv4tel-linux-gnueabi, for example. (I mentioned this already to
>   Steve and Raphaël on some private conversation.)

The challenge, as Matthias points out, is that these triplets are already so
entrenched and there is so much software that handles x86 specially - even
if incorrectly!  - that it's prohibitive to switch back to i386-linux-gnu as
our triplet because of all the re-porting work we'll have to do to get
properly functioning packages on x86.

>   Package build systems might be matching against stuff like
>   arm*-*-linux-gnueabi, so it might require changes to match on -gnueabi*
>   instead, but is more immediate, and not like propagating config.guess
>   all over the place, which we should not need as debian/rules should
>   be passing --build to configure anyway, and config.sub can already
>   canonicalize something like arm-linux-gnueabihf by way of the current
>   patterns.

Yes, package build systems do match on stuff like arm*-*-linux-gnueabi; my
understanding is that this was the pattern match that was propagated when
EABI was introduced on ARM, and as a result there's a fair amount of
software that will misbuild for eabi+hf if we go with -gnueabihf.  So
basically, it's the same problem we have as for switching to i386-linux-gnu
on i386, just on a different set of software in the archive.

How do we locate the software that will be affected by such a change?  I
think this has proven non-trivial in the past.

But ultimately that's for the porters and the GCC maintainers to decide
whether they're happy with that answer; it's an aside to the main point of
my mail, coming up below.  :)

> > > The toolchain has the same triplet for binary incompatible producing
> > > objects, which seems like a bug/misfeature to me, and that's one of
> > > the assumptions dpkg-dev has relied on, in the same way as multiarch
> > > when deciding to use the triplet as a unique token for the installation
> > > directories.

> >  You describe this as a bug/misfeature, that might be true but I don't
> >  think we can challenge this usage of triplets in the upstream
> >  toolchain, and multiarch is moving to having its own tuples instead now
> >  (http://wiki.debian.org/Multiarch/Tuples).

> Oh, but I think I just did. :) Also given the above I don't think it
> makes sense to invent a new set of tuples, the triplets should work
> just fine. In addition those tuples end up relying partly on the
> definition of the ABI the default gcc has for a given target, which
> should not change incompatibly for a given Debian architecture, or we
> are screwed anyway. So I don't see that it buys us much.

The underlying intent of the tuples proposal is that we stop pretending that
GNU triplets provide a valid global one-to-one mapping to ABIs, because we
know from several concrete, high profile examples that this is not true.

You have proposed a patch that Matthias has said "looks ok", but if he adds
it to the Debian/Ubuntu gcc, that only solves the problem for Debian and
Ubuntu.  Multiarch is intended as a cross-vendor standard, fixing the FHS's
inadequate lib directories and