[Bug debuginfod/27399] dpkg-deb/lzma error when indexing .debs

2021-02-13 Thread sergiodj at sergiodj dot net via Elfutils-devel
https://sourceware.org/bugzilla/show_bug.cgi?id=27399

--- Comment #7 from Sergio Durigan Junior  ---
(In reply to Frank Ch. Eigler from comment #6)
> OK, my best guess is a transient low-memory condition.  Just today we found
> bug #27405, which can cause some impressive momentary memory binges in
> elfutils.  Reducing concurrency is one way to limit its impact, as you
> found.  Giving the VM more memory is another: I run it on 16GB 8CPU class
> VMs, scanning even many huge debuginfo files, without a complaint.

I was also considering that this could be a temporarily low-memory thing, but I
don't think so anymore.  I've expanded the VM memory to 32GB now, and I'm still
seeing the errors.

> It might also help reassure here is that if you let debuginfod keep going,
> it'll know that it didn't finish indexing those problematic files, and will
> just retry later.  In the long term (past rescan time), such momentary
> ENOMEMs are not particularly harmful.

OK, good, I thought this would be the case but I did not know for sure.  That's
good to know.

> I can't think of any debuginfod per se changes or bugs in effect here. 
> Maybe scale the default concurrency more conservatively to machine RAM
> rather than solely CPU count?

My current plan is to use the "bsdtar" workaround above, which seems to work
even when I max out the concurrency.  But the debugger in me still wants to
know what is happening...

Anyway, while I agree that this is not a high priority issue (since there are
two known workarounds for it), I think it may be worth mentioning the "bsdtar"
solution more explicitly in the documentation.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug debuginfod/27413] New: use bsdtar for .deb extraction too

2021-02-13 Thread fche at redhat dot com via Elfutils-devel
https://sourceware.org/bugzilla/show_bug.cgi?id=27413

Bug ID: 27413
   Summary: use bsdtar for .deb extraction too
   Product: elfutils
   Version: unspecified
Status: NEW
  Severity: normal
  Priority: P2
 Component: debuginfod
  Assignee: unassigned at sourceware dot org
  Reporter: fche at redhat dot com
CC: elfutils-devel at sourceware dot org
  Target Milestone: ---

Back during bug #25583, we transitioned rpm unpacking to bsdtar (libarchive),
but left deb unpacking to dpkg-deb by default.  One factor in this was the
inconsistent naming convention of the "data.tar*" inner tarball: it can be
.xz or .bz2 or others.  Given a few reports of intermittent crashes via
dpkg-deb (bug #27399), and the ability of bsdtar to take glob patterns,
it's probably time to let bsdtar handle .debs too (and .ipkg).

The extracting pipe command would be:  (bsdtar -O -x -f - 'data.tar*')<

-- 
You are receiving this mail because:
You are on the CC list for the bug.