Package: release.debian.org
Severity: normal
Tags: jessie
User: release.debian....@packages.debian.org
Usertags: pu
Hi,
the APT team (python division) would like to update
python-apt to fix some issues.
A diff against 0.9.3.11 is attached (sans
noise created by the build system - aka updated
mirror lists).
All patches are cherry-picked from the 1.0 beta1
release in unstable which I uploaded 5 days ago. I
have not received any bug reports yet that are
related to those patches, so I think it's a good
time to ask for a stable update permission.
The entire patch set is in
http://anonscm.debian.org/cgit/apt/python-apt.git/log/?h=debian/jessie-pu
if anyone is interested in the individual patches.
I'd like to use 0.9.3.12 as the version number
for the upload, as we moved on from 0.9.3 post
jessie, and this makes it nicer to merge downstream
(e.g. Ubuntu can have a 0.9.3.12ubuntu1 stable update
for vivid).
I put de...@lists.debian.org in X-Debbugs-CC.
Here's an annotated changelog for reference
python-apt (0.9.3.12) jessie; urgency=medium
[ Julian Andres Klode ]
* apt/cache.py: Work around a cyclic reference from Cache to its methods
(Closes: #745487)
-> This one is a bit crazy. Just deleting the cache (that is, cache = None
or 'del cache') did not free the cache because of a cyclic reference,
which is bad, because the cache holds about 20 file descriptors.
There is an explicit close method for the file descriptors (and a
context manager), but nobody is really using it, so it makes a lot
of sense to not leak FDs like that by default (and free the memory,
of course).
* python/arfile.cc: LFS: Use long long instead of long for file sizes
* python/arfile.cc: Do not allow files larger than SIZE_MAX to be mapped
-> Those two work as they should and prevent a crash when a member
is larger than can be read into memory.
* python/tarfile.cc: LFS: Handle too large file
-> The tarfile patch actually does not work entirely, in the first hunk
the first part does nothing (so it does not catch sizes > SIZE_MAX,
as the Size member is still long), but the second part catches too
large allocations, which prevents a crash.
* apt.debfile: Fix splitting of multi-lines Binary fields in dsc files
(Closes: #751770)
-> This one is really simple and fixes a parsing bug, so I'd like to
have it. (If it were more complicated, I'd not include it, but
it's such a tiny patch that it makes no sense to skip it)
* apt/debfile.py: Arch-qualify in compare_to_version_in_cache()
(Closes: #750189)
-> This one is a multi-arch fix. Without it, tools like gdebi will
compare a to be installed deb against the wrong architecture in
the cache
[ Michael Vogt ]
* Fix apt.Package.installed_files for multi-arch packages (LP: #1313699)
-> This multi-arch patch ensures that we look at the correct package
lists for multi-arch packages, and not accidentally, at the native
package.
-- Julian Andres Klode <j...@debian.org> Tue, 16 Jun 2015 11:35:11 +0200
-- System Information:
Debian Release: stretch/sid
APT prefers unstable
APT policy: (990, 'unstable'), (100, 'experimental')
Architecture: amd64 (x86_64)
Foreign Architectures: i386
Kernel: Linux 4.0.0-1-amd64 (SMP w/4 CPU cores)
Locale: LANG=de_DE.utf8, LC_CTYPE=de_DE.utf8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)
--
Julian Andres Klode - Debian Developer, Ubuntu Member
See http://wiki.debian.org/JulianAndresKlode and http://jak-linux.org/.
Be friendly, do not top-post, and follow RFC 1855 "Netiquette".
- If you don't I might ignore you.
diff --git a/apt/cache.py b/apt/cache.py
index 69ee369..b1e3a3e 100644
--- a/apt/cache.py
+++ b/apt/cache.py
@@ -78,8 +78,8 @@ class Cache(object):
self._changes_count = -1
self._sorted_set = None
- self.connect("cache_post_open", self._inc_changes_count)
- self.connect("cache_post_change", self._inc_changes_count)
+ self.connect("cache_post_open", "_inc_changes_count")
+ self.connect("cache_post_change", "_inc_changes_count")
if memonly:
# force apt to build its caches in memory
apt_pkg.config.set("Dir::Cache::pkgcache", "")
@@ -135,7 +135,10 @@ class Cache(object):
""" internal helper to run a callback """
if name in self._callbacks:
for callback in self._callbacks[name]:
- callback()
+ if callback == '_inc_changes_count':
+ self._inc_changes_count()
+ else:
+ callback()
def open(self, progress=None):
""" Open the package cache, after that it can be used like
diff --git a/apt/debfile.py b/apt/debfile.py
index 1f93b7b..26c7e45 100644
--- a/apt/debfile.py
+++ b/apt/debfile.py
@@ -456,6 +456,15 @@ class DebPackage(object):
"""
self._dbg(3, "compare_to_version_in_cache")
pkgname = self._sections["Package"]
+ architecture = self._sections["Architecture"]
+
+ # Architecture all gets mapped to the native architecture internally
+ if architecture == 'all':
+ architecture = apt_pkg.config.find("APT::Architecture")
+
+ # Arch qualify the package name
+ pkgname = ":".join([pkgname, architecture])
+
debver = self._sections["Version"]
self._dbg(1, "debver: %s" % debver)
if pkgname in self._cache:
@@ -734,7 +743,8 @@ class DscSrcPackage(DebPackage):
if 'Source' in sec:
self.pkgname = sec['Source']
if 'Binary' in sec:
- self.binaries = sec['Binary'].split(', ')
+ self.binaries = [b.strip() for b in
+ sec['Binary'].split(',')]
for tag in sec.keys():
if tag in sec:
self._sections[tag] = sec[tag]
diff --git a/apt/package.py b/apt/package.py
index c67540e..810d9dc 100644
--- a/apt/package.py
+++ b/apt/package.py
@@ -904,7 +904,7 @@ class Package(object):
Return a list of unicode names of the files which have
been installed by this package
"""
- for name in self.shortname, self.fullname:
+ for name in self.name, self.fullname:
path = "/var/lib/dpkg/info/%s.list" % name
try:
with open(path, "rb") as file_list:
diff --git a/debian/changelog b/debian/changelog
index f7228a5..0dea0f7 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,21 @@
+python-apt (0.9.3.12) jessie; urgency=medium
+
+ [ Julian Andres Klode ]
+ * apt/cache.py: Work around a cyclic reference from Cache to its methods
+ (Closes: #745487)
+ * python/arfile.cc: LFS: Use long long instead of long for file sizes
+ * python/arfile.cc: Do not allow files larger than SIZE_MAX to be mapped
+ * python/tarfile.cc: LFS: Handle too large file
+ * apt.debfile: Fix splitting of multi-lines Binary fields in dsc files
+ (Closes: #751770)
+ * apt/debfile.py: Arch-qualify in compare_to_version_in_cache()
+ (Closes: #750189)
+
+ [ Michael Vogt ]
+ * Fix apt.Package.installed_files for multi-arch packages (LP: #1313699)
+
+ -- Julian Andres Klode <j...@debian.org> Tue, 16 Jun 2015 11:35:11 +0200
+
python-apt (0.9.3.11) unstable; urgency=low
[ Colin Watson ]
diff --git a/python/arfile.cc b/python/arfile.cc
index 3284ff7..d4d7eed 100644
--- a/python/arfile.cc
+++ b/python/arfile.cc
@@ -179,10 +179,22 @@ static PyObject *ararchive_extractdata(PyArArchiveObject *self, PyObject *args)
PyErr_Format(PyExc_LookupError,"No member named '%s'",name.path);
return 0;
}
+ if (member->Size > SIZE_MAX) {
+ PyErr_Format(PyExc_MemoryError,
+ "Member '%s' is too large to read into memory",name.path);
+ return 0;
+ }
if (!self->Fd.Seek(member->Start))
return HandleErrors();
- char* value = new char[member->Size];
+ char* value;
+ try {
+ value = new char[member->Size];
+ } catch (std::bad_alloc&) {
+ PyErr_Format(PyExc_MemoryError,
+ "Member '%s' is too large to read into memory",name.path);
+ return 0;
+ }
self->Fd.Read(value, member->Size, true);
PyObject *result = PyBytes_FromStringAndSize(value, member->Size);
delete[] value;
@@ -221,14 +233,14 @@ static PyObject *_extract(FileFd &Fd, const ARArchive::Member *member,
// Read 4 KiB from the file, until all of the file is read. Deallocated
// automatically when the function returns.
SPtrArray<char> value = new char[4096];
- unsigned long size = member->Size;
- unsigned long read = 4096;
+ unsigned long long size = member->Size;
+ unsigned long long read = 4096;
while (size > 0) {
if (size < read)
read = size;
if (!Fd.Read(value, read, true))
return HandleErrors();
- if (write(outfd, value, read) != (signed)read)
+ if (write(outfd, value, read) != (signed long long)read)
return PyErr_SetFromErrnoWithFilename(PyExc_OSError, outfile);
size -= read;
}
diff --git a/python/tarfile.cc b/python/tarfile.cc
index adc8e81..0aeae66 100644
--- a/python/tarfile.cc
+++ b/python/tarfile.cc
@@ -43,7 +43,8 @@ public:
PyObject *py_data;
// The requested member or NULL.
const char *member;
- // Set to true if an error occured in the Python callback.
+ // Set to true if an error occured in the Python callback, or a file
+ // was too large to read in extractdata.
bool error;
// Place where the copy of the data is stored.
char *copy;
@@ -76,9 +77,13 @@ bool PyDirStream::DoItem(Item &Itm, int &Fd)
{
if (!member || strcmp(Itm.Name, member) == 0) {
// Allocate a new buffer if the old one is too small.
+ if (Itm.Size > SIZE_MAX)
+ goto to_large;
if (copy == NULL || copy_size < Itm.Size) {
delete[] copy;
- copy = new char[Itm.Size];
+ copy = new (std::nothrow) char[Itm.Size];
+ if (copy == NULL)
+ goto to_large;
copy_size = Itm.Size;
}
Fd = -2;
@@ -86,6 +91,19 @@ bool PyDirStream::DoItem(Item &Itm, int &Fd)
Fd = -1;
}
return true;
+to_large:
+ delete[] copy;
+ copy = NULL;
+ copy_size = 0;
+ /* If we are looking for a specific member, abort reading now */
+ if (member) {
+ error = true;
+ PyErr_Format(PyExc_MemoryError,
+ "The member %s was too large to read into memory",
+ Itm.Name);
+ return false;
+ }
+ return true;
}
#if (APT_PKG_MAJOR >= 4 && APT_PKG_MINOR >= 14)
@@ -96,7 +114,8 @@ bool PyDirStream::Process(Item &Itm,const unsigned char *Data,
unsigned long Size,unsigned long Pos)
#endif
{
- memcpy(copy + Pos, Data,Size);
+ if (copy != NULL)
+ memcpy(copy + Pos, Data,Size);
return true;
}
@@ -107,7 +126,12 @@ bool PyDirStream::FinishedFile(Item &Itm,int Fd)
return true;
Py_XDECREF(py_data);
- py_data = PyBytes_FromStringAndSize(copy, Itm.Size);
+ if (copy == NULL) {
+ Py_INCREF(Py_None);
+ py_data = Py_None;
+ } else {
+ py_data = PyBytes_FromStringAndSize(copy, Itm.Size);
+ }
if (!callback)
return true;
@@ -424,12 +448,12 @@ static PyObject *tarfile_extractdata(PyObject *self, PyObject *args)
// Go through the stream.
GetCpp<ExtractTar*>(self)->Go(stream);
+ if (stream.error)
+ return 0;
+
if (!stream.py_data)
return PyErr_Format(PyExc_LookupError, "There is no member named '%s'",
member.path);
- if (stream.error) {
- return 0;
- }
return Py_INCREF(stream.py_data), stream.py_data;
}
diff --git a/tests/data/test_debs/hello_2.5-1.dsc b/tests/data/test_debs/hello_2.5-1.dsc
index c079f2d..d00db52 100644
--- a/tests/data/test_debs/hello_2.5-1.dsc
+++ b/tests/data/test_debs/hello_2.5-1.dsc
@@ -3,7 +3,8 @@ Hash: SHA256
Format: 1.0
Source: hello
-Binary: hello
+Binary: hello, bello,
+ cello
Architecture: any
Version: 2.5-1
Maintainer: Santiago Vila <sanv...@debian.org>
diff --git a/tests/data/test_debs/large-package-content_1.0_all.deb b/tests/data/test_debs/large-package-content_1.0_all.deb
new file mode 100644
index 0000000..56bdce3
Binary files /dev/null and b/tests/data/test_debs/large-package-content_1.0_all.deb differ
diff --git a/tests/test_debfile.py b/tests/test_debfile.py
index b8418e7..eae8038 100644
--- a/tests/test_debfile.py
+++ b/tests/test_debfile.py
@@ -71,14 +71,14 @@ class TestDebfile(unittest.TestCase):
self.assertEqual(set(deb.missing_deps), missing)
# specialized properties
self.assertEqual(deb.pkgname, "hello")
- self.assertEqual(deb.binaries, ["hello"])
+ self.assertEqual(deb.binaries, ["hello", "bello", "cello"])
self.assertEqual(deb.filelist, ["hello_2.5.orig.tar.gz",
"hello_2.5-1.diff.gz"])
self.assertEqual(deb.depends, [[("autotools-dev", "", "")]])
# tag fields are available as a dict
self.assertEqual(deb["Format"], "1.0")
self.assertEqual(deb["Source"], "hello")
- self.assertEqual(deb["Binary"], "hello")
+ self.assertEqual(deb["Binary"], "hello, bello,\n cello")
self.assertEqual(deb["Architecture"], "any")
self.assertEqual(deb["Version"], "2.5-1")
self.assertEqual(
diff --git a/tests/test_large_file.py b/tests/test_large_file.py
new file mode 100644
index 0000000..852c857
--- /dev/null
+++ b/tests/test_large_file.py
@@ -0,0 +1,23 @@
+#!/usr/bin/python
+
+import sys
+import unittest
+
+import apt_inst
+
+IS_NOT_32BIT = sys.maxsize > 2 ** 32
+
+
+@unittest.skipIf(IS_NOT_32BIT, "Large File support is for 32 bit systems")
+class testHashes(unittest.TestCase):
+ " test the hashsum functions against strings and files "
+
+ LARGE_PACKAGE_CONTENT = "data/test_debs/large-package-content_1.0_all.deb"
+
+ def testExtractData(self):
+ deb = apt_inst.DebFile(self.LARGE_PACKAGE_CONTENT)
+
+ self.assertRaises(MemoryError, deb.data.extractdata, "large-file")
+
+if __name__ == "__main__":
+ unittest.main()
