Package: release.debian.org Severity: normal User: release.debian....@packages.debian.org Usertags: unblock
Hi release team, lets continue our little chit-chat and move it from list to bts, shall we? Attached is a "git log -p" style diff to give you some more context than a debdiff would. Beside some few-liners to fix various regressions this diff is hugely dominated by my attempt to backport a security improvement I (and hopefully you) can't survive without much longer: Up until now "apt-get source" is protected by MD5 only (+ file size) which is changed to "best available (supported) hash". Two things: Making this abicompat for now makes it also really ugly from an API point of view, but so be it. The more 'interesting' point is that this is a backport of the change(s) required to make that work rather than a minimal fix as a) minimal would still be quiet a few lines b) I really would like to avoid a jessie-only API and c) its security related, so it can't hurt using already tested code. But well, if you disagree I will bite the bullet… hence this pre-approval question. (I guess we could further backport this to squeeze/wheezy but I will save this discussion for another day) There might be an update to this than I figured out if and how I am going to backport some apt-key regressions as they were fixed by a complete rewrite… and we would like to include a few translation updates if you don't mind, but lets focus on the security-improvement for now… In regards to the trigger change in dpkg: Nobody commented further, so I feel only half as bad for not having done much about it, but that will surely need quite some changes, so if this isn't magically going forward soon I would post-prune that to 1.0.9.5 for now – just so you know. Best regards David Kalnischkies
commit d94082d5b98cdc10f9bc71377229bb57489ffaab
Author: David Kalnischkies <da...@kalnischkies.de>
Date: Mon Nov 10 17:21:57 2014 +0100
change codenames to jessie as stable POV in docs
diff --git a/doc/apt-verbatim.ent b/doc/apt-verbatim.ent
index 126f26b..f32f667 100644
--- a/doc/apt-verbatim.ent
+++ b/doc/apt-verbatim.ent
@@ -228,10 +228,10 @@
<!ENTITY apt-product-version "1.0.9.1">
<!-- (Code)names for various things used all over the place -->
-<!ENTITY oldstable-codename "squeeze">
-<!ENTITY stable-codename "wheezy">
-<!ENTITY testing-codename "jessie">
-<!ENTITY stable-version "7">
+<!ENTITY oldstable-codename "wheezy">
+<!ENTITY stable-codename "jessie">
+<!ENTITY testing-codename "stretch">
+<!ENTITY stable-version "8">
<!ENTITY ubuntu-codename "trusty">
<!-- good and bad just refers to matching and not matching a pattern…
commit 8cc3535f2cdcfe1301b641dae8dfadf99658c732
Author: David Kalnischkies <da...@kalnischkies.de>
Date: Sat Oct 18 14:44:41 2014 +0200
reenable support for -s (and co) in apt-get source
The conversion to accept only relevant options for commands has
forgotten another one, so adding it again even through the usecase might
very well be equally good served by --print-uris.
Closes: 742578
diff --git a/apt-private/private-cmndline.cc b/apt-private/private-cmndline.cc
index a4490f5..0b5ba5b 100644
--- a/apt-private/private-cmndline.cc
+++ b/apt-private/private-cmndline.cc
@@ -166,7 +166,7 @@ static bool addArgumentsAPTGet(std::vector<CommandLine::Args> &Args, char const
if (CmdMatches("install", "remove", "purge", "upgrade", "dist-upgrade",
"deselect-upgrade", "autoremove", "clean", "autoclean", "check",
- "build-dep", "full-upgrade"))
+ "build-dep", "full-upgrade", "source"))
{
addArg('s', "simulate", "APT::Get::Simulate", 0);
addArg('s', "just-print", "APT::Get::Simulate", 0);
diff --git a/test/integration/test-apt-get-source b/test/integration/test-apt-get-source
index 33bd980..b27cbbe 100755
--- a/test/integration/test-apt-get-source
+++ b/test/integration/test-apt-get-source
@@ -82,3 +82,7 @@ testequal "$HEADER
Need to get 0 B of source archives.
'file://${APTARCHIVE}/foo_0.0.1.dsc' foo_0.0.1.dsc 0 MD5Sum:d41d8cd98f00b204e9800998ecf8427e
'file://${APTARCHIVE}/foo_0.0.1.tar.gz' foo_0.0.1.tar.gz 0 MD5Sum:d41d8cd98f00b204e9800998ecf8427e" aptget source -q --print-uris -t unstable foo=0.0.1
+
+testequal "$HEADER
+Need to get 0 B of source archives.
+Fetch source foo" aptget source -q -s foo
commit c505fa33a6441b451971ce6c636cf2ca4dacdc1d
Author: David Kalnischkies <da...@kalnischkies.de>
Date: Sun Sep 28 01:25:21 2014 +0200
allow options between command and -- on commandline
This used to work before we implemented a stricter commandline parser
and e.g. the dd-schroot-cmd command constructs commandlines like this.
Reported-By: Helmut Grohne
diff --git a/apt-pkg/contrib/cmndline.cc b/apt-pkg/contrib/cmndline.cc
index 3799c82..93c1f46 100644
--- a/apt-pkg/contrib/cmndline.cc
+++ b/apt-pkg/contrib/cmndline.cc
@@ -47,23 +47,26 @@ CommandLine::~CommandLine()
char const * CommandLine::GetCommand(Dispatch const * const Map,
unsigned int const argc, char const * const * const argv)
{
- // if there is a -- on the line there must be the word we search for around it
- // as -- marks the end of the options, just not sure if the command can be
- // considered an option or not, so accept both
+ // if there is a -- on the line there must be the word we search for either
+ // before it (as -- marks the end of the options) or right after it (as we can't
+ // decide if the command is actually an option, given that in theory, you could
+ // have parameters named like commands)
for (size_t i = 1; i < argc; ++i)
{
if (strcmp(argv[i], "--") != 0)
continue;
- ++i;
- if (i < argc)
+ // check if command is before --
+ for (size_t k = 1; k < i; ++k)
for (size_t j = 0; Map[j].Match != NULL; ++j)
- if (strcmp(argv[i], Map[j].Match) == 0)
+ if (strcmp(argv[k], Map[j].Match) == 0)
return Map[j].Match;
- i -= 2;
- if (i != 0)
+ // see if the next token after -- is the command
+ ++i;
+ if (i < argc)
for (size_t j = 0; Map[j].Match != NULL; ++j)
if (strcmp(argv[i], Map[j].Match) == 0)
return Map[j].Match;
+ // we found a --, but not a command
return NULL;
}
// no --, so search for the first word matching a command
diff --git a/test/libapt/commandline_test.cc b/test/libapt/commandline_test.cc
index e403a28..627f1b4 100644
--- a/test/libapt/commandline_test.cc
+++ b/test/libapt/commandline_test.cc
@@ -2,6 +2,7 @@
#include <apt-pkg/cmndline.h>
#include <apt-pkg/configuration.h>
+#include <apt-private/private-cmndline.h>
#include <gtest/gtest.h>
@@ -85,3 +86,70 @@ TEST(CommandLineTest, BoolParsing)
}
}
+
+bool DoVoid(CommandLine &) { return false; }
+
+TEST(CommandLineTest,GetCommand)
+{
+ CommandLine::Dispatch Cmds[] = { {"install",&DoVoid}, {"remove", &DoVoid}, {0,0} };
+ {
+ char const * argv[] = { "apt-get", "-t", "unstable", "remove", "-d", "foo" };
+ char const * com = CommandLine::GetCommand(Cmds, sizeof(argv)/sizeof(argv[0]), argv);
+ EXPECT_STREQ("remove", com);
+ std::vector<CommandLine::Args> Args = getCommandArgs("apt-get", com);
+ ::Configuration c;
+ CommandLine CmdL(Args.data(), &c);
+ ASSERT_TRUE(CmdL.Parse(sizeof(argv)/sizeof(argv[0]), argv));
+ EXPECT_EQ(c.Find("APT::Default-Release"), "unstable");
+ EXPECT_TRUE(c.FindB("APT::Get::Download-Only"));
+ ASSERT_EQ(2, CmdL.FileSize());
+ EXPECT_EQ(std::string(CmdL.FileList[0]), "remove");
+ EXPECT_EQ(std::string(CmdL.FileList[1]), "foo");
+ }
+ {
+ char const * argv[] = {"apt-get", "-t", "unstable", "remove", "--", "-d", "foo" };
+ char const * com = CommandLine::GetCommand(Cmds, sizeof(argv)/sizeof(argv[0]), argv);
+ EXPECT_STREQ("remove", com);
+ std::vector<CommandLine::Args> Args = getCommandArgs("apt-get", com);
+ ::Configuration c;
+ CommandLine CmdL(Args.data(), &c);
+ ASSERT_TRUE(CmdL.Parse(sizeof(argv)/sizeof(argv[0]), argv));
+ EXPECT_EQ(c.Find("APT::Default-Release"), "unstable");
+ EXPECT_FALSE(c.FindB("APT::Get::Download-Only"));
+ ASSERT_EQ(3, CmdL.FileSize());
+ EXPECT_EQ(std::string(CmdL.FileList[0]), "remove");
+ EXPECT_EQ(std::string(CmdL.FileList[1]), "-d");
+ EXPECT_EQ(std::string(CmdL.FileList[2]), "foo");
+ }
+ {
+ char const * argv[] = {"apt-get", "-t", "unstable", "--", "remove", "-d", "foo" };
+ char const * com = CommandLine::GetCommand(Cmds, sizeof(argv)/sizeof(argv[0]), argv);
+ EXPECT_STREQ("remove", com);
+ std::vector<CommandLine::Args> Args = getCommandArgs("apt-get", com);
+ ::Configuration c;
+ CommandLine CmdL(Args.data(), &c);
+ ASSERT_TRUE(CmdL.Parse(sizeof(argv)/sizeof(argv[0]), argv));
+ EXPECT_EQ(c.Find("APT::Default-Release"), "unstable");
+ EXPECT_FALSE(c.FindB("APT::Get::Download-Only"));
+ ASSERT_EQ(CmdL.FileSize(), 3);
+ EXPECT_EQ(std::string(CmdL.FileList[0]), "remove");
+ EXPECT_EQ(std::string(CmdL.FileList[1]), "-d");
+ EXPECT_EQ(std::string(CmdL.FileList[2]), "foo");
+ }
+ {
+ char const * argv[] = {"apt-get", "install", "-t", "unstable", "--", "remove", "-d", "foo" };
+ char const * com = CommandLine::GetCommand(Cmds, sizeof(argv)/sizeof(argv[0]), argv);
+ EXPECT_STREQ("install", com);
+ std::vector<CommandLine::Args> Args = getCommandArgs("apt-get", com);
+ ::Configuration c;
+ CommandLine CmdL(Args.data(), &c);
+ ASSERT_TRUE(CmdL.Parse(sizeof(argv)/sizeof(argv[0]), argv));
+ EXPECT_EQ(c.Find("APT::Default-Release"), "unstable");
+ EXPECT_FALSE(c.FindB("APT::Get::Download-Only"));
+ ASSERT_EQ(CmdL.FileSize(), 4);
+ EXPECT_EQ(std::string(CmdL.FileList[0]), "install");
+ EXPECT_EQ(std::string(CmdL.FileList[1]), "remove");
+ EXPECT_EQ(std::string(CmdL.FileList[2]), "-d");
+ EXPECT_EQ(std::string(CmdL.FileList[3]), "foo");
+ }
+}
diff --git a/test/libapt/makefile b/test/libapt/makefile
index 69a13fd..7f23ace 100644
--- a/test/libapt/makefile
+++ b/test/libapt/makefile
@@ -14,8 +14,8 @@ test: $(BIN)/gtest$(BASENAME)
$(BIN)/gtest$(BASENAME): $(LIB)/gtest.a
PROGRAM = gtest${BASENAME}
-SLIBS = -lapt-pkg -pthread $(LIB)/gtest.a
-LIB_MAKES = apt-pkg/makefile
+SLIBS = -lapt-pkg -lapt-private -pthread $(LIB)/gtest.a
+LIB_MAKES = apt-pkg/makefile apt-private/makefile
SOURCE = gtest_runner.cc $(wildcard *-helpers.cc *_test.cc)
include $(PROGRAM_H)
commit 50ef3344c3afaaf9943142906b2f976a0337d264
Author: David Kalnischkies <da...@kalnischkies.de>
Date: Fri Jun 13 08:35:32 2014 +0200
deprecate the Section member from package struct
A version belongs to a section and has hence a section member of its
own. A package on the other hand can have multiple versions from
different sections. This was "solved" by using the section which was
parsed first as order of sources.list defines, but that is obviously a
horribly unpredictable thing.
Users are way better of with the Section() as returned by the version
they are dealing with. It is likely the same for all versions of a
package, but in the few cases it isn't, it is important (like packages
moving from main/* to contrib/* or into oldlibs …).
Backport of 7a66977 which actually instantly removes the member.
diff --git a/apt-pkg/cacheiterators.h b/apt-pkg/cacheiterators.h
index 2fdf840..513f40f 100644
--- a/apt-pkg/cacheiterators.h
+++ b/apt-pkg/cacheiterators.h
@@ -160,7 +160,9 @@ class pkgCache::PkgIterator: public Iterator<Package, PkgIterator> {
// Accessors
inline const char *Name() const {return S->Name == 0?0:Owner->StrP + S->Name;}
- inline const char *Section() const {return S->Section == 0?0:Owner->StrP + S->Section;}
+ // Versions have sections - and packages can have different versions with different sections
+ // so this interface is broken by design. Run as fast as you can to Version.Section().
+ APT_DEPRECATED inline const char *Section() const {return S->Section == 0?0:Owner->StrP + S->Section;}
inline bool Purge() const {return S->CurrentState == pkgCache::State::Purge ||
(S->CurrentVer == 0 && S->CurrentState == pkgCache::State::NotInstalled);}
inline const char *Arch() const {return S->Arch == 0?0:Owner->StrP + S->Arch;}
diff --git a/apt-pkg/cacheset.h b/apt-pkg/cacheset.h
index 16a3daa..b7229bc 100644
--- a/apt-pkg/cacheset.h
+++ b/apt-pkg/cacheset.h
@@ -118,7 +118,16 @@ public:
inline const char *Name() const {return getPkg().Name(); }
inline std::string FullName(bool const Pretty) const { return getPkg().FullName(Pretty); }
inline std::string FullName() const { return getPkg().FullName(); }
- inline const char *Section() const {return getPkg().Section(); }
+ APT_DEPRECATED inline const char *Section() const {
+#if __GNUC__ >= 4
+ #pragma GCC diagnostic push
+ #pragma GCC diagnostic ignored "-Wdeprecated-declarations"
+#endif
+ return getPkg().Section();
+#if __GNUC__ >= 4
+ #pragma GCC diagnostic pop
+#endif
+ }
inline bool Purge() const {return getPkg().Purge(); }
inline const char *Arch() const {return getPkg().Arch(); }
inline pkgCache::GrpIterator Group() const { return getPkg().Group(); }
diff --git a/apt-pkg/depcache.cc b/apt-pkg/depcache.cc
index 42e3139..16282df 100644
--- a/apt-pkg/depcache.cc
+++ b/apt-pkg/depcache.cc
@@ -1226,7 +1226,7 @@ bool pkgDepCache::MarkInstall(PkgIterator const &Pkg,bool AutoInst,
continue;
}
// now check if we should consider it a automatic dependency or not
- if(InstPkg->CurrentVer == 0 && Pkg->Section != 0 && ConfigValueInSubTree("APT::Never-MarkAuto-Sections", Pkg.Section()))
+ if(InstPkg->CurrentVer == 0 && InstVer->Section != 0 && ConfigValueInSubTree("APT::Never-MarkAuto-Sections", InstVer.Section()))
{
if(DebugAutoInstall == true)
std::clog << OutputInDepth(Depth) << "Setting NOT as auto-installed (direct "
diff --git a/apt-pkg/pkgcache.cc b/apt-pkg/pkgcache.cc
index 58a6345..d7c9656 100644
--- a/apt-pkg/pkgcache.cc
+++ b/apt-pkg/pkgcache.cc
@@ -524,7 +524,10 @@ operator<<(std::ostream& out, pkgCache::PkgIterator Pkg)
out << " -> " << candidate;
if ( newest != "none" && candidate != newest)
out << " | " << newest;
- out << " > ( " << string(Pkg.Section()==0?"none":Pkg.Section()) << " )";
+ if (Pkg->VersionList == 0)
+ out << " > ( none )";
+ else
+ out << " > ( " << string(Pkg.VersionList().Section()==0?"unknown":Pkg.VersionList().Section()) << " )";
return out;
}
/*}}}*/
commit 3a2b39ee602dd5a98b8fdaee2f1c8e0b13a276e2
Author: David Kalnischkies <kalnischk...@gmail.com>
Date: Sun Aug 18 23:27:24 2013 +0200
use 'best' hash for source authentication
Collect all hashes we can get from the source record and put them into a
HashStringList so that 'apt-get source' can use it instead of using
always the MD5sum.
We therefore also deprecate the MD5 struct member in favor of the list.
While at it, the parsing of the Files is enhanced so that records which
miss "Files" (aka MD5 checksums) are still searched for other checksums
as they include just as much data, just not with a nice and catchy name.
This is a cherry-pick of 1262d35 with some dirty tricks to preserve ABI.
LP: 1098738
diff --git a/apt-pkg/deb/debsrcrecords.cc b/apt-pkg/deb/debsrcrecords.cc
index a444cbe..49a348d 100644
--- a/apt-pkg/deb/debsrcrecords.cc
+++ b/apt-pkg/deb/debsrcrecords.cc
@@ -118,13 +118,32 @@ bool debSrcRecordParser::BuildDepends(std::vector<pkgSrcRecords::Parser::BuildDe
// ---------------------------------------------------------------------
/* This parses the list of files and returns it, each file is required to have
a complete source package */
-bool debSrcRecordParser::Files(std::vector<pkgSrcRecords::File> &List)
+bool debSrcRecordParser::Files(std::vector<pkgSrcRecords::File> &F)
{
- List.erase(List.begin(),List.end());
-
- string Files = Sect.FindS("Files");
- if (Files.empty() == true)
+ std::vector<pkgSrcRecords::File2> F2;
+ if (Files2(F2) == false)
return false;
+ for (std::vector<pkgSrcRecords::File2>::const_iterator f2 = F2.begin(); f2 != F2.end(); ++f2)
+ {
+ pkgSrcRecords::File2 f;
+#if __GNUC__ >= 4
+ #pragma GCC diagnostic push
+ #pragma GCC diagnostic ignored "-Wdeprecated-declarations"
+#endif
+ f.MD5Hash = f2->MD5Hash;
+ f.Size = f2->Size;
+#if __GNUC__ >= 4
+ #pragma GCC diagnostic pop
+#endif
+ f.Path = f2->Path;
+ f.Type = f2->Type;
+ F.push_back(f);
+ }
+ return true;
+}
+bool debSrcRecordParser::Files2(std::vector<pkgSrcRecords::File2> &List)
+{
+ List.clear();
// Stash the / terminated directory prefix
string Base = Sect.FindS("Directory");
@@ -133,51 +152,106 @@ bool debSrcRecordParser::Files(std::vector<pkgSrcRecords::File> &List)
std::vector<std::string> const compExts = APT::Configuration::getCompressorExtensions();
- // Iterate over the entire list grabbing each triplet
- const char *C = Files.c_str();
- while (*C != 0)
- {
- pkgSrcRecords::File F;
- string Size;
-
- // Parse each of the elements
- if (ParseQuoteWord(C,F.MD5Hash) == false ||
- ParseQuoteWord(C,Size) == false ||
- ParseQuoteWord(C,F.Path) == false)
- return _error->Error("Error parsing file record");
-
- // Parse the size and append the directory
- F.Size = atoi(Size.c_str());
- F.Path = Base + F.Path;
-
- // Try to guess what sort of file it is we are getting.
- string::size_type Pos = F.Path.length()-1;
- while (1)
+ for (char const * const * type = HashString::SupportedHashes(); *type != NULL; ++type)
+ {
+ // derive field from checksum type
+ std::string checksumField("Checksums-");
+ if (strcmp(*type, "MD5Sum") == 0)
+ checksumField = "Files"; // historic name for MD5 checksums
+ else
+ checksumField.append(*type);
+
+ string const Files = Sect.FindS(checksumField.c_str());
+ if (Files.empty() == true)
+ continue;
+
+ // Iterate over the entire list grabbing each triplet
+ const char *C = Files.c_str();
+ while (*C != 0)
{
- string::size_type Tmp = F.Path.rfind('.',Pos);
- if (Tmp == string::npos)
- break;
- if (F.Type == "tar") {
- // source v3 has extension 'debian.tar.*' instead of 'diff.*'
- if (string(F.Path, Tmp+1, Pos-Tmp) == "debian")
- F.Type = "diff";
- break;
- }
- F.Type = string(F.Path,Tmp+1,Pos-Tmp);
-
- if (std::find(compExts.begin(), compExts.end(), std::string(".").append(F.Type)) != compExts.end() ||
- F.Type == "tar")
+ string hash, size, path;
+
+ // Parse each of the elements
+ if (ParseQuoteWord(C, hash) == false ||
+ ParseQuoteWord(C, size) == false ||
+ ParseQuoteWord(C, path) == false)
+ return _error->Error("Error parsing file record in %s of source package %s", checksumField.c_str(), Package().c_str());
+
+ HashString const hashString(*type, hash);
+ if (Base.empty() == false)
+ path = Base + path;
+
+ // look if we have a record for this file already
+ std::vector<pkgSrcRecords::File2>::iterator file = List.begin();
+ for (; file != List.end(); ++file)
+ if (file->Path == path)
+ break;
+
+ // we have it already, store the new hash and be done
+ if (file != List.end())
{
- Pos = Tmp-1;
+#if __GNUC__ >= 4
+ // set for compatibility only, so warn users not us
+ #pragma GCC diagnostic push
+ #pragma GCC diagnostic ignored "-Wdeprecated-declarations"
+#endif
+ if (checksumField == "Files")
+ file->MD5Hash = hash;
+#if __GNUC__ >= 4
+ #pragma GCC diagnostic pop
+#endif
+ // an error here indicates that we have two different hashes for the same file
+ if (file->Hashes.push_back(hashString) == false)
+ return _error->Error("Error parsing checksum in %s of source package %s", checksumField.c_str(), Package().c_str());
continue;
}
-
- break;
+
+ // we haven't seen this file yet
+ pkgSrcRecords::File2 F;
+ F.Path = path;
+ F.FileSize = strtoull(size.c_str(), NULL, 10);
+ F.Hashes.push_back(hashString);
+
+#if __GNUC__ >= 4
+ // set for compatibility only, so warn users not us
+ #pragma GCC diagnostic push
+ #pragma GCC diagnostic ignored "-Wdeprecated-declarations"
+#endif
+ F.Size = F.FileSize;
+ if (checksumField == "Files")
+ F.MD5Hash = hash;
+#if __GNUC__ >= 4
+ #pragma GCC diagnostic pop
+#endif
+
+ // Try to guess what sort of file it is we are getting.
+ string::size_type Pos = F.Path.length()-1;
+ while (1)
+ {
+ string::size_type Tmp = F.Path.rfind('.',Pos);
+ if (Tmp == string::npos)
+ break;
+ if (F.Type == "tar") {
+ // source v3 has extension 'debian.tar.*' instead of 'diff.*'
+ if (string(F.Path, Tmp+1, Pos-Tmp) == "debian")
+ F.Type = "diff";
+ break;
+ }
+ F.Type = string(F.Path,Tmp+1,Pos-Tmp);
+
+ if (std::find(compExts.begin(), compExts.end(), std::string(".").append(F.Type)) != compExts.end() ||
+ F.Type == "tar")
+ {
+ Pos = Tmp-1;
+ continue;
+ }
+
+ break;
+ }
+ List.push_back(F);
}
-
- List.push_back(F);
}
-
+
return true;
}
/*}}}*/
diff --git a/apt-pkg/deb/debsrcrecords.h b/apt-pkg/deb/debsrcrecords.h
index b65d148..2a3fc86 100644
--- a/apt-pkg/deb/debsrcrecords.h
+++ b/apt-pkg/deb/debsrcrecords.h
@@ -53,6 +53,7 @@ class debSrcRecordParser : public pkgSrcRecords::Parser
return std::string(Start,Stop);
};
virtual bool Files(std::vector<pkgSrcRecords::File> &F);
+ bool Files2(std::vector<pkgSrcRecords::File2> &F);
debSrcRecordParser(std::string const &File,pkgIndexFile const *Index)
: Parser(Index), Fd(File,FileFd::ReadOnly, FileFd::Extension), Tags(&Fd,102400),
diff --git a/apt-pkg/srcrecords.cc b/apt-pkg/srcrecords.cc
index 81b1c54..3175ee7 100644
--- a/apt-pkg/srcrecords.cc
+++ b/apt-pkg/srcrecords.cc
@@ -14,6 +14,7 @@
#include<config.h>
#include <apt-pkg/srcrecords.h>
+#include <apt-pkg/debsrcrecords.h>
#include <apt-pkg/error.h>
#include <apt-pkg/sourcelist.h>
#include <apt-pkg/metaindex.h>
@@ -147,5 +148,33 @@ const char *pkgSrcRecords::Parser::BuildDepType(unsigned char const &Type)
return fields[Type];
}
/*}}}*/
+bool pkgSrcRecords::Parser::Files2(std::vector<pkgSrcRecords::File2> &F2)/*{{{*/
+{
+ debSrcRecordParser * const deb = dynamic_cast<debSrcRecordParser*>(this);
+ if (deb != NULL)
+ return deb->Files2(F2);
-
+ std::vector<pkgSrcRecords::File> F;
+ if (Files(F) == false)
+ return false;
+ for (std::vector<pkgSrcRecords::File>::const_iterator f = F.begin(); f != F.end(); ++f)
+ {
+ pkgSrcRecords::File2 f2;
+#if __GNUC__ >= 4
+ #pragma GCC diagnostic push
+ #pragma GCC diagnostic ignored "-Wdeprecated-declarations"
+#endif
+ f2.MD5Hash = f->MD5Hash;
+ f2.Size = f->Size;
+ f2.Hashes.push_back(HashString("MD5Sum", f->MD5Hash));
+ f2.FileSize = f->Size;
+#if __GNUC__ >= 4
+ #pragma GCC diagnostic pop
+#endif
+ f2.Path = f->Path;
+ f2.Type = f->Type;
+ F2.push_back(f2);
+ }
+ return true;
+}
+ /*}}}*/
diff --git a/apt-pkg/srcrecords.h b/apt-pkg/srcrecords.h
index e000e17..dde22bd 100644
--- a/apt-pkg/srcrecords.h
+++ b/apt-pkg/srcrecords.h
@@ -14,6 +14,7 @@
#define PKGLIB_SRCRECORDS_H
#include <apt-pkg/macros.h>
+#include <apt-pkg/hashes.h>
#include <string>
#include <vector>
@@ -29,15 +30,28 @@ class pkgSrcRecords
{
public:
+#if __GNUC__ >= 4
+ // ensure that con- & de-structor don't trigger this warning
+ #pragma GCC diagnostic push
+ #pragma GCC diagnostic ignored "-Wdeprecated-declarations"
+#endif
// Describes a single file
struct File
{
- std::string MD5Hash;
- unsigned long Size;
+ APT_DEPRECATED std::string MD5Hash;
+ APT_DEPRECATED unsigned long Size;
std::string Path;
std::string Type;
};
-
+ struct File2 : public File
+ {
+ unsigned long long FileSize;
+ HashStringList Hashes;
+ };
+#if __GNUC__ >= 4
+ #pragma GCC diagnostic pop
+#endif
+
// Abstract parser for each source record
class Parser
{
@@ -77,6 +91,7 @@ class pkgSrcRecords
static const char *BuildDepType(unsigned char const &Type) APT_PURE;
virtual bool Files(std::vector<pkgSrcRecords::File> &F) = 0;
+ bool Files2(std::vector<pkgSrcRecords::File2> &F);
Parser(const pkgIndexFile *Index) : iIndex(Index) {};
virtual ~Parser() {};
diff --git a/cmdline/apt-get.cc b/cmdline/apt-get.cc
index cfa7933..a285377 100644
--- a/cmdline/apt-get.cc
+++ b/cmdline/apt-get.cc
@@ -797,13 +797,13 @@ static bool DoSource(CommandLine &CmdL)
}
// Back track
- vector<pkgSrcRecords::File> Lst;
- if (Last->Files(Lst) == false) {
+ vector<pkgSrcRecords::File2> Lst;
+ if (Last->Files2(Lst) == false) {
return false;
}
// Load them into the fetcher
- for (vector<pkgSrcRecords::File>::const_iterator I = Lst.begin();
+ for (vector<pkgSrcRecords::File2>::const_iterator I = Lst.begin();
I != Lst.end(); ++I)
{
// Try to guess what sort of file it is we are getting.
@@ -832,22 +832,26 @@ static bool DoSource(CommandLine &CmdL)
queued.insert(Last->Index().ArchiveURI(I->Path));
// check if we have a file with that md5 sum already localy
- if(!I->MD5Hash.empty() && FileExists(flNotDir(I->Path)))
- {
- FileFd Fd(flNotDir(I->Path), FileFd::ReadOnly);
- MD5Summation sum;
- sum.AddFD(Fd.Fd(), Fd.Size());
- Fd.Close();
- if((string)sum.Result() == I->MD5Hash)
+ std::string localFile = flNotDir(I->Path);
+ if (FileExists(localFile) == true)
+ if(I->Hashes.VerifyFile(localFile) == true)
{
ioprintf(c1out,_("Skipping already downloaded file '%s'\n"),
- flNotDir(I->Path).c_str());
+ localFile.c_str());
continue;
}
+
+ // see if we have a hash (Acquire::ForceHash is the only way to have none)
+ HashString const * const hs = I->Hashes.find(NULL);
+ if (hs == NULL && _config->FindB("APT::Get::AllowUnauthenticated",false) == false)
+ {
+ ioprintf(c1out, "Skipping download of file '%s' as requested hashsum is not available for authentication\n",
+ localFile.c_str());
+ continue;
}
new pkgAcqFile(&Fetcher,Last->Index().ArchiveURI(I->Path),
- I->MD5Hash,I->Size,
+ hs != NULL ? hs->toStr() : "", I->FileSize,
Last->Index().SourceInfo(*Last,*I),Src);
}
}
diff --git a/debian/libapt-pkg4.12.symbols b/debian/libapt-pkg4.12.symbols
index d89f07b..d481e51 100644
--- a/debian/libapt-pkg4.12.symbols
+++ b/debian/libapt-pkg4.12.symbols
@@ -1587,6 +1587,8 @@ libapt-pkg.so.4.12 libapt-pkg4.12 #MINVER#
(c++)"HashStringList::VerifyFile(std::basic_string<char, std::char_traits<char>, std::allocator<char> >) const@Base" 1.0.9.4
(c++)"HashString::operator==(HashString const&) const@Base" 1.0.9.4
(c++)"HashString::operator!=(HashString const&) const@Base" 1.0.9.4
+ (c++)"pkgSrcRecords::Parser::Files2(std::vector<pkgSrcRecords::File2, std::allocator<pkgSrcRecords::File2> >&)@Base" 1.0.9.4
+ (c++)"debSrcRecordParser::Files2(std::vector<pkgSrcRecords::File2, std::allocator<pkgSrcRecords::File2> >&)@Base" 1.0.9.4
### demangle strangeness - buildd report it as MISSING and as new…
(c++)"pkgAcqMetaSig::pkgAcqMetaSig(pkgAcquire*, std::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::vector<IndexTarget*, std::allocator<IndexTarget*> > const*, indexRecords*)@Base" 0.8.0
### gcc-4.6 artefacts
diff --git a/test/integration/test-ubuntu-bug-1098738-apt-get-source-md5sum b/test/integration/test-ubuntu-bug-1098738-apt-get-source-md5sum
new file mode 100755
index 0000000..9bdc812
--- /dev/null
+++ b/test/integration/test-ubuntu-bug-1098738-apt-get-source-md5sum
@@ -0,0 +1,260 @@
+#!/bin/sh
+set -e
+
+TESTDIR=$(readlink -f $(dirname $0))
+. $TESTDIR/framework
+
+setupenvironment
+configarchitecture 'native'
+
+cat > aptarchive/Sources <<EOF
+Package: pkg-md5-ok
+Binary: pkg-md5-ok
+Version: 1.0
+Maintainer: Joe Sixpack <j...@example.org>
+Architecture: all
+Files:
+ d41d8cd98f00b204e9800998ecf8427e 0 pkg-md5-ok_1.0.dsc
+ d41d8cd98f00b204e9800998ecf8427e 0 pkg-md5-ok_1.0.tar.gz
+
+Package: pkg-sha256-ok
+Binary: pkg-sha256-ok
+Version: 1.0
+Maintainer: Joe Sixpack <j...@example.org>
+Architecture: all
+Files:
+ d41d8cd98f00b204e9800998ecf8427e 0 pkg-sha256-ok_1.0.dsc
+ d41d8cd98f00b204e9800998ecf8427e 0 pkg-sha256-ok_1.0.tar.gz
+Checksums-Sha1:
+ da39a3ee5e6b4b0d3255bfef95601890afd80709 0 pkg-sha256-ok_1.0.dsc
+ da39a3ee5e6b4b0d3255bfef95601890afd80709 0 pkg-sha256-ok_1.0.tar.gz
+Checksums-Sha256:
+ e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 0 pkg-sha256-ok_1.0.dsc
+ e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 0 pkg-sha256-ok_1.0.tar.gz
+
+Package: pkg-sha256-bad
+Binary: pkg-sha256-bad
+Version: 1.0
+Maintainer: Joe Sixpack <j...@example.org>
+Architecture: all
+Files:
+ d41d8cd98f00b204e9800998ecf8427e 0 pkg-sha256-bad_1.0.dsc
+ d41d8cd98f00b204e9800998ecf8427e 0 pkg-sha256-bad_1.0.tar.gz
+Checksums-Sha1:
+ da39a3ee5e6b4b0d3255bfef95601890afd80709 0 pkg-sha256-bad_1.0.dsc
+ da39a3ee5e6b4b0d3255bfef95601890afd80709 0 pkg-sha256-bad_1.0.tar.gz
+Checksums-Sha256:
+ aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa 0 pkg-sha256-bad_1.0.dsc
+ bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb 0 pkg-sha256-bad_1.0.tar.gz
+
+Package: pkg-no-md5
+Binary: pkg-no-md5
+Version: 1.0
+Maintainer: Joe Sixpack <j...@example.org>
+Architecture: all
+Checksums-Sha1:
+ da39a3ee5e6b4b0d3255bfef95601890afd80709 0 pkg-no-md5_1.0.dsc
+ da39a3ee5e6b4b0d3255bfef95601890afd80709 0 pkg-no-md5_1.0.tar.gz
+Checksums-Sha256:
+ e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 0 pkg-no-md5_1.0.dsc
+ e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 0 pkg-no-md5_1.0.tar.gz
+
+Package: pkg-mixed-ok
+Binary: pkg-mixed-ok
+Version: 1.0
+Maintainer: Joe Sixpack <j...@example.org>
+Architecture: all
+Checksums-Sha1:
+ da39a3ee5e6b4b0d3255bfef95601890afd80709 0 pkg-mixed-ok_1.0.tar.gz
+Checksums-Sha256:
+ e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 0 pkg-mixed-ok_1.0.dsc
+
+Package: pkg-mixed-sha1-bad
+Binary: pkg-mixed-sha1-bad
+Version: 1.0
+Maintainer: Joe Sixpack <j...@example.org>
+Architecture: all
+Checksums-Sha1:
+ aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa 0 pkg-mixed-sha1-bad_1.0.dsc
+Checksums-Sha256:
+ e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 0 pkg-mixed-sha1-bad_1.0.tar.gz
+
+Package: pkg-mixed-sha2-bad
+Binary: pkg-mixed-sha2-bad
+Version: 1.0
+Maintainer: Joe Sixpack <j...@example.org>
+Architecture: all
+Checksums-Sha1:
+ da39a3ee5e6b4b0d3255bfef95601890afd80709 0 pkg-mixed-sha2-bad_1.0.dsc
+Checksums-Sha256:
+ bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb 0 pkg-mixed-sha2-bad_1.0.tar.gz
+
+Package: pkg-md5-disagree
+Binary: pkg-md5-disagree
+Version: 1.0
+Maintainer: Joe Sixpack <j...@example.org>
+Architecture: all
+Files:
+ d41d8cd98f00b204e9800998ecf8427e 0 pkg-md5-disagree_1.0.dsc
+ d41d8cd98f00b204e9800998ecf8427e 0 pkg-md5-disagree_1.0.tar.gz
+ aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa 0 pkg-md5-disagree_1.0.dsc
+ bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb 0 pkg-md5-disagree_1.0.tar.gz
+
+Package: pkg-md5-agree
+Binary: pkg-md5-agree
+Version: 1.0
+Maintainer: Joe Sixpack <j...@example.org>
+Architecture: all
+Files:
+ d41d8cd98f00b204e9800998ecf8427e 0 pkg-md5-agree_1.0.dsc
+ d41d8cd98f00b204e9800998ecf8427e 0 pkg-md5-agree_1.0.tar.gz
+ d41d8cd98f00b204e9800998ecf8427e 0 pkg-md5-agree_1.0.tar.gz
+ d41d8cd98f00b204e9800998ecf8427e 0 pkg-md5-agree_1.0.dsc
+
+Package: pkg-sha256-disagree
+Binary: pkg-sha256-disagree
+Version: 1.0
+Maintainer: Joe Sixpack <j...@example.org>
+Architecture: all
+Files:
+ d41d8cd98f00b204e9800998ecf8427e 0 pkg-sha256-disagree_1.0.dsc
+ d41d8cd98f00b204e9800998ecf8427e 0 pkg-sha256-disagree_1.0.tar.gz
+Checksums-Sha1:
+ da39a3ee5e6b4b0d3255bfef95601890afd80709 0 pkg-sha256-disagree_1.0.dsc
+ da39a3ee5e6b4b0d3255bfef95601890afd80709 0 pkg-sha256-disagree_1.0.tar.gz
+Checksums-Sha256:
+ e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 0 pkg-sha256-disagree_1.0.dsc
+ e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 0 pkg-sha256-disagree_1.0.tar.gz
+ aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa 0 pkg-sha256-disagree_1.0.dsc
+ bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb 0 pkg-sha256-disagree_1.0.tar.gz
+EOF
+
+# create fetchable files
+for x in 'pkg-md5-ok' 'pkg-sha256-ok' 'pkg-sha256-bad' 'pkg-no-md5' \
+ 'pkg-mixed-ok' 'pkg-mixed-sha1-bad' 'pkg-mixed-sha2-bad' \
+ 'pkg-md5-agree' 'pkg-md5-disagree' 'pkg-sha256-disagree'; do
+ touch aptarchive/${x}_1.0.dsc aptarchive/${x}_1.0.tar.gz
+done
+
+setupaptarchive
+changetowebserver
+testsuccess aptget update
+
+testok() {
+ rm -f ${1}_1.0.dsc ${1}_1.0.tar.gz
+ testequal "Reading package lists...
+Building dependency tree...
+Need to get 0 B of source archives.
+Get:1 http://localhost:8080/ $1 1.0 (dsc)
+Get:2 http://localhost:8080/ $1 1.0 (tar)
+Download complete and in download only mode" aptget source -d "$@"
+ msgtest 'Files were successfully downloaded for' "$1"
+ testsuccess --nomsg test -e ${1}_1.0.dsc -a -e ${1}_1.0.tar.gz
+ rm -f ${1}_1.0.dsc ${1}_1.0.tar.gz
+}
+
+testkeep() {
+ touch ${1}_1.0.dsc ${1}_1.0.tar.gz
+ testequal "Reading package lists...
+Building dependency tree...
+Skipping already downloaded file '${1}_1.0.dsc'
+Skipping already downloaded file '${1}_1.0.tar.gz'
+Need to get 0 B of source archives.
+Download complete and in download only mode" aptget source -d "$@"
+ msgtest 'Files already downloaded are kept for' "$1"
+ testsuccess --nomsg test -e ${1}_1.0.dsc -a -e ${1}_1.0.tar.gz
+ rm -f ${1}_1.0.dsc ${1}_1.0.tar.gz
+}
+
+testmismatch() {
+ rm -f ${1}_1.0.dsc ${1}_1.0.tar.gz
+ testequal "Reading package lists...
+Building dependency tree...
+Need to get 0 B of source archives.
+Get:1 http://localhost:8080/ $1 1.0 (dsc)
+Get:2 http://localhost:8080/ $1 1.0 (tar)
+E: Failed to fetch http://localhost:8080/${1}_1.0.dsc Hash Sum mismatch
+
+E: Failed to fetch http://localhost:8080/${1}_1.0.tar.gz Hash Sum mismatch
+
+E: Failed to fetch some archives." aptget source -d "$@"
+ msgtest 'Files were not download as they have hashsum mismatches for' "$1"
+ testfailure --nomsg test -e ${1}_1.0.dsc -a -e ${1}_1.0.tar.gz
+
+ rm -f ${1}_1.0.dsc ${1}_1.0.tar.gz
+ testequal "Reading package lists...
+Building dependency tree...
+Skipping download of file 'pkg-sha256-bad_1.0.dsc' as requested hashsum is not available for authentication
+Skipping download of file 'pkg-sha256-bad_1.0.tar.gz' as requested hashsum is not available for authentication
+Need to get 0 B of source archives.
+Download complete and in download only mode" aptget source -d "$@" -o Acquire::ForceHash=ROT26
+ msgtest 'Files were not download as hash is unavailable for' "$1"
+ testfailure --nomsg test -e ${1}_1.0.dsc -a -e ${1}_1.0.tar.gz
+
+ rm -f ${1}_1.0.dsc ${1}_1.0.tar.gz
+ testequal "Reading package lists...
+Building dependency tree...
+Need to get 0 B of source archives.
+Get:1 http://localhost:8080/ $1 1.0 (dsc)
+Get:2 http://localhost:8080/ $1 1.0 (tar)
+Download complete and in download only mode" aptget source --allow-unauthenticated -d "$@" -o Acquire::ForceHash=ROT26
+ msgtest 'Files were downloaded unauthenticated as user allowed it' "$1"
+ testsuccess --nomsg test -e ${1}_1.0.dsc -a -e ${1}_1.0.tar.gz
+}
+
+testok pkg-md5-ok
+testkeep pkg-md5-ok
+testok pkg-sha256-ok
+testkeep pkg-sha256-ok
+
+# pkg-sha256-bad has a bad SHA sum, but good MD5 sum. If apt is
+# checking the best available hash (as it should), this will trigger
+# a hash mismatch.
+testmismatch pkg-sha256-bad
+testmismatch pkg-sha256-bad
+testok pkg-sha256-bad -o Acquire::ForceHash=MD5Sum
+
+# not having MD5 sum doesn't mean the file doesn't exist at all …
+testok pkg-no-md5
+testok pkg-no-md5 -o Acquire::ForceHash=SHA256
+testequal "Reading package lists...
+Building dependency tree...
+Skipping download of file 'pkg-no-md5_1.0.dsc' as requested hashsum is not available for authentication
+Skipping download of file 'pkg-no-md5_1.0.tar.gz' as requested hashsum is not available for authentication
+Need to get 0 B of source archives.
+Download complete and in download only mode" aptget source -d pkg-no-md5 -o Acquire::ForceHash=MD5Sum
+msgtest 'Files were not download as MD5 is not available for this package' 'pkg-no-md5'
+testfailure --nomsg test -e pkg-no-md5_1.0.dsc -a -e pkg-no-md5_1.0.tar.gz
+
+# deal with cases in which we haven't for all files the same checksum type
+# mostly pathologic as this shouldn't happen, but just to be sure
+testok pkg-mixed-ok
+testequal 'Reading package lists...
+Building dependency tree...
+Need to get 0 B of source archives.
+Get:1 http://localhost:8080/ pkg-mixed-sha1-bad 1.0 (tar)
+Get:2 http://localhost:8080/ pkg-mixed-sha1-bad 1.0 (dsc)
+E: Failed to fetch http://localhost:8080/pkg-mixed-sha1-bad_1.0.dsc Hash Sum mismatch
+
+E: Failed to fetch some archives.' aptget source -d pkg-mixed-sha1-bad
+msgtest 'Only tar file is downloaded as the dsc has hashsum mismatch' 'pkg-mixed-sha1-bad'
+testsuccess --nomsg test ! -e pkg-mixed-sha1-bad_1.0.dsc -a -e pkg-mixed-sha1-bad_1.0.tar.gz
+testequal 'Reading package lists...
+Building dependency tree...
+Need to get 0 B of source archives.
+Get:1 http://localhost:8080/ pkg-mixed-sha2-bad 1.0 (tar)
+Get:2 http://localhost:8080/ pkg-mixed-sha2-bad 1.0 (dsc)
+E: Failed to fetch http://localhost:8080/pkg-mixed-sha2-bad_1.0.tar.gz Hash Sum mismatch
+
+E: Failed to fetch some archives.' aptget source -d pkg-mixed-sha2-bad
+msgtest 'Only dsc file is downloaded as the tar has hashsum mismatch' 'pkg-mixed-sha2-bad'
+testsuccess --nomsg test -e pkg-mixed-sha2-bad_1.0.dsc -a ! -e pkg-mixed-sha2-bad_1.0.tar.gz
+
+# it gets even more pathologic: multiple entries for one file, some even disagreeing!
+testok pkg-md5-agree
+testequal 'Reading package lists...
+Building dependency tree...
+E: Error parsing checksum in Files of source package pkg-md5-disagree' aptget source -d pkg-md5-disagree
+testequal 'Reading package lists...
+Building dependency tree...
+E: Error parsing checksum in Checksums-SHA256 of source package pkg-sha256-disagree' aptget source -d pkg-sha256-disagree
commit 3f439e2b7126fb82952cd7bc12b8d6cb01352219
Author: David Kalnischkies <kalnischk...@gmail.com>
Date: Sun Aug 18 23:17:05 2013 +0200
add a simple container for HashStrings
APT supports more than just one HashString and even allows to enforce
the usage of a specific hash. This class is intended to help with
storage and passing around of the HashStrings.
The cherry-pick here the un-const-ification of HashType() compared to
f4c3850ea335545e297504941dc8c7a8f1c83358. The point of this commit is
adding infrastructure for the next one. All by itself, it just adds new
symbols.
Git-Dch: Ignore
diff --git a/apt-pkg/contrib/hashes.cc b/apt-pkg/contrib/hashes.cc
index 15f8361..bb11a3f 100644
--- a/apt-pkg/contrib/hashes.cc
+++ b/apt-pkg/contrib/hashes.cc
@@ -27,7 +27,7 @@
#include <iostream>
/*}}}*/
-const char* HashString::_SupportedHashes[] =
+const char * HashString::_SupportedHashes[] =
{
"SHA512", "SHA256", "SHA1", "MD5Sum", NULL
};
@@ -42,11 +42,16 @@ HashString::HashString(std::string Type, std::string Hash) : Type(Type), Hash(Ha
HashString::HashString(std::string StringedHash) /*{{{*/
{
- // legacy: md5sum without "MD5Sum:" prefix
- if (StringedHash.find(":") == std::string::npos && StringedHash.size() == 32)
+ if (StringedHash.find(":") == std::string::npos)
{
- Type = "MD5Sum";
- Hash = StringedHash;
+ // legacy: md5sum without "MD5Sum:" prefix
+ if (StringedHash.size() == 32)
+ {
+ Type = "MD5Sum";
+ Hash = StringedHash;
+ }
+ if(_config->FindB("Debug::Hashes",false) == true)
+ std::clog << "HashString(string): invalid StringedHash " << StringedHash << std::endl;
return;
}
std::string::size_type pos = StringedHash.find(":");
@@ -82,25 +87,25 @@ std::string HashString::GetHashForFile(std::string filename) const /*{{{*/
std::string fileHash;
FileFd Fd(filename, FileFd::ReadOnly);
- if(Type == "MD5Sum")
+ if(strcasecmp(Type.c_str(), "MD5Sum") == 0)
{
MD5Summation MD5;
MD5.AddFD(Fd);
fileHash = (std::string)MD5.Result();
}
- else if (Type == "SHA1")
+ else if (strcasecmp(Type.c_str(), "SHA1") == 0)
{
SHA1Summation SHA1;
SHA1.AddFD(Fd);
fileHash = (std::string)SHA1.Result();
}
- else if (Type == "SHA256")
+ else if (strcasecmp(Type.c_str(), "SHA256") == 0)
{
SHA256Summation SHA256;
SHA256.AddFD(Fd);
fileHash = (std::string)SHA256.Result();
}
- else if (Type == "SHA512")
+ else if (strcasecmp(Type.c_str(), "SHA512") == 0)
{
SHA512Summation SHA512;
SHA512.AddFD(Fd);
@@ -111,20 +116,105 @@ std::string HashString::GetHashForFile(std::string filename) const /*{{{*/
return fileHash;
}
/*}}}*/
-const char** HashString::SupportedHashes()
+const char** HashString::SupportedHashes() /*{{{*/
{
return _SupportedHashes;
}
-
-APT_PURE bool HashString::empty() const
+ /*}}}*/
+APT_PURE bool HashString::empty() const /*{{{*/
{
return (Type.empty() || Hash.empty());
}
+ /*}}}*/
+std::string HashString::toStr() const /*{{{*/
+{
+ return Type + ":" + Hash;
+}
+ /*}}}*/
+APT_PURE bool HashString::operator==(HashString const &other) const /*{{{*/
+{
+ return (strcasecmp(Type.c_str(), other.Type.c_str()) == 0 && Hash == other.Hash);
+}
+APT_PURE bool HashString::operator!=(HashString const &other) const
+{
+ return !(*this == other);
+}
+ /*}}}*/
+
+HashString const * HashStringList::find(char const * const type) const /*{{{*/
+{
+ if (type == NULL || type[0] == '\0')
+ {
+ std::string forcedType = _config->Find("Acquire::ForceHash", "");
+ if (forcedType.empty() == false)
+ return find(forcedType.c_str());
+ for (char const * const * t = HashString::SupportedHashes(); *t != NULL; ++t)
+ for (std::vector<HashString>::const_iterator hs = list.begin(); hs != list.end(); ++hs)
+ if (strcasecmp(hs->HashType().c_str(), *t) == 0)
+ return &*hs;
+ return NULL;
+ }
+ for (std::vector<HashString>::const_iterator hs = list.begin(); hs != list.end(); ++hs)
+ if (strcasecmp(hs->HashType().c_str(), type) == 0)
+ return &*hs;
+ return NULL;
+}
+ /*}}}*/
+bool HashStringList::supported(char const * const type) /*{{{*/
+{
+ for (char const * const * t = HashString::SupportedHashes(); *t != NULL; ++t)
+ if (strcasecmp(*t, type) == 0)
+ return true;
+ return false;
+}
+ /*}}}*/
+bool HashStringList::push_back(const HashString &hashString) /*{{{*/
+{
+ if (hashString.HashType().empty() == true ||
+ hashString.HashValue().empty() == true ||
+ supported(hashString.HashType().c_str()) == false)
+ return false;
+
+ // ensure that each type is added only once
+ HashString const * const hs = find(hashString.HashType().c_str());
+ if (hs != NULL)
+ return *hs == hashString;
-std::string HashString::toStr() const
+ list.push_back(hashString);
+ return true;
+}
+ /*}}}*/
+bool HashStringList::VerifyFile(std::string filename) const /*{{{*/
{
- return Type + std::string(":") + Hash;
+ if (list.empty() == true)
+ return false;
+ HashString const * const hs = find(NULL);
+ if (hs == NULL || hs->VerifyFile(filename) == false)
+ return false;
+ return true;
}
+ /*}}}*/
+bool HashStringList::operator==(HashStringList const &other) const /*{{{*/
+{
+ short matches = 0;
+ for (const_iterator hs = begin(); hs != end(); ++hs)
+ {
+ HashString const * const ohs = other.find(hs->HashType());
+ if (ohs == NULL)
+ continue;
+ if (*hs != *ohs)
+ return false;
+ ++matches;
+ }
+ if (matches == 0)
+ return false;
+ return true;
+}
+bool HashStringList::operator!=(HashStringList const &other) const
+{
+ return !(*this == other);
+}
+ /*}}}*/
// Hashes::AddFD - Add the contents of the FD /*{{{*/
// ---------------------------------------------------------------------
diff --git a/apt-pkg/contrib/hashes.h b/apt-pkg/contrib/hashes.h
index 7a62f8a..5a42138 100644
--- a/apt-pkg/contrib/hashes.h
+++ b/apt-pkg/contrib/hashes.h
@@ -17,6 +17,7 @@
#include <apt-pkg/md5.h>
#include <apt-pkg/sha1.h>
#include <apt-pkg/sha2.h>
+#include <apt-pkg/macros.h>
#include <cstring>
#include <string>
@@ -41,7 +42,7 @@ class HashString
protected:
std::string Type;
std::string Hash;
- static const char* _SupportedHashes[10];
+ static const char * _SupportedHashes[10];
// internal helper
std::string GetHashForFile(std::string filename) const;
@@ -53,6 +54,8 @@ class HashString
// get hash type used
std::string HashType() { return Type; };
+ std::string HashType() const { return Type; };
+ std::string HashValue() const { return Hash; };
// verify the given filename against the currently loaded hash
bool VerifyFile(std::string filename) const;
@@ -64,11 +67,90 @@ class HashString
// helper
std::string toStr() const; // convert to str as "type:hash"
bool empty() const;
+ bool operator==(HashString const &other) const;
+ bool operator!=(HashString const &other) const;
// return the list of hashes we support
static APT_CONST const char** SupportedHashes();
};
+class HashStringList
+{
+ public:
+ /** find best hash if no specific one is requested
+ *
+ * @param type of the checksum to return, can be \b NULL
+ * @return If type is \b NULL (or the empty string) it will
+ * return the 'best' hash; otherwise the hash which was
+ * specifically requested. If no hash is found \b NULL will be returned.
+ */
+ HashString const * find(char const * const type) const;
+ HashString const * find(std::string const &type) const { return find(type.c_str()); }
+ /** check if the given hash type is supported
+ *
+ * @param type to check
+ * @return true if supported, otherwise false
+ */
+ static APT_PURE bool supported(char const * const type);
+ /** add the given #HashString to the list
+ *
+ * @param hashString to add
+ * @return true if the hash is added because it is supported and
+ * not already a different hash of the same type included, otherwise false
+ */
+ bool push_back(const HashString &hashString);
+ /** @return size of the list of HashStrings */
+ size_t size() const { return list.size(); }
+
+ /** take the 'best' hash and verify file with it
+ *
+ * @param filename to verify
+ * @return true if the file matches the hashsum, otherwise false
+ */
+ bool VerifyFile(std::string filename) const;
+
+ /** is the list empty ?
+ *
+ * @return \b true if the list is empty, otherwise \b false
+ */
+ bool empty() const { return list.empty(); }
+
+ typedef std::vector<HashString>::const_iterator const_iterator;
+
+ /** iterator to the first element */
+ const_iterator begin() const { return list.begin(); }
+
+ /** iterator to the end element */
+ const_iterator end() const { return list.end(); }
+
+ /** start fresh with a clear list */
+ void clear() { list.clear(); }
+
+ /** compare two HashStringList for similarity.
+ *
+ * Two lists are similar if at least one hashtype is in both lists
+ * and the hashsum matches. All hashes are checked, if one doesn't
+ * match false is returned regardless of how many matched before.
+ */
+ bool operator==(HashStringList const &other) const;
+ bool operator!=(HashStringList const &other) const;
+
+ HashStringList() {}
+
+ // simplifying API-compatibility constructors
+ HashStringList(std::string const &hash) {
+ if (hash.empty() == false)
+ list.push_back(HashString(hash));
+ }
+ HashStringList(char const * const hash) {
+ if (hash != NULL && hash[0] != '\0')
+ list.push_back(HashString(hash));
+ }
+
+ private:
+ std::vector<HashString> list;
+};
+
class Hashes
{
public:
diff --git a/debian/libapt-pkg4.12.symbols b/debian/libapt-pkg4.12.symbols
index 3fa128c..d89f07b 100644
--- a/debian/libapt-pkg4.12.symbols
+++ b/debian/libapt-pkg4.12.symbols
@@ -1579,6 +1579,14 @@ libapt-pkg.so.4.12 libapt-pkg4.12 #MINVER#
(c++)"typeinfo for debTranslationsParser@Base" 1.0.4
(c++)"typeinfo name for debTranslationsParser@Base" 1.0.4
(c++)"vtable for debTranslationsParser@Base" 1.0.4
+ (c++)"HashStringList::find(char const*) const@Base" 1.0.9.4
+ (c++)"HashStringList::operator==(HashStringList const&) const@Base" 1.0.9.4
+ (c++)"HashStringList::operator!=(HashStringList const&) const@Base" 1.0.9.4
+ (c++)"HashStringList::push_back(HashString const&)@Base" 1.0.9.4
+ (c++)"HashStringList::supported(char const*)@Base" 1.0.9.4
+ (c++)"HashStringList::VerifyFile(std::basic_string<char, std::char_traits<char>, std::allocator<char> >) const@Base" 1.0.9.4
+ (c++)"HashString::operator==(HashString const&) const@Base" 1.0.9.4
+ (c++)"HashString::operator!=(HashString const&) const@Base" 1.0.9.4
### demangle strangeness - buildd report it as MISSING and as new…
(c++)"pkgAcqMetaSig::pkgAcqMetaSig(pkgAcquire*, std::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::vector<IndexTarget*, std::allocator<IndexTarget*> > const*, indexRecords*)@Base" 0.8.0
### gcc-4.6 artefacts
diff --git a/test/libapt/hashsums_test.cc b/test/libapt/hashsums_test.cc
index c06d85e..ac7d415 100644
--- a/test/libapt/hashsums_test.cc
+++ b/test/libapt/hashsums_test.cc
@@ -207,16 +207,56 @@ TEST(HashSumsTest, FileBased)
}
fd.Close();
- {
- HashString sha2("SHA256", sha256.Value());
- EXPECT_TRUE(sha2.VerifyFile(__FILE__));
- }
- {
- HashString sha2("SHA512", sha512.Value());
- EXPECT_TRUE(sha2.VerifyFile(__FILE__));
- }
- {
- HashString sha2("SHA256:" + sha256.Value());
- EXPECT_TRUE(sha2.VerifyFile(__FILE__));
- }
+ HashString sha2file("SHA512", sha512.Value());
+ EXPECT_TRUE(sha2file.VerifyFile(__FILE__));
+ HashString sha2wrong("SHA512", "00000000000");
+ EXPECT_FALSE(sha2wrong.VerifyFile(__FILE__));
+ EXPECT_EQ(sha2file, sha2file);
+ EXPECT_TRUE(sha2file == sha2file);
+ EXPECT_NE(sha2file, sha2wrong);
+ EXPECT_TRUE(sha2file != sha2wrong);
+
+ HashString sha2big("SHA256", sha256.Value());
+ EXPECT_TRUE(sha2big.VerifyFile(__FILE__));
+ HashString sha2small("sha256:" + sha256.Value());
+ EXPECT_TRUE(sha2small.VerifyFile(__FILE__));
+ EXPECT_EQ(sha2big, sha2small);
+ EXPECT_TRUE(sha2big == sha2small);
+ EXPECT_FALSE(sha2big != sha2small);
+
+ HashStringList hashes;
+ EXPECT_TRUE(hashes.empty());
+ EXPECT_TRUE(hashes.push_back(sha2file));
+ EXPECT_FALSE(hashes.empty());
+ EXPECT_EQ(1, hashes.size());
+
+ HashStringList wrong;
+ EXPECT_TRUE(wrong.push_back(sha2wrong));
+ EXPECT_NE(wrong, hashes);
+ EXPECT_FALSE(wrong == hashes);
+ EXPECT_TRUE(wrong != hashes);
+
+ HashStringList similar;
+ EXPECT_TRUE(similar.push_back(sha2big));
+ EXPECT_NE(similar, hashes);
+ EXPECT_FALSE(similar == hashes);
+ EXPECT_TRUE(similar != hashes);
+
+ EXPECT_TRUE(hashes.push_back(sha2big));
+ EXPECT_EQ(2, hashes.size());
+ EXPECT_TRUE(hashes.push_back(sha2small));
+ EXPECT_EQ(2, hashes.size());
+ EXPECT_FALSE(hashes.push_back(sha2wrong));
+ EXPECT_EQ(2, hashes.size());
+ EXPECT_TRUE(hashes.VerifyFile(__FILE__));
+
+ EXPECT_EQ(similar, hashes);
+ EXPECT_TRUE(similar == hashes);
+ EXPECT_FALSE(similar != hashes);
+ similar.clear();
+ EXPECT_TRUE(similar.empty());
+ EXPECT_EQ(0, similar.size());
+ EXPECT_NE(similar, hashes);
+ EXPECT_FALSE(similar == hashes);
+ EXPECT_TRUE(similar != hashes);
}
commit 28460cb27846b2437010b08adf10bde18e370974
Author: Michael Vogt <m...@debian.org>
Date: Thu Oct 23 14:32:01 2014 -0400
Fix incorrect comparison between signed/unsigned
Git-Dch: ignore
diff --git a/apt-pkg/deb/dpkgpm.cc b/apt-pkg/deb/dpkgpm.cc
index 2d26493..7bbf18c 100644
--- a/apt-pkg/deb/dpkgpm.cc
+++ b/apt-pkg/deb/dpkgpm.cc
@@ -1252,7 +1252,7 @@ bool pkgDPkgPM::GoNoABIBreak(APT::Progress::PackageManager *progress)
unsigned int const MaxArgs = _config->FindI("Dpkg::MaxArgs",32*1024);
// try to figure out the max environment size
- unsigned int OSArgMax = sysconf(_SC_ARG_MAX);
+ int OSArgMax = sysconf(_SC_ARG_MAX);
if(OSArgMax < 0)
OSArgMax = 32*1024;
OSArgMax -= EnvironmentSize() - 2*1024;
commit a3cada6abc42a2966c427a3b0731977ecfa7edcb
Author: Michael Vogt <m...@debian.org>
Date: Thu Oct 23 14:19:32 2014 -0400
Use sysconf(_SC_ARG_MAX) to find the size of Dpkg::MaxArgBytes
Instead of hardcoding Dpkg::MaxArgBytes find out about it using
the sysconf(_SC_ARG_MAX) call.
diff --git a/apt-pkg/deb/dpkgpm.cc b/apt-pkg/deb/dpkgpm.cc
index 04a13a8..2d26493 100644
--- a/apt-pkg/deb/dpkgpm.cc
+++ b/apt-pkg/deb/dpkgpm.cc
@@ -55,6 +55,18 @@
using namespace std;
+APT_PURE static unsigned int
+EnvironmentSize()
+{
+ unsigned int size = 0;
+ char **envp = environ;
+
+ while (*envp != NULL)
+ size += strlen (*envp++) + 1;
+
+ return size;
+}
+
class pkgDPkgPMPrivate
{
public:
@@ -1236,8 +1248,15 @@ bool pkgDPkgPM::GoNoABIBreak(APT::Progress::PackageManager *progress)
fd_set rfds;
struct timespec tv;
- unsigned int const MaxArgs = _config->FindI("Dpkg::MaxArgs",8*1024);
- unsigned int const MaxArgBytes = _config->FindI("Dpkg::MaxArgBytes",32*1024);
+ // FIXME: do we really need this limit when we have MaxArgBytes?
+ unsigned int const MaxArgs = _config->FindI("Dpkg::MaxArgs",32*1024);
+
+ // try to figure out the max environment size
+ unsigned int OSArgMax = sysconf(_SC_ARG_MAX);
+ if(OSArgMax < 0)
+ OSArgMax = 32*1024;
+ OSArgMax -= EnvironmentSize() - 2*1024;
+ unsigned int const MaxArgBytes = _config->FindI("Dpkg::MaxArgBytes", OSArgMax);
bool const NoTriggers = _config->FindB("DPkg::NoTriggers", false);
if (RunScripts("DPkg::Pre-Invoke") == false)
commit ee877edd6a92c094c6e70e392f3f6e64756281e2
Author: Michael Vogt <m...@debian.org>
Date: Wed Oct 22 17:54:08 2014 -0400
doc/examples/configure-index: make "Dpkg::Max{Arg,ArgBytes} match reality
Git-Dch: ignore
diff --git a/doc/examples/configure-index b/doc/examples/configure-index
index 2d9f829..56e7e1a 100644
--- a/doc/examples/configure-index
+++ b/doc/examples/configure-index
@@ -414,8 +414,8 @@ DPkg
FlushSTDIN "true";
// Control the size of the command line passed to dpkg.
- MaxBytes 1024;
- MaxArgs 350;
+ MaxArgBytes 32768;
+ MaxArgs 8192;
// controls if apt will apport on the first dpkg error or if it
// tries to install as many packages as possible
signature.asc
Description: Digital signature
