Re: [gentoo-dev] Manifest2 hashes, take n+1-th
Hi, So here's my proposed plan, after considering all the replies. Immediately after accepting --- a. Revbump Portage to add pyblake2 dep (to ensure BLAKE2 is supported on py<3.6) and request stabilizing this version. b. Create a git update hook that rejects Manifest entries that contain SHA512 only, to prevent a bug in current versions of Portage, that causes it to skip BLAKE2 when no implementation is installed instead of complaining [optional]. Now, let T = day when the new version is stable on amd64. T + 7 days -- Set: manifest-hashes = BLAKE2B SHA512 manifest-required-hashes = SHA512 New Manifest entries will use the new hashes but Portage will keep the old hash set whenever it would need to refetch old distfiles. T + 3 months Set: manifest-required-hashes = BLAKE2B Portage will now request updating hashes for all files, including old distfiles. We will start proactively updating Manifests here, and file bugs for fetch-restricted packages. T + 6 months All Manifests should use the new hashes by this time. The remaining fetch-restricted packages should be last-rited. T + 36 months - Set: manifest-hashes = BLAKE2B Remove SHA512 from all Manifests. -- Best regards, Michał Górny
Re: [gentoo-dev] Manifest2 hashes, take n+1-th
+1 overall, just one timeline clarification. On Mon, Nov 06, 2017 at 05:58:21PM +0100, Michał Górny wrote: > T + 7 days > -- > Set: > manifest-hashes = BLAKE2B SHA512 > manifest-required-hashes = SHA512 > > New Manifest entries will use the new hashes but Portage will keep the > old hash set whenever it would need to refetch old distfiles. Query: Do we need to wait for it to be stable before making this change? Shouldn't old stable versions of Portage continue to verify SHA512 fine? Mostly I think devs need to be using a new enough Portage that can generate the BLAKE2B entries, but it shouldn't impact user Portage versions. -- Robin Hugh Johnson Gentoo Linux: Dev, Infra Lead, Foundation Asst. Treasurer E-Mail : robb...@gentoo.org GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85 GnuPG FP : 7D0B3CEB E9B85B1F 825BCECF EE05E6F6 A48F6136 signature.asc Description: Digital signature
Re: [gentoo-dev] Manifest2 hashes, take n+1-th
On Mon, Nov 6, 2017 at 2:13 PM, Robin H. Johnson wrote: > +1 overall, just one timeline clarification. > > On Mon, Nov 06, 2017 at 05:58:21PM +0100, Michał Górny wrote: >> T + 7 days >> -- >> Set: >> manifest-hashes = BLAKE2B SHA512 >> manifest-required-hashes = SHA512 >> >> New Manifest entries will use the new hashes but Portage will keep the >> old hash set whenever it would need to refetch old distfiles. > Query: > Do we need to wait for it to be stable before making this change? > Shouldn't old stable versions of Portage continue to verify SHA512 fine? > Mostly I think devs need to be using a new enough Portage that can > generate the BLAKE2B entries, but it shouldn't impact user Portage > versions. Quite a few devs use stable versions of Portage and Repoman when committing.
Re: [gentoo-dev] Manifest2 hashes, take n+1-th
W dniu pon, 06.11.2017 o godzinie 19∶13 +, użytkownik Robin H. Johnson napisał: > +1 overall, just one timeline clarification. > > On Mon, Nov 06, 2017 at 05:58:21PM +0100, Michał Górny wrote: > > T + 7 days > > -- > > Set: > > manifest-hashes = BLAKE2B SHA512 > > manifest-required-hashes = SHA512 > > > > New Manifest entries will use the new hashes but Portage will keep the > > old hash set whenever it would need to refetch old distfiles. > > Query: > Do we need to wait for it to be stable before making this change? > Shouldn't old stable versions of Portage continue to verify SHA512 fine? > Mostly I think devs need to be using a new enough Portage that can > generate the BLAKE2B entries, but it shouldn't impact user Portage > versions. > Devs are who I'm worried about. Those 7 days should give them enough time to upgrade their stable Portage. -- Best regards, Michał Górny
Re: [gentoo-dev] [v1.0.3] GLEP 74: Full-tree verification using Manifest files
On Sun, Nov 05, 2017 at 10:10:32PM +0100, Michał Górny wrote: > > Nits: > > - please stick to ASCII ellipsis. The unicode ellipsis is unreadable in > > some monospace fonts. > Done. Also replaced '—' for consistency. I wasn't even aware you had used a different dash, it was rendered identically here, definitely thanks for fixing that too. > > Further items inline: > > > Directory tree coverage > > > --- > I've went for something even more explicit: > | If files or directories that are not otherwise ignored reside > | on a different filesystem, or symbolic links point to targets > | on a different filesystem, they must be explicitly excluded > | via ``IGNORE``. +1, resolves the concern very well, nice and clear. > > > Tree layout restrictions > > > > > 'common' in the second sentence seems odd. What about uncommon > > filenames? Maybe just s/other common filenames/other filenames/. > Done. The idea was to say 'do not put IGNORE for corner cases which are > better handled via PM config' but I guess it's not necessary here. Yes. Generally, IGNORE entries in Manifest should be for files distributed alongside the Manifest. We're say as common special cases, that local/distfiles/packages/lost+found are also known for ignore, since they have previously-defined meaning in the repo (along with the old timestamp files). > > > Non-strict Manifest verification > > > > Rewritten to: > | It is much more common for users to strip whole packages > | or categories. The ``MISC`` type is not suitable for that, > | and so a dedicated package manager mechanism needs to be developed > | instead; possibly combining it with rsync exclusion list. The same > | mechanism can also handle files that historically used the ``MISC`` > | type. > But it's merely a rationale, so I'd rather not spend another hour trying > to cover every corner case in it. +1. Maybe cover it with a single sentence, "As an example, the package manager may choose to generate both the rsync exclusion list and Manifest IGNORE based on a source list" -- Robin Hugh Johnson Gentoo Linux: Dev, Infra Lead, Foundation Asst. Treasurer E-Mail : robb...@gentoo.org GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85 GnuPG FP : 7D0B3CEB E9B85B1F 825BCECF EE05E6F6 A48F6136 signature.asc Description: Digital signature
[gentoo-dev] dev-php/PEAR-HTTP_Download
# Brian Evans signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] [v1.0.4] GLEP 74: Full-tree verification using Manifest files
Hopefully the last version, after getting all the suggestions from Robin. W dniu czw, 26.10.2017 o godzinie 22∶12 +0200, użytkownik Michał Górny napisał: > > ReST: https://dev.gentoo.org/~mgorny/tmp/glep-0074.rst > HTML: https://dev.gentoo.org/~mgorny/tmp/glep-0074.html > impl: https://github.com/mgorny/gemato/ > --- GLEP: 74 Title: Full-tree verification using Manifest files Author: Michał Górny , Robin Hugh Johnson , Ulrich Müller Type: Standards Track Status: Draft Version: 1 Created: 2017-10-21 Last-Modified: 2017-11-06 Post-History: 2017-10-26 Content-Type: text/x-rst Requires: 59, 61 Replaces: 44, 58, 60 --- Abstract This GLEP extends the Manifest file format to cover full-tree file integrity and authenticity checks.The format aims to be future-proof, efficient and provide means of backwards compatibility. Motivation == The Manifest files as defined by GLEP 44 [#GLEP44]_ provide the current means of verifying the integrity of distfiles and package files in Gentoo. Combined with OpenPGP signatures, they provide means to ensure the authenticity of the covered files. However, as noted in GLEP 57 [#GLEP57]_ they lack the ability to provide full-tree authenticity verification as they do not cover any files outside the package directory. In particular, they provide multiple ways for a third party to inject malicious code into the ebuild environment. Historically, the topic of providing authenticity coverage for the whole repository has been mentioned multiple times. The most noteworthy effort are GLEPs 58 [#GLEP58]_ and 60 [#GLEP60]_ by Robin H. Johnson from 2008. They were accepted by the Council in 2010 but have never been implemented. When potential implementation work started in 2017, a new discussion about the specification arose. It prompted the creation of a competing GLEP that would provide a redesigned alternative to the old GLEPs. This specification is designed with the following goals in mind: 1. It should provide means to ensure the authenticity of the complete repository, including preventing the injection of additional files. 2. The format should be universal enough to work both for the Gentoo repository and third-party repositories of different characteristics. 3. The Manifest files should be verifiable stand-alone, that is without knowing any details about the underlying repository format. Specification = Manifest file format This specification reuses and extends the Manifest file format defined in GLEP 44 [#GLEP44]_. For the purpose of it, the *file type* field is repurposed as a generic *tag* that could also indicate additional (non-checksum) metadata. Appropriately, those tags can be followed by other space-separated values. Unless specified otherwise, the paths used in the Manifest files are relative to the directory containing the Manifest file. The paths must not reference the parent directory (``..``). Manifest file locations and nesting --- The ``Manifest`` file located in the root directory of the repository is called top-level Manifest, and it is used to perform the full-tree verification. In order to verify the authenticity, it must be signed using OpenPGP, using the armored cleartext format. The top-level Manifest may reference sub-Manifests contained in subdirectories of the repository. The sub-Manifests are traditionally named ``Manifest``; however, the implementation must support arbitrary names, including the possibility of multiple (split) Manifests for a single directory. The sub-Manifest can only cover the files inside the directory tree where it resides. The sub-Manifest can also be signed using OpenPGP armored cleartext format. However, the signature verification can be omitted if it is covered by a signed top-level Manifest. Directory tree coverage --- The specification provides three ways of skipping Manifest verification of specific files and directories (recursively): 1. explicit ``IGNORE`` entries in Manifest files, 2. injected ignore paths via package manager configuration, 3. using names starting with a dot (``.``) which are always skipped. All files that are not ignored must be covered by at least one of the Manifests. A single file may be matched by multiple identical or equivalent Manifest entries, if and only if the entries have the same semantics, specify the same size and the checksums common to both entries match. It is an error for a single file to be matched by multiple entries of different semantics, file size or checksum values. It is an error to specify another entry for a file matching ``IGNORE``, or one of its subdirectories. The file entries (except for ``IGNORE``) can be specified for regular files only. Symbolic links are followed when opening files and traversing directories. It is an error to specify an entry for a different file type. If the tree contain files of other types that are n
