Re: Three steps we could take to make supply chain attacks a bit harder

2024-04-01 Thread François Rigault
To echo > To trust code, it needs to be reviewed. > If the code is reviewed, and the build system is sane, [..] I deduce from your response that the binary tests committed in systemd were not reviewed neither by co-maintainers nor by downstream package maintainers. I understand that the build

Re: Three steps we could take to make supply chain attacks a bit harder

2024-04-01 Thread Zbigniew Jędrzejewski-Szmek
On Sun, Mar 31, 2024 at 07:54:08PM +0200, Kevin Kofler via devel wrote: > Adam Williamson wrote: > > Maybe this needs to go on the growing pile of reasons why the > > traditional Linux model *does* need to go away. Maybe Fedora, with its > > foundation of First, should be kind of at the forefront o

Re: Three steps we could take to make supply chain attacks a bit harder

2024-04-01 Thread Zbigniew Jędrzejewski-Szmek
On Mon, Apr 01, 2024 at 08:46:39AM -, François Rigault wrote: > To echo > > > To trust code, it needs to be reviewed. > > If the code is reviewed, and the build system is sane, [..] > > I deduce from your response that the binary tests committed in > systemd were not reviewed neither by co-

Re: Three steps we could take to make supply chain attacks a bit harder

2024-04-01 Thread Zbigniew Jędrzejewski-Szmek
On Mon, Apr 01, 2024 at 09:06:16AM +0900, Dominique Martinet wrote: > Scott Schmit wrote on Sun, Mar 31, 2024 at 05:02:44PM -0400: > > Deleting the tests makes no sense to me either, but it seems like a > > mechanism that ensures the test code can't change the build outputs (or > > a mechanism to d

Re: Three steps we could take to make supply chain attacks a bit harder

2024-04-01 Thread Zbigniew Jędrzejewski-Szmek
On Sun, Mar 31, 2024 at 11:20:17PM -0700, Adam Williamson wrote: > On Sun, 2024-03-31 at 22:13 -0700, Carlos Rodriguez-Fernandez wrote: > > Adam, > > > > Is there a way already to achieve test isolation during the rpm build? > > Nothing systematic that I'm aware of, no. It would be tricky because

Re: Three steps we could take to make supply chain attacks a bit harder

2024-04-01 Thread Neal Gompa
On Mon, Apr 1, 2024 at 7:38 AM Zbigniew Jędrzejewski-Szmek wrote: > > On Sun, Mar 31, 2024 at 11:20:17PM -0700, Adam Williamson wrote: > > On Sun, 2024-03-31 at 22:13 -0700, Carlos Rodriguez-Fernandez wrote: > > > Adam, > > > > > > Is there a way already to achieve test isolation during the rpm bu

Re: Three steps we could take to make supply chain attacks a bit harder

2024-04-01 Thread Stephen Smoogen
On Mon, 1 Apr 2024 at 04:47, François Rigault wrote: > To echo > > > To trust code, it needs to be reviewed. > > If the code is reviewed, and the build system is sane, [..] > > I deduce from your response that the binary tests committed in systemd > were not reviewed neither by co-maintainers nor

Re: Three steps we could take to make supply chain attacks a bit harder

2024-04-01 Thread Sam Varshavchik
Christoph Erhardt writes: I strongly oppose this suggestion. While it would have prevented this particular backdoor as a side-effect, the primary effect of going without unit tests would be an outsize hole in Fedora's QA. There have been several suggestions here for ways that this specific

Re: xz backdoor

2024-04-01 Thread Christopher Klooz
On 31/03/2024 21.33, Sandro wrote: On 31-03-2024 20:54, Christopher Klooz wrote: On 31/03/2024 20.52, Christopher Klooz wrote: On 31/03/2024 20.21, Michael Catanzaro wrote: On Sun, Mar 31 2024 at 09:56:04 AM -05:00:00, Michael Catanzaro wrote: I'm really frustrated with our communication r

Re: Three steps we could take to make supply chain attacks a bit harder

2024-04-01 Thread Carlos Rodriguez-Fernandez
Test isolation is still assuming the attack comes in the test phase. The attack can come in the `make`, or in the `make install` too. That's why the idea of other techniques being discussed are still valid, but perhaps not abstracted out enough for a wider defense. However, the test isolatio

Fedora rawhide compose report: 20240401.n.0 changes

2024-04-01 Thread Fedora Rawhide Report
OLD: Fedora-Rawhide-20240331.n.0 NEW: Fedora-Rawhide-20240401.n.0 = SUMMARY = Added images:3 Dropped images: 1 Added packages: 1 Dropped packages:0 Upgraded packages: 17 Downgraded packages: 0 Size of added packages: 68.66 KiB Size of dropped packages:0 B

Re: Three steps we could take to make supply chain attacks a bit harder

2024-04-01 Thread François Rigault
> Those blobs were not in systemd, that was not my point, nevertheless putting it this way: nobody knows. For the example about compression methods you could generate your binary using a piece of code, that can be reviewed (maybe using a fixed seed as inspired by https://git.rootprojects.org/roo

Re: xz backdoor

2024-04-01 Thread Michael Catanzaro
On Sun, Mar 31 2024 at 06:52:53 PM +00:00:00, Christopher Klooz wrote: "Fedora Linux 40 branched users (i.e. pre-Beta) likely received the potentially vulnerable 5.6.0-2.fc40 build if the system updated between March 2nd and March 6th. Fedora Linux 40 Beta users only using stable repositories

Re: Three steps we could take to make supply chain attacks a bit harder

2024-04-01 Thread Carlos Rodriguez-Fernandez
I created a discussion issue for this idea: https://github.com/rpm-software-management/rpm/discussions/3009 I think it worth pursuing further. On 4/1/24 04:46, Neal Gompa wrote: On Mon, Apr 1, 2024 at 7:38 AM Zbigniew Jędrzejewski-Szmek wrote: On Sun, Mar 31, 2024 at 11:20:17PM -0700, Adam

Re: Three steps we could take to make supply chain attacks a bit harder

2024-04-01 Thread Carlos Rodriguez-Fernandez
Definitely this attack leveraged places where eyes don't look: distributed tar.gz and blobs. I put the PoC to flag those two in github[1] Example output: $ ./rpmseclint tests/rpmseclint-test.spec -Diff- ~ test.txt + additional.txt + blob.txt.gz -Blobs application/gzip blob

Fedora 40 compose report: 20240401.n.0 changes

2024-04-01 Thread Fedora Branched Report
OLD: Fedora-40-20240331.n.0 NEW: Fedora-40-20240401.n.0 = SUMMARY = Added images:3 Dropped images: 0 Added packages: 4 Dropped packages:0 Upgraded packages: 82 Downgraded packages: 0 Size of added packages: 4.82 MiB Size of dropped packages:0 B Size of

Outage alert: openQA result reporting (affects critical path gating), nightly page updating, candidate compose nominations

2024-04-01 Thread Adam Williamson
Hi folks! I just discovered this so I'm still investigating it, but wanted to give a quick heads-up. It looks like the message consumers on openqa01 all broke on Saturday when a fedora-messaging update landed. This affects a lot of things, but by far the most important is that openQA test results

Re: Outage alert: openQA result reporting (affects critical path gating), nightly page updating, candidate compose nominations

2024-04-01 Thread Adam Williamson
On Mon, 2024-04-01 at 08:53 -0700, Adam Williamson wrote: > as a stopgap I > will manually trigger submission of all reports from the last couple of > days shortly. correction: I won't do this right away, as there would be a flood of duplicate reports if I did then fix the consumers. If I can't fi

Re: Three steps we could take to make supply chain attacks a bit harder

2024-04-01 Thread Miroslav Suchý
Dne 01. 04. 24 v 3:16 dop. Kilian Hanich via devel napsal(a): Also, I have seen build setups which encode the status of tests in the eventual binary and as such info page or integrated bug report generators. Often because some distros sometimes turned them off or ships software even with failed t

Re: Three steps we could take to make supply chain attacks a bit harder

2024-04-01 Thread Scott Schmit
On Mon, Apr 01, 2024 at 02:23:19PM -, François Rigault wrote: > > Those blobs were not in systemd, > > that was not my point, nevertheless putting it this way: nobody knows. > > For the example about compression methods you could generate your binary > using a piece of code, that can be revi

Re: Three steps we could take to make supply chain attacks a bit harder

2024-04-01 Thread Adam Williamson
On Mon, 2024-04-01 at 10:56 +, Zbigniew Jędrzejewski-Szmek wrote: > On Sun, Mar 31, 2024 at 07:54:08PM +0200, Kevin Kofler via devel wrote: > > Adam Williamson wrote: > > > Maybe this needs to go on the growing pile of reasons why the > > > traditional Linux model *does* need to go away. Maybe

Re: Three steps we could take to make supply chain attacks a bit harder

2024-04-01 Thread Neal Gompa
On Mon, Apr 1, 2024 at 12:22 PM Adam Williamson wrote: > > On Mon, 2024-04-01 at 10:56 +, Zbigniew Jędrzejewski-Szmek wrote: > > On Sun, Mar 31, 2024 at 07:54:08PM +0200, Kevin Kofler via devel wrote: > > > Adam Williamson wrote: > > > > Maybe this needs to go on the growing pile of reasons wh

Re: Three steps we could take to make supply chain attacks a bit harder

2024-04-01 Thread Adam Williamson
On Mon, 2024-04-01 at 05:58 -0700, Carlos Rodriguez-Fernandez wrote: > Test isolation is still assuming the attack comes in the test phase. As I initially suggested it, it does not. My suggestion was that we ensure the test code is not available to the prep / build / install phases *at all*, and i

Schedule for Monday's FESCo Meeting (2024-04-01)

2024-04-01 Thread Josh Stone
Following is the list of topics that will be discussed in the FESCo meeting Monday at 19:30 UTC in #meeting:fedoraproject.org on Matrix. To convert UTC to your local time, take a look at http://fedoraproject.org/wiki/UTCHowto or run: date -d '2024-04-01 19:30 UTC' Links to all issues to be d

What we mean when we talk about "supply chains" [was Re: Three steps we could take to make supply chain attacks a bit harder]

2024-04-01 Thread Adam Williamson
On Mon, 2024-04-01 at 12:27 -0400, Neal Gompa wrote: > > > > ii) the fact that this attack reinforces the painful truth that > > sophisticated attackers *are* extremely interested in attacking the > > supply chain of which we form a significant component > > Can we please reframe it for what it a

Re: xz backdoor

2024-04-01 Thread Christopher Klooz
On 01/04/2024 16.32, Michael Catanzaro wrote: On Sun, Mar 31 2024 at 06:52:53 PM +00:00:00, Christopher Klooz wrote: "Fedora Linux 40 branched users (i.e. pre-Beta) likely received the potentially vulnerable 5.6.0-2.fc40 build if the system updated between March 2nd and March 6th. Fedora Li

Re: Three steps we could take to make supply chain attacks a bit harder

2024-04-01 Thread Carlos Rodriguez-Fernandez
Understood. However, at least for those unit tests run in the %check, it is going to be almost unfeasible, because of the variability of the way things are done in the different programming ecosystems. In Java, unit tests are nicely separated in a different folder (i.e., `src/test`), but in gol

Re: xz backdoor

2024-04-01 Thread Christopher Klooz
On 31/03/2024 23.08, Kevin Fenzi wrote: On Sun, Mar 31, 2024 at 10:30:23PM +0200, Leon Fauster via devel wrote: Not sure, if it was already mentioned -> containers. I had here a toolbox environment with F40. That I had not in my first actions on the screen. The last state had 5.6.0-3 installed

Re: What we mean when we talk about "supply chains" [was Re: Three steps we could take to make supply chain attacks a bit harder]

2024-04-01 Thread Gary Buhrmaster
On Mon, Apr 1, 2024 at 4:42 PM Adam Williamson wrote: > I think we *are* part of a supply chain, regardless of any handwaving > about The Open Source Model. And, more importantly, the industry has agreed to use the term supply chain. Is the term perhaps overloaded, or perhaps too ill-defined/im

Re: xz backdoor

2024-04-01 Thread Adam Williamson
On Mon, 2024-04-01 at 09:32 -0500, Michael Catanzaro wrote: > On Sun, Mar 31 2024 at 06:52:53 PM +00:00:00, Christopher Klooz > wrote: > > "Fedora Linux 40 branched users (i.e. pre-Beta) likely received the > > potentially vulnerable 5.6.0-2.fc40 build if the system updated > > between March 2n

Re: xz backdoor

2024-04-01 Thread Michael Catanzaro
On Mon, Apr 1 2024 at 10:12:55 AM -07:00:00, Adam Williamson wrote: This is not really correct, or at least at all relevant. The bug wasn't in F40 Beta simply because the update never made it to 'stable'. Only 'stable' packages go into *composes*. However, saying that is not really useful becau

Re: xz backdoor

2024-04-01 Thread Adam Williamson
On Mon, 2024-04-01 at 12:16 -0500, Michael Catanzaro wrote: > On Mon, Apr 1 2024 at 10:12:55 AM -07:00:00, Adam Williamson > wrote: > > This is not really correct, or at least at all relevant. The bug > > wasn't > > in F40 Beta simply because the update never made it to 'stable'. Only > > 'stabl

Re: xz backdoor

2024-04-01 Thread Kevin Fenzi
On Mon, Apr 01, 2024 at 05:07:13PM +, Christopher Klooz wrote: > > On 31/03/2024 23.08, Kevin Fenzi wrote: > > On Sun, Mar 31, 2024 at 10:30:23PM +0200, Leon Fauster via devel wrote: > > > Not sure, if it was already mentioned -> containers. I had here a toolbox > > > environment with F40. Tha

Re: xz backdoor

2024-04-01 Thread Christopher Klooz
On 01/04/2024 19.27, Kevin Fenzi wrote: On Mon, Apr 01, 2024 at 05:07:13PM +, Christopher Klooz wrote: On 31/03/2024 23.08, Kevin Fenzi wrote: On Sun, Mar 31, 2024 at 10:30:23PM +0200, Leon Fauster via devel wrote: Not sure, if it was already mentioned -> containers. I had here a toolbox

Re: Three steps we could take to make supply chain attacks a bit harder

2024-04-01 Thread Peter Jones
> (3) We should have a "security path", like "critical path". > > sshd is linked to a lot of libraries: > > /lib64/libaudit.so.1audit-libs > /lib64/libc.so.6glibc > /lib64/libcap-ng.so.0 libcap-ng > /lib64/libcap.so.2 libcap > /lib64/libcom_err.so.2 libcom_

Re: xz backdoor

2024-04-01 Thread Gary Buhrmaster
On Mon, Apr 1, 2024 at 5:27 PM Kevin Fenzi wrote: > Yes. The downgrade was pushed out on friday along with the f40 one. Of course, your mirror may vary as to availability (as I recall, in my particular case, my test VM for rawhide did not get the update for a day or so). It does bring up a pote

Re: xz backdoor

2024-04-01 Thread Sandro
On 01-04-2024 19:12, Adam Williamson wrote: On Mon, 2024-04-01 at 09:32 -0500, Michael Catanzaro wrote: On Sun, Mar 31 2024 at 06:52:53 PM +00:00:00, Christopher Klooz wrote: "Fedora Linux 40 branched users (i.e. pre-Beta) likely received the potentially vulnerable 5.6.0-2.fc40 build if the sy

Re: Outage alert: openQA result reporting (affects critical path gating), nightly page updating, candidate compose nominations

2024-04-01 Thread Adam Williamson
On Mon, 2024-04-01 at 08:53 -0700, Adam Williamson wrote: > Hi folks! I just discovered this so I'm still investigating it, but > wanted to give a quick heads-up. > > It looks like the message consumers on openqa01 all broke on Saturday > when a fedora-messaging update landed. This affects a lot o

Re: xz backdoor

2024-04-01 Thread Michael Catanzaro
On Mon, Apr 1 2024 at 10:25:16 AM -07:00:00, Adam Williamson wrote: Oh, ISWYM. Well, I suppose yes, that does happen to be true. We could communicate that if it's done very carefully and made really clear that it's about the *time frame*, nothing to do with the repositories. It's been brough

Summary/Minutes from today's FESCo Meeting (2024-04-01)

2024-04-01 Thread Josh Stone
(sorry for the crazy links -- meetbot didn't grab the intended name) Text Log: https://meetbot.fedoraproject.org/meeting_matrix_fedoraproject-org/2024-04-01/fesco-chairs-conan-kudo-matrix-org-ngompa-fedora-im-nirik-matrix-scrye-com-humaton-fedora-im-zbyszek-fedora-im-sgallagh-fedora-im-jistone-fed

Re: Three steps we could take to make supply chain attacks a bit harder

2024-04-01 Thread Matthew Miller
On Sat, Mar 30, 2024 at 08:11:38PM +0100, Kevin Kofler via devel wrote: > Unit tests are something for upstream developers. They should NEVER be run > in a distribution build. Even in the few little packages I'm still responsible for, I sometimes see unit test failures. The developer ran the test

Re: xz backdoor

2024-04-01 Thread Matthew Miller
On Mon, Apr 01, 2024 at 05:47:10PM +, Gary Buhrmaster wrote: > It does bring up a potential point that perhaps > Fedora should have an additional repo (let's > call it "emergency fixes") that is not community > mirrored (so any mirrors for load sharing > would be fully controlled by the project

Re: Three steps we could take to make supply chain attacks a bit harder

2024-04-01 Thread Jakub Jelinek
On Mon, Apr 01, 2024 at 01:36:48PM -0400, Peter Jones wrote: > Unrelated to the idea that some packages are special in this way, it's > probably worth writing some static analysis tools we could put into > rpm-inspect to detect when (a) a binary grows new public keys it didn't > have before, and (b

Re: Three steps we could take to make supply chain attacks a bit harder

2024-04-01 Thread Kevin Kofler via devel
Adam Williamson wrote: >> * Deleting ALL files automatically generated or imported by autotools in >> %prep, THEN running "autoreconf -i -f". (DO NOT trust autoreconf, it >> would NOT have done the right thing here. Delete the files, THEN run >> autoreconf.) > > No. This would not have avoided the

Re: xz backdoor

2024-04-01 Thread Gary Buhrmaster
On Mon, Apr 1, 2024 at 9:17 PM Matthew Miller wrote: > > On Mon, Apr 01, 2024 at 05:47:10PM +, Gary Buhrmaster wrote: > > It does bring up a potential point that perhaps > > Fedora should have an additional repo (let's > > call it "emergency fixes") that is not community > > mirrored (so any m

Re: Three steps we could take to make supply chain attacks a bit harder

2024-04-01 Thread Adam Williamson
On Mon, 2024-04-01 at 23:37 +0200, Kevin Kofler via devel wrote: > Adam Williamson wrote: > > > * Deleting ALL files automatically generated or imported by autotools in > > > %prep, THEN running "autoreconf -i -f". (DO NOT trust autoreconf, it > > > would NOT have done the right thing here. Delete

Re: Three steps we could take to make supply chain attacks a bit harder

2024-04-01 Thread Gabriel Somlo
> On Mon, Apr 1, 2024 at 17:11:46 -0400, Matthew Miller via devel wrote: > On Sat, Mar 30, 2024 at 08:11:38PM +0100, Kevin Kofler via devel wrote: > > Unit tests are something for upstream developers. They should NEVER be run > > in a distribution build. > > Even in the few little packages I'm st

Re: Three steps we could take to make supply chain attacks a bit harder

2024-04-01 Thread Chris Adams
Once upon a time, Gabriel Somlo said: > IMHO, there's no good way to *programmatically* protect ourselves > from a malicious upstream on which we depend. If their goal is to > compromise us, they will work around whatever programmatic/technical > measures we happen to have in place at the time the

Re: Three steps we could take to make supply chain attacks a bit harder

2024-04-01 Thread Carlos Rodriguez-Fernandez
Chris, The specific points of entry were evading the strength of open source: many skilled eyes. Therefore, there is value in programmatically enforcing that everything used as an input in a build must have been exposed to *normal opensource workflows*. It is a very simple principle, yet very

Re: Three steps we could take to make supply chain attacks a bit harder

2024-04-01 Thread Carlos Rodriguez-Fernandez
Matthew Miller, Unit tests, even though in theory developer should mock dependencies to isolate their code to the maximum, in reality, it is not that clear cut. Therefore, those unit tests do serve to some extent as a validation that their code works with the system libraries and platforms pre

Re: Three steps we could take to make supply chain attacks a bit harder

2024-04-01 Thread Andreas Schneider
On Saturday, 30 March 2024 10:37:44 CEST Richard W.M. Jones wrote: > These are just my thoughts on a Saturday morning. Feedback welcome of > course. I find the use of the ifunc attribute is really uncommon at this place. I would expect it in ffmpeg or some media codecs. In xz it looks like it is

[Test Day] Fedora CoreOS 40 Test Week

2024-04-01 Thread Sumantro Mukherjee
Hey All, During the week of April 1- April 7 2024, the Fedora 40 CoreOS Test Week will be happening. The test week is an opportunity for the community to test FCOS based on Fedora 40 content before it is released as part of the `testing` and `stable` streams. As part of the Test Week, we'll host a

Re: Three steps we could take to make supply chain attacks a bit harder

2024-04-01 Thread Gordon Messmer
On 2024-03-30 13:18, Gordon Messmer wrote: The write up describing the back door indicates that the malicious xz library "changes the value of rsa_public_decr...@plt to point to its own code."  So the back door has pointed one of the symbols that should point to a page mapped to OpenSSL's