Yes, please use the existing fuzz-testing list. It’s basically a notifications list at this point due to differences in memory safety between Java and the C family making fuzzing a little trickier to reproduce security issues. — Matt Sicker
> On Nov 23, 2022, at 08:58, Mark Thomas <ma...@apache.org> wrote: > > On 21/11/2022 04:22, Oliver Chang wrote: >> Hi Mark, >> Thanks for the early feedback. >> Re a), unfortunately I'm not aware of an easy way to do this with our >> current bug tracking system (Monorail). If it's an important feature to >> have, one way to achieve this may be to set up a separate " >> security-oss-fuzz-not...@commons.apache.org" group or something similar to >> be CCed on all issues, which just forwards any notifications to the main " >> secur...@commons.apache.org" list. The main list can then filter out emails >> based on the recipient to avoid duplication. Would that work? > > Given that Monorail is a Google owned / controlled project I'd hope that such > a feature addition would be possible. > >> Re b), thank you for the feedback. We will be working on making our bug >> reports contain more actionable context in the notifications themselves. > > Thank you. > > > I have just finished reviewing approximately 50 oss-fuzz reports for Commons. > Give the excessive noise to signal ratio, the Apache Commons project has > disabled all email notifications from monorail to our security team unless we > explicitly mark the issue of interest. > > That gets us to a position where our security mailing list isn't swamped. > > We will continue to receive notifications for all issues at > fuzz-test...@commons.apache.org > > If you could ensure that fuzz-test...@commons.apache.org is on the CC for all > Apache Commons components that would ensure we don't miss anything. > > On reflection, it would probably be better if fuzz-test...@commons.apache.org > was the primary contact for all Commons components and > secur...@commons.apache.org was on the CC list. > > > The remaining major point is triage of discovered issues. We are still > putting together our thoughts on that given the high number of issues and > high false positive rate. > > Mark > > >> Best, >> Oliver >> On Sun, 20 Nov 2022 at 21:24, Mark Thomas <ma...@apache.org> wrote: >>> Hi Oliver, >>> >>> The following are a couple of (hopefully) low hanging fruit that will >>> smooth a couple of rough edges. These aren't the biggest issues - just >>> something to get started with. >>> >>> a) It would be very helpful if there was an option to enable sending of >>> notifications for your own comments. >>> >>> b) It would be helpful if more (actually all) of the issue detail was >>> included in the notification emails. >>> >>> Mark >>> >>> >>> On 18/11/2022 00:02, Oliver Chang wrote: >>>> Thanks Mark. >>>> >>>> Please let us know how we can help make this fuzzing experience better >>>> for you. We're also happy to jump on a call to walk through your >>>> concerns and reach a good outcome. >>>> >>>> Best regards, >>>> -- >>>> Oliver >>>> >>>> >>>> On Thu, 17 Nov 2022 at 06:56, Mark Thomas <ma...@apache.org >>>> <mailto:ma...@apache.org>> wrote: >>>> >>>> I haven't forgotten about this. I am currently working through the >>> open >>>> issues. I want to complete first that so feedback isn't skewed by a >>>> single project. >>>> >>>> Mark >>>> >>>> >>>> On 11/11/2022 14:45, Roman Wagner wrote: >>>> > Hi Mark, >>>> > >>>> > I think the best way forward is to collaborate and have a short >>>> feedback >>>> > loop. >>>> > >>>> > Did you mean build failures by “Invalid due to broken test”? If >>>> yes, I am >>>> > not sure what we can do about the broken tests since those are >>>> already >>>> > executed and tested by check build scripts locally and in a CI/CD >>>> pipeline. >>>> > Build and Coverage failures are sometimes supposed to happen if >>>> there are >>>> > changes in the target repository itself or if there are >>>> infrastructure >>>> > issues in OSS-Fuzz. We will investigate those issues in more >>>> detail. Maybe >>>> > some filter in the apache mailing list is helpful for you in the >>>> short >>>> > term, Fuzzing and Coverage build issues have a "build failure" >>>> string in >>>> > the subject. That would enable you to focus on the reports only. >>>> > >>>> > In order to make sure that we get high-quality tests and results, >>>> > maintainer feedback from apache will be very valuable and >>>> welcome. You have >>>> > the best domain knowledge about your code base and can give >>>> valuable hints >>>> > on which APIs to tackle best. There was already some valuable >>>> feedback for >>>> > Apache Tomcat in >>>> https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=53153 >>>> <https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=53153>. >>>> > Let us extend this collaboration. We can discuss and agree on >>>> the attack >>>> > vectors in apache-commons components. >>>> > >>>> > Best regards >>>> > Roman >>>> > >>>> > On Thu, Nov 10, 2022 at 10:29 AM Mark Thomas <ma...@apache.org >>>> <mailto:ma...@apache.org>> wrote: >>>> > >>>> >> Oliver, >>>> >> >>>> >> My requirements regarding configuration are: >>>> >> >>>> >> - secur...@commons.apache.org >>>> <mailto:secur...@commons.apache.org> MUST be notified of all >>> security >>>> >> vulnerability reports for all Apache Commons components >>>> >> >>>> >> - a mechanism MUST be provided for the >>>> secur...@commons.apache.org <mailto:secur...@commons.apache.org> >>>> >> Google user to view all historical reports that were not >>>> previously >>>> >> notified to that address >>>> >> >>>> >> - if any further Apache Commons components get added to oss-fuzz >>>> >> then secur...@commons.apache.org >>>> <mailto:secur...@commons.apache.org> MUST receive notification of >>> any >>>> >> issues as they are found >>>> >> >>>> >> - more generally, if *any* Apache Software Foundation project is >>>> added >>>> >> to oss-fuzz then the notifications for that project MUST >>>> include the >>>> >> relevant security team for that project >>>> >> >>>> >> If you can achieve the above with the current structure then >>> great. >>>> >> >>>> >> >>>> >> Separately, there is a concern regarding the false positive >>>> rate. With >>>> >> the oss-fuzz integration provided by Code Intelligence we have >>>> seen the >>>> >> following with Apache Tomcat in a little under 3 months. >>>> >> >>>> >> Total "vulnerability" reports: 39 >>>> >> >>>> >> Invalid due to broken test: 31% >>>> >> False positive: 52% >>>> >> Bugs: 18% >>>> >> Valid security issues: 0% >>>> >> >>>> >> To add some commentary: >>>> >> - the bugs were minor / extreme edge cases users were unlikely >>>> to hit >>>> >> - false positives were all due to the tests being based on >>> invalid >>>> >> assumptions regarding whether input was expected to be >>>> trusted or not >>>> >> >>>> >> >>>> >> If those statistics were repeated across multiple Apache Commons >>>> >> components, the volume of invalid reports would be more than the >>>> >> volunteers of the Apache Commons security team could handle. >>>> >> >>>> >> I have no objection to being overwhelmed with valid security >>>> >> vulnerability reports. If that ever happened, we would find a >>> way to >>>> >> deal with it. >>>> >> >>>> >> I do have very strong objections to being overwhelmed with >>> invalid >>>> >> security vulnerability reports. If we see the same false >>>> positive rate >>>> >> repeated across the Apache Commons components that has been >>> observed >>>> >> with Apache Tomcat then I don't see that Apache Commons would >>>> have any >>>> >> choice but to request the removal of all Apache Commons >>>> Components from >>>> >> oss-fuzz. >>>> >> >>>> >> Mark >>>> >> >>>> >> >>>> >> >>>> >> >>>> >> >>>> >> On 10/11/2022 04:19, Oliver Chang wrote: >>>> >>> Hi Mark, >>>> >>> >>>> >>> In addition to the reasons Roman listed, the current structure >>> also >>>> >>> allows us to allocate more compute resources to all of these >>> Apache >>>> >>> packages, rather than all of them sharing the CPUs allocated >>> for a >>>> >>> single OSS-Fuzz "project". >>>> >>> >>>> >>> We can definitely ensure that secur...@commons.apache.org >>>> <mailto:secur...@commons.apache.org> >>>> >>> <mailto:secur...@commons.apache.org >>>> <mailto:secur...@commons.apache.org>> is included on all relevant >>> Apache >>>> >>> projects going forward, and other than that I believe there's >>>> not much >>>> >>> other difference in terms of the end result (i.e. bug reports) >>>> that end >>>> >>> up getting filed. >>>> >>> >>>> >>> Does that sound OK to you? Or did you have other concerns >>>> around the >>>> >>> current directory structure? >>>> >>> >>>> >>> Best regards, >>>> >>> -- >>>> >>> Oliver >>>> >>> >>>> >>> >>>> >>> On Wed, 9 Nov 2022 at 21:31, Roman Wagner >>>> <wag...@code-intelligence.com <mailto:wag...@code-intelligence.com> >>>> >>> <mailto:wag...@code-intelligence.com >>>> <mailto:wag...@code-intelligence.com>>> wrote: >>>> >>> >>>> >>> Hi Mark, >>>> >>> >>>> >>> I have added @Oliver Chang <mailto:och...@google.com >>>> <mailto:och...@google.com>> from the >>>> >>> Google OSS-Fuzz to the thread. >>>> >>> >>>> >>> I had a short discussion with Oliver. There could be >>> different >>>> >>> issues in OSS-Fuzz by design If all apache-commons >>>> components will >>>> >>> move under apache-commons directory: >>>> >>> >>>> >>> * it is not scalable and will slow down both fuzzing and >>>> triage >>>> >>> (e.g. automated bisections, fix verification) >>>> >>> * changing the structure this way will invalidate all >>>> existing >>>> >>> open testcases, and cause new ones to be filed, which >>> will >>>> >>> result in a fair bit of spam. >>>> >>> >>>> >>> My proposal would be that "secur...@commons.apache.org >>>> <mailto:secur...@commons.apache.org> >>>> >>> <mailto:secur...@commons.apache.org >>>> <mailto:secur...@commons.apache.org>>" is added to all individual >>>> >>> apache-commons components. >>>> >>> I am not sure how it is possible to ensure that future >>>> onboardings >>>> >>> of apache-commons components will automatically have >>>> >>> "secur...@commons.apache.org >>>> <mailto:secur...@commons.apache.org> >>>> <mailto:secur...@commons.apache.org >>>> <mailto:secur...@commons.apache.org>>" >>>> >>> as primary contact. OSS-Fuzz could have some additional >>>> >>> documentation for that. @Oliver Chang >>>> <mailto:och...@google.com <mailto:och...@google.com>> do >>>> >>> you have any ideas here? >>>> >>> >>>> >>> Best regards >>>> >>> Roman >>>> >>> >>>> >>> On Tue, Nov 8, 2022 at 5:56 PM Mark Thomas >>>> <ma...@apache.org <mailto:ma...@apache.org> >>>> >>> <mailto:ma...@apache.org <mailto:ma...@apache.org>>> >>> wrote: >>>> >>> >>>> >>> Thanks for the update. >>>> >>> >>>> >>> I'll wait for that PR to be resolved before taking any >>>> further >>>> >>> action. >>>> >>> >>>> >>> Mark >>>> >>> >>>> >>> >>>> >>> On 08/11/2022 16:42, Roman Wagner wrote: >>>> >>> > Hi Mark, >>>> >>> > >>>> >>> > there is a PR open in oss-fuzz >>>> >>> https://github.com/google/oss-fuzz/pull/8933 >>>> <https://github.com/google/oss-fuzz/pull/8933> >>>> >>> <https://github.com/google/oss-fuzz/pull/8933 >>>> <https://github.com/google/oss-fuzz/pull/8933>> >>>> >>> > . >>>> >>> > >>>> >>> > Best regards >>>> >>> > Roman >>>> >>> > >>>> >>> > On Tue, Nov 8, 2022 at 4:15 PM Gary Gregory >>>> >>> <garydgreg...@gmail.com >>>> <mailto:garydgreg...@gmail.com> <mailto:garydgreg...@gmail.com >>>> <mailto:garydgreg...@gmail.com>>> wrote: >>>> >>> > >>>> >>> >> Sounds good. >>>> >>> >> >>>> >>> >> Gary >>>> >>> >> >>>> >>> >> On Tue, Nov 8, 2022, 10:07 Mark Thomas >>>> <ma...@apache.org <mailto:ma...@apache.org> >>>> >>> <mailto:ma...@apache.org <mailto:ma...@apache.org>>> >>>> wrote: >>>> >>> >> >>>> >>> >>> There has been no response to this email from >>>> anyone from >>>> >> Code >>>> >>> >>> Intelligence. >>>> >>> >>> >>>> >>> >>> Unless there are objections from the Apache >>> Commons >>>> >>> Community my next >>>> >>> >>> step will be to submit a PR to have the following >>>> modules >>>> >>> removed from >>>> >>> >>> oss-fuzz: >>>> >>> >>> >>>> >>> >>> apache-commons-bcel >>>> >>> >>> apache-commons-beanutils >>>> >>> >>> apache-commons-cli >>>> >>> >>> apache-commons-codec >>>> >>> >>> apache-commons-collections >>>> >>> >>> apache-commons-configuration >>>> >>> >>> apache-commons-io >>>> >>> >>> apache-commons-jxpath >>>> >>> >>> apache-commons-lang >>>> >>> >>> apache-commons-logging >>>> >>> >>> >>>> >>> >>> Code Intelligence (or anyone else) will remain >>>> free to add >>>> >>> them back in >>>> >>> >>> the right place - under apache-commons should >>>> they wish to >>>> >>> do so. >>>> >>> >>> >>>> >>> >>> Mark >>>> >>> >>> >>>> >>> >>> >>>> >>> >>> >>>> >>> >>> On 19/10/2022 10:56, Mark Thomas wrote: >>>> >>> >>>> Hi, >>>> >>> >>>> >>>> >>> >>>> You are receiving this email as you are currently >>>> >>> configured as the >>>> >>> >>>> recipients for oss-fuzz reports for Apache >>>> Commons JXPath. >>>> >>> >>>> >>>> >>> >>>> As per the discussion on the Apache Commons dev >>>> list[1], >>>> >>> please make >>>> >>> >>> the >>>> >>> >>>> following configuration changes to the oss-fuzz >>>> >>> integrations with >>>> >>> >>>> immediate effect: >>>> >>> >>>> >>>> >>> >>>> - Move all oss-fuzz integrations added for *ALL* >>>> Apache >>>> >>> Commons >>>> >>> >>>> components to the oss-fuzz module for >>>> Apache-Commons: >>>> >>> >>>> >>>> >>> >>>> >>>> >>> >>> >>>> >>> >>>> >> >>>> >>> https://github.com/google/oss-fuzz/tree/master/projects/apache-commons < >>> https://github.com/google/oss-fuzz/tree/master/projects/apache-commons> < >>>> >> >>>> >>> https://github.com/google/oss-fuzz/tree/master/projects/apache-commons < >>> https://github.com/google/oss-fuzz/tree/master/projects/apache-commons>> >>>> >>> >>>> >>>> >>> >>>> There should *NOT* be separate oss-fuzz >>>> modules for >>>> >>> each component >>>> >>> >>>> >>>> >>> >>>> >>>> >>> >>>> - Add the Google account for >>>> "secur...@commons.apache.org <mailto:secur...@commons.apache.org> >>>> >>> <mailto:secur...@commons.apache.org >>>> <mailto:secur...@commons.apache.org>>" to >>>> >>> >>>> - the notifications for these issues >>>> >>> >>>> - the ACL to enable this account to access >>>> the details >>>> >>> for each >>>> >>> >>> report >>>> >>> >>>> >>>> >>> >>>> >>>> >>> >>>> Please notify dev@commons.apache.org >>>> <mailto:dev@commons.apache.org> >>>> >>> <mailto:dev@commons.apache.org >>>> <mailto:dev@commons.apache.org>> and secur...@commons.apache.org >>>> <mailto:secur...@commons.apache.org> >>>> >>> <mailto:secur...@commons.apache.org >>>> <mailto:secur...@commons.apache.org>> >>>> >>> >>>> when these changes have been completed. >>>> >>> >>>> >>>> >>> >>>> Thanks, >>>> >>> >>>> >>>> >>> >>>> Mark >>>> >>> >>>> >>>> >>> >>>> >>>> >>> >>>> >>>> >>> >>>> [1] >>>> >>> >>>> https://lists.apache.org/thread/53vwy3g8w3f8nydz7jvxm8snrqx7msln >>>> <https://lists.apache.org/thread/53vwy3g8w3f8nydz7jvxm8snrqx7msln> >>>> >>> < >>>> >> https://lists.apache.org/thread/53vwy3g8w3f8nydz7jvxm8snrqx7msln >>>> <https://lists.apache.org/thread/53vwy3g8w3f8nydz7jvxm8snrqx7msln>> >>>> >>> >>>> >>>> >>> >>>> >>>> >>> >>>> >> >>>> >>> --------------------------------------------------------------------- >>>> >>> >>>> To unsubscribe, e-mail: >>>> dev-unsubscr...@commons.apache.org >>>> <mailto:dev-unsubscr...@commons.apache.org> >>>> >>> <mailto:dev-unsubscr...@commons.apache.org >>>> <mailto:dev-unsubscr...@commons.apache.org>> >>>> >>> >>>> For additional commands, e-mail: >>>> >>> dev-h...@commons.apache.org >>>> <mailto:dev-h...@commons.apache.org> >>>> <mailto:dev-h...@commons.apache.org >>>> <mailto:dev-h...@commons.apache.org>> >>>> >>> >>>> >>>> >>> >>> >>>> >>> >>> >>>> >>> >>>> >> >>>> >>> --------------------------------------------------------------------- >>>> >>> >>> To unsubscribe, e-mail: >>>> dev-unsubscr...@commons.apache.org >>>> <mailto:dev-unsubscr...@commons.apache.org> >>>> >>> <mailto:dev-unsubscr...@commons.apache.org >>>> <mailto:dev-unsubscr...@commons.apache.org>> >>>> >>> >>> For additional commands, e-mail: >>>> >>> dev-h...@commons.apache.org >>>> <mailto:dev-h...@commons.apache.org> >>>> <mailto:dev-h...@commons.apache.org >>>> <mailto:dev-h...@commons.apache.org>> >>>> >>> >>> >>>> >>> >>> >>>> >>> > >>>> >>> >>>> >>> >>>> >> >>>> >>> --------------------------------------------------------------------- >>>> >>> To unsubscribe, e-mail: >>>> dev-unsubscr...@commons.apache.org >>>> <mailto:dev-unsubscr...@commons.apache.org> >>>> >>> <mailto:dev-unsubscr...@commons.apache.org >>>> <mailto:dev-unsubscr...@commons.apache.org>> >>>> >>> For additional commands, e-mail: >>>> dev-h...@commons.apache.org <mailto:dev-h...@commons.apache.org> >>>> >>> <mailto:dev-h...@commons.apache.org >>>> <mailto:dev-h...@commons.apache.org>> >>>> >>> >>>> >>> >>>> >>> >>>> >>> -- >>>> >>> >>>> >>> Roman Wagner >>>> >>> Application Security Engineer >>>> >>> >>>> >>> Code Intelligence >>>> >>> Rheinwerkallee 6 >>>> >>> 53227 Bonn >>>> >>> >>>> >>> Amtsgericht Bonn >>>> >>> HRB 23408 >>>> >>> >>>> >>> Geschäftsführer: Sergej Dechand, Dr. Khaled Yakdan >>>> >>> >>>> >> >>>> > >>>> > >>>> >>> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org > For additional commands, e-mail: dev-h...@commons.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org For additional commands, e-mail: dev-h...@commons.apache.org
