Hi Oliver,
The following are a couple of (hopefully) low hanging fruit that will
smooth a couple of rough edges. These aren't the biggest issues - just
something to get started with.
a) It would be very helpful if there was an option to enable sending of
notifications for your own comments.
b) It would be helpful if more (actually all) of the issue detail was
included in the notification emails.
Mark
On 18/11/2022 00:02, Oliver Chang wrote:
Thanks Mark.
Please let us know how we can help make this fuzzing experience better
for you. We're also happy to jump on a call to walk through your
concerns and reach a good outcome.
Best regards,
--
Oliver
On Thu, 17 Nov 2022 at 06:56, Mark Thomas <ma...@apache.org
<mailto:ma...@apache.org>> wrote:
I haven't forgotten about this. I am currently working through the
open
issues. I want to complete first that so feedback isn't skewed by a
single project.
Mark
On 11/11/2022 14:45, Roman Wagner wrote:
> Hi Mark,
>
> I think the best way forward is to collaborate and have a short
feedback
> loop.
>
> Did you mean build failures by “Invalid due to broken test”? If
yes, I am
> not sure what we can do about the broken tests since those are
already
> executed and tested by check build scripts locally and in a CI/CD
pipeline.
> Build and Coverage failures are sometimes supposed to happen if
there are
> changes in the target repository itself or if there are
infrastructure
> issues in OSS-Fuzz. We will investigate those issues in more
detail. Maybe
> some filter in the apache mailing list is helpful for you in the
short
> term, Fuzzing and Coverage build issues have a "build failure"
string in
> the subject. That would enable you to focus on the reports only.
>
> In order to make sure that we get high-quality tests and results,
> maintainer feedback from apache will be very valuable and
welcome. You have
> the best domain knowledge about your code base and can give
valuable hints
> on which APIs to tackle best. There was already some valuable
feedback for
> Apache Tomcat in
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=53153
<https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=53153>.
> Let us extend this collaboration. We can discuss and agree on
the attack
> vectors in apache-commons components.
>
> Best regards
> Roman
>
> On Thu, Nov 10, 2022 at 10:29 AM Mark Thomas <ma...@apache.org
<mailto:ma...@apache.org>> wrote:
>
>> Oliver,
>>
>> My requirements regarding configuration are:
>>
>> - secur...@commons.apache.org
<mailto:secur...@commons.apache.org> MUST be notified of all
security
>> vulnerability reports for all Apache Commons components
>>
>> - a mechanism MUST be provided for the
secur...@commons.apache.org <mailto:secur...@commons.apache.org>
>> Google user to view all historical reports that were not
previously
>> notified to that address
>>
>> - if any further Apache Commons components get added to oss-fuzz
>> then secur...@commons.apache.org
<mailto:secur...@commons.apache.org> MUST receive notification of
any
>> issues as they are found
>>
>> - more generally, if *any* Apache Software Foundation project is
added
>> to oss-fuzz then the notifications for that project MUST
include the
>> relevant security team for that project
>>
>> If you can achieve the above with the current structure then
great.
>>
>>
>> Separately, there is a concern regarding the false positive
rate. With
>> the oss-fuzz integration provided by Code Intelligence we have
seen the
>> following with Apache Tomcat in a little under 3 months.
>>
>> Total "vulnerability" reports: 39
>>
>> Invalid due to broken test: 31%
>> False positive: 52%
>> Bugs: 18%
>> Valid security issues: 0%
>>
>> To add some commentary:
>> - the bugs were minor / extreme edge cases users were unlikely
to hit
>> - false positives were all due to the tests being based on
invalid
>> assumptions regarding whether input was expected to be
trusted or not
>>
>>
>> If those statistics were repeated across multiple Apache Commons
>> components, the volume of invalid reports would be more than the
>> volunteers of the Apache Commons security team could handle.
>>
>> I have no objection to being overwhelmed with valid security
>> vulnerability reports. If that ever happened, we would find a
way to
>> deal with it.
>>
>> I do have very strong objections to being overwhelmed with
invalid
>> security vulnerability reports. If we see the same false
positive rate
>> repeated across the Apache Commons components that has been
observed
>> with Apache Tomcat then I don't see that Apache Commons would
have any
>> choice but to request the removal of all Apache Commons
Components from
>> oss-fuzz.
>>
>> Mark
>>
>>
>>
>>
>>
>> On 10/11/2022 04:19, Oliver Chang wrote:
>>> Hi Mark,
>>>
>>> In addition to the reasons Roman listed, the current structure
also
>>> allows us to allocate more compute resources to all of these
Apache
>>> packages, rather than all of them sharing the CPUs allocated
for a
>>> single OSS-Fuzz "project".
>>>
>>> We can definitely ensure that secur...@commons.apache.org
<mailto:secur...@commons.apache.org>
>>> <mailto:secur...@commons.apache.org
<mailto:secur...@commons.apache.org>> is included on all relevant
Apache
>>> projects going forward, and other than that I believe there's
not much
>>> other difference in terms of the end result (i.e. bug reports)
that end
>>> up getting filed.
>>>
>>> Does that sound OK to you? Or did you have other concerns
around the
>>> current directory structure?
>>>
>>> Best regards,
>>> --
>>> Oliver
>>>
>>>
>>> On Wed, 9 Nov 2022 at 21:31, Roman Wagner
<wag...@code-intelligence.com <mailto:wag...@code-intelligence.com>
>>> <mailto:wag...@code-intelligence.com
<mailto:wag...@code-intelligence.com>>> wrote:
>>>
>>> Hi Mark,
>>>
>>> I have added @Oliver Chang <mailto:och...@google.com
<mailto:och...@google.com>> from the
>>> Google OSS-Fuzz to the thread.
>>>
>>> I had a short discussion with Oliver. There could be
different
>>> issues in OSS-Fuzz by design If all apache-commons
components will
>>> move under apache-commons directory:
>>>
>>> * it is not scalable and will slow down both fuzzing and
triage
>>> (e.g. automated bisections, fix verification)
>>> * changing the structure this way will invalidate all
existing
>>> open testcases, and cause new ones to be filed, which
will
>>> result in a fair bit of spam.
>>>
>>> My proposal would be that "secur...@commons.apache.org
<mailto:secur...@commons.apache.org>
>>> <mailto:secur...@commons.apache.org
<mailto:secur...@commons.apache.org>>" is added to all individual
>>> apache-commons components.
>>> I am not sure how it is possible to ensure that future
onboardings
>>> of apache-commons components will automatically have
>>> "secur...@commons.apache.org
<mailto:secur...@commons.apache.org>
<mailto:secur...@commons.apache.org
<mailto:secur...@commons.apache.org>>"
>>> as primary contact. OSS-Fuzz could have some additional
>>> documentation for that. @Oliver Chang
<mailto:och...@google.com <mailto:och...@google.com>> do
>>> you have any ideas here?
>>>
>>> Best regards
>>> Roman
>>>
>>> On Tue, Nov 8, 2022 at 5:56 PM Mark Thomas
<ma...@apache.org <mailto:ma...@apache.org>
>>> <mailto:ma...@apache.org <mailto:ma...@apache.org>>>
wrote:
>>>
>>> Thanks for the update.
>>>
>>> I'll wait for that PR to be resolved before taking any
further
>>> action.
>>>
>>> Mark
>>>
>>>
>>> On 08/11/2022 16:42, Roman Wagner wrote:
>>> > Hi Mark,
>>> >
>>> > there is a PR open in oss-fuzz
>>> https://github.com/google/oss-fuzz/pull/8933
<https://github.com/google/oss-fuzz/pull/8933>
>>> <https://github.com/google/oss-fuzz/pull/8933
<https://github.com/google/oss-fuzz/pull/8933>>
>>> > .
>>> >
>>> > Best regards
>>> > Roman
>>> >
>>> > On Tue, Nov 8, 2022 at 4:15 PM Gary Gregory
>>> <garydgreg...@gmail.com
<mailto:garydgreg...@gmail.com> <mailto:garydgreg...@gmail.com
<mailto:garydgreg...@gmail.com>>> wrote:
>>> >
>>> >> Sounds good.
>>> >>
>>> >> Gary
>>> >>
>>> >> On Tue, Nov 8, 2022, 10:07 Mark Thomas
<ma...@apache.org <mailto:ma...@apache.org>
>>> <mailto:ma...@apache.org <mailto:ma...@apache.org>>>
wrote:
>>> >>
>>> >>> There has been no response to this email from
anyone from
>> Code
>>> >>> Intelligence.
>>> >>>
>>> >>> Unless there are objections from the Apache
Commons
>>> Community my next
>>> >>> step will be to submit a PR to have the following
modules
>>> removed from
>>> >>> oss-fuzz:
>>> >>>
>>> >>> apache-commons-bcel
>>> >>> apache-commons-beanutils
>>> >>> apache-commons-cli
>>> >>> apache-commons-codec
>>> >>> apache-commons-collections
>>> >>> apache-commons-configuration
>>> >>> apache-commons-io
>>> >>> apache-commons-jxpath
>>> >>> apache-commons-lang
>>> >>> apache-commons-logging
>>> >>>
>>> >>> Code Intelligence (or anyone else) will remain
free to add
>>> them back in
>>> >>> the right place - under apache-commons should
they wish to
>>> do so.
>>> >>>
>>> >>> Mark
>>> >>>
>>> >>>
>>> >>>
>>> >>> On 19/10/2022 10:56, Mark Thomas wrote:
>>> >>>> Hi,
>>> >>>>
>>> >>>> You are receiving this email as you are currently
>>> configured as the
>>> >>>> recipients for oss-fuzz reports for Apache
Commons JXPath.
>>> >>>>
>>> >>>> As per the discussion on the Apache Commons dev
list[1],
>>> please make
>>> >>> the
>>> >>>> following configuration changes to the oss-fuzz
>>> integrations with
>>> >>>> immediate effect:
>>> >>>>
>>> >>>> - Move all oss-fuzz integrations added for *ALL*
Apache
>>> Commons
>>> >>>> components to the oss-fuzz module for
Apache-Commons:
>>> >>>>
>>> >>>>
>>> >>>
>>>
>>
https://github.com/google/oss-fuzz/tree/master/projects/apache-commons <
https://github.com/google/oss-fuzz/tree/master/projects/apache-commons> <
>>
https://github.com/google/oss-fuzz/tree/master/projects/apache-commons <
https://github.com/google/oss-fuzz/tree/master/projects/apache-commons>>
>>> >>>>
>>> >>>> There should *NOT* be separate oss-fuzz
modules for
>>> each component
>>> >>>>
>>> >>>>
>>> >>>> - Add the Google account for
"secur...@commons.apache.org <mailto:secur...@commons.apache.org>
>>> <mailto:secur...@commons.apache.org
<mailto:secur...@commons.apache.org>>" to
>>> >>>> - the notifications for these issues
>>> >>>> - the ACL to enable this account to access
the details
>>> for each
>>> >>> report
>>> >>>>
>>> >>>>
>>> >>>> Please notify dev@commons.apache.org
<mailto:dev@commons.apache.org>
>>> <mailto:dev@commons.apache.org
<mailto:dev@commons.apache.org>> and secur...@commons.apache.org
<mailto:secur...@commons.apache.org>
>>> <mailto:secur...@commons.apache.org
<mailto:secur...@commons.apache.org>>
>>> >>>> when these changes have been completed.
>>> >>>>
>>> >>>> Thanks,
>>> >>>>
>>> >>>> Mark
>>> >>>>
>>> >>>>
>>> >>>>
>>> >>>> [1]
>>>
https://lists.apache.org/thread/53vwy3g8w3f8nydz7jvxm8snrqx7msln
<https://lists.apache.org/thread/53vwy3g8w3f8nydz7jvxm8snrqx7msln>
>>> <
>> https://lists.apache.org/thread/53vwy3g8w3f8nydz7jvxm8snrqx7msln
<https://lists.apache.org/thread/53vwy3g8w3f8nydz7jvxm8snrqx7msln>>
>>> >>>>
>>> >>>>
>>>
>>
---------------------------------------------------------------------
>>> >>>> To unsubscribe, e-mail:
dev-unsubscr...@commons.apache.org
<mailto:dev-unsubscr...@commons.apache.org>
>>> <mailto:dev-unsubscr...@commons.apache.org
<mailto:dev-unsubscr...@commons.apache.org>>
>>> >>>> For additional commands, e-mail:
>>> dev-h...@commons.apache.org
<mailto:dev-h...@commons.apache.org>
<mailto:dev-h...@commons.apache.org
<mailto:dev-h...@commons.apache.org>>
>>> >>>>
>>> >>>
>>> >>>
>>>
>>
---------------------------------------------------------------------
>>> >>> To unsubscribe, e-mail:
dev-unsubscr...@commons.apache.org
<mailto:dev-unsubscr...@commons.apache.org>
>>> <mailto:dev-unsubscr...@commons.apache.org
<mailto:dev-unsubscr...@commons.apache.org>>
>>> >>> For additional commands, e-mail:
>>> dev-h...@commons.apache.org
<mailto:dev-h...@commons.apache.org>
<mailto:dev-h...@commons.apache.org
<mailto:dev-h...@commons.apache.org>>
>>> >>>
>>> >>>
>>> >
>>>
>>>
>>
---------------------------------------------------------------------
>>> To unsubscribe, e-mail:
dev-unsubscr...@commons.apache.org
<mailto:dev-unsubscr...@commons.apache.org>
>>> <mailto:dev-unsubscr...@commons.apache.org
<mailto:dev-unsubscr...@commons.apache.org>>
>>> For additional commands, e-mail:
dev-h...@commons.apache.org <mailto:dev-h...@commons.apache.org>
>>> <mailto:dev-h...@commons.apache.org
<mailto:dev-h...@commons.apache.org>>
>>>
>>>
>>>
>>> --
>>>
>>> Roman Wagner
>>> Application Security Engineer
>>>
>>> Code Intelligence
>>> Rheinwerkallee 6
>>> 53227 Bonn
>>>
>>> Amtsgericht Bonn
>>> HRB 23408
>>>
>>> Geschäftsführer: Sergej Dechand, Dr. Khaled Yakdan
>>>
>>
>
>
