On 28/11/2015 19:42, Dan Stillman wrote:
On 11/28/15 5:06 AM, Gijs Kruitbosch wrote:
On 27/11/2015 23:46, [email protected] wrote:
The issue here is that this new system -- specifically, an automated
scanner sending extensions to manual review -- has been defended by
Jorge's saying, from March when I first brought this up until
yesterday on the hardening bug [1], that he believes the scanner can
"block the majority of malware".
Funny how you omit part of the quote you've listed elsewhere, namely:
"block the majority of malware, but it will never be perfect".
You assert the majority of malware will be 'smarter' than the
validator expects (possibly after initial rejection) and bypass it.
Jorge asserts, from years of experience, that malware authors are lazy
and the validator has already been helpful, in conjunction with manual
review. It's not helpful to say that what Jorge is saying is "not
true" - you mean different things when you say "the majority of malware".
I've addressed this repeatedly. In my view, saying "it will never be
perfect" is a misleading statement that betrays a misunderstanding of
the technical issues.
The validator has been used for years for AMO-published add-ons. It is
in that context that it did what it did, and it did so reasonably well -
there was always manual review for what the validator didn't catch. Not
all add-ons would be published on AMO, and there was no reason for
malware authors to publish on AMO except if they wanted to frontload it
and have the slightly-less-bumpy install flow that AMO offers because of
its default whitelist status in terms of sources of XPIs. Some people
did try this. I have no comprehensive data or anything, but I do not
believe that there was much if any such malware that made it to "fully
reviewed" status on AMO.
In that context, saying that the validator blocked the majority of
malware and would never be perfect was a perfectly valid statement and
does not imply any lack of technical understanding.
The change that muddies the waters here is that we're now signing
add-ons, and we're signing the ones that aren't distributed on AMO. We
do not have source code for non-AMO distributed add-ons pre-signing, and
so we have no way of knowing how much of that malware would or would not
be picked up by the validator as-is. It seems likely that, as-is, a lot
of it would be, because they have had no cause to use any techniques of
circumvention. It seems equally likely that they would proceed to use
such techniques in order to get signed anyway.
IOW, I think you're both right, and it would be helpful if you stopped
attacking Jorge and other folks because I don't think that is a
constructive discussion to be having.
If the person who's been defending this system for the last year
Jorge does not make decisions in a vacuum, wasn't the only person who
architected the current solution, isn't some kind of lone dictator,
doesn't get to decide what the Firefox team does (the team that actually
implemented signing on the browser side), and he isn't the only person
to have "defended the system" for whatever definition of system you're
using here (it's not clear) -- and really, as I have pointed out before,
even if he was all those things, it would not make "the system" any
better or worse. Stop making this all about Jorge and his supposed lack
of understanding. As I have repeatedly said, it is not helpful.
As for what, if anything, should block release without override, I'm
happy to talk specifics, but we can't have a discussion about that
without even agreeing on the point of the validator,
Why do we have to agree (and who are 'we'?) on the 'point' of the
validator? As it is, the people in this discussion without access to the
validator's results for zotero (which is almost everybody) have no idea
what you're running into and which things are bothering you because they
flag up false positives. For all we know you obfuscate all your code and
use eval(btoa(...)) all over the place. More seriously, all I remember
explicitly being mentioned is assigning to innerHTML in documents that
are not (and won't be) in any docshell and therefore shouldn't be
exploitable. It would be good to get a broader idea of the issues you're
actually running into (as I'm assuming there are more than just this
one, particularly because this one would be pretty easy to fix on your
side, as you say the line in question only runs under other browsers).
In any case, if you want to insist, here's my view: the point of the
validator is to raise the bar for both malware and for trivially-found
security holes in otherwise benign (or seemingly benign / greyware, if
you assume collusion between the 'benign' add-on submitted and the
website that will use that add-on's security hole for privilege
escalation of their website / remote code) add-ons. Raising the bar is
helpful so that editors don't waste time reviewing script kiddie or
copy-pasted / metasploit-style submissions of bad add-ons (assuming the
validator gets updated consistently, anything with 0 creativity compared
with a previous submission should be detectable in some way). The
validator won't be able to defeat a concerted malicious actor by itself,
and we should therefore manually review all add-ons in addition to what
the validator does. Yes, that means delays for everyone. I personally do
not believe that's avoidable if we want to make any kind of useful
guarantee about the add-ons.
For AMO-based add-ons, the validator should additionally help people
deal with Firefox code changes to interfaces, e10s, etc.
In my view, if the scanner can be trivially bypassed by malware authors
and is just an advisory tool, there's no justification for blocking
release.
Can we be specific, please? Release of new add-ons? Updates? Both? Just
AMO add-ons? Frontloaded and sideloaded non-AMO add-ons?
It should be seen as a linter, providing conscientious
developers with an opportunity to fix potential (but rarely unambiguous)
issues and flagging them for later review by AMO editors. If AMO editors
feel that developers are ignoring legitimate security issues, they could
temporarily rescind the ability to publish without review. Essentially,
I'm calling for whitelist-by-default.
As I've said before, I think this provides no improvement over the
pre-signing era. If we allow just anyone to register as a new developer
with a throwaway email address and automatically get whatever kind of
rubbish signed, with an API to do that to boot, I don't see what the
point of signing would be. Getting post-facto blocking in place for
those add-ons is of comparatively little use if (a) by that time the
add-on can e.g. stop Firefox updating and/or fetching the blocklist; (b)
people can resubmit the exact same thing with a different id and get
signed - all fully automatically (note that you're saying that the
validator shouldn't be allowed to stop signing, so even if we had
patterns of bad add-ons encoded in the validator, this would be possible).
And yes, using the automated scanner to try to combat malware is, in my
view, security theater: "the practice of investing in countermeasures
intended to provide the feeling of improved security while doing little
or nothing to actually achieve it" [2]. It may be a harsh assessment,
but I don't think it's unfair.
It is unfair when people have repeatedly pointed out that it has
actually achieved improved security. Perhaps not by as much as you would
like (ie not enough to dispense with manual review), but it raises the bar.
What you have now is a system that is extremely
disruptive to legitimate developers
I will just point out that not all legitimate developers seem to be
struggling as much with it as you do, so I don't know that your
generalization is justified. Struggling with signing, privately-run
add-ons, modifying public add-ons, the overall debate and its
consequences wrt e.g. government surveillance, centralizing a bunch of
infrastructure that used to be distributed - yes. Struggling
specifically with the automated portion of the review system for
frontloaded, non-AMO add-ons... not so much.
I don't know how many extensions are being flagged for manual review,
true.
This seems like something we should be able to get data about. (I do not
have such data.) Have you asked anyone?
But some certainly are, and for them it's extremely disruptive, to
the point where, in Zotero's case, we've decided that we would need to
cease development rather than be in a position where we couldn't release
timely updates to our users.
You have also dismissed all suggestions that you could fix at least some
of the issues the validator flags up (as noted above, I don't know what
all of them are, so I don't know if *all* of them are fixable, though
that seems likely enough to me), and that in case of "emergency" fixes,
you (and anyone else to whom this would apply) could email the amo
editors list to get an expedited review. We have reviewers in a number
of timezones, both paid and volunteer, and so response times are usually
pretty quick.
In other words, it is not like we have not provided you options. You
don't think the options are good enough, but it's disingenuous to
suggest that we haven't been trying to help. What we haven't been
willing to do is dispense with automated and manual review altogether.
~ Gijs
_______________________________________________
dev-platform mailing list
[email protected]
https://lists.mozilla.org/listinfo/dev-platform
