On Tue, Dec 5, 2017 at 1:35 PM, Eric Rescorla <e...@rtfm.com> wrote: > Hi folks, > > I now have some preliminary numbers to share with the group based on > our Firefox experiments. The executive summary is that our data > confirms Google's results. More detail below. > > > EXPERIMENTAL DESIGN > This is a forced experiment in which each client tries all the > variants. The experiment is deployed via a system add-on (a remotely > deployable, centrally managed piece of JavaScript code), and then > takes measurements by trying to do an XHR to a given URL > (https://mail.google.com/robots.txt) with a specific set of flags. We > do the following three measurements: > > - TLS 1.2 > - TLS 1.3 draft-18 > - TLS 1.3 draft-18 with (approximately) PR#1092 ("7e02") > > We take five trials for each measurement, randomly shuffling the > measurement order and then repeating the shuffled pattern five > times. Each trial is done with a different connection and we declare > "success" when any of the five trials succeeds. > > > RESULTS > This experiment was run on a 2% sample of the Firefox Beta population > who have locale set to en-US, which we selected because of very > high GMail blocking rates in some locales, which is a potential > confounding factor. The experimen started 11/27 and has been running > through today. > > This gave us an initial population of 161578, of whom 160809 (99.5% > completed the experiment and reported results). This produced the > following results: > > Success Failure Fail Rate > -------------------------------------------------------- > > TLS 1.2 158260 2549 .0158 > TLS 1.3-18 158194 4743 .0291 >
Oops. This first number should be 156066. This is what happens when you cut and paste from your notebook. -Ekr TLS 1.3-Experiment 158194 2615 .0163 > > For the statistics minded, the difference between -18 and 1.2 is > significant at p < .001 and the 95% confidence interval of the failure > rate difference is .0122-.0143 (using R's prop.test). There is no > significant difference between 1.2 and 1.3-experiment (p = .36). > > We've got a -22 experiment in flight now, but it will only be on > Nightly, so this is probably the strongest data we will have for > a while. > > -Ekr > > > ADDITIONAL DETAILS > The relevant NSS version: https://dxr.mozilla.org/ > mozilla-beta/source/security/nss/lib/ssl > Experimental code: https://github.com/mozilla/one-off-system-add-ons/tree/ > master/addons/tls13-middlebox-ghack > iPython Notebook with analysis: https://gist.github.com/ekr/ > 598208b5399faf303453b10cb11647bf > > > > > > >
_______________________________________________ TLS mailing list TLS@ietf.org https://www.ietf.org/mailman/listinfo/tls
