Hi folks, Here are the results of our experiment with Firefox Nightly (draft-22) against Facebook.
EXPERIMENTAL DESIGN This is a forced experiment in which each client tries all the variants. The experiment is deployed via a system add-on (a remotely deployable, centrally managed piece of JavaScript code), and then takes measurements by trying to do an XHR to a given URL (https://www.tls13.facebook.com/) with a specific set of flags. We do the following three measurements: - TLS 1.2 - TLS 1.3 draft-22 - TLS 1.3 draft-22 in compat mode. We take five trials for each measurement, randomly shuffling the measurement order and then repeating the shuffled pattern five times. Each trial is done with a different connection and we declare "success" when any of the five trials succeeds. RESULTS This experiment was run on a 40% sample of the Firefox Nightly population who have locale set to en-US. The data below is taken from the period 20171216 to 20171222. There's a bit of contamination in the targeting because we temporarily failed to filter on "en-US", but that should mostly only affect the first day. 37716 clients started the experiment and 37430 completed it (99.2%). The results are: Success Fail Rate fb-tls12 35615 1815 0.048491 fb-tls13-draft-22 35552 1878 0.050174 fb-tls13-draft22-compat 35630 1800 0.048090 The overall failure numbers here are a lot higher than with our Beta experiment, which may be a result of different targeting on Nightly versus Beta. In particular, I'm still seeing a lot of data from China and Vietnam, which seem to have high blocking rates in general (i.e., not just for TLS 1.3). If I restrict to non China and non-Vietnam, we get: Success Fail Rate fb-tls12 35034 1176 0.032477 fb-tls13-draft-22 34960 1250 0.034521 fb-tls13-draft22-compat 35037 1173 0.032394 None of these differences are statistically significant (in the second data set, the p value for 1.2 versus -22 is .13), but this all seems consistent with saying that that -22 compat mode isn't significantly worse than TLS 1.2 and that normal -22 may be somewhat worse (unfortunately, we don't have -18 in this experiment). Taken together with the results David has reported and our previously reported Beta results, this seems fairly encouraging. We'll probably let the Nightly experiment run a little longer to see if we hit significance, but after that will start looking at a rollout of -22 to Release. -Ekr ADDITIONAL DETAILS Experimental code: https://github.com/mozilla/one-off-system-add-ons/tree/master/addons/tls13-middlebox-draft22
_______________________________________________ TLS mailing list TLS@ietf.org https://www.ietf.org/mailman/listinfo/tls
