I’ve continued analyzing UKM results and have now reached n=40 as Rick suggested:
| origin | link to analysis | continues to work with the change? | | :--------------------------------------- | :---------------------------- | :--------------------------------- | | https://account.amazon.jobs | https://crbug.com/1412729#c23 | ✅ | | https://code.earthengine.google.com | https://crbug.com/1412729#c41 | ✅ | | https://ekartlogistics.com | https://crbug.com/1412729#c52 | ✅ | | https://go.thepersonalfinancialguide.com | https://crbug.com/1412729#c29 | ✅ | | https://heliyatra.irctc.co.in | https://crbug.com/1412729#c47 | ✅ | | https://identity.appen.com | https://crbug.com/1412729#c11 | ✅ | | https://login.vitalsource.com | https://crbug.com/1412729#c17 | ✅ | | https://login.yahoo.com | https://crbug.com/1412729#c16 | ✅ | | https://m.betpix365.com | https://crbug.com/1412729#c27 | ✅ | | https://m.esportesdasorte.com | https://crbug.com/1412729#c21 | ✅ | | https://m.estrelabet.com | https://crbug.com/1412729#c22 | ✅ | | https://m.vaidebet.com | https://crbug.com/1412729#c32 | ✅ | | https://my1.konami.net | https://crbug.com/1412729#c33 | ✅ | | https://particuliers.engie.fr | https://crbug.com/1412729#c34 | ✅ | | https://register.betway.de | https://crbug.com/1412729#c25 | ✅ | | https://renegociacao.itau.com.br | https://crbug.com/1412729#c51 | ✅ | | https://signin.costco.com | https://crbug.com/1412729#c12 | ✅ | | https://smartkey.xertica.com | https://crbug.com/1412729#c19 | ✅ | | https://sports.coral.co.uk | https://crbug.com/1412729#c35 | ✅ | | https://sports.ladbrokes.com | https://crbug.com/1412729#c31 | ✅ | | https://tazkarti.com | https://crbug.com/1412729#c36 | ✅ | | https://voterportal.eci.gov.in | https://crbug.com/1412729#c53 | ✅ | | https://www.acesso.gov.pt | https://crbug.com/1412729#c26 | ✅ | | https://www.air.irctc.co.in | https://crbug.com/1412729#c37 | ✅ | | https://www.bancoestado.cl | https://crbug.com/1412729#c15 | ✅ | | https://www.bienlinea.bi.com.gt | https://crbug.com/1412729#c50 | ✅ | | https://www.coachoutlet.com | https://crbug.com/1412729#c49 | ✅ | | https://www.enterprise.de | https://crbug.com/1412729#c40 | ✅ | | https://www.etopaz.az | https://crbug.com/1412729#c44 | ✅ | | https://www.gamestop.com | https://crbug.com/1412729#c38 | ✅ | | https://www.iliad.it | https://crbug.com/1412729#c39 | ✅ | | https://www.irctctourism.com | https://crbug.com/1412729#c42 | ✅ | | https://www.leroymerlin.fr | https://crbug.com/1412729#c18 | ✅ | | https://www.mediafire.com | https://crbug.com/1412729#c20 | ✅ | | https://www.midatacredito.com | https://crbug.com/1412729#c43 | ✅ | | https://www.milanuncios.com | https://crbug.com/1412729#c30 | ✅ | | https://www.portaleargo.it | https://crbug.com/1412729#c14 | ✅ | | https://www.readworks.org | https://crbug.com/1412729#c48 | ✅ | | https://www.saksfifthavenue.com | https://crbug.com/1412729#c46 | ✅ | | https://www.thebay.com | https://crbug.com/1412729#c45 | ✅ | This is not quite a "random" sample, as I instead started with the most common sources of UseCounter hits and then went down the list. I looked at a total of 58 unique origins, but some of the sites failed to load for me, or the offending `pattern` was hidden behind a paywall or login that I couldn’t get past. WDYT? On Thu, Apr 20, 2023 at 1:09 PM Mathias Bynens <[email protected]> wrote: > > > On Wed, Apr 19, 2023 at 9:41 PM Philip Jägenstedt <[email protected]> > wrote: > >> I wonder if we can get enough confidence with less work than >> investigating 40 randomly chosen sites from UseCounter hits. >> >> This is a population proportion problem, and >> https://sample-size.net/confidence-interval-proportion/ is a useful >> tool. If you check 40 cases and find no breakage (N=40, x=0) that gives us >> 95% confidence that breakage is less than 7.2% of samples in this data set. >> If it's useful to check that much depends on the value of the use counter. >> >> Is https://chromestatus.com/metrics/feature/timeline/popularity/4463 the >> right use counter, and has it reached stable yet? Why is marked as obsolete? >> > > Yes, that’s the correct use counter, and yes it’s in M112. It’s marked > “obsolete” on the site because we removed it from the source tree as part > of M113, and ChromeStatus uses the latest HEAD as the source of truth for > the UseCounter labels. The rationale for removing the code triggering this > UseCounter was that a full milestone cycle (M112) should be more than > enough data to make a decision in this particular case. > > >> For purposes of illustration, let's use 0.04% from earlier in the thread >> and say we want to be (95%) confident that real breakage is less than >> 0.01%. Then we just need to get below 25% in the linked tool, and checking >> 11 samples and finding nothing is enough to do this. >> >> On Wed, Apr 19, 2023 at 5:43 PM Mathias Bynens <[email protected]> wrote: >> >>> Thanks for the guidance, Rick. I’ve prepared a CL moving the flag to >>> status=experimental >>> <https://chromium-review.googlesource.com/c/chromium/src/+/4447958> and >>> I can commit to investigating 40 unique UseCounter hits and summarizing my >>> findings. Fingers crossed the trend of “no actual breakage detected” >>> continues. I’ll keep you posted. >>> >>> On Wed, Apr 19, 2023 at 5:26 PM Rick Byers <[email protected]> wrote: >>> >>>> Thanks for doing a thorough compat analysis of this Mathias. I can >>>> totally see this being one where all the examples we can find don't seem to >>>> cause breakage in practice. I know it's a lot, but if we looked at 40 >>>> random examples and found none of them to break, that would suggest an >>>> upper bound of <0.001% of pages impacted (probably much lower) and I'd be >>>> OK giving this a shot with a finch killswitch ready in case of reports of >>>> serious breakage. Does that sound reasonable to you? >>>> >>>> Also feel free to set your flag >>>> <https://source.chromium.org/chromium/chromium/src/+/main:third_party/blink/renderer/platform/runtime_enabled_features.json5;l=1912?q=HTMLPatternRegExpUnicodeSets%20file:.json5&ss=chromium> >>>> to status=experimental, that'll get us some additional usage coverage (from >>>> the small population that runs with >>>> --enable-experimental-web-platform-features) and also signal that this is >>>> close to becoming shipping behavior. >>>> >>>> Rick >>>> >>>> On Mon, Apr 17, 2023 at 7:03 AM 'Mathias Bynens' via blink-dev < >>>> [email protected]> wrote: >>>> >>>>> So far, none of the UseCounter hits I investigated constitute any >>>>> actual breakage. The vast majority of hits seem to be login forms backed >>>>> by >>>>> server-side validation. I’ll keep looking though. >>>>> >>>>> In the meantime, this feature is now >>>>> <https://chromium-review.googlesource.com/c/chromium/src/+/4414859> >>>>> available behind the >>>>> `--enable-blink-features=HTMLPatternRegExpUnicodeSets` >>>>> flag (disabled by default). >>>>> >>>>> On Wednesday, April 5, 2023 at 5:53:10 PM UTC+2 Mathias Bynens wrote: >>>>> >>>>>> On Wed, Apr 5, 2023 at 5:23 PM Alex Russell <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> I don't understand why TAG review is not applicable for this intent. >>>>>>> >>>>>> >>>>>> Fair enough. I’ve filed a TAG review request here: >>>>>> https://github.com/w3ctag/design-reviews/issues/832 I’ll update the >>>>>> ChromeStatus entry to refer to it. >>>>>> >>>>>> On Tuesday, April 4, 2023 at 5:21:16 AM UTC-7 [email protected] wrote: >>>>>>> >>>>>> Thanks to the UseCounter + UKM + M112 hitting Stable, more results >>>>>>>> are starting to come in. I’ll be collecting public examples of >>>>>>>> potential >>>>>>>> incompatibilities here: >>>>>>>> https://bugs.chromium.org/p/chromium/issues/detail?id=1412729#c11 >>>>>>>> So far 0 out of the 2 examples cause any actual breakage — fingers >>>>>>>> crossed >>>>>>>> that trend continues. >>>>>>>> >>>>>>>> On Mon, Apr 3, 2023 at 10:26 AM Philip Jägenstedt < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>> I took a look at https://github.com/whatwg/html/pull/7908 and it >>>>>>>>> looks like there's agreement to merge it, but it's waiting on this >>>>>>>>> intent >>>>>>>>> to be approved. Normally we block in the other direction, but that's >>>>>>>>> fine, >>>>>>>>> as long as the spec change is merged. >>>>>>>>> >>>>>>>>> Looks like there's broad support for this change, and it's just a >>>>>>>>> question of the site compat risk. ~0.04% as an upper bound is quite >>>>>>>>> high. >>>>>>>>> Can we wait until the use counter is in stable and look at a random >>>>>>>>> set of >>>>>>>>> sites hitting the use counter to determine what the real-world >>>>>>>>> breakage >>>>>>>>> looks like? >>>>>>>>> >>>>>>>>> On Fri, Mar 31, 2023 at 5:07 PM 'Mathias Bynens' via blink-dev < >>>>>>>>> [email protected]> wrote: >>>>>>>>> >>>>>>>> On Fri, Mar 31, 2023 at 4:35 PM Mike Taylor <[email protected]> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> Hey Mathias, >>>>>>>>>>> On 3/31/23 5:56 AM, Mathias Bynens wrote: >>>>>>>>>>> >>>>>>>>>>> Contact emails >>>>>>>>>>> >>>>>>>>>>> [email protected] >>>>>>>>>>> >>>>>>>>>>> Specification >>>>>>>>>>> >>>>>>>>>>> https://github.com/whatwg/html/pull/7908 >>>>>>>>>>> >>>>>>>>>>> Summary >>>>>>>>>>> >>>>>>>>>>> The <input pattern> attribute allows developers to specify a >>>>>>>>>>> regular expression pattern against which the input’s values are >>>>>>>>>>> checked for >>>>>>>>>>> validity. >>>>>>>>>>> >>>>>>>>>>> <label> >>>>>>>>>>> >>>>>>>>>>> Part number: >>>>>>>>>>> >>>>>>>>>>> <input pattern="[0-9][A-Z]{3}" name="part" >>>>>>>>>>> >>>>>>>>>>> title="A part number is a digit followed by three >>>>>>>>>>> uppercase letters."> >>>>>>>>>>> >>>>>>>>>>> </label> >>>>>>>>>>> >>>>>>>>>>> When the pattern attribute was first implemented, these regular >>>>>>>>>>> expressions were compiled without any RegExp flags. In 2014, the >>>>>>>>>>> HTML >>>>>>>>>>> Standard changed this by implicitly enabling the u flag for the >>>>>>>>>>> pattern attribute, enabling better Unicode support (including >>>>>>>>>>> support for >>>>>>>>>>> Unicode character properties like \p{Letter}). This change >>>>>>>>>>> shipped in Chrome 53. >>>>>>>>>>> <https://chromestatus.com/feature/4753420745441280> >>>>>>>>>>> >>>>>>>>>>> Now, we’re taking this to the next level by enabling the new >>>>>>>>>>> RegExp v flag <https://v8.dev/features/regexp-v-flag> instead >>>>>>>>>>> of u, enabling the use of set notation, string literal syntax, >>>>>>>>>>> and Unicode properties of strings. >>>>>>>>>>> >>>>>>>>>>> (Context: The RegExp v flag is a JavaScript language feature >>>>>>>>>>> which previously went through the Blink Intents process and shipped >>>>>>>>>>> in Chrome 112 >>>>>>>>>>> <https://chromestatus.com/feature/5144156542861312>. This new >>>>>>>>>>> ChromeStatus entry is specifically about integrating it with the >>>>>>>>>>> HTML >>>>>>>>>>> pattern attribute.) >>>>>>>>>>> >>>>>>>>>>> Blink component >>>>>>>>>>> >>>>>>>>>>> Blink>Forms >>>>>>>>>>> <https://bugs.chromium.org/p/chromium/issues/list?q=component:Blink%3EForms> >>>>>>>>>>> >>>>>>>>>>> Search tags >>>>>>>>>>> >>>>>>>>>>> unicode <https://chromestatus.com/features#tags:unicode>, regexp >>>>>>>>>>> <https://chromestatus.com/features#tags:regexp>, pattern >>>>>>>>>>> <https://chromestatus.com/features#tags:pattern>, validation >>>>>>>>>>> <https://chromestatus.com/features#tags:validation> >>>>>>>>>>> >>>>>>>>>>> TAG review >>>>>>>>>>> TAG review status >>>>>>>>>>> >>>>>>>>>>> Not applicable >>>>>>>>>>> >>>>>>>>>>> Risks >>>>>>>>>>> Interoperability and Compatibility >>>>>>>>>>> >>>>>>>>>>> The spec patch at https://github.com/whatwg/html/pull/7908 >>>>>>>>>>> lists the potentially breaking changes. Some patterns that >>>>>>>>>>> previously would >>>>>>>>>>> compile, now throw an early error with the v flag — >>>>>>>>>>> specifically those with a character class including either an >>>>>>>>>>> unescaped >>>>>>>>>>> special character or a double punctuator. >>>>>>>>>>> >>>>>>>>>>> We expect such patterns to be rare. To validate this assumption >>>>>>>>>>> we’ve added a UseCounter called >>>>>>>>>>> HTMLPatternRegExpUnicodeSetIncompatibilitiesWithUnicodeMode >>>>>>>>>>> <https://chromestatus.com/metrics/feature/popularity#HTMLPatternRegExpUnicodeSetIncompatibilitiesWithUnicodeMode> >>>>>>>>>>> in M112, which tracks patterns in any JavaScript u RegExps >>>>>>>>>>> generated via the HTML pattern attribute that would throw if >>>>>>>>>>> they were used with the v flag. >>>>>>>>>>> >>>>>>>>>>> Importantly, note that any throwing pattern gracefully degrades >>>>>>>>>>> — it simply behaves as if the pattern attribute wasn’t present, >>>>>>>>>>> resulting in inputElement.validity.valid === true for any input >>>>>>>>>>> value. Consequently, the only compatibility risk is that some >>>>>>>>>>> value/pattern >>>>>>>>>>> combinations that would previously result in >>>>>>>>>>> inputElement.validity.valid being false now result in it being >>>>>>>>>>> true. Thus, for every UseCounter hit, it could still be that >>>>>>>>>>> there is no actual breakage — the UseCounter just gives us the >>>>>>>>>>> upper bound. The currently available data from Beta suggests >>>>>>>>>>> the UseCounter hits for 0.0393% of Chrome page loads. >>>>>>>>>>> >>>>>>>>>>> I'm somewhat curious to see how much that UseCounter will grow >>>>>>>>>>> (if at all) when 112 goes to stable next week... >>>>>>>>>>> >>>>>>>>>> Me too, and FWIW I'd understand if you and the other API owners >>>>>>>>>> prefer to wait until there’s some data for Stable before responding >>>>>>>>>> to this >>>>>>>>>> Intent. >>>>>>>>>> >>>>>>>>>>> Do you have any concerns about certain inputs being sent to a >>>>>>>>>>> server that might not have any backend validation, that would >>>>>>>>>>> previously be >>>>>>>>>>> prevented by the u-vintage validation? >>>>>>>>>>> >>>>>>>>>> That’s indeed the only scenario in which there would be breakage. >>>>>>>>>> So far we haven’t heard of such cases in the wild. (Arguably, such >>>>>>>>>> web >>>>>>>>>> pages are already broken, since DevTools could easily be used to >>>>>>>>>> remove the >>>>>>>>>> `pattern` attribute, or requests could be made with tools like >>>>>>>>>> `curl`.) >>>>>>>>>> FWIW, there was a similar discussion in this old blink-dev thread: >>>>>>>>>> https://groups.google.com/a/chromium.org/g/blink-dev/c/XUNMtri0tI4/m/mjPkwXKNAQAJ >>>>>>>>>> >>>>>>>>>> I forgot to mention that we explicitly added a console warning in >>>>>>>>>> M112 for any `pattern` attribute values that would be affected by >>>>>>>>>> this >>>>>>>>>> change, to help developers prepare for the potential change. One >>>>>>>>>> developer >>>>>>>>>> reported seeing the warning and adjusting their `pattern` attribute >>>>>>>>>> values >>>>>>>>>> accordingly, but it’s unclear whether inaction would have really >>>>>>>>>> broken >>>>>>>>>> their web page: >>>>>>>>>> https://bugs.chromium.org/p/chromium/issues/detail?id=1412729#c7 >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>>>> Gecko: Positive (Mozilla standards position request >>>>>>>>>>> <https://github.com/mozilla/standards-positions/issues/745>, >>>>>>>>>>> implementation >>>>>>>>>>> tracking issue >>>>>>>>>>> <https://bugzilla.mozilla.org/show_bug.cgi?id=pattern-v>) >>>>>>>>>>> >>>>>>>>>>> WebKit: Positive (WebKit standards position request >>>>>>>>>>> <https://github.com/WebKit/standards-positions/issues/132>, >>>>>>>>>>> implementation >>>>>>>>>>> tracking issue >>>>>>>>>>> <https://bugs.webkit.org/show_bug.cgi?id=pattern-v>) >>>>>>>>>>> >>>>>>>>>>> Web developers: No signals >>>>>>>>>>> >>>>>>>>>>> Other signals: >>>>>>>>>>> >>>>>>>>>>> Debuggability >>>>>>>>>>> >>>>>>>>>>> The pattern attribute is already well-supported in DevTools and >>>>>>>>>>> other tooling; no changes are necessary. >>>>>>>>>>> >>>>>>>>>>> Will this feature be supported on all six Blink platforms >>>>>>>>>>> (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)? >>>>>>>>>>> >>>>>>>>>>> Yes >>>>>>>>>>> >>>>>>>>>>> Is this feature fully tested by web-platform-tests >>>>>>>>>>> <https://chromium.googlesource.com/chromium/src/+/main/docs/testing/web_platform_tests.md> >>>>>>>>>>> ? >>>>>>>>>>> >>>>>>>>>>> Pull Request: >>>>>>>>>>> https://github.com/web-platform-tests/wpt/pull/38547 >>>>>>>>>>> >>>>>>>>>>> Flag name >>>>>>>>>>> >>>>>>>>>>> N/A >>>>>>>>>>> >>>>>>>>>>> Requires code in //chrome? >>>>>>>>>>> >>>>>>>>>>> False >>>>>>>>>>> >>>>>>>>>>> Tracking bug >>>>>>>>>>> >>>>>>>>>>> https://bugs.chromium.org/p/chromium/issues/detail?id=1412729 >>>>>>>>>>> >>>>>>>>>>> Sample links >>>>>>>>>>> >>>>>>>>>>> https://mathiasbynens.be/demo/pattern-u-vs-v >>>>>>>>>>> >>>>>>>>>>> Estimated milestones >>>>>>>>>>> >>>>>>>>>>> M114 >>>>>>>>>>> >>>>>>>>>>> Link to entry on the Chrome Platform Status >>>>>>>>>>> >>>>>>>>>>> https://chromestatus.com/feature/5149507107422208 >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> You received this message because you are subscribed to the >>>>>>>>>>> Google Groups "blink-dev" group. >>>>>>>>>>> >>>>>>>>>>> To unsubscribe from this group and stop receiving emails from >>>>>>>>>>> it, send an email to [email protected]. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> To view this discussion on the web visit >>>>>>>>>>> https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CADizRgaAq4FwzJbUqLQVo%2BQdd_V0PT7rBr510OGe8fenHA%3D3HQ%40mail.gmail.com >>>>>>>>>>> <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CADizRgaAq4FwzJbUqLQVo%2BQdd_V0PT7rBr510OGe8fenHA%3D3HQ%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>>>>>>>> . >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> You received this message because you are subscribed to the >>>>>>>>>>> Google Groups "blink-dev" group. >>>>>>>>>>> >>>>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>>>>> send an email to [email protected]. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> To view this discussion on the web visit >>>>>>>>>>> https://groups.google.com/a/chromium.org/d/msgid/blink-dev/c9571b2a-a35b-3824-0f37-c93a9bb522fc%40chromium.org >>>>>>>>>>> <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/c9571b2a-a35b-3824-0f37-c93a9bb522fc%40chromium.org?utm_medium=email&utm_source=footer> >>>>>>>>>>> . >>>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> You received this message because you are subscribed to the >>>>>>>>>> Google Groups "blink-dev" group. >>>>>>>>>> >>>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>>>> send an email to [email protected]. >>>>>>>>> >>>>>>>>> >>>>>>>>>> To view this discussion on the web visit >>>>>>>>>> https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CADizRgYxU2v2ANQgzNiLD%2B4P-qJHxzTYJfRDsKNCtY0Yb_0bdg%40mail.gmail.com >>>>>>>>>> <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CADizRgYxU2v2ANQgzNiLD%2B4P-qJHxzTYJfRDsKNCtY0Yb_0bdg%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>>>>>>> . >>>>>>>>>> >>>>>>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "blink-dev" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> To view this discussion on the web visit >>>>> https://groups.google.com/a/chromium.org/d/msgid/blink-dev/bf73fe5b-fde2-42df-90f0-582a905d1948n%40chromium.org >>>>> <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/bf73fe5b-fde2-42df-90f0-582a905d1948n%40chromium.org?utm_medium=email&utm_source=footer> >>>>> . >>>>> >>>> -- You received this message because you are subscribed to the Google Groups "blink-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CADizRgbDGB0udDHwia31UWS0CM_rN4y_EOGgVBArMGh4nyz-xQ%40mail.gmail.com.
