LGTM1 On Thursday, December 14, 2023 at 8:00:41 AM UTC-8 Rick Byers wrote:
> Sounds good Henrik! Yes, from our brief discussion in the API owners > meeting I believe you have support from at least 3 API owners to proceed in > this direction. It's important to us that we engage constructively in the > standards process even in areas of differing opinion, but it's also crucial > to our process that we not let such differences in opinion and priority be > an effective veto power over what we choose to ship in Chromium. I'd > encourage you and the WebRTC group to formalize processes for amicably > agreeing to disagree on the importance of different use cases, while still > being open to technical feedback and doing what we reasonably can to > maximize the chance that the APIs we ship can eventually be interoperable > if priorities of the other engines someday change [*]. API owners would > also be interested to hear any other arguments for why Chromium shipping > these APIs would be bad for the web (on this list, or anywhere else). I > know there's a messy history with WebRTC in particular and services coming > to depend on Chromium-only APIs when suitable standards-track alternatives > are available in other engines. That's IMHO definitely not a pattern we > want to risk repeating. > > Of course you also need Chrome privacy and security reviews, since it's > important that features like this don't create a hole in our careful > balance of side channel attack mitigations. But I see you already have > privacy approval so hopefully security isn't too far off. You might want to > wait for a signal there before starting implementation. > > Personally I'm also less concerned about interoperability risks when it > comes to metrics API. It's already the case that our top > performance metrics (Core Web Vitals) have APIs exposed only in Chromium. > There's certainly some interop risk there, eg. of sites optimizing in > engine-specific ways. But in practice we've seen developers use these APIs > mostly to make their sites faster in ways that generally apply to all > engines. So in that case, Safari and Firefox are getting most of the > benefit of these APIs existing without having to incur most of the cost, > which seems like a fine outcome to me. Also, I'm confident that if we > eventually agree with other engines on some better way to expose the same > information, then we can deprecate and remove any API shape we ship today > and customers can migrate over without causing user-visible breakage > (worst-case we just return dummy values on the deprecated APIs). > > Rick > > [*] My favorite example of this is Pointer Events where Apple was opposed > to the use case, but also had good technical critiques of the API. We > eventually (after a lot of research, open debate, and some flip-flopping by > me) shipped a version of the API that addressed the legitimate technical > concerns without addressing Apple's objections around the use case. Years > later when a use case materialized for Apple (the Apple Pen), they just > shipped the API in a fully interoperable fashion. That (as well as cases of > the inverse where we realize we were wrong and unship) is what we mean by > the blink process being designed for "eventual interoperability". > > > On Thu, Dec 14, 2023 at 4:34 AM Henrik Boström <h...@chromium.org> wrote: > >> Thanks, Rick. Responses inline. >> >> On Wednesday, December 13, 2023 at 6:39:48 PM UTC+1 Rick Byers wrote: >> >> I looked into the details of the standards debate on this issue. It >> sounds like it's still unclear whether the spec for this has WG support or >> not, right? I certainly wouldn't want to mislead anyone as to API maturity >> / likely interoperability by shipping based on a WebRTC WG specification if >> there is an unresolved process concern. >> >> >> I think it's safe to say we don't have consensus on the frame counters >> (exposing dropped/glitches), while capture latency hopefully be less >> contentious. >> >> >> That said, I think Chromium's position on the technical debate here is >> pretty clear - we do believe there's value in such stats APIs, even IF they >> can only represent browser bugs (it's why we ship a crash reporting API >> <https://wicg.github.io/crash-reporting/>, which has been similarly >> controversial with Mozilla and Apple). We disagree that "there's nothing >> web developers can do about it". Maybe that's true for Apple and Mozilla, >> but for Google we rely critically on developer feedback through both open ( >> crbug.com) and private (Google partnerships) channels and we want to >> make it as easy as possible for developers to report an issue they're >> seeing to us in a way that's actionable. Our privacy policy limits Chrome's >> visibility into what's going on in the wild. So usually we find this >> requires a mix of both site-acquired telemetry and browser-required >> telemetry, and find the two can often complement each other nicely. >> >> Henrik, my advice if the WG doesn't have consensus for this API is to >> move it to some incubation venue (like a WICG group). You clearly have a >> community of web developers who want it, so it's probably more productive >> to focus standards energy among allies who share an interest for the use >> case, right? If you're willing to promptly move this to a WICG spec in the >> event the WG asks to remove it from their spec, then I don't think this >> debate changes anything from a blink API owner's perspective so I'm OK >> treating it as non-blocking. A subset of API owners met today (Daniel, Mike >> Taylor, Philip and I), and agreed with this stance. WDYT? >> >> >> Me moving this to a WICG spec in the event that the WG asks to remove >> them from mediacapture-extensions sounds good to me. >> If me and/or the WebRTC Audio Team has access to a WICG spec for these >> things, that may also give us a venue for, in the future, exploring >> *playout* glitch metrics in a more enthusiastic setting, which is in the >> early stages of discussions internally. >> >> >> In terms of testing, normally we ask to see the tests land before >> approving an I2S. Any reason we shouldn't wait for that here? >> >> >> Given the "controversy" around the glitch metrics, the WebRTC Audio Team >> wanted to get some Blink owners signals before they spend the engineering >> efforts to implement this (including WPTs). >> But if Blink owners also see the value in these metrics and we have a >> plan (= move to WICG in the event that they are removed) I see no reason >> not to ask them to start implementation today. >> >> For reference, see the WPTs we added for the video track stats here >> <https://source.chromium.org/chromium/chromium/src/+/main:third_party/blink/web_tests/external/wpt/mediacapture-extensions/MediaStreamTrack-video-stats.https.html>. >> >> The audio track stats WPTs should be similar and also complemented by C++ >> unit tests in lower layers. >> >> >> Thanks, >> Rick >> >> >> >> On Fri, Dec 8, 2023 at 11:53 AM Henrik Boström <h...@chromium.org> wrote: >> >> Contact emails >> h...@chromium.org, o...@chromium.org, h...@chromium.org >> >> Specification >> https://w3c.github.io/mediacapture-extensions/#the- >> mediastreamtrackaudiostats-interface >> >> Summary >> >> The MediaStreamTrack Statistics API, or `track.stats`, has already >> shipped for video tracks: see previous I2S here >> <https://groups.google.com/a/chromium.org/g/blink-dev/c/ttzYv-30gY4/m/2FvJpxqMGQAJ> >> . >> >> >> This is the same API but for audio tracks, also motivated by the app's >> desire to measure capture quality. This is very important for conferencing >> websites such as Google Meet, Microsoft Teams, Zoom, Goto Meetings, etc. >> All of which has expressed an interest in the audio portion of this API. >> >> >> The API is only available for getUserMedia() sourced audio tracks, i.e. >> microphone, so the API is behind a user prompt and only available during >> capture. >> >> >> The new interface we want to ship is MediaStreamTrackAudioStats >> <https://w3c.github.io/mediacapture-extensions/#the-mediastreamtrackaudiostats-interface> >> >> which allow measuring two things from the audio capture pipeline: >> >> 1. The number of audio frames, including if any audio frames are dropped >> by the device, OS or User Agent. This allows measuring glitches in captured >> audio. >> >> 2. The input latency, such as due to buffering or other delays in >> delivering the audio frames to the track's sinks. >> >> Blink component >> Blink>GetUserMedia >> <https://bugs.chromium.org/p/chromium/issues/list?q=component:Blink%3EGetUserMedia> >> >> Risks >> Interoperability and Compatibility >> >> Because the API provides *statistics* about capture quality, rather than >> providing capture *functionality, *the interop/compatibility risk is >> small. >> >> *Gecko*: Standards position issue >> <https://github.com/mozilla/standards-positions/issues/935> >> *WebKit*: Standards position issue >> <https://github.com/WebKit/standards-positions/issues/260> >> >> Standardization >> While the audio stats API is written by the W3C WebRTC Working Group and >> track statistics overall is not controversial, there is an ongoing >> disagreement with Mozilla about whether or not dropped frames (= >> totalFrames - deliveredFrames) should be exposed to the web in the audio >> case. The disagreement is tracked by this issue >> <https://github.com/w3c/mediacapture-extensions/issues/129>. Our need >> for this metric has been discussed at Virtual Interims, see recording >> with 10:39 timestamp <https://www.youtube.com/watch?v=xJMXnf3Qwh8&t=639s> >> but >> no consensus could be reached (rough consensus was reached between everyone >> except Mozilla). Youenn (Apple) has shown that frames can be dropped due >> to Bluetooth connection >> <https://github.com/w3c/mediacapture-extensions/issues/129#issuecomment-1822624904> >> and >> is not just "measuring browser bugs". >> >> *Web developers*: Positive >> - The I2P >> <https://groups.google.com/a/chromium.org/g/blink-dev/c/vUbD_psbPL8/m/wqq3kmZFBwAJ> >> shows >> support from Teams, Zoom and GoTo meetings. >> - In the spec issue >> <https://github.com/w3c/mediacapture-extensions/issues/129> regarding >> the disagreement, more developer support is expressed (e.g. alfredh from >> Nvidia and steely-glint). >> >> WebView application risks >> >> None >> >> Will this feature be supported on all six Blink platforms (Windows, Mac, >> Linux, ChromeOS, Android, and Android WebView)? >> Yes >> >> Is this feature fully tested by web-platform-tests >> <https://chromium.googlesource.com/chromium/src/+/main/docs/testing/web_platform_tests.md> >> ? >> Yes and WPTs will be written as part of implementing this, however unit >> tests will also be needed to verify accuracy of metrics on a lower level. >> WPTs will be more general like "frame counters increase over time during >> capture" and that run-to-completion semantics are preserved. >> >> Link to entry on the Chrome Platform Status >> https://chromestatus.com/feature/5141112910249984 >> >> Links to previous Intent discussions >> Intent to prototype: >> https://groups.google.com/a/chromium.org/d/msgid/blink- >> dev/bb6c1af3-9eb3-4c6f-a136-dee709b7f906n%40chromium.org >> >> -- >> You received this message because you are subscribed to the Google Groups >> "blink-dev" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to blink-dev+unsubscr...@chromium.org. >> To view this discussion on the web visit https://groups.google.com/a/ >> chromium.org/d/msgid/blink-dev/a594fc55-8030-4848-9fe6- >> 549eccfdd8a8n%40chromium.org >> <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/a594fc55-8030-4848-9fe6-549eccfdd8a8n%40chromium.org?utm_medium=email&utm_source=footer> >> . >> >> -- You received this message because you are subscribed to the Google Groups "blink-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscr...@chromium.org. To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/2e451af1-5e0a-4dd9-a12c-fc30c7dff7dbn%40chromium.org.
