Support on update Service delivery Manager
On 13-Feb-2024 10:48 PM, "James Hartig" <fastest...@gmail.com> wrote: My employer ran into the window size during our pre-production validation and it was difficult to debug since it was working in cURL, the zstd CLI, and only presented itself on certain URLs. I appreciate Nidhi responding to our issue so quickly and updating Chrome to have a more helpful error message in the future. The Go package we use already updated their default <https://github.com/klauspost/compress/pull/913> to 8MB (without any awareness to Chrome's size) which should help future users of that package but there might be other packages out there that might not have a low enough default. The updated Chrome error message will help but only if you run into that error message when testing; which might not if you happen to be testing with small responses. I'm not sure where developers should be looking to be aware of the window size. Does it make sense to mention in the Chrome Status entry? If the spec is updated that might be good enough but I just wanted to discuss other avenues that might be more developer-aware. On Tuesday, February 13, 2024 at 4:24:55 AM UTC-6 Asif jutt Jutt wrote: > S > > Service delivery Manager > > On Tue, 13 Feb 2024, 5:11 AM Nidhi Jaju, <nidh...@chromium.org> wrote: > >> Contact emails >> >> nidh...@chromium.org >> >> Explainer >> >> >> https://docs.google.com/document/d/1aDyUw4mAzRdLyZyXpVgWvO-eLpc4ERz7I_7VDIPo9Hc/edit?usp=sharing >> >> Specification >> >> https://datatracker.ietf.org/doc/html/rfc8878 >> >> Design docs >> >> >> https://docs.google.com/document/d/14dbzMpsYPfkefAJos124uPrlkvW7jyPJhzjujSWws2k/edit?usp=sharing >> >> Summary >> >> Zstandard, or “zstd”, is a data compression mechanism described in >> RFC8878. It is a fast lossless compression algorithm, targeting real-time >> compression scenarios at zlib-level and better compression ratios. The >> "zstd" token was added as an IANA-registered Content-Encoding token as per >> https://datatracker.ietf.org/doc/html/rfc8878#name-content-encoding. >> >> Adding support for "zstd" as a Content-Encoding will help load pages >> faster and use less bandwidth, and spend less time and CPU/power on >> compression on our servers, resulting in reduced server costs. >> >> Blink component >> >> Internals>Network >> <https://bugs.chromium.org/p/chromium/issues/list?q=component:Internals%3ENetwork> >> >> TAG review >> >> https://github.com/w3ctag/design-reviews/issues/930 >> >> TAG review status >> >> Pending >> >> Risks >> Interoperability and Compatibility >> >> Servers that have a broken implementation of zstd might exist, but the >> risk of this is small. Additionally, middleware and middleboxes like virus >> checkers that intercept HTTPS connections might not support zstd, but might >> fail to remove it from the Accept-Encoding header in the request. >> >> Another known risk is interoperability between clients that support zstd >> regarding window frame sizes. In Chrome, we limit the window frame size to >> 8MB to prevent excessive memory usage, but this limit does not exist in >> curl and when using zstd directly. We have seen very few sites that use a >> window size larger than 8MB which causes decoding errors, but we have added >> new net error codes and debugging messages to help them understand what to >> do in this situation. >> >> Gecko: Positive ( >> https://github.com/mozilla/standards-positions/issues/775) >> >> WebKit: Positive ( >> https://github.com/WebKit/standards-positions/issues/168) >> >> Web developers: Positive (https://crbug.com/1246971) Meta (Yann and >> Felix) and Akamai (Nic) are positive about zstd content-encoding on the >> browser. Meta has collaborated with us to improve the compression ratios >> for Meta origins during the experiment and is seeing positive user-level >> results. Alibaba is also supportive of shipping zstd support as they saw >> massive savings on their origins in terms of server CPU cost. >> >> Other signals: >> >> Ergonomics >> >> While both Zstandard and Brotli are clear wins over gzip >> content-encoding, which of Zstandard or Brotli to use depends on many >> factors, and site authors may need to experiment to identify the optimal >> choice for their content. >> >> Zstandard uses more memory for decompression than gzip. However, this is >> also true for Brotli, and we haven't seen any problems in practice. >> >> Activation >> >> The "zstd" Content-Encoding is not as widely supported by HTTP servers as >> gzip. Of the top 5 web servers, Nginx has a third-party module, which >> should also work for OpenResty (untested). Apache, IIS, and LiteSpeed >> appear to have no support. Explicit server support is often only necessary >> for dynamic content. For static (pre-compressed) content, Zstandard can >> often be supported just by configuration. >> >> Only one public CDN is known to be able to compress Zstandard itself, and >> some CDN's may require custom configuration to pass-through Zstandard >> correctly. >> >> Zstd support is not particularly difficult to implement for a server that >> already implements multiple content encodings. The C implementation has a >> straightforward API and there are implementations for many other languages. >> There is also a lively community of Zstandard enthusiasts which should help >> accelerate adoption. >> >> Security >> >> CRIME <https://en.wikipedia.org/wiki/CRIME> and BREACH >> <https://en.wikipedia.org/wiki/BREACH> mean that the resource being >> compressed can be considered readable by the document deploying them. That >> is bad if any of them contains information that the document cannot already >> obtain by other means. An attacker may provide correctly formed compressed >> frames with unreasonable memory requirements, and dictionaries may interact >> unexpectedly with a decoder, leading to possible memory or other >> resource-exhaustion attacks. It is possible to store arbitrary user >> metadata in skippable frames, so they can be used as a watermark to track >> the path of the compressed payload. It is important to note that these >> concerns apply to all compression formats, not just zstd. >> >> To mitigate these risks, similar to Brotli, we'll be advertising support >> for "zstd" encoding only if transferred data is opaque to proxies, to >> ensure that resources don't contain private data that the origin cannot >> read otherwise. >> >> Adding zstd to third_party/ in Chromium adds a large new code surface >> that processes untrusted data, which inevitably brings risks of new >> security holes. However, this is mitigated by the extensive fuzzing and >> security analysis done on zstd by Google and other community members. >> >> Furthermore, zstd is implemented in C, which is not a memory-safe >> language, and the network service is not yet sandboxed on all platforms. >> >> WebView application risks >> >> Does this intent deprecate or change behavior of existing APIs, such that >> it has potentially high risk for Android WebView-based applications? >> >> Apps which use a WebView to display content from Meta's servers will >> suddenly start using Zstandard. Since we've already extensively tested our >> implementation against Meta's servers in Chrome, no problems are expected. >> There is a killswitch. No special treatment should be needed. >> >> >> Debuggability >> >> No special support needed. >> >> Zstd content-encoding support is exposed to the devtools protocol, so >> developers are able to override it and view the headers from the inspector. >> >> A new net error has been added for decoding errors related to window >> frame size. >> >> Will this feature be supported on all six Blink platforms (Windows, Mac, >> Linux, ChromeOS, Android, and Android WebView)? >> >> Yes >> >> Is this feature fully tested by web-platform-tests >> <https://chromium.googlesource.com/chromium/src/+/main/docs/testing/web_platform_tests.md> >> ? >> >> Yes (https://wpt.fyi/results/fetch/content-encoding/zstd >> <https://wpt.fyi/results/fetch/content-encoding/zstd?label=experimental&label=master&aligned> >> ) >> >> Flag name on chrome://flags >> >> enable-zstd-content-encoding >> >> Finch feature name >> >> ZstdContentEncoding >> >> Requires code in //chrome? >> >> False >> >> Tracking bug >> >> https://bugs.chromium.org/p/chromium/issues/detail?id=1246971 >> >> Launch bug >> >> https://launch.corp.google.com/launch/4266275 >> >> Measurement >> >> https://chromestatus.com/metrics/feature/timeline/popularity/4629 >> >> Adoption plan >> >> In our experimental group, around 1% of responses use "zstd" >> content-encoding. Given the significant benefits of zstandard over gzip, >> we'd like to see it increase to 10% within 2 years. >> >> Estimated milestones >> >> Shipping on desktop >> >> 123 >> >> DevTrial on desktop >> >> 117 >> >> Shipping on Android >> >> 123 >> >> DevTrial on Android >> >> 117 >> >> Shipping on WebView >> >> 123 >> >> >> Anticipated spec changes >> >> Open questions about a feature may be a source of future web compat or >> interop issues. Please list open issues (e.g. links to known github issues >> in the project for the feature specification) whose resolution may >> introduce web compat/interop risk (e.g., changing to naming or structure of >> the API in a non-backward-compatible way). >> >> The current standard, RFC8878, doesn't require a limit on the window size >> used by HTTP servers when compressing Zstandard. An update of some form >> will be needed to ensure interoperability. >> >> Link to entry on the Chrome Platform Status >> >> https://chromestatus.com/feature/6186023867908096 >> >> Links to previous Intent discussions >> >> Intent to Prototype: >> https://groups.google.com/a/chromium.org/g/blink-dev/c/GDsI0Hw-jYk/m/Yc5QZWD-AwAJ >> >> Intent to Experiment: >> https://groups.google.com/a/chromium.org/g/blink-dev/c/I6IWfl95gRU >> >> >> This intent message was generated by Chrome Platform Status >> <https://chromestatus.com/>. >> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "blink-dev" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to blink-dev+...@chromium.org. >> >> To view this discussion on the web visit >> https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAMZNYAN7VRca4VfRqP7pi%2BnqwDuor4ZVjF9yDNH1mZcXteQURw%40mail.gmail.com >> <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAMZNYAN7VRca4VfRqP7pi%2BnqwDuor4ZVjF9yDNH1mZcXteQURw%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> > On 13-Feb-2024 5:11 AM, "Nidhi Jaju" <nidh...@chromium.org> wrote: > > Contact emails > > nidh...@chromium.org > > Explainer > > > https://docs.google.com/document/d/1aDyUw4mAzRdLyZyXpVgWvO-eLpc4ERz7I_7VDIPo9Hc/edit?usp=sharing > > Specification > > https://datatracker.ietf.org/doc/html/rfc8878 > > Design docs > > > https://docs.google.com/document/d/14dbzMpsYPfkefAJos124uPrlkvW7jyPJhzjujSWws2k/edit?usp=sharing > > Summary > > Zstandard, or “zstd”, is a data compression mechanism described in > RFC8878. It is a fast lossless compression algorithm, targeting real-time > compression scenarios at zlib-level and better compression ratios. The > "zstd" token was added as an IANA-registered Content-Encoding token as per > https://datatracker.ietf.org/doc/html/rfc8878#name-content-encoding. > > Adding support for "zstd" as a Content-Encoding will help load pages > faster and use less bandwidth, and spend less time and CPU/power on > compression on our servers, resulting in reduced server costs. > > Blink component > > Internals>Network > <https://bugs.chromium.org/p/chromium/issues/list?q=component:Internals%3ENetwork> > > TAG review > > https://github.com/w3ctag/design-reviews/issues/930 > > TAG review status > > Pending > > Risks > Interoperability and Compatibility > > Servers that have a broken implementation of zstd might exist, but the > risk of this is small. Additionally, middleware and middleboxes like virus > checkers that intercept HTTPS connections might not support zstd, but might > fail to remove it from the Accept-Encoding header in the request. > > Another known risk is interoperability between clients that support zstd > regarding window frame sizes. In Chrome, we limit the window frame size to > 8MB to prevent excessive memory usage, but this limit does not exist in > curl and when using zstd directly. We have seen very few sites that use a > window size larger than 8MB which causes decoding errors, but we have added > new net error codes and debugging messages to help them understand what to > do in this situation. > > Gecko: Positive (https://github.com/mozilla/standards-positions/issues/775 > ) > > WebKit: Positive (https://github.com/WebKit/standards-positions/issues/168 > ) > > Web developers: Positive (https://crbug.com/1246971) Meta (Yann and > Felix) and Akamai (Nic) are positive about zstd content-encoding on the > browser. Meta has collaborated with us to improve the compression ratios > for Meta origins during the experiment and is seeing positive user-level > results. Alibaba is also supportive of shipping zstd support as they saw > massive savings on their origins in terms of server CPU cost. > > Other signals: > > Ergonomics > > While both Zstandard and Brotli are clear wins over gzip content-encoding, > which of Zstandard or Brotli to use depends on many factors, and site > authors may need to experiment to identify the optimal choice for their > content. > > Zstandard uses more memory for decompression than gzip. However, this is > also true for Brotli, and we haven't seen any problems in practice. > > Activation > > The "zstd" Content-Encoding is not as widely supported by HTTP servers as > gzip. Of the top 5 web servers, Nginx has a third-party module, which > should also work for OpenResty (untested). Apache, IIS, and LiteSpeed > appear to have no support. Explicit server support is often only necessary > for dynamic content. For static (pre-compressed) content, Zstandard can > often be supported just by configuration. > > Only one public CDN is known to be able to compress Zstandard itself, and > some CDN's may require custom configuration to pass-through Zstandard > correctly. > > Zstd support is not particularly difficult to implement for a server that > already implements multiple content encodings. The C implementation has a > straightforward API and there are implementations for many other languages. > There is also a lively community of Zstandard enthusiasts which should help > accelerate adoption. > > Security > > CRIME <https://en.wikipedia.org/wiki/CRIME> and BREACH > <https://en.wikipedia.org/wiki/BREACH> mean that the resource being > compressed can be considered readable by the document deploying them. That > is bad if any of them contains information that the document cannot already > obtain by other means. An attacker may provide correctly formed compressed > frames with unreasonable memory requirements, and dictionaries may interact > unexpectedly with a decoder, leading to possible memory or other > resource-exhaustion attacks. It is possible to store arbitrary user > metadata in skippable frames, so they can be used as a watermark to track > the path of the compressed payload. It is important to note that these > concerns apply to all compression formats, not just zstd. > > To mitigate these risks, similar to Brotli, we'll be advertising support > for "zstd" encoding only if transferred data is opaque to proxies, to > ensure that resources don't contain private data that the origin cannot > read otherwise. > > Adding zstd to third_party/ in Chromium adds a large new code surface that > processes untrusted data, which inevitably brings risks of new security > holes. However, this is mitigated by the extensive fuzzing and security > analysis done on zstd by Google and other community members. > > Furthermore, zstd is implemented in C, which is not a memory-safe > language, and the network service is not yet sandboxed on all platforms. > > WebView application risks > > Does this intent deprecate or change behavior of existing APIs, such that > it has potentially high risk for Android WebView-based applications? > > Apps which use a WebView to display content from Meta's servers will > suddenly start using Zstandard. Since we've already extensively tested our > implementation against Meta's servers in Chrome, no problems are expected. > There is a killswitch. No special treatment should be needed. > > > Debuggability > > No special support needed. > > Zstd content-encoding support is exposed to the devtools protocol, so > developers are able to override it and view the headers from the inspector. > > A new net error has been added for decoding errors related to window frame > size. > > Will this feature be supported on all six Blink platforms (Windows, Mac, > Linux, ChromeOS, Android, and Android WebView)? > > Yes > > Is this feature fully tested by web-platform-tests > <https://chromium.googlesource.com/chromium/src/+/main/docs/testing/web_platform_tests.md> > ? > > Yes (https://wpt.fyi/results/fetch/content-encoding/zstd > <https://wpt.fyi/results/fetch/content-encoding/zstd?label=experimental&label=master&aligned> > ) > > Flag name on chrome://flags > > enable-zstd-content-encoding > > Finch feature name > > ZstdContentEncoding > > Requires code in //chrome? > > False > > Tracking bug > > https://bugs.chromium.org/p/chromium/issues/detail?id=1246971 > > Launch bug > > https://launch.corp.google.com/launch/4266275 > > Measurement > > https://chromestatus.com/metrics/feature/timeline/popularity/4629 > > Adoption plan > > In our experimental group, around 1% of responses use "zstd" > content-encoding. Given the significant benefits of zstandard over gzip, > we'd like to see it increase to 10% within 2 years. > > Estimated milestones > > Shipping on desktop > > 123 > > DevTrial on desktop > > 117 > > Shipping on Android > > 123 > > DevTrial on Android > > 117 > > Shipping on WebView > > 123 > > > Anticipated spec changes > > Open questions about a feature may be a source of future web compat or > interop issues. Please list open issues (e.g. links to known github issues > in the project for the feature specification) whose resolution may > introduce web compat/interop risk (e.g., changing to naming or structure of > the API in a non-backward-compatible way). > > The current standard, RFC8878, doesn't require a limit on the window size > used by HTTP servers when compressing Zstandard. An update of some form > will be needed to ensure interoperability. > > Link to entry on the Chrome Platform Status > > https://chromestatus.com/feature/6186023867908096 > > Links to previous Intent discussions > > Intent to Prototype: > https://groups.google.com/a/chromium.org/g/blink-dev/c/GDsI0Hw-jYk/m/Yc5QZWD-AwAJ > > Intent to Experiment: > <https://groups.google.com/a/chromium.org/g/blink-dev/c/I6IWfl95gRU> > > -- You received this message because you are subscribed to the Google Groups "blink-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscr...@chromium.org. To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAH6zh9v8HSHc0sSOC6o9Gj9qRubzfQa%2BERTpWsdMCVLixAZgBg%40mail.gmail.com.
