SO post discuss about the compressing the response, so not directly
relevant here. Anyway our client is a browser, so user-agent is
"sensible" already like in case with Chrome.
user-agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/97.0.4692.71 Safari/537.36
btw; We are seeing success and failing requests with the same browser,
so I do not suspect that the user-agent header has anything to with this
issue as it is the same in both cases.
Content type is sent like this:
content-type=application/json;charset=utf-8
We do have logging in place but we have not done any tests what the
exact limit is when it starts to fail. We have seen failing requests
where sizes have been about 1.5Mb compressed / ~10Mb uncompressed.
Majority of requests are much less than that and we haven't had any
problems with those smaller requests. That is why we suspect that it
really has something to do with the requests size but as 10Mb is far
less 32Mb which we found from one document, we are not sure. And it was
a bit of surprise that sometimes the requests is not uncompressed like
in normal cases. So we weren't prepared for that.
This is easy to handle as when the request is uncompressed
content-encoding=gzip header is still in place. So such request can be
detected and uncompressed in the application. But if AppEngine should
take care of it (should it?) why we need backup processing in the
application? So really would want to understand what are the limits and
what we should be prepared for.
On 21.1.2022 6.43, M M wrote:
Could you check this SO link
<https://stackoverflow.com/questions/9772061/app-engine-accept-encoding>,
specifically Nick Johnson's comment related to using a "sensible"
user-agent in addition to content type? I suspect these are being
explicitly defined in your POST request. In addition, are you logging
the exceptions when these compressed packages arrives to GAE? Are you
capturing the size of the load of these packages? It would help here
if those compressed packages that make it through have certain file
size or if it fails to uncompressed for any size and it is more
random. For example, these packages usually have 1MB of size or less?
You specified that the packages are not more that 32MB. Uncompressed,
are they more than this limit (Maybe not relevant? I am not in
capacity to reproduce this problem, but this would be a good detail to
share here, if possible)
Cheers,
On Thursday, January 20, 2022 at 1:44:54 AM UTC-5
[email protected] wrote:
As said earlier, this has nothing to do with responses from
AppEngine. There are not any extra players in front of AppEngine
either. This is a very basic case, a client making a request to
AppEngine running Java 8 application.
[Client] -> [Compressed request] -> [Google AppEngine ***** -> [
AppEngine Java 8 Application ]]
I am really looking for the information or documentation about
what happens in AppEngine request processing (***) before the
request reaches the Java application running in the AppEngine.
The request is normally uncompressed automatically. Sometimes it
is not and I am looking for the root cause of what causes it NOT
to be un-compressed.
On Wednesday, 19 January 2022 at 18:25:55 UTC+2
[email protected] wrote:
Hmmm... Judging by this document
<https://cloud.google.com/appengine/docs/standard/go/reference/request-response-headers#headers_added>
it
seems that occasionally compression is expected. It states
that "depending on the request headers and response
/Content-Type/, the server may automatically compress the
response body". Doesn't specify which, so I understand that
App Engine Java may respond as needed unless you specify a
response Content-Type header.
Though this is from your client to GAE, I don't see why this
couldn't occur in the opposite direction.
What are you using as frontend? Is there a reason you're
handling de-compression there?
On Tuesday, January 18, 2022 at 3:22:01 PM UTC+1
[email protected] wrote:
Just the opposite. All of the requests we are sending to
App Engine are compressed. Normal case is that our App
Engine Java app will receive them as being uncompressed.
But occasionally App Engine Java app will receive them as
still being uncompressed.
On 18.1.2022 15.05, 'Felipe Bergallo Corral' via Google
App Engine wrote:
So, just to get this straight because I can be a little
slow on the uptake sometimes:
All of the requests you are sending to App Engine are
uncompressed, but occassionally App Engine will receive
them as being compressed, is that correct?
On Wednesday, January 12, 2022 at 3:36:26 PM UTC+1
[email protected] wrote:
Sorry not to be clear enough. I am talking about
inbound traffic here, so HTTP POST requests to the
App Engine. Not HTTP responses from the App Engine.
Our responses from App Engine are zipped just fine by
the platform, no problems there.
The case here really is HTTP POSTs to the App Engine
when the HTTP request POST body is compressed. Normal
cases are that App Engine will get the request as
already uncompressed but sometimes the request is
coming as is thus still compression in place. I am
suspecting that it has something to do with the
request size. I am not sure though.
On Wednesday, 12 January 2022 at 13:20:22 UTC+2
[email protected] wrote:
My understanding is that that 32MB constraint is
only if the responses are not on Cloud Storage or
Legacy Blobstore. If you're requesting
gzip-compressed responses, then the responses
shouldn't be uncompressed, as far as I understand
it - according to this
<https://cloud.google.com/appengine/docs/standard/java/how-requests-are-handled#response_compression>,
if the request doesn't specify compressed
responses ( by having the header
/Accept-Encoding: gzip/) it should return
uncompressed responses. The likely reason for
some responses being compressed and others not is
due to caching
<https://cloud.google.com/appengine/docs/standard/java/how-requests-are-handled#response_caching>
if you've configured, if you've configured it
(though I believe there is a predetermined
configuration).
Though I'm not sure how to answer your third
question, would you mind rephrasing it? I'm not
sure what you mean
On Wednesday, January 12, 2022 at 9:38:57 AM
UTC+1 [email protected] wrote:
We use App Engine standard with Java 8. We
have a client that sends HTTP POST (post body
as JSON) requests as gzip-compressed. In
normal cases, everything works just fine. The
frontend will uncompress the requests and the
App Engine will see the uncompressed request.
But with some requests (large ones?)
uncompressing is not done by the frontend.
This is seen as the request header
/content-encoding=[gzip] /is still in the
request headers and content is binary when
the request arrives in App Engine. Our
application is not prepared to uncompress the
request but relies on the frontend to
uncompress it. We couldn't find much
information about the rules of uncompressing
requests. The page
https://cloud.google.com/appengine/docs/standard/java/how-requests-are-handled#request_limits
says only that the request limit is 32
megabytes. Our failing requests are far below
that limit in both compressed and uncompressed.
* Is there a way to find out why some
requests are not uncompressed?
* If it is the size of the request, can the
limit be configured/increased?
* Is the best practice that the application
must handle compressed requests also even
frontend is taking care of the majority
of the compressed requests?
* Is there documentation somewhere telling
more about compressed request
limits/handling in App Engine standard?
--
You received this message because you are subscribed to a
topic in the Google Groups "Google App Engine" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/google-appengine/nbzRV2WctMY/unsubscribe.
To unsubscribe from this group and all its topics, send
an email to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/google-appengine/ee3fd588-dfb1-4935-b33e-a8cfe7e26a2cn%40googlegroups.com
<https://groups.google.com/d/msgid/google-appengine/ee3fd588-dfb1-4935-b33e-a8cfe7e26a2cn%40googlegroups.com?utm_medium=email&utm_source=footer>.
--
You received this message because you are subscribed to a topic in the
Google Groups "Google App Engine" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/google-appengine/nbzRV2WctMY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
[email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/google-appengine/eaf7fb6a-20cf-413f-88bf-39e5ac55d91en%40googlegroups.com
<https://groups.google.com/d/msgid/google-appengine/eaf7fb6a-20cf-413f-88bf-39e5ac55d91en%40googlegroups.com?utm_medium=email&utm_source=footer>.
--
You received this message because you are subscribed to the Google Groups "Google
App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/google-appengine/df385a9d-c009-cd79-469d-22e2cfe1ef8b%40koivusolutions.com.
