Michael Henriksen pointed out an issue with this approach.
If the web server is actually generating the content on the fly then it may send it as chunked encoding, and in HTTP/1.1 it's not required that the Content-Length field is present (since it may not be known when the server begins sending the content): https://www.rfc-editor.org/rfc/rfc2616#section-4.4 The only way to know the true length would be to download all the content, which we definitely don't want to do. I'm not totally clear if Content-Length, if present, must be valid. Curl source code seems to imply not: https://github.com/curl/curl/blob/6e4fedeef7ff2aefaa4841a4710d6b8a37c6ae7c/lib/http.c#L3868 but maybe they are just being over-cautious? The RFC is a bit confusing. AWS itself _does_ send Content-Length and it appears to be valid ... So one approach might be to assume it is valid, which I believe is what the current patch series does. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com libguestfs lets you edit virtual machines. Supports shell scripting, bindings from many languages. http://libguestfs.org _______________________________________________ Libguestfs mailing list Libguestfs@redhat.com https://listman.redhat.com/mailman/listinfo/libguestfs
