[
https://issues.apache.org/jira/browse/CAMEL-13433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16823747#comment-16823747
]
Claus Ibsen commented on CAMEL-13433:
-------------------------------------
Yeah good idea to grab the content-length if its already provided as a header
and use that. And maybe add a little DEBUG logging that we are doing this, or
that the length is to be determined by reading the data into memory.
Also if you stream from eg http to s3 then we can maybe use the content-length
header from HTTP, and read the stream as-is. There is an
Exchange.CONTENT-LENGTH constant AFAIR.
> S3: Exchange body stream is loaded into memory to calculate content length
> which is already set via headers
> -----------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-13433
> URL: https://issues.apache.org/jira/browse/CAMEL-13433
> Project: Camel
> Issue Type: Bug
> Components: camel-aws
> Affects Versions: 2.22.1, 2.23.2, 3.0.0-M1
> Reporter: MykhailoVlakh
> Assignee: Andrea Cosentino
> Priority: Major
>
> My code uploads large files to S3 storage (500 MB and bigger). I create an
> exchange with properly set meta information headers including the
> _CamelAwsS3ContentLength_, as a body I provide a stream. Everything worked
> well in Camel 2.14.3 but when I upgraded to Camel ti version 2.22.1 I
> suddenly began getting OOM issues from time to time. I did some debugging and
> found that the issue is in *S3Producer* that loads content of the body stream
> into memory.
> {code:java}
> public void processSingleOp(final Exchange exchange) throws Exception {
> ObjectMetadata objectMetadata = determineMetadata(exchange); <== here we
> already set length
> File filePayload = null;
> InputStream is = null;
> ByteArrayOutputStream baos = null;
> Object obj = exchange.getIn().getMandatoryBody();
> PutObjectRequest putObjectRequest = null;
> // Need to check if the message body is WrappedFile
> if (obj instanceof WrappedFile) {
> obj = ((WrappedFile<?>)obj).getFile();
> }
> if (obj instanceof File) {
> filePayload = (File)obj;
> is = new FileInputStream(filePayload);
> } else {
> is = exchange.getIn().getMandatoryBody(InputStream.class);
> baos = determineLengthInputStream(is); <====== the issue is here
> objectMetadata.setContentLength(baos.size());
> is = new ByteArrayInputStream(baos.toByteArray());
> }
> ...
> {code}
> This is a defect since, as I mentioned, I already provide content length via
> headers and in the first line of the method populates that information into
> object metadata.
> I think there should be some if check that skips content length size
> calculation if it is already known.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
