[
https://issues.apache.org/jira/browse/CAMEL-13433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16821931#comment-16821931
]
Andrea Cosentino commented on CAMEL-13433:
------------------------------------------
I think we still need to know the size if it's not provided, but yes, we can
skip that part if the content length has been already provided. I'll try to
push a fix next week.
> S3: Exchange body stream is loaded into memory to calculate content length
> which is already set via headers
> -----------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-13433
> URL: https://issues.apache.org/jira/browse/CAMEL-13433
> Project: Camel
> Issue Type: Bug
> Components: camel-aws
> Affects Versions: 2.22.1, 2.23.2, 3.0.0-M1
> Reporter: MykhailoVlakh
> Assignee: Andrea Cosentino
> Priority: Major
>
> My code uploads large files to S3 storage (500 MB and bigger). I create an
> exchange with properly set meta information headers including the
> _CamelAwsS3ContentLength_, as a body I provide a stream. Everything worked
> well in Camel 2.14.3 but when I upgraded to Camel ti version 2.22.1 I
> suddenly began getting OOM issues from time to time. I did some debugging and
> found that the issue is in *S3Producer* that loads content of the body stream
> into memory.
> {code:java}
> public void processSingleOp(final Exchange exchange) throws Exception {
> ObjectMetadata objectMetadata = determineMetadata(exchange); <== here we
> already set length
> File filePayload = null;
> InputStream is = null;
> ByteArrayOutputStream baos = null;
> Object obj = exchange.getIn().getMandatoryBody();
> PutObjectRequest putObjectRequest = null;
> // Need to check if the message body is WrappedFile
> if (obj instanceof WrappedFile) {
> obj = ((WrappedFile<?>)obj).getFile();
> }
> if (obj instanceof File) {
> filePayload = (File)obj;
> is = new FileInputStream(filePayload);
> } else {
> is = exchange.getIn().getMandatoryBody(InputStream.class);
> baos = determineLengthInputStream(is); <====== the issue is here
> objectMetadata.setContentLength(baos.size());
> is = new ByteArrayInputStream(baos.toByteArray());
> }
> ...
> {code}
> This is a defect since, as I mentioned, I already provide content length via
> headers and in the first line of the method populates that information into
> object metadata.
> I think there should be some if check that skips content length size
> calculation if it is already known.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
