dsmiley commented on code in PR #2259:
URL: https://github.com/apache/solr/pull/2259#discussion_r1500124827
##########
solr/solrj/src/java/org/apache/solr/client/solrj/impl/HttpSolrJdkClient.java:
##########
@@ -188,33 +210,52 @@ private HttpResponse<InputStream> doPutOrPost(
}
HttpRequest.BodyPublisher bodyPublisher;
-
+ final PipedOutputStream source;
if (contentWriter != null) {
- // TODO: There is likely a more memory-efficient way to do this!
- ByteArrayOutputStream baos = new ByteArrayOutputStream();
- contentWriter.write(baos);
- byte[] bytes = baos.toByteArray();
- bodyPublisher = HttpRequest.BodyPublishers.ofByteArray(bytes);
+ source = new PipedOutputStream();
+ PipedInputStream sink = new PipedInputStream(source);
+ bodyPublisher = HttpRequest.BodyPublishers.ofInputStream(() -> sink);
+ // NOCOMMIT - this does not work. The call to "client.send" sometimes
will throw an
+ // IOException
+ // because our thread closes the outputstream too early. But if we wait
to close it until
Review Comment:
Hmm; doesn't make sense to me. Writer should close to signify there's no
more data. The reader will eventually see this and return -1 bytes read to
signify EOF. I reviewed the code for that as well.
##########
solr/solrj/src/java/org/apache/solr/client/solrj/impl/HttpSolrJdkClient.java:
##########
@@ -188,33 +210,52 @@ private HttpResponse<InputStream> doPutOrPost(
}
HttpRequest.BodyPublisher bodyPublisher;
-
+ final PipedOutputStream source;
if (contentWriter != null) {
- // TODO: There is likely a more memory-efficient way to do this!
- ByteArrayOutputStream baos = new ByteArrayOutputStream();
- contentWriter.write(baos);
- byte[] bytes = baos.toByteArray();
- bodyPublisher = HttpRequest.BodyPublishers.ofByteArray(bytes);
+ source = new PipedOutputStream();
+ PipedInputStream sink = new PipedInputStream(source);
+ bodyPublisher = HttpRequest.BodyPublishers.ofInputStream(() -> sink);
+ // NOCOMMIT - this does not work. The call to "client.send" sometimes
will throw an
+ // IOException
+ // because our thread closes the outputstream too early. But if we wait
to close it until
+ // after the response returns, "client.send" blocks.
+ executor.submit(
+ () -> {
+ try (source) {
+ contentWriter.write(source);
+ } catch (Exception e) {
+ log.error("Cannot write Content Stream", e);
+ }
+ });
} else if (streams != null && streams.size() == 1) {
- ContentStream contentStream = streams.iterator().next();
- InputStream is = contentStream.getStream();
+ source = null;
+ InputStream is = streams.iterator().next().getStream();
bodyPublisher = HttpRequest.BodyPublishers.ofInputStream(() -> is);
} else if (queryParams != null && urlParamNames != null) {
+ source = null;
ModifiableSolrParams requestParams = queryParams;
queryParams = calculateQueryParams(urlParamNames, requestParams);
queryParams.add(calculateQueryParams(solrRequest.getQueryParams(),
requestParams));
bodyPublisher =
HttpRequest.BodyPublishers.ofString(requestParams.toString());
} else {
+ source = null;
bodyPublisher = HttpRequest.BodyPublishers.noBody();
}
- if(isPut) {
+ if (isPut) {
reqb.PUT(bodyPublisher);
} else {
reqb.POST(bodyPublisher);
}
decorateRequest(reqb, solrRequest);
reqb.uri(new URI(url + "?" + queryParams));
- return client.send(reqb.build(),
HttpResponse.BodyHandlers.ofInputStream());
+ HttpResponse<InputStream> response =
+ client.send(reqb.build(), HttpResponse.BodyHandlers.ofInputStream());
+ try {
+ source.close();
+ } catch (Exception e1) {
+ // ignore
+ }
Review Comment:
I don't think this is necessary; source should already be closed. But a
concern is that, what if send() or other code fails... we want that executor
thread to cancel/stop. I think a Future should be kept so that we can cancel
it in a try-finally.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
