[
https://issues.apache.org/jira/browse/SOLR-9296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15675970#comment-15675970
]
Noble Paul edited comment on SOLR-9296 at 11/18/16 6:50 AM:
------------------------------------------------------------
The {{PushWriter}} interface and the associated {{MapWriter/IteratorWriter}}
interfaces are created to avoid creating objects and stream bytes out as soon
as they are available. The binary implementation is already taking advantage of
that. The json implementation doesn't. However, it is easy to make json
efficient as well. Soon (with SOLR-9721) the communication can be totally
switched to binary format which is far more efficient on the wire as well as
writing/reading. However, at the client side, the reading is not designed to be
efficient on object creation. The way tuples are created makes it unnecessarily
inefficient. Anyway, I'll choose to fight that battle another day.
was (Author: noble.paul):
The {{PushWriter}} interface and the associated {{MapWriter/IteratorWriter}}
interfaces are created to avoid creating objects and stream bytes out as soon
as they are available. The binary implementation is already taking advantage of
that. The json implementation doesn't. However, it is easy to make json
efficient as well. Soon (with SOLR-9721) the communication can be totally
switched to binary format which is far more efficient on the wire as well as
writing/reading. However, at the client side, the reading is not designed to be
efficient on object creation. Anyway, I'll choose to fight that battle another
day.
> Examine SortingResponseWriter with an eye towards removing extra object
> creation
> --------------------------------------------------------------------------------
>
> Key: SOLR-9296
> URL: https://issues.apache.org/jira/browse/SOLR-9296
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Affects Versions: 6.2, master (7.0)
> Reporter: Erick Erickson
> Assignee: Erick Erickson
> Attachments: SOLR-9296.patch
>
>
> Assigning to myself just to keep from losing track it. Anyone who wants to
> take it, please feel free!
> While looking at SOLR-9166 I noticed that SortingResponseWriter does a
> toString for each field it writes out. At a _very_ preliminary examination it
> seems like we create a lot of String objects that need to be GC'd. Could we
> reduce this by using some kind of CharsRef/ByteBuffer/Whatever?
> I've only looked at this briefly, not quite sure what the gotchas are but
> throwing it out for discussion.
> Some initial thoughts:
> 1> for the fixed types (numerics, dates, booleans) there's a strict upper
> limit on the size of each value so we can allocate something up-front.
> 2> for string fields, we already get a chars ref so just pass that through?
> 3> must make sure that whatever does the actual writing transfers all the
> bytes before returning.
> I'm sure I won't get to this for a week or perhaps more, so grab it if you
> have the bandwidth.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
