[
https://issues.apache.org/jira/browse/SOLR-18415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18112646#comment-18112646
]
Andrzej Bialecki commented on SOLR-18415:
-----------------------------------------
After further analysis this looks somewhat better than I thought, especially
for updates that are less than 8192 byte in size (serialized), because for
these the array is resized to the actual size of the payload (as accumulated in
the {{FastOutputStream}}'s buffer that the {{JavaBinCodec}} uses). In this case
no extra padding bytes are wasted.
Since the producer side ({{MirroringUpdateProcessor / KafkaMirroringSink}})
doesn't accumulate individual requests in batches and in a typical scenario
most documents are smaller than 8kB serialized, this optimization will mostly
hold.
This is not the case when sending larger documents or when sending
MirroredConfigSetRequest-s that have multiple content streams. Here's the
breakdown:
{noformat}
actualSize=61 byteArraySize=64 waste=3 (4.7% of
transmitted bytes)
actualSize=153 byteArraySize=153 waste=0 (0.0% of
transmitted bytes)
actualSize=604 byteArraySize=604 waste=0 (0.0% of
transmitted bytes)
actualSize=4104 byteArraySize=4104 waste=0 (0.0% of
transmitted bytes)
actualSize=6104 byteArraySize=6104 waste=0 (0.0% of
transmitted bytes)
actualSize=8104 byteArraySize=8104 waste=0 (0.0% of
transmitted bytes)
actualSize=8304 byteArraySize=16464 waste=8160 (49.6% of
transmitted bytes)
actualSize=10104 byteArraySize=20064 waste=9960 (49.6% of
transmitted bytes)
actualSize=15104 byteArraySize=30064 waste=14960 (49.8% of
transmitted bytes)
actualSize=20105 byteArraySize=40066 waste=19961 (49.8% of
transmitted bytes)
actualSize=30105 byteArraySize=32768 waste=2663 (8.1% of
transmitted bytes)
actualSize=50105 byteArraySize=65536 waste=15431 (23.5% of
transmitted bytes)
actualSize=100105 byteArraySize=131072 waste=30967 (23.6% of
transmitted bytes)
{noformat}
The {{actualSize}} represents the size of serialized data, {{byteArraySize}} is
the size of the returned byte array that contains the serialized data. Up until
the boundary of 8192 bytes the optimization works very well, but above this
threshold it produces a lot of waste.
I propose to add a conditional in {{ExposedByteArrayOutputStream.byteArray()}}
to return the raw buffer only if it's at most 10% larger than the size of
serialized data. This will cover all small-doc cases and avoid an array copy
for some less egregious cases for larger docs.
Here's the breakdown with the proposed optimization:
{noformat}
actualSize=61 byteArraySize=64 waste=3 (4.7% of
transmitted bytes)
actualSize=153 byteArraySize=153 waste=0 (0.0% of
transmitted bytes)
actualSize=604 byteArraySize=604 waste=0 (0.0% of
transmitted bytes)
actualSize=4104 byteArraySize=4104 waste=0 (0.0% of
transmitted bytes)
actualSize=6104 byteArraySize=6104 waste=0 (0.0% of
transmitted bytes)
actualSize=8104 byteArraySize=8104 waste=0 (0.0% of
transmitted bytes)
actualSize=8304 byteArraySize=8304 waste=0 (0.0% of
transmitted bytes)
actualSize=17386 byteArraySize=17386 waste=0 (0.0% of
transmitted bytes)
actualSize=24700 byteArraySize=24700 waste=0 (0.0% of
transmitted bytes)
actualSize=28550 byteArraySize=28550 waste=0 (0.0% of
transmitted bytes)
actualSize=31026 byteArraySize=32768 waste=1742 (5.3% of
transmitted bytes)
actualSize=36877 byteArraySize=36877 waste=0 (0.0% of
transmitted bytes)
actualSize=43667 byteArraySize=43667 waste=0 (0.0% of
transmitted bytes)
actualSize=47067 byteArraySize=47067 waste=0 (0.0% of
transmitted bytes)
actualSize=50696 byteArraySize=50696 waste=0 (0.0% of
transmitted bytes)
actualSize=57516 byteArraySize=57516 waste=0 (0.0% of
transmitted bytes)
actualSize=61110 byteArraySize=65536 waste=4426 (6.8% of
transmitted bytes)
actualSize=69145 byteArraySize=69145 waste=0 (0.0% of
transmitted bytes)
actualSize=79692 byteArraySize=79692 waste=0 (0.0% of
transmitted bytes)
actualSize=89342 byteArraySize=89342 waste=0 (0.0% of
transmitted bytes)
actualSize=91734 byteArraySize=91734 waste=0 (0.0% of
transmitted bytes)
actualSize=93769 byteArraySize=93769 waste=0 (0.0% of
transmitted bytes)
{noformat}
> CrossDC Common: ExposedByteArrayOutputStream optimizes wrong aspect
> -------------------------------------------------------------------
>
> Key: SOLR-18415
> URL: https://issues.apache.org/jira/browse/SOLR-18415
> Project: Solr
> Issue Type: Bug
> Components: module - crossDC
> Affects Versions: 10.0, 9.10.1
> Reporter: Andrzej Bialecki
> Assignee: Andrzej Bialecki
> Priority: Major
> Fix For: main(11.0), 10.1
>
>
> {{MirroredSolrRequestSerializer}} uses a subclass of
> {{ByteArrayOutputStream}} in an attempt to optimize data transfer by avoiding
> array copy - instead it passes the whole buffer directly as the output bytes
> from {{{}serialize(){}}}.
> This sounds like a plausible optimization. However, the initial size of the
> buffer (32 bytes) is doubled on each overflow as the codec writes to the
> stream, which means that on average the actual serialized data occupies
> around half of the buffer, the rest are zeroes. Deserialization will handle
> these trailing zeros just fine, it's the cost of transmitting roughly twice
> as much data over the wire that is questionable - this buffer is then passed
> as-is to the output stream and eventually ends up as network traffic.
> This "optimization" avoids one array copy cost (which reduces GC) but
> generates multiple costs on the way - not only the empty data has to be sent
> over the network but also it needs to be stored in source Kafka, in
> MirrorMaker and in target Kafka, to be finally retrieved by the Consumer
> (which has to allocate a buffer sufficiently large to fit also the zeroes),
> only to discard the zeroes during deserialization.
> I propose to get rid of this optimization and use the plain
> {{ByteArrayOutputStream}} instead.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
