Hi Vadim, According to these results, I don't think it makes sense to make this change in the product. I closed the ticket.
-Val On Fri, Mar 10, 2017 at 12:28 PM, Вадим Опольский <[email protected]> wrote: > Hi Valentin, > > I cant find out why it's better on long string, but worse on a short > string. May be it needs to use special tools like such as oracle solaris > studio performance analyzer or other. > > I've added links to the benchmark, unit test and results to the ticket and > switched status to patch available. > > Vadim Opolski > > > > 2017-03-09 13:57 GMT+03:00 Valentin Kulichenko < > [email protected]>: > >> Hi Vadim, >> >> Results are a bit confusing. Any idea why it's better on long string, but >> worse on a short string? If that's actually the case, there is no any >> reason to make the change and I would just close the ticket. >> >> -Val >> >> On Thu, Mar 9, 2017 at 9:20 AM, Вадим Опольский <[email protected]> >> wrote: >> >>> Hello everyone! >>> >>> Colleagues, take a look please at the results of measuring. >>> >>> Can I close this ticket ? >>> >>> Should I add JMH benchmark and unit test to Ignite project ? >>> >>> Results of measuring >>> https://github.com/javaller/mybenchmark/blob/master/out.txt >>> >>> Benchmark >>> https://github.com/javaller/mybenchmark/blob/master/src/main >>> /java/org/sample/ExampleTest.java >>> >>> UTest >>> https://github.com/javaller/mybenchmark/blob/master/src/main >>> /java/org/sample/BinaryMarshallerSelfTest.java >>> >>> *results of measuring* >>> Benchmark >>> (message) Mode Cnt >>> Score Error Units >>> LatchBenchmark.binaryHeapOutputStreamDirect >>> TestTestTestTestTestTestTestTestTest avgt 50 128,036 ± 4,360 ns/op >>> LatchBenchmark.binaryHeapOutputStreamDirect >>> TestTestTest avgt 50 44,934 ± 1,463 ns/op >>> LatchBenchmark.binaryHeapOutputStreamDirect >>> Test avgt 50 21,254 ± 0,776 ns/op >>> LatchBenchmark.binaryHeapOutputStreamInDirect >>> TestTestTestTestTestTestTestTestTest avgt 50 83,262 ± 2,264 ns/op >>> LatchBenchmark.binaryHeapOutputStreamInDirect >>> TestTestTest avgt 50 58,975 ± 1,559 ns/op >>> LatchBenchmark.binaryHeapOutputStreamInDirect >>> Test avgt 50 48,506 ± 1,116 ns/op >>> >>> >>> Vadim >>> >>> 2017-03-06 19:42 GMT+03:00 Вадим Опольский <[email protected]>: >>> >>>> Hello, everybody! >>>> >>>> Valentin, I've corrected benchmark and received the results: >>>> >>>> Benchmark >>>> (message) Mode Cnt >>>> Score Error Units >>>> LatchBenchmark.binaryHeapOutputStreamDirect >>>> TestTestTestTestTestTestTestTestTest avgt 50 128,036 ± 4,360 ns/op >>>> LatchBenchmark.binaryHeapOutputStreamDirect >>>> TestTestTest avgt 50 44,934 ± 1,463 ns/op >>>> LatchBenchmark.binaryHeapOutputStreamDirect >>>> Test avgt 50 21,254 ± 0,776 ns/op >>>> LatchBenchmark.binaryHeapOutputStreamInDirect >>>> TestTestTestTestTestTestTestTestTest avgt 50 83,262 ± 2,264 ns/op >>>> LatchBenchmark.binaryHeapOutputStreamInDirect >>>> TestTestTest avgt 50 58,975 ± 1,559 ns/op >>>> LatchBenchmark.binaryHeapOutputStreamInDirect >>>> Test avgt 50 48,506 ± 1,116 ns/op >>>> >>>> https://github.com/javaller/MyBenchmark/blob/master/out_06_03_17_2.txt >>>> >>>> Whats the next step ? >>>> >>>> Do I have to add benchmark to Ignite project ? >>>> >>>> Vadim Opolskiy >>>> >>>> 2017-03-03 21:11 GMT+03:00 Valentin Kulichenko < >>>> [email protected]>: >>>> >>>>> Hi Vadim, >>>>> >>>>> What do you mean by "copied benchmarks"? What changed singe previous >>>>> iteration and why results are so different? >>>>> >>>>> As for duplicated loop, you don't need it. BinaryOutputStream allows >>>>> to write a value to a particular position (even before already written >>>>> data). So you can reserve 4 bytes for length, remember position, calculate >>>>> length while encoding and writing bytes, and then write length. >>>>> >>>>> -Val >>>>> >>>>> On Fri, Mar 3, 2017 at 12:45 AM, Вадим Опольский <[email protected] >>>>> > wrote: >>>>> >>>>>> Valentin, >>>>>> >>>>>> What do you think about duplicated cycle in strToBinaryOutputStream ? >>>>>> >>>>>> How to calculate StrLen для outBinaryHeap without this cycle ? >>>>>> >>>>>> public class BinaryUtilsNew extends BinaryUtils { >>>>>> >>>>>> public static int getStrLen(String val) { >>>>>> int strLen = val.length(); >>>>>> int utfLen = 0; >>>>>> int c; >>>>>> >>>>>> // Determine length of resulting byte array. >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> *for (int cnt = 0; cnt < strLen; cnt++) { c = >>>>>> val.charAt(cnt); if (c >= 0x0001 && c <= 0x007F)* >>>>>> utfLen++; >>>>>> * else if (c > 0x07FF)* >>>>>> utfLen += 3; >>>>>> else >>>>>> utfLen += 2; >>>>>> } >>>>>> >>>>>> return utfLen; >>>>>> } >>>>>> >>>>>> public static void strToUtf8BytesDirect(BinaryOutputStream >>>>>> outBinaryHeap, String val) { >>>>>> >>>>>> int strLen = val.length(); >>>>>> int c, cnt; >>>>>> >>>>>> int position = 0; >>>>>> >>>>>> outBinaryHeap.unsafeEnsure(1 + 4); >>>>>> >>>>>> * outBinaryHeap.unsafeWriteByte(GridBinaryMarshaller.STRING); >>>>>> outBinaryHeap.unsafeWriteInt(getStrLen(val));* >>>>>> >>>>>> >>>>>> >>>>>> * for (cnt = 0; cnt < strLen; cnt++) { c = val.charAt(cnt);* >>>>>> * if (c >= 0x0001 && c <= 0x007F)* >>>>>> outBinaryHeap.writeByte((byte) c); >>>>>> * else if (c > 0x07FF) {* >>>>>> outBinaryHeap.writeByte((byte)(0xE0 | (c >> 12) & 0x0F)); >>>>>> outBinaryHeap.writeByte((byte)(0x80 | (c >> 6) & 0x3F)); >>>>>> outBinaryHeap.writeByte((byte)(0x80 | (c & 0x3F))); >>>>>> } >>>>>> else { >>>>>> outBinaryHeap.writeByte((byte)(0xC0 | ((c >> 6) & >>>>>> 0x1F))); >>>>>> outBinaryHeap.writeByte((byte)(0x80 | (c & 0x3F))); >>>>>> } >>>>>> } >>>>>> } >>>>>> >>>>>> >>>>>> Vadim >>>>>> >>>>>> >>>>>> >>>>>> 2017-03-03 2:00 GMT+03:00 Valentin Kulichenko < >>>>>> [email protected]>: >>>>>> >>>>>>> Vadim, >>>>>>> >>>>>>> Looks better now. Can you also try to modify the benchmark so that >>>>>>> marshaller and writer are created outside of the measured method? I.e. >>>>>>> the >>>>>>> benchmark methods should be as simple as this: >>>>>>> >>>>>>> @Benchmark >>>>>>> public void binaryHeapOutputStreamDirect() throws Exception { >>>>>>> writer.doWriteStringDirect(message); >>>>>>> } >>>>>>> >>>>>>> @Benchmark >>>>>>> public void binaryHeapOutputStreamInDirect() throws Exception { >>>>>>> writer.doWriteString(message); >>>>>>> } >>>>>>> >>>>>>> In any case, do I understand correctly that it didn't actually make >>>>>>> any performance difference? If so, I think we can close the ticket. >>>>>>> >>>>>>> Vova, can you also take a look and provide your thoughts? >>>>>>> >>>>>>> -Val >>>>>>> >>>>>>> On Thu, Mar 2, 2017 at 1:27 PM, Вадим Опольский < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> Hi Valentin! >>>>>>>> >>>>>>>> I've created: >>>>>>>> >>>>>>>> new method strToUtf8BytesDirect in BinaryUtilsNew >>>>>>>> https://github.com/javaller/MyBenchmark/blob/master/src/main >>>>>>>> /java/org/sample/BinaryUtilsNew.java >>>>>>>> >>>>>>>> new method doWriteStringDirect in BinaryWriterExImplNew >>>>>>>> https://github.com/javaller/MyBenchmark/blob/master/src/main >>>>>>>> /java/org/sample/BinaryWriterExImplNew.java >>>>>>>> >>>>>>>> benchmarks for BinaryWriterExImpl doWriteString and >>>>>>>> BinaryWriterExImplNew doWriteStringDirect >>>>>>>> https://github.com/javaller/MyBenchmark/blob/master/src/main >>>>>>>> /java/org/sample/ExampleTest.java >>>>>>>> >>>>>>>> This is a result of comparing: >>>>>>>> >>>>>>>> Benchmark >>>>>>>> Mode Cnt Score Error >>>>>>>> UnitsExampleTest.binaryHeapOutputStreamDirect >>>>>>>> avgt 50 1128448,743 ± 13536,689 >>>>>>>> ns/opExampleTest.binaryHeapOutputStreamInDirect >>>>>>>> avgt 50 1127270,695 ± 17309,256 ns/op >>>>>>>> >>>>>>>> Vadim >>>>>>>> >>>>>>>> 2017-03-02 1:02 GMT+03:00 Valentin Kulichenko < >>>>>>>> [email protected]>: >>>>>>>> >>>>>>>>> Hi Vadim, >>>>>>>>> >>>>>>>>> We're getting closer :) I would actually like to see the test for >>>>>>>>> actual implementation of BinaryWriterExImpl#doWriteString method. >>>>>>>>> Logic in binaryHeapOutputInDirect() confuses me a bit and I'm not sure >>>>>>>>> comparison is valid. >>>>>>>>> >>>>>>>>> Can you please do the following: >>>>>>>>> >>>>>>>>> 1. Create new BinaryUtils#strToUtf8BytesDirect method, copy-paste >>>>>>>>> the code from existing BinaryUtils#strToUtf8Bytes and modify it so >>>>>>>>> that it >>>>>>>>> takes BinaryOutputStream as an argument and writes to it directly. Do >>>>>>>>> not >>>>>>>>> create stream inside this method, as it's the same as creating new >>>>>>>>> array. >>>>>>>>> 2. Create new BinaryWriterExImpl#doWriteStringDirect, copy-paste >>>>>>>>> the code from existing BinaryWriterExImpl#doWriteString and >>>>>>>>> modify it so that it uses BinaryUtils#strToUtf8BytesDirect and >>>>>>>>> doesn't call out.writeByteArray. >>>>>>>>> 3. Create benchmark for BinaryWriterExImpl#doWriteString method. >>>>>>>>> I.e., create an instance of BinaryWriterExImpl and call >>>>>>>>> doWriteString() in >>>>>>>>> benchmark method. >>>>>>>>> 4. Similarly, create benchmark for BinaryWriterExImpl#doWriteStri >>>>>>>>> ngDirect. >>>>>>>>> 5. Compare results. >>>>>>>>> >>>>>>>>> This will give us clear picture of how these two approaches >>>>>>>>> perform. Your current results are actually promising, but I would >>>>>>>>> like to >>>>>>>>> confirm them. >>>>>>>>> >>>>>>>>> -Val >>>>>>>>> >>>>>>>>> On Wed, Mar 1, 2017 at 6:17 AM, Вадим Опольский < >>>>>>>>> [email protected]> wrote: >>>>>>>>> >>>>>>>>>> Hi Valentin! >>>>>>>>>> >>>>>>>>>> Thank you for comments. >>>>>>>>>> >>>>>>>>>> There is a new method which writes directly to BinaryOutputStream >>>>>>>>>> instead of intermediate array. >>>>>>>>>> https://github.com/javaller/MyBenchmark/blob/master/src/main >>>>>>>>>> /java/org/sample/BinaryUtilsNew.java >>>>>>>>>> >>>>>>>>>> There is benchmark. >>>>>>>>>> https://github.com/javaller/MyBenchmark/blob/master/src/main >>>>>>>>>> /java/org/sample/MyBenchmark.java >>>>>>>>>> >>>>>>>>>> Unit test >>>>>>>>>> https://github.com/javaller/MyBenchmark/blob/master/src/main >>>>>>>>>> /java/org/sample/BinaryOutputStreamTest.java >>>>>>>>>> >>>>>>>>>> Statistics >>>>>>>>>> https://github.com/javaller/MyBenchmark/blob/master/out_01_0 >>>>>>>>>> 3_17.txt >>>>>>>>>> >>>>>>>>>> Benchmark >>>>>>>>>> Mode Cnt Score Error Units >>>>>>>>>> MyBenchmark.binaryHeapOutputIn >>>>>>>>>> Direct avgt 50 111,337 ± 0,742 ns/op >>>>>>>>>> MyBenchmark.binaryHeapOutputStreamDirect avgt 50 >>>>>>>>>> 23,847 ± 0,303 ns/op >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Vadim >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> 2017-02-28 4:29 GMT+03:00 Valentin Kulichenko < >>>>>>>>>> [email protected]>: >>>>>>>>>> >>>>>>>>>>> Hi Vadim, >>>>>>>>>>> >>>>>>>>>>> Looks like you accidentally removed dev list from the thread, >>>>>>>>>>> adding it back. >>>>>>>>>>> >>>>>>>>>>> I think there is still misunderstanding. What I propose is to >>>>>>>>>>> modify the BinaryUtils#strToUtf8Bytes so that it writes directly to >>>>>>>>>>> BinaryOutputStream >>>>>>>>>>> instead of intermediate array. This should decrease memory >>>>>>>>>>> consumption and >>>>>>>>>>> can also increase performance as we will avoid 'writeByteArray' >>>>>>>>>>> step at the end. >>>>>>>>>>> >>>>>>>>>>> Does it make sense to you? >>>>>>>>>>> >>>>>>>>>>> -Val >>>>>>>>>>> >>>>>>>>>>> On Mon, Feb 27, 2017 at 6:55 AM, Вадим Опольский < >>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>> >>>>>>>>>>>> Hi, Valentin! >>>>>>>>>>>> >>>>>>>>>>>> What do you think about using the methods of BinaryOutputStream: >>>>>>>>>>>> >>>>>>>>>>>> 1) writeByteArray(byte[] val) >>>>>>>>>>>> 2) writeCharArray(char[] val) >>>>>>>>>>>> 3) write (byte[] arr, int off, int len) >>>>>>>>>>>> >>>>>>>>>>>> String val = "Test"; >>>>>>>>>>>> out.writeByteArray( val.getBytes(UTF_8)); >>>>>>>>>>>> >>>>>>>>>>>> String val = "Test"; >>>>>>>>>>>> out.writeCharArray(str.toCharArray()); >>>>>>>>>>>> >>>>>>>>>>>> String val = "Test" >>>>>>>>>>>> InputStream stream = new ByteArrayInputStream( >>>>>>>>>>>> exampleString.getBytes(StandartCharsets.UTF_8)); >>>>>>>>>>>> byte[] buffer = new byte[1024]; >>>>>>>>>>>> while ((buffer = stream.read()) != -1) { >>>>>>>>>>>> out.writeByteArray(buffer); >>>>>>>>>>>> } >>>>>>>>>>>> >>>>>>>>>>>> What else can we use ? >>>>>>>>>>>> >>>>>>>>>>>> Vadim >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> 2017-02-25 2:21 GMT+03:00 Valentin Kulichenko < >>>>>>>>>>>> [email protected]>: >>>>>>>>>>>> >>>>>>>>>>>>> Hi Vadim, >>>>>>>>>>>>> >>>>>>>>>>>>> Which method implements the approach described in the ticket? >>>>>>>>>>>>> From what I see, all writeToStringX versions are still encoding >>>>>>>>>>>>> into an >>>>>>>>>>>>> intermediate array and then call out.writeByteArray. What we need >>>>>>>>>>>>> to test >>>>>>>>>>>>> is the approach where bytes are written directly into the stream >>>>>>>>>>>>> during >>>>>>>>>>>>> encoding. Encoding algorithm itself should stay the same for now, >>>>>>>>>>>>> otherwise >>>>>>>>>>>>> we will not know how to interpret the result. >>>>>>>>>>>>> >>>>>>>>>>>>> It looks like there is some misunderstanding here, so please >>>>>>>>>>>>> let me know anything is still unclear. I will be happy to answer >>>>>>>>>>>>> your >>>>>>>>>>>>> questions. >>>>>>>>>>>>> >>>>>>>>>>>>> -Val >>>>>>>>>>>>> >>>>>>>>>>>>> On Wed, Feb 22, 2017 at 7:22 PM, Valentin Kulichenko < >>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Hi Vadim, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, I will review this week. >>>>>>>>>>>>>> >>>>>>>>>>>>>> -Val >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Wed, Feb 22, 2017 at 2:28 AM, Вадим Опольский < >>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hi Valentin! >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/IGNITE-13 >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I created BinaryWriterExImplNew (extended of >>>>>>>>>>>>>>> BinaryWriterExImpl) and added new methods with changes >>>>>>>>>>>>>>> described in the ticket >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> https://github.com/javaller/MyBenchmark/blob/master/src/main >>>>>>>>>>>>>>> /java/org/sample/BinaryWriterExImplNew.java >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I created a benchmark for BinaryWriterExImplNew >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> https://github.com/javaller/MyBenchmark/blob/master/src/main >>>>>>>>>>>>>>> /java/org/sample/ExampleTest.java >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I run benchmark and compared results >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> https://github.com/javaller/MyBenchmark/blob/master/totalsta >>>>>>>>>>>>>>> t.txt >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> # Run complete. Total time: 00:10:24 >>>>>>>>>>>>>>> Benchmark Mode >>>>>>>>>>>>>>> Cnt Score Error Units >>>>>>>>>>>>>>> ExampleTest.binaryHeapOutputStream1 avgt 50 >>>>>>>>>>>>>>> 1114999,207 ± 16756,776 ns/op >>>>>>>>>>>>>>> ExampleTest.binaryHeapOutputStream2 avgt 50 >>>>>>>>>>>>>>> 1118149,320 ± 17515,961 ns/op >>>>>>>>>>>>>>> ExampleTest.binaryHeapOutputStream3 avgt 50 >>>>>>>>>>>>>>> 1113678,657 ± 17652,314 ns/op >>>>>>>>>>>>>>> ExampleTest.binaryHeapOutputStream4 avgt 50 >>>>>>>>>>>>>>> 1112415,051 ± 18273,874 ns/op >>>>>>>>>>>>>>> ExampleTest.binaryHeapOutputStream5 avgt 50 >>>>>>>>>>>>>>> 1111366,583 ± 18282,829 ns/op >>>>>>>>>>>>>>> ExampleTest.binaryHeapOutputStreamACSII avgt 50 >>>>>>>>>>>>>>> 1112079,667 ± 16659,532 ns/op >>>>>>>>>>>>>>> ExampleTest.binaryHeapOutputStreamUTFCustom avgt 50 >>>>>>>>>>>>>>> 1114949,759 ± 16809,669 ns/op >>>>>>>>>>>>>>> ExampleTest.binaryHeapOutputStreamUTFNIO avgt 50 >>>>>>>>>>>>>>> 1121462,325 ± 19836,466 ns/op >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Is it OK? Whats the next step? Do I have to move this >>>>>>>>>>>>>>> JMH benchmark to the Ignite project ? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Vadim Opolski >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> 2017-02-21 1:06 GMT+03:00 Valentin Kulichenko < >>>>>>>>>>>>>>> [email protected]>: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Hi Vadim, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I'm not sure I understand your benchmarks and how they >>>>>>>>>>>>>>>> verify the optimization discussed here. Basically, here is >>>>>>>>>>>>>>>> what needs to be >>>>>>>>>>>>>>>> done: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> 1. Create a benchmark for BinaryWriterExImpl#doWriteString >>>>>>>>>>>>>>>> method. >>>>>>>>>>>>>>>> 2. Run the benchmark with current implementation. >>>>>>>>>>>>>>>> 3. Make the change described in the ticket. >>>>>>>>>>>>>>>> 4. Run the benchmark with these changes. >>>>>>>>>>>>>>>> 5. Compare results. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Makes sense? Let me know if anything is unclear. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> -Val >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On Mon, Feb 20, 2017 at 8:51 AM, Вадим Опольский < >>>>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Hello everybody! >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/IGNITE-13 >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Valentin, I just have finished benchmark (with JMH) - >>>>>>>>>>>>>>>>> https://github.com/javaller/MyBenchmark.git >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> It collect data about time working of serialization. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> For instance - https://github.com/javaller/My >>>>>>>>>>>>>>>>> Benchmark/blob/master/out200217.txt >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> To start it you have to do next: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> 1) clone it - git colne https://github.com/javal >>>>>>>>>>>>>>>>> ler/MyBenchmark.git >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> 2) install it - mvn install >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> 3) run benchmarks - java -Xms1024m -Xmx4096m -jar >>>>>>>>>>>>>>>>> target\benchmarks.jar >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Vadim Opolski >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> 2017-02-15 0:52 GMT+03:00 Valentin Kulichenko < >>>>>>>>>>>>>>>>> [email protected]>: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Vladimir, >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> I think we misunderstood each other. My understanding of >>>>>>>>>>>>>>>>>> this optimization is the following. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Currently string serialization is done in two steps (see >>>>>>>>>>>>>>>>>> BinaryWriterExImpl#doWriteString): >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> strArr = BinaryUtils.strToUtf8Bytes(val); // Encode >>>>>>>>>>>>>>>>>> string into byte array. >>>>>>>>>>>>>>>>>> out.writeByteArray(strArr); // Write >>>>>>>>>>>>>>>>>> byte array into stream. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> What this ticket suggests is to write directly into >>>>>>>>>>>>>>>>>> stream while string is encoded, without intermediate array. >>>>>>>>>>>>>>>>>> This both >>>>>>>>>>>>>>>>>> reduces memory consumption and eliminates array copy step. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> I updated the ticket and added this explanation there. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Vadim, can you create a micro benchmark and check if it >>>>>>>>>>>>>>>>>> gives any improvement? >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> -Val >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> On Sun, Feb 12, 2017 at 10:38 PM, Vladimir Ozerov < >>>>>>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> It is hard to say whether it makes sense or not. No >>>>>>>>>>>>>>>>>>> doubt, it could speed up marshalling process at the cost of >>>>>>>>>>>>>>>>>>> 2x memory >>>>>>>>>>>>>>>>>>> required for strings. From my previous experience with >>>>>>>>>>>>>>>>>>> marshalling >>>>>>>>>>>>>>>>>>> micro-optimizations, we will hardly ever notice speedup in >>>>>>>>>>>>>>>>>>> distributed >>>>>>>>>>>>>>>>>>> environment. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> But, there is another sied - it could speedup our >>>>>>>>>>>>>>>>>>> queries, because we will not have to unmarshal string on >>>>>>>>>>>>>>>>>>> every field >>>>>>>>>>>>>>>>>>> access. So I would try to make this optimization optional >>>>>>>>>>>>>>>>>>> and then measure >>>>>>>>>>>>>>>>>>> query performance with classes having lots of strings. It >>>>>>>>>>>>>>>>>>> could give us >>>>>>>>>>>>>>>>>>> interesting results. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> On Mon, Feb 13, 2017 at 5:37 AM, Valentin Kulichenko < >>>>>>>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Vladimir, >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Can you please take a look and provide your thoughts? >>>>>>>>>>>>>>>>>>>> Can this be applied to binary marshaller? From what I >>>>>>>>>>>>>>>>>>>> recall, it serializes >>>>>>>>>>>>>>>>>>>> string a bit differently from optimized marshaller, so I'm >>>>>>>>>>>>>>>>>>>> not sure. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> -Val >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> On Fri, Feb 10, 2017 at 5:16 PM, Dmitriy Setrakyan < >>>>>>>>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> On Thu, Feb 9, 2017 at 11:26 PM, Valentin Kulichenko < >>>>>>>>>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> > Hi Vadim, >>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>> > I don't think it makes much sense to invest into >>>>>>>>>>>>>>>>>>>>> OptimizedMarshaller. >>>>>>>>>>>>>>>>>>>>> > However, I would check if this optimization is >>>>>>>>>>>>>>>>>>>>> applicable to >>>>>>>>>>>>>>>>>>>>> > BinaryMarshaller, and if yes, implement it. >>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Val, in this case can you please update the ticket? >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>> > -Val >>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>> > On Thu, Feb 9, 2017 at 11:05 PM, Вадим Опольский < >>>>>>>>>>>>>>>>>>>>> [email protected]> >>>>>>>>>>>>>>>>>>>>> > wrote: >>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>> > > Dear sirs! >>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>> > > I want to resolve issue IGNITE-13 - >>>>>>>>>>>>>>>>>>>>> > > https://issues.apache.org/jira/browse/IGNITE-13 >>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>> > > Is it actual? >>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>> > > Vadim Opolski >>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> >
