Hi again, So I made the memory mapped file changes to the Go version, and the results:
insert time 10000000 records = 17890 ms, usec per op 1.7890143 close time 8477 ms scan time 2887 ms, usec per op 0.2887559 scan time 50% 81 ms, usec per op 0.162584 random access time 3.508029 us per get close with merge 1 time 0.148 ms scan time 2887 ms, usec per op 0.2887248 scan time 50% 85 ms, usec per op 0.171406 random access time 3.487226 us per get Improving the Go performance to exceed Java’s in the random access, and improving the sequential performance by nearly 5x. > On Jan 1, 2021, at 5:20 PM, robert engels <reng...@ix.netcom.com> wrote: > > Hi all, > > I had a opportunity to change the Java version to use memory mapped files, > using GraalVM: > > insert time 10000000 records = 15443ms, usec per op 1.5443 > close time 4954ms > scan time 1934ms, usec per op 0.1934 > scan time 50% 81ms, usec per op 0.162 > random access time 6.264us per get > close with merge 1 time 0ms > scan time 2077ms, usec per op 0.2077 > scan time 50% 67ms, usec per op 0.134 > random access time 6.083us per get > > The Java version is now more than 7x faster than Go in most sequential reads, > and 33% faster in random. > > If I find the time I will add memory mapped files to the Go version as well. > > I offer these metrics to illustrate that usually the IO algorithm is going to > affect performance more than micro differences in JVM/Go/Native binaries with > modern development platforms. > >> On Dec 26, 2020, at 5:23 PM, robert engels <reng...@ix.netcom.com >> <mailto:reng...@ix.netcom.com>> wrote: >> >> Ask and ye shall receive… >> >> So, first with running using the GraalVM JDK 11 Community Edition: >> >> insert time 10000000 records = 19305ms, usec per op 1.9305 >> close time 12313ms >> scan time 10695ms, usec per op 1.0695 >> scan time 50% 461ms, usec per op 0.922 >> random access time 12.014us per get >> close with merge 1 time 0ms >> scan time 9685ms, usec per op 0.9685 >> scan time 50% 456ms, usec per op 0.912 >> random access time 11.986us per get >> >> then I created the statically compiled binary: >> >> insert time 10000000 records = 33182ms, usec per op 3.3182 >> close time 23602ms >> scan time 15104ms, usec per op 1.5104 >> scan time 50% 670ms, usec per op 1.34 >> random access time 23.85us per get >> close with merge 1 time 0ms >> scan time 15093ms, usec per op 1.5093 >> scan time 50% 705ms, usec per op 1.41 >> random access time 24.271us per get >> >> oops, not so good. So then I downloaded the enterprise versions and created >> a profile guided optimized static binary: >> >> insert time 10000000 records = 21705ms, usec per op 2.1705 >> close time 13792ms >> scan time 10415ms, usec per op 1.0415 >> scan time 50% 479ms, usec per op 0.958 >> random access time 13.582us per get >> close with merge 1 time 0ms >> scan time 10391ms, usec per op 1.0391 >> scan time 50% 479ms, usec per op 0.958 >> random access time 13.557us per get >> >> The binary size of the last is 8.5 MB. For comparison, the Go binary is 3MB. >> >> >> >> >>> On Dec 22, 2020, at 10:55 AM, flying_dutchman <younisas...@gmail.com >>> <mailto:younisas...@gmail.com>> wrote: >>> >>> I think it'd be a good idea if you can compile your Java port using GraalVM >>> and benchmark the generated executable. >>> >>> On Monday, December 14, 2020 at 2:58:15 AM UTC+5:30 ren...@ix.netcom.com >>> <http://ix.netcom.com/> wrote: >>> I did not, and honestly it is probably not a great comparison. >>> >>> Java requires the JVM - which is separate - so executables sizes are hard >>> to compare. The Java ‘executable code’ is tiny. >>> >>> As for runtime memory usage, it is fairly trivial since the data is stored >>> on disk with an in-memory ’skip index’. The skip index is nearly identical >>> between the two - Java probably having a bit more pointer overhead - but >>> the size of the in-memory index is configurable - so trading memory for >>> speed is up to the user. >>> >>> There is no way to cap the heap size in Go to offer an apples-2-apples >>> comparison. >>> >>> >>> >>> >>> >>>> On Dec 13, 2020, at 3:08 PM, da...@suarezhouse.net >>>> <http://suarezhouse.net/> <da...@suarezhouse.net >>>> <applewebdata://F5002F7B-0E75-40B1-AB08-F7A75D36160F>> wrote: >>>> >>> >>>> Super interesting. Did you happen to catch the runtime memory avg, >>>> median, max and current "executable" file size difference? >>>> >>>> On Saturday, December 12, 2020 at 1:04:42 PM UTC-6 ren...@ix.netcom.com >>>> <http://ix.netcom.com/> wrote: >>>> Hi all, >>>> >>>> I thought this might be of interest to some. I released a Java version of >>>> keydb <https://github.com/robaho/keydb> at jkeydb >>>> <https://github.com/robaho/jkeydb>. I primarily did the exercise to keep >>>> my Java skills fresh and do an updated performance comparison between Go >>>> and Java. >>>> >>>> Tests performed using OSX Big Sur. >>>> >>>> Using Go 1.15.5: >>>> >>>> insert time 10000000 records = 24670 ms, usec per op 2.4670965 >>>> close time 16945 ms >>>> scan time 10631 ms, usec per op 1.063149 >>>> scan time 50% 470 ms, usec per op 0.941686 >>>> random access time 9.658001 us per get >>>> close with merge 1 time 0.681 ms >>>> scan time 11253 ms, usec per op 1.1253718 >>>> scan time 50% 471 ms, usec per op 0.942876 >>>> random access time 9.702651 us per get >>>> >>>> Using Java 1.15: >>>> >>>> insert time 10000000 records = 24102ms, usec per op 2.4102 >>>> close time 13564ms >>>> scan time 10259ms, usec per op 1.0259 >>>> scan time 50% 474ms, usec per op 0.948 >>>> random access time 13.209us per get >>>> close with merge 1 time 0ms >>>> scan time 10142ms, usec per op 1.0142 >>>> scan time 50% 501ms, usec per op 1.002 >>>> random access time 13.28us per get >>>> >>>> Performance is very similar, except that Go is significantly faster in the >>>> random access tests. I attribute this to the JNI overhead in making lots >>>> of small IO requests. In a previous life I wrote some custom JNI code for >>>> ultrafast IO and I might resurrect that to see if it makes a difference. >>>> >>>> You can vary the ‘keyIndexInterval’ to trade memory for speed which >>>> significantly helps the Java version by reducing the IO. >>>> >>>> There are significantly fewer source (non test) code files in the Go >>>> version, 10 vs. 26 which highlights the simplicity of Go. >>>> >>>> Anyway, feel free to ask any questions if you wish. >>>> >>>> >>>> >>> >>>> -- >>>> You received this message because you are subscribed to the Google Groups >>>> "golang-nuts" group. >>>> To unsubscribe from this group and stop receiving emails from it, send an >>>> email to golang-nuts...@googlegroups.com >>>> <applewebdata://F5002F7B-0E75-40B1-AB08-F7A75D36160F>. >>> >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/golang-nuts/9079e608-2c6c-4547-8c27-ebbb48a2fe61n%40googlegroups.com >>>> >>>> <https://groups.google.com/d/msgid/golang-nuts/9079e608-2c6c-4547-8c27-ebbb48a2fe61n%40googlegroups.com?utm_medium=email&utm_source=footer>. >>> >>> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "golang-nuts" group. >>> To unsubscribe from this group and stop receiving emails from it, send an >>> email to golang-nuts+unsubscr...@googlegroups.com >>> <mailto:golang-nuts+unsubscr...@googlegroups.com>. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/golang-nuts/37374167-c575-407f-971b-78f2b1c3131bn%40googlegroups.com >>> >>> <https://groups.google.com/d/msgid/golang-nuts/37374167-c575-407f-971b-78f2b1c3131bn%40googlegroups.com?utm_medium=email&utm_source=footer>. > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/9151521A-5002-45E0-B56A-9168C2965B1D%40ix.netcom.com.
