Re: indexing performance 6.6 vs 7.1

2018-01-29 Thread Rob Audenaerde
Hi Uwe, Thanks for the reply. We commit often. Actually, in the benchmark, we commit every 60 documents (but we will run a larger set with less commits). The number of commits we call does not change between 6.6. and 7.1. In our production systems we commit every 5000 documents. We dug deeper in

RE: indexing performance 6.6 vs 7.1

2018-01-29 Thread Uwe Schindler
Hi, How often do you commit? If you index the data initially (that's the case where indexing needs to be fast), one would call commit at the end of the whole job, so the actual time it takes is not so important. If you have a system where the index is updated all the time, then of course commi

Re: indexing performance 6.6 vs 7.1

2018-01-29 Thread Rob Audenaerde
Hi all, Some follow up (sorry for the delay). We built a benchmark in our application, and profiled it (on a smallish data set). What we currently see in the profiler is that in Lucene 7.1 the calls to `commit()` take much longer. The self-time committing in 6.6: 3,215 ms The self-time committin