Hi Ian, Yes I/O ->CPU, but read on... The throughput is about the same with similar i/o and 20% higher CPU. We have plenty of CPU and I would rather be cpu bound, thean i/o bound.
However, we have increased the number of indexing threads in our application. On Lucene 2.3.2, this gave no performance improvement as the I/O just increased. However, lucene 3.6.0 ramps up the CPU (user) usage while I/O only increases marginally. Our throughput has trebled !! - amazing improvement, and all in CPU space. So it appears there have been a lot of performance improvements made in the new version that I was not previously using, the Ferrari was in first gear and now we are taking advantage of these features. This is truly phenominal step-change in throughput, thank you Lucene developers ! Clive ________________________________ From: Ian Lea <ian....@gmail.com> To: java-user@lucene.apache.org; kiwi clive <kiwi_cl...@yahoo.com> Sent: Friday, November 9, 2012 10:04 AM Subject: Re: Lucene 3.6.0 high CPU usage Are you getting the same, improved or worse performance/throughput? Has the bottleneck switched from IO to CPU? -- Ian. On Thu, Nov 8, 2012 at 12:40 PM, kiwi clive <kiwi_cl...@yahoo.com> wrote: > Having played with merge parameters and various index parameters, it seems > possible to change the I/O usage at the cost of the number of index files. > However, it does appear that this version of lucene is using more CPU. Is > there any reason for this ? is it normal ? We can push a large amount of > documents through the indexer and it seams to work admirably although I would > be happier if the load average came down. > > Any lucene devs out there who could shed some light on this behaviour ? > > > Thanks, > Clive > > > ________________________________ > From: kiwi clive <kiwi_cl...@yahoo.com> > To: "java-user@lucene.apache.org" <java-user@lucene.apache.org> > Sent: Wednesday, November 7, 2012 10:58 AM > Subject: Lucene 3.6.0 high CPU usage > > Hi Guys, > > Having upgraded from lucene2.3.2 to lucene3.6.0 and jumping through a few > hoops, we note that the CPU usage on the new version is perhaps 20-30% > higher. The i/o wait appears about the same on both versions. > > Can anyone suggest why this should be so or is the newer lucene version just > more CPU hungry? If not, any tuning suggestions to reduce CPU usage would be > gratefully received ! > > Thanks > Clive --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
