Michael McCandless wrote: > I don't like that the answer is different... but it's really really > odd that it's different-yet-almost-the-same. > > Mark, were these 4 results on a normal (ext4) filesystem, or tmpfs? > (Because the top 2 entries of your 4 results match the first set of 2 > entries you sent... so I'm thinking these 4 were actually tmpfs not > ext4). > Those 4 were tmpfs - I mention ext4 at the end because I had just given a feel for the hardrive tests and wanted to note it was from ext4 - the results are def ramdisk though. > What JRE/OS, linux, kernel versions, and hardware, are you running on? > These are on: Ubuntu Karmic Koala 9.10, currently updated java-1.5.0-sun-1.5.0.20 2.6.31-10-generic
RAM: 3.9 Gig 4 core Intel Core2 duo 2.0GHz Slow 5200 rpm laptop drives. > The gains of SeparateFile over all else are stunning. And, quite > different from the linux tests I had run under LUCENE-753. Maybe we > need to revert FSDir.open to return SimpleFSDir again, on non-Windows > hosts. But then we don't have good concurrency... > > Mike > > On Tue, Sep 15, 2009 at 2:59 PM, Yonik Seeley > <yonik.see...@lucidimagination.com> wrote: > >> It's been a while since I wrote that benchmarker... is it OK that the >> answer is different? Did you use the same test file? >> >> -Yonik >> http://www.lucidimagination.com >> >> >> >> On Tue, Sep 15, 2009 at 2:18 PM, Mark Miller <markrmil...@gmail.com> wrote: >> >>> The results: >>> >>> config: impl=SeparateFile serial=false nThreads=4 iterations=100 >>> bufsize=1024 poolsize=2 filelen=730554368 >>> answer=-282295611, ms=173550, MB/sec=1683.7899579371938 >>> >>> config: impl=ChannelFile serial=false nThreads=4 iterations=100 >>> bufsize=1024 poolsize=2 filelen=730554368 >>> answer=-282295361, ms=1377768, MB/sec=212.09793463050383 >>> >>> config: impl=ChannelPread serial=false nThreads=4 iterations=100 >>> bufsize=1024 poolsize=2 filelen=730554368 >>> answer=-282295361, ms=632253, MB/sec=462.19115955163517 >>> >>> config: impl=PooledPread serial=false nThreads=4 iterations=100 >>> bufsize=1024 poolsize=2 filelen=730554368 >>> answer=-282295361, ms=774664, MB/sec=377.2238637654518 >>> >>> ClassicFile was heading for the same fate as ChannelFile. >>> >>> >>> I'll have to check what its like on the file system - but it appears >>> just ridiculously slower. Even with SeparateFile, All 4 cores are bouncing >>> from 0-12% independently and really favoring the low end of that. >>> ChannelPread appears no better. >>> >>> There are results from other OS's/setups in the JIRA issue. >>> >>> I'm using ext4. >>> >>> Uwe Schindler wrote: >>> >>>> How does a conventional file system compare? >>>> >>>> ----- >>>> Uwe Schindler >>>> H.-H.-Meier-Allee 63, D-28213 Bremen >>>> http://www.thetaphi.de >>>> eMail: u...@thetaphi.de >>>> >>>> >>>> >>>>> -----Original Message----- >>>>> From: Mark Miller [mailto:markrmil...@gmail.com] >>>>> Sent: Tuesday, September 15, 2009 7:15 PM >>>>> To: java-user@lucene.apache.org >>>>> Subject: Re: lucene 2.9.0RC4 slower than 2.4.1? >>>>> >>>>> Mark Miller wrote: >>>>> >>>>> >>>>>> Indeed - I just ran the FileReaderTest on a Linux tmpfs ramdisk - with >>>>>> SeparateFile all 4 of my cores are immediately pinned and remain so. >>>>>> With ChannelFile, all 4 cores hover 20-30%. >>>>>> >>>>>> It would appear it may not be a good idea to use NIOFSDirectory on >>>>>> >>>>>> >>>>> ramdisks. >>>>> >>>>> >>>>>> Even still though - it looks like you have a further issue - your Lucene >>>>>> 2.9 old-api results don't use it, and are still not good. >>>>>> >>>>>> >>>>>> >>>>>> >>>>> The quick results: >>>>> >>>>> ramdisk: sudo mount -t tmpfs tmpfs /tmp/space -o >>>>> size=1G,nr_inodes=200k,mode=01777 >>>>> >>>>> config: impl=SeparateFile serial=false nThreads=4 iterations=100 >>>>> bufsize=1024 poolsize=2 filelen=730554368 >>>>> answer=-282295611, ms=173550, MB/sec=1683.7899579371938 >>>>> >>>>> config: impl=ChannelFile serial=false nThreads=4 iterations=100 >>>>> bufsize=1024 poolsize=2 filelen=730554368 >>>>> answer=-282295361, ms=1377768, MB/sec=212.09793463050383 >>>>> >>>>> >>>>> -- >>>>> - Mark >>>>> >>>>> http://www.lucidimagination.com >>>>> >>>>> >>>>> >>>>> >>>>> --------------------------------------------------------------------- >>>>> To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org >>>>> For additional commands, e-mail: java-user-h...@lucene.apache.org >>>>> >>>>> >>>> >>>> --------------------------------------------------------------------- >>>> To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org >>>> For additional commands, e-mail: java-user-h...@lucene.apache.org >>>> >>>> >>>> >>> -- >>> - Mark >>> >>> http://www.lucidimagination.com >>> >>> >>> >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org >>> For additional commands, e-mail: java-user-h...@lucene.apache.org >>> >>> >>> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org >> For additional commands, e-mail: java-user-h...@lucene.apache.org >> >> >> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > For additional commands, e-mail: java-user-h...@lucene.apache.org > > -- - Mark http://www.lucidimagination.com --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
