Thanks for the info! Our servers have SSDs and plenty of RAM. So it sounds like possibly putting the index files in tmp on a separate disk might add some performance improvement ...though the SSDs by themselves will probably be the biggest improvement.
On Thu, Feb 19, 2015 at 4:48 PM, Paul A <pau...@navalmarinearchive.com> wrote: > At 01:26 PM 2/20/2015 +1300, Robin Sheat wrote: > >> Chad Roseburg schreef op do 19-02-2015 om 15:14 [-0800]: >> > We are considering mounting the index files to a separate >> high-performance >> > disk array in hopes of improving performance during the indexing >> process -- >> > has anyone experimented with something like this? >> >> It'd be worth seeing where the slowdown in the indexing actually is: is >> the exporting the slow part, or the merging into the index? >> >> Though, I've never tried with zebra indicies on a high speed disk, I'd >> be interested to see some real benchmarks. >> > > Not absolute, but... our latest server uses SSD (raided 240 Gig Kingstons) > rather than HDD (raided 500 Gig Seagates) with very slightly faster 8-core > CPU, same 18 Gigs RAM; and the "read speed" (counting by 100's) of a total > re-index is about twice as fast, with authorities faster than biblios. > However, the "Cleaning" stage is disappointingly about the same speed, > maybe a tad faster but barely perceptible. Memcached enabled on both boxes, > same version of Zebra. I get the *impression* that Zebra sorts (or does > some other process) during the final write stage, otherwise we'd have seen > an improvement. > > However, we rarely use the full re-index, having got the Cron job fully > sorted every minute (including minor tweaking of > biblio-zebra-indexdefs.xsl) -- and that is as near instantaneous as anyone > could wish. > > Best -- Paul > > Also, more RAM might be >> useful for disk caching, and you could experiment with disk fsync() >> commit options (i.e. by making it not require data to be written back >> before returning.) This has risk though, but if you're careful, it could >> well be acceptable, for example just needing to a full reindex in case >> of a power outage. >> >> -- >> Robin Sheat >> Catalyst IT Ltd. >> ✆ +64 4 803 2204 >> GPG: 5FA7 4B49 1E4D CAA4 4C38 8505 77F5 B724 F871 3BDF >> >> _______________________________________________ >> Koha mailing list http://koha-community.org >> Koha@lists.katipo.co.nz >> http://lists.katipo.co.nz/mailman/listinfo/koha >> > > --- > Maritime heritage and history, preservation and conservation, > research and education through the written word and the arts. > <http://NavalMarineArchive.com> and <http://UltraMarine.ca> > > _______________________________________________ > Koha mailing list http://koha-community.org > Koha@lists.katipo.co.nz > http://lists.katipo.co.nz/mailman/listinfo/koha > -- Chad Roseburg Automation Dept. North Central Regional Library _______________________________________________ Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz http://lists.katipo.co.nz/mailman/listinfo/koha
