On Sun, Jun 21, 2009 at 11:05:28PM +0200, Rocco Rutte wrote: > I think making it faster is always good. However, there're some more
eheh me too ;) > points that would have to be solved to get the patch ready for > inclusion. For example test if posix_fadvise() is available, why only That requires a configure knob? > do that for tokyocabinet, etc. Note that we're in a feature-freeze to No good reason to do it only for tokyocabinet indeed. But if one wants performance he should use something with compression so that one reads 1/5th of the data at 3 times the speed with fadvise than without fadvise. Cumulatively compression and fadvise reduce the opening time tenfold, ignoring the time it actually takes to process the data by the db lib after the I/O is complete. I agree we should add fadvise to all backends even if it makes little sense to use anything but tokyocabinet. I can try to look into preparing a more suitable and complete patch than the proof of concept hack that I'm using right now next week. If somebody wants to takeover this task you're welcome ;). > It's there because mutt displays what the mailbox has, not what the > cache has. The cur/ directory may change greatly between to mutt > sessions (e.g. a sync using unison, rsync, etc). I can't imagine > dropping it (except for a replacement mechanism that tells us what files > there are). What if I can guarantee you that I don't modify the 'cur' directory under 'mutt'. The only thing that touches my maildir are procmail in 'tmp' and 'new' and mutt in 'cur' and 'new'. Nothing else ever touches my maildir, so I definitely need a knob to disable readdir on 'cur' and to only have readdir on 'new' when opening a maildir so with the addition of fadvise and tokyocabinet compression things will fly even for large folders with 150k mails ;). If I can remove readdir opening a 150k folder on cold cache after boot, with fadvise and tokyocabinet (55M hcache file) will be only 2 seconds slower than opening it from totally hot filesystem caches (after those 2 seconds of I/O inside fadvise syscall reading the hcache file at >30M/sec the all wait time will be spent at 100% cpu load). On small folders (<10k emails) with fadvise and compression things are now almost immediate even keeping readdir with cold cache too. My hd on the laptop is very slow, on real fast harddisk the hcache will be loaded at 100M/sec with fadvise, so then it almost won't be noticeable to open a 150k email folder on hot or cold kernel filesystem caches if we've a knob to disable readdir on 'cur'. (this assuming 'new' has no lots of mails but that's fine if it slowdown a bit if you don't open it for a while ;)
