On Tue, Nov 29, 2005 at 02:53:36PM +1100, Gavin Sherry wrote: > The second idea is using posix async IO at key points within the system > to better parallelise CPU and IO work. There areas I think we could use > async IO are: during sequential scans, use async IO to do pre-fetching of > blocks; inside WAL, begin flushing WAL buffers to disk before we commit; > and, inside the background writer/check point process, asynchronously > write out pages and, potentially, asynchronously build new checkpoint > segments.
I actually worked on this and got it to the stage where it wouldn't crash anymore. It basically added a command to bufmgr.c called PrefetchBuffer() which would initiate a request but not block. I then hooked a few strategic places to call this. In particular during an index scan, it would prefetch the next index block and the next few data blocks and then return them in order as they came in. Unfortunatly I can't really test it at it's full potential because it uses glibc's default POSIX AIO which is *lame*. No more than one outstanding request per fd which for PostgreSQL is crappy. There was some evidence that in an index scan of a highly uncorrelated index that it did make a small difference, but I never got around to testing it fully. But bitmap scans already hugely reduce the cost of uncorrelated indexes. It doesn't pass regression because index_getmulti doesn't do backward scans. Everything else works though. If anyone is interested in the code I can send it to them. The results on my system just wern't good enough to justify a lot more effort. Have a nice day, -- Martijn van Oosterhout <kleptog@svana.org> http://svana.org/kleptog/ > Patent. n. Genius is 5% inspiration and 95% perspiration. A patent is a > tool for doing 5% of the work and then sitting around waiting for someone > else to do the other 95% so you can sue them.
pgpMQJGyCGvx0.pgp
Description: PGP signature
