--------------------------- Derek Tracy [EMAIL PROTECTED] ---------------------------
On Apr 3, 2008, at 3:03 AM, Paul Rubin <"http:// phr.cx"@NOSPAM.invalid> wrote: > Derek Martin <[EMAIL PROTECTED]> writes: >>> Both are clocking in at the same time (1m 5sec for 2.6Gb), are there >>> any ways I can optimize either solution? > > Getting 40+ MB/sec through a file system is pretty impressive. > Sounds like a RAID? > >> That said, due to normal I/O generally involving double-buffering, >> you >> might be able to speed things up noticably by using Memory-Mapped I/O >> (MMIO). It depends on whether or not the implementation of the >> Python >> things you're using already use MMIO under the hood, and whether or >> not MMIO happens to be broken in your OS. :) > > Python has the mmap module and I use it sometimes, but it's not > necessarily the right thing for something like this. Each page you > try to read from results in own delay while the resulting page fault > is serviced, so any overlapped i/o you get comes from the OS being > nice enough to do some predictive readahead for you on sequential > access if it does that. By coincidence there are a couple other > threads mentioning AIO which is a somewhat more powerful mechanism. > > -- > http://mail.python.org/mailman/listinfo/python-list -- http://mail.python.org/mailman/listinfo/python-list
