Ulrich Eckhardt wrote:
Dan Stromberg wrote:
My new version formats an SD card and preallocates some file space in
about 3 minutes with "Optimize Performance" selected, and in about 30
minutes with "Optimize for Quick Removal" selected.  Needless to say, I
don't like the 27 minute penalty much.

For performance, the driver will probably use delayed write operations,
buffering etc. For quick removal, it will probably make all operations
atomic, i.e. perform a write operation and not return until the data has
really reached the SD card.
The driver may delay writes, but I'd think it more likely that the filesystem or buffer cache would be doing so.
But the old version of the program formats and preallocates in 3 minutes
irrespective of whether the drive is in "optimize performance" or
"optimize for quick removal".

Somehow, I guess the new version does many small writes while the old one
doesn't. When optimizing for quick removal, those operations add up,
otherwise their overhead is negligible.
Nope, same block size.
one_meg = 'a'*2**20

That's one mib, not one meg. ;)

You're aware that a lot of people are ignoring the new units?
file = open('remove-me', 'w')

Try adding the 'b' flag, too. I wouldn't expect it to affect the IO speed,
but it is one factor that adds up.
This appears to have wholly accounted for the problem!  Thank you very much.

Adding a "b" to my open sped up the writes by a factor of about 15.
 Otherwise, I looked over your program
and couldn't find anything that would explain a problem. Just wondering,
what transfer speed do you get with the two versions? What is the
theoretical maximum for the SD card?
I don't know what the theoretical max write speeds of the USB bus, card reader and card are, but I was getting over 10x faster using the same card and card reader on an ancient Thinkpad running Ubuntu. With the "b" specified, the XP system is now faster than the Thinkpad.

Interesting that the "b" would make such a performance difference. You'd think it'd just be, in C, looking for newlines and adding a carriage return after them into a buffer of potentially double the size, then writing as usual. The data I've been writing so far -has- no newlines.

Thanks!

--
http://mail.python.org/mailman/listinfo/python-list

Reply via email to