On 2016-12-06 13:08, eryk sun wrote:
On Tue, Dec 6, 2016 at 12:26 PM, Chris Angelico <ros...@gmail.com> wrote:
On Tue, Dec 6, 2016 at 10:56 PM, BartC <b...@freeuk.com> wrote:
In that directory (which was on Windows but accessible via a virtual Linux),
typing any Linux command followed by * would have required all 3.4 million
directory entries to be accessed in order to build a 3.4 million-element
argv list. I've no idea how long that would have taken.
I just asked Python to build me a 4-million-element list, and it took
no visible time - a small fraction of a second. Don't be afraid of
large argument lists. We're not writing 8088 Assembly Language
programs in 64KB of working memory here.
The problem isn't building an arbitrary list with millions of
elements. The problem is the time it would take to read millions of
filenames from a directory. It depends on the performance of the disk
and filesystem. Be careful with globbing. Think about the consequences
before running a command, especially if you're in the habit of
creating directories with hundreds of thousands, or millions, of
files. It's not a problem that I've ever had to deal with.
Many years ago I was working with a database application running on
MSDOS that stored predefined queries in files, one query per file. There
were many queries (though fewer than a thousand), resulting in many
small files in a single directory. Fetching one of those predefined
queries was surprisingly slow.
I found that it was a lot faster to put them into a single file and then
call an external program to extract the one wanted. It also took up a
lot less disk space!
--
https://mail.python.org/mailman/listinfo/python-list
