Re: [dev] Re: coreutils / moreutils - DC a directory counter

2013-08-06 Thread Martti Kühne
On Mon, Jul 29, 2013 at 6:39 AM, Paul Hoffman wrote: > > I just read something about using LD_PRELOAD for this. Write a library > that implements open(2), munging the file path and then calling the > "real" open(2). Then you just set LD_PRELOAD in the environment of the > scripts and Bob's your

Re: [dev] Re: coreutils / moreutils - DC a directory counter

2013-07-29 Thread Bjartur Thorlacius
On mán 29.júl 2013 11:38, Thorsten Glaser wrote: Are they, still? I thought they had the equivalent of UFS_DIRHASH nowadays… Ext4 does, optionally.

Re: [dev] Re: coreutils / moreutils - DC a directory counter

2013-07-29 Thread Thorsten Glaser
Bjartur Thorlacius dixit: > by the censor. In short, ext2/3 directories are linked lists. You can traverse Are they, still? I thought they had the equivalent of UFS_DIRHASH nowadays… bye, //mirabilos -- [...] if maybe ext3fs wasn't a better pick, or jfs, or maybe reiserfs, oh but what about xfs

Re: [dev] Re: coreutils / moreutils - DC a directory counter

2013-07-29 Thread Bjartur Thorlacius
On mán 29.júl 2013 04:39, Paul Hoffman wrote: Their 100+ Perl and bash scripts are slow because they're opening files in a humongous directory. They can't subdivide the directory because they're afraid that they will break the scripts when modifying them. I posted a comprehensive comment on the

Re: [dev] Re: coreutils / moreutils - DC a directory counter

2013-07-28 Thread Paul Hoffman
On Fri, Jul 26, 2013 at 01:17:54AM +, Thorsten Glaser wrote: > Calvin Morrison dixit: > > >I was sick of ls | wc -l being so damned slow on large directories, so > > What, besides the printing and sorting, is the slow part anyway? > Is it the VFS API or just the filesystem code? > > In the l

Re: [dev] Re: coreutils / moreutils - DC a directory counter

2013-07-26 Thread Calvin Morrison
Yes master On Jul 26, 2013 3:40 AM, "Thorsten Glaser" wrote: > Calvin Morrison dixit: > > >Its called unionfs if I recall > > No. Go read it again. > > >On Jul 25, 2013 9:28 PM, "Thorsten Glaser" wrote: > > And stop top-posting and full-quoting. > > Read http://www.afaik.de/usenet/faq/zitieren/

Re: [dev] Re: coreutils / moreutils - DC a directory counter

2013-07-26 Thread Thorsten Glaser
Calvin Morrison dixit: >Its called unionfs if I recall No. Go read it again. >On Jul 25, 2013 9:28 PM, "Thorsten Glaser" wrote: And stop top-posting and full-quoting. Read http://www.afaik.de/usenet/faq/zitieren/ (it’s in German, English and Dutch, so no excuses). bye, //mirabilos -- > emac

Re: [dev] Re: coreutils / moreutils - DC a directory counter

2013-07-25 Thread Calvin Morrison
Its called unionfs if I recall On Jul 25, 2013 9:28 PM, "Thorsten Glaser" wrote: > Calvin Morrison dixit: > > >I was sick of ls | wc -l being so damned slow on large directories, so > > What, besides the printing and sorting, is the slow part anyway? > Is it the VFS API or just the filesystem cod

[dev] Re: coreutils / moreutils - DC a directory counter

2013-07-25 Thread Thorsten Glaser
Calvin Morrison dixit: >I was sick of ls | wc -l being so damned slow on large directories, so What, besides the printing and sorting, is the slow part anyway? Is it the VFS API or just the filesystem code? In the latter case… could workarounds exist? Someone asked this… http://fenski.pl/2013/07

Re: [dev] Re: coreutils / moreutils - DC a directory counter

2013-07-18 Thread Szabolcs Nagy
* Calvin Morrison [2013-07-17 16:43:00 -0400]: > On 17 July 2013 16:32, Christian Neukirchen wrote: > >> calvin@ecoli:~/big_folder> time ls file2v1dir/ | wc -l > >> 687560 > >> > >> real0m7.798s > >> user0m7.317s > >> sys 0m0.700s > >> > >> calvin@ecoli:~/big_folder> time ~/bin/dc fil

Re: [dev] Re: coreutils / moreutils - DC a directory counter

2013-07-18 Thread Robert Ransom
On 7/17/13, Calvin Morrison wrote: > On 17 July 2013 16:32, Christian Neukirchen wrote: >> What's the bottle neck here? > > Looking up the filenames and reading them, printing them to standard > out and then wc parsing for all the \n characters. Most ls implementations also sort the list of fil

Re: [dev] Re: coreutils / moreutils - DC a directory counter

2013-07-18 Thread Troels Henriksen
Bjartur Thorlacius writes: > On 07/17/2013 09:02 PM, Calvin Morrison wrote: >> So it seems a good deal of that time is ls >> > Wait, sbase ls doesn't seem to implement -f. Are you sorting the > directory entries? Using -f improves performance a bit, but no matter what, an 'ls -whatever|wc -l' pi

Re: [dev] Re: coreutils / moreutils - DC a directory counter

2013-07-17 Thread Bjartur Thorlacius
On 07/17/2013 09:02 PM, Calvin Morrison wrote: So it seems a good deal of that time is ls Wait, sbase ls doesn't seem to implement -f. Are you sorting the directory entries?

Re: [dev] Re: coreutils / moreutils - DC a directory counter

2013-07-17 Thread Calvin Morrison
On 17 July 2013 16:58, Christian Neukirchen wrote: > Calvin Morrison writes: > >> On 17 July 2013 16:32, Christian Neukirchen wrote: >>> Calvin Morrison writes: >>> Hi guys, I came up with a utility[0] that i think could be useful, and I sent it to the moreutils page, but ma

[dev] Re: coreutils / moreutils - DC a directory counter

2013-07-17 Thread Christian Neukirchen
Calvin Morrison writes: > On 17 July 2013 16:32, Christian Neukirchen wrote: >> Calvin Morrison writes: >> >>> Hi guys, >>> >>> I came up with a utility[0] that i think could be useful, and I sent >>> it to the moreutils page, but maybe it might fit better here. All it >>> does is give a count

Re: [dev] Re: coreutils / moreutils - DC a directory counter

2013-07-17 Thread Calvin Morrison
On 17 July 2013 16:32, Christian Neukirchen wrote: > Calvin Morrison writes: > >> Hi guys, >> >> I came up with a utility[0] that i think could be useful, and I sent >> it to the moreutils page, but maybe it might fit better here. All it >> does is give a count of files in a directory. >> >> I wa

[dev] Re: coreutils / moreutils - DC a directory counter

2013-07-17 Thread Christian Neukirchen
Calvin Morrison writes: > Hi guys, > > I came up with a utility[0] that i think could be useful, and I sent > it to the moreutils page, but maybe it might fit better here. All it > does is give a count of files in a directory. > > I was sick of ls | wc -l being so damned slow on large directories