Re: [go-nuts] Re: os.remove is slow

2017-12-04 Thread Reto Brunner
But why? Why does it matter if you are in the same dir or not when you glob? On Mon, Dec 4, 2017, 20:34 Gabriel Forster wrote: > Ding ding ding! We have a winner! Thank you so much. That made a huge > difference. It is now running ~20 seconds. > > The key is being in the same directory. > > > O

Re: [go-nuts] Re: os.remove is slow

2017-12-04 Thread Gabriel Forster
Ding ding ding! We have a winner! Thank you so much. That made a huge difference. It is now running ~20 seconds. The key is being in the same directory. On Monday, December 4, 2017 at 2:17:03 PM UTC-5, Patrick Smith wrote: > > Does it make a difference if you first change directory to > /var/s

Re: [go-nuts] Re: os.remove is slow

2017-12-04 Thread Patrick Smith
Does it make a difference if you first change directory to /var/spool/directory, then glob * and unlink the resulting filenames, without prepending the directory? On Mon, Dec 4, 2017 at 11:05 AM, Gabriel Forster wrote: > Readdirnames wasn't much better. Now using globbing and syscall.Unlink > wh

[go-nuts] Re: os.remove is slow

2017-12-04 Thread Gabriel Forster
Readdirnames wasn't much better. Now using globbing and syscall.Unlink which take ~1minute 30seconds. It is much better, but still a long way from perl's sub-20 seconds. package main import ( "fmt" "path/filepath" "syscall" ) func main() { upperDirPattern := "/var/spool/di

Re: [go-nuts] Re: os.remove is slow

2017-12-04 Thread Justin Israel
On Tue, Dec 5, 2017, 7:13 AM Gabriel Forster wrote: > As far as I understand, that perl line is by far the fastest way to delete > files in linux. We're talking over 500k files on hdd. > It may be the 500k stats that are killing you. From the looks of your code, the stat seems unnecessary as yo

[go-nuts] Re: os.remove is slow

2017-12-04 Thread Gabriel Forster
As far as I understand, that perl line is by far the fastest way to delete files in linux. We're talking over 500k files on hdd. On Monday, December 4, 2017 at 12:50:54 PM UTC-5, Gabriel Forster wrote: > > What takes 18 seconds in a perl command: > perl -e 'for(<*>){((stat)[9]<(unlink))}' > > i

[go-nuts] Re: os.remove is slow

2017-12-04 Thread Hotei
Gabriel - Thank you for reminding me that perl is a write-only language. It's been a few years since I had to deal with it. How many files are we talking about and on what media? On Monday, December 4, 2017 at 12:50:54 PM UTC-5, Gabriel Forster wrote: > > What takes 18 seconds in a perl comman