On Wed, 10 Jan 2018 15:31:26 -0800 Amandeep Gautam <amandeepgaut...@gmail.com>
wrote:
Amandeep Gautam writes:
>
> Is there a equivalent of telldir in golang which can page reading of a
> directory.
> Just using filepath.walk would be inefficient for large file systems.
>
> >From one of the answers to this question
> https://stackoverflow.com/questions/39583522/how-do-i-use-seekdir-telldir-in-
> golang
>
> it looks like there is nothing for this use case, which brings me to my
> next question: Any ideas about how to go about dealing such use case.
The only reason I can think of wanting this is where you ^C a
long running treewalker before it has finished and then on
restart you want it to continue from where it left off.
Useful when you are scanning a large filesystem in a
background process and want to be able to terminate the
scanner and restart. [If you just want to be able to stop the
scanner but not terminate/restart, your solution can be
simpler.]
If this is your use case, read on.
If your current directory is N levels deep, you have to save
the state of N partial walks. On exit, save this state and on
restart restore from this state. One suggestion:
func TreeWalk(state []string, callbacks ...) []string
For a brand new walk, TreeWalk's state would be {path:0}. If
terminated prematurely, returned value would be a slice of
partial walks. If the tree was completely walked, TreeWalk
returns nil. For a restart, state would be the result of the
previous walk. The saved state can be put in a file, one per
line:
path:pos
To terminate early you'd need a way to tell TreeWalk to save
state and give up. This can be via a signal handler that
stuffs a value in a channel or update an atomic variable or
something.
Another way is to replace callback functions for processing
dir/files with a channel and pass appropriate data for each
dir/file for concurrent processing but saving partial state
would get slightly trickier.
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
