On Tue, 5 Jul 2016 02:38:28 -0700 (PDT) Pierre Durand <pierredur...@gmail.com> wrote:
> My code: https://play.golang.org/p/pg-p17UuEW > I'm trying to append lines to a file concurrently. > > My first write() function is buggy, because WriteString() are called > by several goroutines in unexpected order. > > So, I've written writeMutex(), that uses a mutex. > It works as expected, but in my real code, I'd prefer to avoid mutex. > (in my real code write() can write to different files; the file's > name is given as argument) > > I've also written writeBuffer(). > Is it OK to call f.Write() 1 time from several goroutines? > It looks like it works, but I'm not 100% sure this is correct. It's hard for me to discern your real problem from your statement, but I'll do a shot in the dark: if your concern is that having a single mutex will possibly create unnecessary contention when writing in separate files, just have a separate mutex per file. If the entry point to your writing machinery receives the file's name as its argument, have a map which assigns mutexes to file names. Be sure to also delete the matching entry from this map when a particular file is finished dealing with. You might also consider a different approach. Say, you might have a manager goroutine running forever and doing two things: maintaining a list of active file writes and an input channel receiving tasks in the form of tuples (file_name, data_to_write). If it detects there's no in progress write operation happens for the file from the next task, it spawns a goroutine writing data to that file and creates an entry in its internal map marking that file as "busy". It then waits for the signal of that working goroutine that it has finished work and removes the "busy mark". Yet another approach is to have such manager goroutine fork a long-running worker goroutine per named file: from each incoming task it fetches the file's name and submits its data to write to appropriate worker goroutine on a dedicated channel. But before you "go parallel" take into account that writing data to files is an I/O-bound operation and if your files reside on the same filesystem (and especially if that filesystem is backed by rotating medium), or on different filesystem backed by the same meduim, you probably won't get the speedup you're seeking: the write operations will be naturally serialized "back" by the filesystem layer of your OS. Hence rather than have N "parallel" writes been stalled waiting on the OS to cope with them, it might have more sense to just have a single dedicated writing goroutine which will receive "writing tasks" (see above) and perform they in FIFO order. To reiterate: being able to perform certain operations in parallel or concurrently is cool but always keep in mind that after "leaving" the runtime they hit CPU and OS where things suddenly stop being "pure", theoretical and ideal. -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
