For that case, I'd suggest you pass a channel to each worker. It can either close the channel, or send a specific message down it, to indicate its termination.
This may also be an X-Y problem. Generally speaking, "worker pools" are an anti-pattern in Go. The startup and termination overhead of a goroutine is so tiny that often it's better just to start a goroutine when there's work to be done, and let it run to completion. The other reason you might want to have a worker pool is to limit concurrency, but there are better ways to do that in Go: e.g. you can have a buffered channel with N slots, which the goroutine writes to before starting work and reads from when it's finished. That will limit your concurrency to N tasks. Bryan C Mills discusses this in detail in this excellent video on "Rethinking Classical Concurrency Patterns": https://www.youtube.com/watch?v=5zXAHh5tJqQ It took me several views with pauses to fully understand it, but it was worth the effort :-) On Monday, 30 August 2021 at 12:29:26 UTC+1 sansaid wrote: > Appreciated, thanks Brian and Ian. > > On the subject of sync.WaitGroups, I haven't tackled this yet for my use > case, but will use the opportunity to brainstorm some options, if that's > okay. I want to be able to terminate each worker independently of others > (i.e. close on worker, wait for it to finish its tasks, but allow other > workers to run if they haven't been signalled to stop). Should I create a > wait group per worker and store that in a map? This sounds bloated to me, > especially since it's always going to be a wait group of 1, but I haven't > come across better alternatives yet (relatively new to concurrency myself). > > If that's too broad a question, can come back once I've explored some > options. > > On Mon, 30 Aug 2021, 12:20 Brian Candler, <b.ca...@pobox.com> wrote: > >> Also: you can use the same context for all the workers (if that's >> appropriate); or if you want each worker to have a different context, you >> can make those contexts children of the same top-level context. In that >> case, you only need to cancel one context and all the workers will stop, >> without having to keep a map of them. >> >> Do bear in mind that you may wish to wait for the workers to finish what >> they're currently doing, before the program exits. One way to do that is >> with a sync.WaitGroup, which you increment before starting each worker, and >> each worker decrements just before it terminates. Then in your main >> program you wait on the WaitGroup. >> >> Typical pattern: https://play.golang.org/p/51oy7ySbL4T >> >> On Monday, 30 August 2021 at 10:01:08 UTC+1 Ian Davis wrote: >> >>> >>> On Sun, 29 Aug 2021, at 10:45 PM, sansaid wrote: >>> >>> Hello, >>> >>> Does anybody know of some reference code or common patterns I can use to >>> keep track of worker goroutines? For context, I want a user to be able to >>> issue a "stop" command using a CLI tool which will prompt my server to >>> gracefully terminate one of my workers. >>> >>> For example, when a user issues a "start" command through a CLI tool, >>> this will signal my server to spawn a goroutine that perpetually runs and >>> polls an HTTP endpoint until a user initiates a "stop" command. When the >>> user issues a "stop" command through the same CLI tool, I want the server >>> to be able to signal the goroutine to stop. Any reference code would be >>> much appreciated! >>> >>> One approach I thought of was by passing a `context.WithCancel()` and >>> holding a reference to the cancel function in a global map (with the worker >>> ID as keys). When the user issues a "stop" command against the worker ID, >>> another function is executed which calls the context's cancel function. >>> Example code below (appreciate this is horrendously breaking a lot of >>> rules, but I want to focus only on the elements around passing the cancel >>> function around - happy to discuss anything else that is of concern to >>> someone though): >>> >>> ``` >>> // invoked when a CLI start subcommand is issued to my main CLI tool >>> func (w *WorkerGroup) Start(ctx context.Context, id string) { >>> _, cancel := context.WithCancel(ctx) >>> >>> w.Workers.Lock() >>> w.Workers[id] := cancel >>> w.Workers.Unlock() >>> >>> go startWorker(id) >>> } >>> >>> // invoked when a CLI stop subcommand is issued to my main CLI tool >>> func (w *WorkerGroup) Stop(id string) { >>> cancelWorker := w.Workers[id] >>> >>> cancelWorker() >>> } >>> ``` >>> >>> I have read that passing a context's cancel function around is a bad >>> idea, but in an example like this, is it justified? >>> >>> >>> Passing the cancel function is fine and storing it for use later is fine >>> too. >>> >>> -- >> You received this message because you are subscribed to the Google Groups >> "golang-nuts" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to golang-nuts...@googlegroups.com. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/golang-nuts/fdff94c4-c0e4-4418-abb3-eba91298fbb7n%40googlegroups.com >> >> <https://groups.google.com/d/msgid/golang-nuts/fdff94c4-c0e4-4418-abb3-eba91298fbb7n%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/49a392dc-b0de-4187-89bd-6db2e8fb9758n%40googlegroups.com.
