FWIW I wrote *https://godoc.org/github.com/juju/utils/parallel#Run <https://godoc.org/github.com/juju/utils/parallel#Run>* for exactly this kind of use case. It doesn't explicitly mention the values in the API because it's easier (and more type safe) to let the body of the closure write directly to a result slice.
Example: https://play.golang.org/p/ffj_Eb_0B2q On Fri, 25 Jan 2019, 10:48 pm <twpa...@gmail.com wrote: > For what it's worth > http://www.golangpatterns.info/concurrency/parallel-for-loop > implements an order-preserving parallel map, but does not limit the number > of workers. > > In my case, I want to limit the number of workers because I'm making a lot > of system calls and don't want to overload the kernel. runtime.NumCPU() > seems like a reasonable limit. > > > > On Friday, January 25, 2019 at 8:04:31 PM UTC+1, twp...@gmail.com wrote: >> >> Hi, >> >> I have a number of slow tasks that I want to run concurrently across >> runtime.NumCPU() workers in a single process. The tasks have a specific >> input order, but they are completely independent of each other and can >> execute in any order. I would like to print the output of each task in the >> same order as the input order of tasks. >> >> This can be implemented by including each task's index in the input order >> as it is distributed via a channel to the workers, and the final collection >> of results assembled using these task indexes before the results are >> printed. >> >> Assumptions: >> - Small number of tasks (~10,000 max), i.e. this easily fits in memory. >> - Single Go process, i.e. I don't want/need a distributed system. >> >> This feels like it should be common problem and there's probably either a >> library or a standard Go pattern out there which can do it. My web search >> skills didn't find such a library though. Do you know of one? >> >> Cheers, >> Tom >> >> >> Background info to avoid the XY problem <http://xyproblem.info/>: this >> is to make chezmoi <https://github.com/twpayne/chezmoi> run faster. I >> want to run the doctor checks >> <https://github.com/twpayne/chezmoi/blob/ed27b49f9ca4cd3662e6a59908dee24b0d295b79/cmd/doctor.go#L102-L163> >> (basically os.Exec'ing a whole load of binaries to get their versions) >> concurrently in the short term. In the long term I want to make chezmoi's >> apply concurrent, so it runs faster too. In the first case, the order >> requirement is because I want all users to see the output in the same order >> so that it's easy to compare. In the second case, the order requirement >> comes because I need to ensure that parent directories are in the correct >> state before checking their children. >> > -- > You received this message because you are subscribed to the Google Groups > "golang-nuts" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to golang-nuts+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
