code snippet: https://play.golang.org/p/pphzr6QKiQ3
here I have an existing serial process (using channels in preparation) for a run-time switch between serial and parallel evaluation. the snippet is that switch. N desired to be used cpus N worker goroutines N work chan N answer chan distribute work to job%N gather answers from job%N results are faster and in same total order On Fri, Jan 25, 2019 at 12:57 PM robert engels <reng...@ix.netcom.com> wrote: > Also you need to do is read the results (each result object should have > the task ID of the submission)from the channel, add the result to slice of > results, when result count == task count, sort the results by task id, and > output each one. Straightforward. > > A single Go routine reads from all of the “output channels “ (one per > number of CPU), and builds the result set. > > On Jan 25, 2019, at 2:39 PM, twpa...@gmail.com wrote: > > The output is a series of strings written to the console. I think this is > independent of the problem description though: once you have either a slice > of results in order, or a channel returning results in order, then the > problem is solved. > > On Friday, January 25, 2019 at 9:36:53 PM UTC+1, twp...@gmail.com wrote: >> >> For task submission, I have a slice of tasks. The easy way to submit them >> is to spin up a goroutine that writes them sequentially to a channel and >> then closes the channel when there are no more tasks. >> >> On Friday, January 25, 2019 at 9:29:43 PM UTC+1, robert engels wrote: >>> >>> I think the reason you haven’t found a library, is that given the >>> specifications, it is probably less than 100 lines of Go code to accomplish. >>> >>> The important elements lacking in your specification though are “how are >>> the tasks submitted” (e.g. http? in code ?, etc.) , , and “how is the >>> output presented” (e.g. to the console, each task decides, etc .) >>> >>> On Jan 25, 2019, at 1:04 PM, twp...@gmail.com wrote: >>> >>> Hi, >>> >>> I have a number of slow tasks that I want to run concurrently across >>> runtime.NumCPU() workers in a single process. The tasks have a specific >>> input order, but they are completely independent of each other and can >>> execute in any order. I would like to print the output of each task in the >>> same order as the input order of tasks. >>> >>> This can be implemented by including each task's index in the input >>> order as it is distributed via a channel to the workers, and the final >>> collection of results assembled using these task indexes before the results >>> are printed. >>> >>> Assumptions: >>> - Small number of tasks (~10,000 max), i.e. this easily fits in memory. >>> - Single Go process, i.e. I don't want/need a distributed system. >>> >>> This feels like it should be common problem and there's probably either >>> a library or a standard Go pattern out there which can do it. My web search >>> skills didn't find such a library though. Do you know of one? >>> >>> Cheers, >>> Tom >>> >>> >>> Background info to avoid the XY problem <http://xyproblem.info/>: this >>> is to make chezmoi <https://github.com/twpayne/chezmoi> run faster. I >>> want to run the doctor checks >>> <https://github.com/twpayne/chezmoi/blob/ed27b49f9ca4cd3662e6a59908dee24b0d295b79/cmd/doctor.go#L102-L163> >>> (basically os.Exec'ing a whole load of binaries to get their versions) >>> concurrently in the short term. In the long term I want to make chezmoi's >>> apply concurrent, so it runs faster too. In the first case, the order >>> requirement is because I want all users to see the output in the same order >>> so that it's easy to compare. In the second case, the order requirement >>> comes because I need to ensure that parent directories are in the correct >>> state before checking their children. >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "golang-nuts" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to golang-nuts...@googlegroups.com. >>> For more options, visit https://groups.google.com/d/optout. >>> >>> >>> > -- > You received this message because you are subscribed to the Google Groups > "golang-nuts" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to golang-nuts+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > > > -- > You received this message because you are subscribed to the Google Groups > "golang-nuts" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to golang-nuts+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -- *Michael T. jonesmichael.jo...@gmail.com <michael.jo...@gmail.com>* -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
