Isn't Ian's solution basically the same, but using a Waitgroup instead of
an Errgroup?
I think the Fetch() is just a placeholder for "this is where the
implementation detail of the task would go"
On Sunday, 21 May 2023 at 21:54:39 UTC+1 Andrew Harris wrote:
> The recipe on 10:26 should work but it's definitely contrived - using
> Fetch looks strange to me, and the cancellation behavior of errgroup is
> another moving part that adds some overhead. If it's sufficient, Ian's
> solution groks better :)
>
>
> On Sunday, May 21, 2023 at 7:38:31 AM UTC-7 Tobias Klausmann wrote:
>
>> Hi!
>>
>> First off, thanks both Ian and Andrew!
>>
>> On Sat, 20 May 2023, Ian Lance Taylor wrote:
>> > On Sat, May 20, 2023 at 1:23 AM Tobias Klausmann
>> > <klau...@schwarzvogel.de> wrote:
>> > > What I wonder about is the collection stage: the assumption that
>> there
>> > > will be exactly as many results as inputs seems brittle, and the only
>> > > real way to handle errors is to put them into the result struct and
>> > > examining them later.
>> > >
>> > > I also usually prefer handling workers with a `waitgroup`, but that
>> only
>> > > really works if there is no return channel to handle, either because
>> > > the workers do the output/final processing, or because results are
>> > > written to a global. The former is not always possible, and the
>> latter
>> > > is ugly.
>> > >
>> > > There is also the option of just making the result channel big enough
>> > > to hold all results, but that seems a bit ugly as well.
>> > >
>> > > So what's the most idiomatic way of doing this?
>> >
>> > The pattern I tend to prefer is
>> >
>> > rc := make(chan result, numprocs)
>> > var wg sync.WaitGroup
>> > for i := 0; i < numprocs; i++ {
>> > wg.Add(1)
>> > go func() {
>> > defer wg.Done()
>> > sendResults(rc)
>> > }()
>> > }
>> > go func() {
>> > wg.Wait()
>> > close(rc)
>> > }()
>> > This ensures that the channel is closed when all the results are sent.
>> > Then you can collect the results using one or more goroutines with
>> > "for result := range rc".
>> >
>> > Note that for this kind of pattern it's usually a good idea to use a
>> > buffered channel to hold the results, to avoid tight synchronization
>> > between the senders and the processors. Of course sometimes you want
>> > that tight synchronization, but usually you don't. The exact size of
>> > the buffer doesn't matter too much.
>>
>> I'm now using this approach, and my mind is much more at ease about the
>> robustness. As for the channel size, I've tried numproc, *2, *10 and
>> *100, and it all makes no practical difference time-wise, so I'll just
>> stay
>> with numproc
>>
>> > You may also want to watch Bryan's talk at
>> > https://www.youtube.com/watch?v=5zXAHh5tJqQ .
>>
>> Started watching and tried the recipe outlined/on screen at 10:26, but
>> couldn't get it to work: the result-gathering loop reading from c would
>> do zero iterations. Not sure what was going on ther. But the above
>> solution is one I grok, so I'm staying with that :)
>>
>>
>> Best,
>> Tobias
>>
>
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/golang-nuts/6c410c55-1a5e-4a54-8fed-d77eddcbf94an%40googlegroups.com.
