There is also a hybrid model:
Model A: for each work item, create a goroutine. This is easy to program,
but uses additional resources for the goroutines.
Model B: keep a worker pool and feed a channel. This bounds the goroutine
count and requires more code to pull off and make correct.
I'm going
So, it looks to me like 1 million URLS will spin up 1 million goroutines.
That seems unnecessary, and likely to be a problem. I usually default to
the simplest model, unless there is a reason to do otherwise. IMHO, here
that would be to spin up as many goroutines as you want to be working at
on
