Hi Burak,
Thank you for your reply,
Inside the HydrateCollection function there are database queries (at least
three), run using the same *sql.DB throughout the application.
Is it possible that
> whatever connection HydrateCollection is operating on cannot handle
> multiple concurrent requests?
>
In that case, what should I look out for? I've always assumed that using a
global *sql.DB will always suffice.
On Tuesday, May 22, 2018 at 12:36:25 PM UTC+7, Burak Serdar wrote:
>
> On Mon, May 21, 2018 at 11:12 PM, Hawari Rahman
> <hawari....@gmail.com <javascript:>> wrote:
> > Hi Everyone,
> >
> > So for the background, I have an API for retrieving data from a
> database.
> > For an endpoint there exists two function calls:
> > - SelectCollections - for retrieving the collection records
> > - HydrateCollection - for hydrating each collections (the second
> function is
> > called on each results of the SelectCollections function).
> >
> > Previously we run the HydrateCollections sequentially using a for loop,
> and
> > it results in the following:
> >
> > 2018/05/22 10:59:05 HydrateCollection took 41.497134ms
> > 2018/05/22 10:59:05 HydrateCollection took 42.695152ms
> > 2018/05/22 10:59:05 HydrateCollection took 22.870913ms
> > 2018/05/22 10:59:05 HydrateCollection took 17.833512ms
> > 2018/05/22 10:59:05 HydrateCollection took 18.765984ms
> > 2018/05/22 10:59:05 HydrateCollection took 16.338906ms
> > 2018/05/22 10:59:05 HydrateCollection took 14.660654ms
> > 2018/05/22 10:59:05 HydrateCollection took 20.818716ms
> > 2018/05/22 10:59:05 HydrateCollection took 18.57054ms
> > 2018/05/22 10:59:05 Resolving Items took 172.9182ms
> > 2018/05/22 10:59:05 SelectCollections took 173.992051ms
> >
> > After using goroutines to run HydrateCollections now the results is as
> > follows:
> >
> > 2018/05/22 11:21:36 HydrateCollection took 9.187861ms
> > 2018/05/22 11:21:36 HydrateCollection took 63.651507ms
> > 2018/05/22 11:21:36 HydrateCollection took 79.199976ms
> > 2018/05/22 11:21:36 HydrateCollection took 122.349986ms
> > 2018/05/22 11:21:37 HydrateCollection took 150.627746ms
> > 2018/05/22 11:21:37 HydrateCollection took 168.432517ms
> > 2018/05/22 11:21:37 HydrateCollection took 171.602705ms
> > 2018/05/22 11:21:37 HydrateCollection took 179.127794ms
> > 2018/05/22 11:21:37 HydrateCollection took 185.137562ms
> > 2018/05/22 11:21:37 Resolving items took 185.434821ms
> > 2018/05/22 11:21:37 SelectCollections took 187.358141ms
>
> It is difficult to guess without knowing what HydrateCollection does.
> But these numbers are increasing monotonically. Is it possible that
> whatever connection HydrateCollection is operating on cannot handle
> multiple concurrent requests? Maybe some transaction isolation
> problem? So all HydrateCollection instances start running at the same
> time, but the second one cannot actually start its work until the
> first is done, and third cannot start until the second is done, etc.?
> You can test if that's the case by logging the start-end times of
> HydrateCollections and the collections each are working on.
>
> >
> >
> > Why does the HydrateCollection take a significant performance hit now
> that
> > I'm using goroutine?
> >
> > HydrateCollection function takes a pointer to Collection struct, and
> > mutating it inside the goroutine, so I think that it may be a problem.
> Also
> > the HydrateCollection itself is a method of a Repository pointer which
> holds
> > a database connection, the same Repository pointer is used by
> > SelectCollections.
> >
> > hydrateChan := make(chan error, len(collections))
> > for index := range collections {
> > go func(coll *Collection) {
> > hydrateChan <- r.hydrateCollection(coll)
> > }(&collections[index])
> > }
> >
> > for i := 0; i < len(collections); i++ {
> > err := <-hydrateChan
> > if err != nil {
> > logrus.Error(err)
> > }
> > }
> >
> > Is there something that I need to address in the previous code snippet?
> >
> > --
> > You received this message because you are subscribed to the Google
> Groups
> > "golang-nuts" group.
> > To unsubscribe from this group and stop receiving emails from it, send
> an
> > email to golang-nuts...@googlegroups.com <javascript:>.
> > For more options, visit https://groups.google.com/d/optout.
>
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
