Do you mind explain a bit more as to what is linear view and what is a serial view?
Also I think syncWaitGroup should be fine if I stick a context to exec.Command and guarantee it's execution time. On Tuesday, July 25, 2017 at 6:57:19 PM UTC+8, Jesper Louis Andersen wrote: > > On Tue, Jul 25, 2017 at 6:56 AM Glen Huang <hey...@gmail.com <javascript:>> > wrote: > >> >> My current design is that uploading is finished as soon as the image is >> uploaded to the server, after which I spawn a goroutine to exec a command >> to do the compression. When the user requests the image , I'll wait for the >> command to finish if it still on going, after which I send the compressed >> image back. >> >> > An image has three possible states: > > NOT_UPLOADED > UPLOADED > COMPRESSED > > Assume you have some kind of registry, perhaps via syncmap which can > faithfully tell you the state of an image. Then the cases NOT_UPLOADED and > COMPRESSED are trivial: return an error, or return the compressed file. The > last state requires some kind of signalling to tell when the file > transitions from UPLOADED to COMPRESSED. > > The faithfulness is important and you have to be wary of introducing some > kind of data race if you don't go to the syncmap for every state lookup. If > you have little to no contention in the system, using a global lock is > probably the easiest way to ensure linearization[0]. Optimizations are > possible because they can rely on the fact that the state transitions are > one-way and that you can recheck a condition after going to the > lock/syncmap. > > As for the signalling, I'd use a channel and close it when done. A > sync.WaitGroup solution might seem tempting (add 1 worker, let all the > other Wait on that worker being done), but it will block, prying you from > other events. Looking a bit ahead in the code and being visionary, chances > are you want to attach a context to your system and in this case, the > channel neatly fits into a select(-ive recieve) with the context as well. > This allows a waiting request to time out as well. > > singleflight is up for consideration in the parallel problem: how do you > intent to guard against the same image being uploaded and processed twice. > One simple way is to use a content-addressing of the image: its SHA256 > checksum is the key on which you add singleflight work. It has to weaved > into the syncmap solution above however, so perhaps it is easier to > approach it layered: solve one problem at a time. > > [0] I think a linear view is enough here, though obviously a serial view > could also be used. > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
