On Tue, Jul 25, 2017 at 6:56 AM Glen Huang <hey....@gmail.com> wrote:
> > My current design is that uploading is finished as soon as the image is > uploaded to the server, after which I spawn a goroutine to exec a command > to do the compression. When the user requests the image , I'll wait for the > command to finish if it still on going, after which I send the compressed > image back. > > An image has three possible states: NOT_UPLOADED UPLOADED COMPRESSED Assume you have some kind of registry, perhaps via syncmap which can faithfully tell you the state of an image. Then the cases NOT_UPLOADED and COMPRESSED are trivial: return an error, or return the compressed file. The last state requires some kind of signalling to tell when the file transitions from UPLOADED to COMPRESSED. The faithfulness is important and you have to be wary of introducing some kind of data race if you don't go to the syncmap for every state lookup. If you have little to no contention in the system, using a global lock is probably the easiest way to ensure linearization[0]. Optimizations are possible because they can rely on the fact that the state transitions are one-way and that you can recheck a condition after going to the lock/syncmap. As for the signalling, I'd use a channel and close it when done. A sync.WaitGroup solution might seem tempting (add 1 worker, let all the other Wait on that worker being done), but it will block, prying you from other events. Looking a bit ahead in the code and being visionary, chances are you want to attach a context to your system and in this case, the channel neatly fits into a select(-ive recieve) with the context as well. This allows a waiting request to time out as well. singleflight is up for consideration in the parallel problem: how do you intent to guard against the same image being uploaded and processed twice. One simple way is to use a content-addressing of the image: its SHA256 checksum is the key on which you add singleflight work. It has to weaved into the syncmap solution above however, so perhaps it is easier to approach it layered: solve one problem at a time. [0] I think a linear view is enough here, though obviously a serial view could also be used. -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
