I’d like to write a circuit breaker wrapper to use with services that I 
call from inside a Django app, and I’d like to get some pointers on the 
following questions:

   1. What’s the most global, persisting, shared-across-requests type of 
   data I can read & write to memory?
   2. However I can do this, does this require using `threading.Lock()`? 
   (looking around `dispatch.Signal` I see some use of it)
   3. If this data is shared per-process, what’s a good way to see how many 
   processes it would be (for uwsgi is that just the # of worker processes)?

The term “circuit breaker” is referring to service clients that cut 
themselves off from making future requests to a service after the service 
has failed to connect or read/write after a certain error threshold. They 
can employ retries to eventually right themselves. If a service continues 
to fail, even despite strict timeouts, it can significantly slow down a 
site or even cause the webserver to run out of threads and it will 
effectively reach a DoS scenario. The clients are per-webserver (or maybe 
in this case, per-process) and all independently can open or close their 
“circuits” based on what they’re experiencing.

To effectively track the effectiveness of service calls without having a 
performance impact on the webserver, it really needs to store this info in 
memory—which is the source of my original questions—and I assume this means 
I’m stuck with only info per-process? I’m OK with the info disappearing 
after server restarts, but might there be some gotchas around other 
configurations, like the `--harakiri` option for uwsgi? As stated above, 
I’m also a little unclear on when using threading locks would be required 
vs not, given that python uses the GIL—but maybe certain configurations do 
allow for non-thread-safe things to happen?

I found 2 barebones approaches on github, but the correctness/effectiveness 
of each was quite unclear:

   - 
   
https://github.com/cuker/django-patchboard/blob/master/patchboard/circuitbreaker.py
   - 
   
https://github.com/globocom/memcached_memoize/blob/master/memcached_memoize/decorators/circuit_breaker.py
   
Any help would be appreciated—whatever I build for this would definitely be 
open-sourced to the community!

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/35f791f2-d857-44de-a9e7-e52e346728b9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Reply via email to