I'm architecting a system that will send and receive messages to/from our SMS gateway vendor, and I'm trying to figure out the best way to accomplish asynchronous web service calls inside a Django view. I'd like the view to create a job and notify another process that an item has been added to its queue to process.
Here's the background: Our SMS gateway provider accepts SMS messages to our short code and turns them into a SOAP-like message sent to my server via HTTP Post. I need to accept the request, decide what action needs to be taken, get the appropriate data, then return a response to the customer. For example, I receive messages such as "WEATHER UPDATE 33609" for our local weather, or "TAMPA BREAKING NEWS" to get the latest breaking news story links. Instead of duplicating our enterprise content on this server, I will use web service calls to query our various servers for the necessary information. I get the data back, package it up in the vendor's message format, then send it back to the SMS gateway where it is delivered to the customer. 10-60 second response time from SMS message sent to SMS response received is adequate for our needs. When the gateway sends its initial POST, if I send any text in the server's response it will be delivered to the customer as an SMS message. If I return a blank http 200 response, no message is generated, and the request is successfully terminated. I'd like to use Django, as we use it heavily in our enterprise and I'm very comfortable with its HTTP handling and the convenience of the ORM versus writing a custom HTTP server in Twisted to accomplish this task. Here's the tricky part: Most of these web service calls are very fast, but I am concerned about possible latency. In my experience, making a view dependant on an external URL can really tie up Apache when the calls time out or are slow, since that Apache process is not able to serve other requests until it is finished, and there are only a finite number of processes available (I'm using mod-prefork). I also don't want to tie up the vendor's servers by leaving a bunch of connections open waiting for a response. Here's what I think would be ideal: It would be great if the view could accept the request then open an asynchronous subprocess or send a signal to a listener that there is a "job" waiting in the queue. Unfortunately, my research indicates that Django signals are not asynchronous, and that using Popen() under Mod Python is buggy (in that you don't truly end up with an asynchronous process or thread). If anyone disagrees, please let me know, I'm certainly no expert. Here's my best solution: The best solution I can come up with would be to have my view send a message to a listening socket server. That socket server could use Twisted to do asynchronous to query the web service, package up the response, and send the HTTP POST back to the gateway, leaving Apache to focus on just accepting the requests. The Twisted app can use the Django API to log the response. My only big concern is what happens in a swarm of requests; if we put up a contest message at an NFL game on the big screen, we could have thousands of requests in a short amount of time. Even if the machine and Apache is tuned to accept that many requests, I'm afraid that relying on a socket server to do the queueing would result in too many concurrent threads. (and I am not very well-versed in Twisted) The only other thing I can think of is to have Django accept the initial HTTP Post from the gateway, and have a daemon running on the server that would continuously check for unprocessed items in the database. This seems pretty inefficient as I would be querying the database continuously. If anyone has any clever ideas or suggestions, I'd love to hear them. I'm really not sure what the best way to go forward is. Thanks! Scott --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~----------~----~----~----~------~----~------~--~---
