I've got some database transactional code running within a thread via RunInteraction (it needs to rollback on error).
One of the features on the logic is that it interacts with a 3rd party API, which may or may not have a ratelimit in effect. Most 3rd party APIs i consume have a minute/hourly based limit -- so I just do a graceful fail on the transaction to requeue it, and set a local timeout. This particular API has a per-second meter in effect. It is considerably more efficient to just sleep for a second and "try again" a second later. I end up being able to process over twice as many API calls in a given period with that strategy. The problem I ran into, is that I need a second database connection to note that a ratelimit has been created in the database -- and i'd like to pull that db connection out of the twisted connection pool to simplify my connection management. - I can't do it within the existing connection, because I am in a transaction and the limit won't appear to other twisted jobs. - I can't use runInteraction() off the pool, because that will defer to a thread and I am in a thread, and the last time I did something like that... everything broke. - the stopgap I have right now is just creating/closing a raw pscycopg2 connection and using that. it works, but it requires a bit more management of postgres to ensure I can balance the right number of connections. has anyone had a need like this before? if so, what did you do? i basically just need to make two "INSERT" statements from a second connection. _______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
