I am really hitting a wall on this issue. The problem occurs when a transaction is shut down due to some error. It is possible for the inbound (client, UA) side to be handled by thread 1 and the outbound (origin server) side by thread 2. In such a case the shutdown logic in thread 1 shuts down the outbound VC without locking it. If the timing is bad this can happen simultaneously with a timeout or other operation in thread 2. In fact it seems quite possible that thread 2 could shutdown the VC, put it on the thread 2 free list, and then re-use it before thread 1 notices. I presume there must be something that prevents that but I can't find it.
It might seem like the obvious solution would be to lock the VC but that seems to create a significant risk for deadlock in this should happen symmetrically. If we use just a TRY_LOCK then one must answer the question "what if you don't get the lock?". I have been looking doing a variant of that 1) Try to lock the VC. If that works, shut it down. 2) If no lock, schedule a continuation on thread 2 to do the shutdown. But this leaves me concerned about what happens if whatever it is on thread 2 happens between the scheduling and execution of the callback. How can the continuation code even know? I think that adding a generation number to the VC objects might be a reasonable approach. This would be incremented each time the VC is put on the free list. The continuation (or any other structure that holds a pointer to the VC) can cache the generation number and validate it when useful. Zwoop is working on / has a patch that puts the VCs on the same thread which might make all of this moot. I don't know if that is a fully general solution that handles the connection sharing (one outbound connection feeding multiple inbound requests).
