#37033: Psycopg2 has different semantics than psycopg3 and should not be
deprecated
--------------------------------+--------------------------------------
Reporter: Cameron Gorrie | Owner: (none)
Type: Uncategorized | Status: new
Component: Uncategorized | Version: 6.0
Severity: Normal | Resolution:
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
--------------------------------+--------------------------------------
Comment (by Cameron Gorrie):
Thanks for taking a look, Simon. Agreed that we're asking a lot of the
libraries / frameworks here w.r.t. idempotency.
Warm shutdown seems like a good plan: check e.g. `self.terminate = True`
periodically. In the context we're supporting this system, it would
require us to change an uncomfortable amount of code, but it is doable. I
thought about doing it in a thread, but I think `SystemExit` would have
the exact same problem. I have a feeling we'd have options if we had async
handlers, but we're not on 6.0 yet and even then it's a big change... can
you think of anything I've missed?
I've opened a ticket with psycopg and am about to with celery. I don't
think this issue is resolved by writing code -- but to collaborate in
ensuring the message is out to help those who--perhaps naively--believe
this combination will work. I opened it here too because the root cause is
two-fold:
1. celery soft / cold shutdown (i.e. exception propagation)
2. I/O "commits" that require transactional properties across Python
bytecode
Django ORM's `on_commit` semantics hit the latter: when we can't be sure
if our COMMIT was successful or not, we cannot run code that depends on
the result. It results in situations that seem impossible until you look
at the problem from the right lens.
In my case I inherited parts of this system and am still in some ways
finding my footing; opening a ticket with this problem in itself makes it
more discoverable, but I think you're right -- it's Celery's soft shutdown
that intrinsically breaks some of the lower-level stuff in the python
stack.
--
Ticket URL: <https://code.djangoproject.com/ticket/37033#comment:2>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion visit
https://groups.google.com/d/msgid/django-updates/0107019d897ba800-2691e8b5-5d07-4f5e-a722-312b88f8d5e9-000000%40eu-central-1.amazonses.com.
