#31373: Django 3.x / uwsgi is leaking connections
-------------------------------+--------------------------------------
Reporter: Marcin Nowak | Owner: nobody
Type: Uncategorized | Status: new
Component: Uncategorized | Version: 3.0
Severity: Normal | Resolution:
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------+--------------------------------------
Description changed by Marcin Nowak:
Old description:
> Project upgraded from 2.x to 3.0.4 is leaking PostgreSQL database
> connections.
> ATOMIC_REQUESTS=False, AUTOCOMMIT=True, CONN_MAX_AGE=0
> UWSGi threads are enabled (`--enable-threads`)
> Python 3.8.1 and 3.7.3
>
> After few hours all posgtress connections are used (too many connections
> error).
> pg_stat_activity is reporting idle connections with part of queries
> visible.
>
> I would try to deliver more details later.
>
> [[Image(Firefox_Screenshot_2020-03-17T15-05-35.104Z.png)]]
>
> ----
>
> Facts:
> - setting up CONN_MAX_AGE > 0 does not help
> - CONN_MAX_AGE = 30, starting `runserver` and making many calls with `ab
> -c 10 -n 100 http://127.0.0.1:8000/` uses all available connections and
> ends with error 500 (due to no available connections)
> - CONN_MAX_AGE = 30, starting `runserver --nothreading` and making calls
> with `ab -c 100 -n 100 http://127.0.0.1:8000/` works fine
> - CONN_MAX_AGE=30 + runserver + ab scenario is reproducible on clean
> install. Called view must access database.
> - CONN_MAX_AGE=0 + runserver + ab -> cannot reproduce.
> - Django 2.x has similar issue for CONN_MAX_AGE=30, but can reuse
> connections (still there are several errors on high traffic, but app
> server can "recover" and handle bunch of incoming requests)
> - Looks like Django 3.x cannot reuse connections
>
> ----
>
> Runserver (with threads) test results:
>
> `ab -c 10 -n 200 ` Django 3.0.4:
> Complete requests: 200
> Failed requests: 108
>
> `ab -c 10 -n 1000` Django 2.2.11:
> Complete requests: 1000
> Failed requests: 37
>
> ----
>
> There may be two separate cases:
> 1. not reusing connection when CONN_MAX_AGE is greater than 0
> 2. leaking (not closing?) connections on uwsgi with enabled threads and
> CONN_MAX_AGE set to 0
New description:
Project upgraded from 2.x to 3.0.4 is leaking PostgreSQL database
connections.
ATOMIC_REQUESTS=False, AUTOCOMMIT=True, CONN_MAX_AGE=0
UWSGi threads are enabled (`--enable-threads`)
Python 3.8.1 and 3.7.3
After few hours all posgtress connections are used (too many connections
error).
pg_stat_activity is reporting idle connections with part of queries
visible.
I would try to deliver more details later.
[[Image(Firefox_Screenshot_2020-03-17T15-05-35.104Z.png)]]
----
Facts:
- setting up CONN_MAX_AGE > 0 does not help
- CONN_MAX_AGE = 30, starting `runserver` and making many calls with `ab
-c 10 -n 100 http://127.0.0.1:8000/` uses all available connections and
ends with error 500 (due to no available connections)
- CONN_MAX_AGE = 30, starting `runserver --nothreading` and making calls
with `ab -c 100 -n 100 http://127.0.0.1:8000/` works fine
- CONN_MAX_AGE=30 + runserver + ab scenario is reproducible on clean
install. Called view must access database.
- CONN_MAX_AGE=0 + runserver + ab -> cannot reproduce.
- Django 2.x has similar issue for CONN_MAX_AGE=30, but can reuse
connections (still there are several errors on high traffic, but app
server can "recover" and handle bunch of incoming requests)
- Looks like Django 3.x cannot reuse connections
- uwsgi + threads + CONN_MAX_AGE=0 on localhost -> cannot reproduce
----
Runserver (with threads) test results:
`ab -c 10 -n 200 ` Django 3.0.4:
Complete requests: 200
Failed requests: 108
`ab -c 10 -n 1000` Django 2.2.11:
Complete requests: 1000
Failed requests: 37
----
There may be two separate cases:
1. not reusing connection when CONN_MAX_AGE is greater than 0
2. leaking (not closing?) connections on uwsgi (?) with enabled threads
(?) and CONN_MAX_AGE set to 0
--
--
Ticket URL: <https://code.djangoproject.com/ticket/31373#comment:8>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-updates/065.b8b4652e1d087176f11e86872222afb4%40djangoproject.com.
