Hi All, I'm trying to troubleshoot an issue and i'm posting here because its caused by connecting to Riak even if i may miss some configuration on uwsgi. This is my enviroment:
nginx + uwsgi + flask app The flask app uses Riak and Redis. The connection between nginx and uwsgi is via unix socket. If i use only one process in uwsgi i can easily run simultaneous requests without hitting the issue i'm having. When i add even only one more process all the workers gets busy and the app hangs. If i remove the riak code part it's working fine, so the issue has to be somewhere on the connection pooling or something else. I'm experiencing the same issues as this user: http://lists.basho.com/pipermail/riak-users_lists.basho.com/2014-January/014387.html If i use protobuf protocol i hit the DecodeErrors messages, sometimes i don't get any error the app just hangs. If i use the http protocol with riak, i don't get any exception but it just hangs. It hangs on a simple snippet: user_bucket = riak_client.bucket_type('user_type').bucket('users') user_info = user_bucket.get(user_id) I'm using Locust to generate traffic 1 uwsgi worker, locust 10 users hatch 2 seconds = no issues 2+ uwsgi worker, locust 10 users hatch 2 seconds = app hangs after few minutes For Riak i have 3 nodes running on the same box, i'm using the latest version from git. The app hangs in any of those connection scenarios: riak_client = riak.RiakClient(host='127.0.0.1', pb_port=10017, protocol='pbc') riak_client = riak.RiakClient(protocol='pbc', nodes=[{'host':'127.0.0.1', 'pb_port':10017},{'host':'127.0.0.1', 'pb_port':10027},{'host':'127.0.0.1', 'pb_port':10037}]) riak_client = riak.RiakClient(protocol='http', http_port=10018, host='127.0.0.1') riak_client = riak.RiakClient(protocol='http', nodes=[{'host':'127.0.0.1', 'http_port':10018},{'host':'127.0.0.1', 'http_port':10028},{'host':'127.0.0.1', 'http_port':10038}]) My uwsgi config is the following: [uwsgi] vhost = true socket = /tmp/app.sock venv = /opt/app/venv chdir = /opt/app/ module = myapp callable = app processes = 2 master = true close-on-exec=true master = true post-buffering = 1 carbon = 127.0.0.1:2003 stats = /tmp/stats.sock If i sniff the network traffic, when it hangs uwsgi basically stops sending any request to riak, all the workers becomes busy and the only way to restore it it's a restart of uwsgi. My SW versions are the following: Riak latest from git. Python libs: riak (2.2.0) riak-pb (2.0.0.16) protobuf (2.5.0) UWSGI: 2.0.10 Any idea on how i can troubleshoot this issue? It seems related to uwsgi but it's happening only when using the Riak connection. Thank you :tele _______________________________________________ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
