We run into problems with connection pools on Heroku.
In Postgres a connection is very expensive. The cheap/free plans on Heroku Postgres all have very small connection limits. The hobby plan is limited to 20 connections https://devcenter.heroku.com/articles/heroku-postgres-plans. Some people will build really small rack apps that use active record, and want to max out their processes/threads. It's not that common but it happens. More commonly people don't know about this behavior and get ActiveRecord::ConnectionTimeoutError because they're using default Puma which is set to 16 threads and default AR which is set to 5 connections. I wrote these docs to help people understand the connection requirements of AR https://devcenter.heroku.com/articles/concurrency-and-database-connections Our larger customers end up hitting our max conneciton limit of our "production" plans which is 500 connections. They end up having to run PgBouncer which does per machine connection multiplexing https://devcenter.heroku.com/articles/concurrency-and-database-connections#limit-connections-with-pgbouncer ---Richard Schneeman http://www.schneems.com On Thu, Oct 1, 2015 at 9:45 AM, Matt Jones <[email protected]> wrote: > If the concern is long-running threads holding DB connections they don't > need, wouldn't a simpler solution be to explicitly return connections to > the pool (via release_connection) before doing a long-running non-DB > operation? Checking out and back in on most operations seems like > optimizing for that uncommon case at a runtime cost for the common one. > --Matt Jones > On Thu, Oct 1, 2015 at 4:25 AM, Xavier Noria <[email protected]> wrote: >> The only time I've seen one connection per thread being an issue was in >> one app that ran many processes and it started to reach the connection >> limit of their db server in traffic peaks. Even in that scenario, >> contention is also a risk if you reduce the pool. >> >> Other than the mental image that you're using less resources (at the price >> of contention), I am not sure there is going to be any significant >> practical win. It could be, I am not saying it wouldn't (I have not done >> your study), but at first sight the cost/benefit is not clear to me in >> practical terms. >> >> Regarding transactions, #execute is public AR interface, and >> >> AR::Base.connection.execute('START TRANSACTION') >> >> is valid AR code, I am not sure if drivers know the connection is in a >> transaction and have API to check, but #transaction_open? returns false as >> of this writing. Why would anybody do that? I don't know, maybe because >> they are writing a heavy SQL oriented script and doing that manually feels >> natural... it doesn't matter, it can be done. >> >> Another practical point is that when a connection is checked out the >> connection pool tests if it is alive issuing for example SELECT 1. That >> would mean that if a request performs today 200 queries, they would become >> 400 queries. I don't know if the alive check could be rethought to run less >> frequently, but it probably should to avoid that 2x. >> >> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Ruby on Rails: Core" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To post to this group, send email to [email protected]. >> Visit this group at http://groups.google.com/group/rubyonrails-core. >> For more options, visit https://groups.google.com/d/optout. >> > -- > You received this message because you are subscribed to the Google Groups > "Ruby on Rails: Core" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at http://groups.google.com/group/rubyonrails-core. > For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Core" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/rubyonrails-core. For more options, visit https://groups.google.com/d/optout.
