On Mon, Mar 16, 2015 at 6:59 AM, Robert Kaye wrote:
>
> 4. Linux 3.2 apparently has some less than desirable swap behaviours. Once
> we started swapping, everything went nuts.
On older machines I used to just turn off swap altogether. Esp if I
wasn't running out of memory but swap was engaging an
On 3/15/15 7:17 PM, mich...@sqlexec.com wrote:
Please avoid top-posting.
I agree with your counter argument about how high max_connections "can"
cause problems, but max_connections may not part of the problem here.
There's a bunch of "depends stuff" in there based on workload details, #
cpus, RA
Robert many thanks for feedback!!
Could you post your new pgbouncer config file??
How many postgresql process do you have now at OS with this new conf??
How many clients from app server hit your pgbouncer??
Regards,
Regards,
2015-03-16 11:32 GMT-03:00 Robert Kaye :
>
> > On Mar 16, 2015, at
On 03/16/2015 05:59 AM, Robert Kaye wrote:
> 4. Linux 3.2 apparently has some less than desirable swap behaviours.
> Once we started swapping, everything went nuts.
Relevant to this:
http://www.databasesoup.com/2014/09/why-you-need-to-avoid-linux-kernel-32.html
Anybody who is on Linux Kernels 3
> On Mar 16, 2015, at 2:22 PM, Thomas Kellerer wrote:
>
> I think it would be nice if you can amend your blog posting to include the
> solution that you found.
>
> Otherwise this will simply stick around as yet another unsolved performance
> problem
Good thinking:
http://blog.musicbrain
Robert Kaye wrote:
> However, I am glad to report that our problems are fixed and that our server
> is
> back to humming along nicely.
>
> What we changed:
>
> 1. As it was pointed out here, max_connections of 500 was in fact insanely
> high, especially in light of using PGbouncer. Before we
Robert Kaye schrieb am 16.03.2015 um 13:59:
> However, I am glad to report that our problems are fixed and that our
> server is back to humming along nicely.
>
> And as I said to Josh earlier: "Postgres rocks our world. I’m
> immensely pleased that once again the problems were our own stupidity
>
On March 16, 2015 at 3:24:34 AM, Roxanne Reid-Bennett (r...@tara-lu.com) wrote:
Robert,
Wow - You've engaged the wizards indeed.
I haven't heard or seen anything that would answer my *second* question if
faced with this (my first would have been "what changed")
Yes, indeed — I feel honored
On 3/15/2015 6:54 AM, Robert Kaye wrote:
Hi!
We at MusicBrainz have been having trouble with our Postgres install
for the past few days. I’ve collected all the relevant information here:
http://blog.musicbrainz.org/2015/03/15/postgres-troubles/
If anyone could provide tips, suggestions or ot
On 16/03/15 13:07, Tomas Vondra wrote:
On 16.3.2015 00:55, mich...@sqlexec.com wrote:
Why is 500 connections "insane". We got 32 CPU with 96GB and 3000
max connections, and we are doing fine, even when hitting our max
concurrent connection peaks around 4500. At a previous site, we were
using 200
I agree with your counter argument about how high max_connections "can"
cause problems, but max_connections may not part of the problem here.
There's a bunch of "depends stuff" in there based on workload details, #
cpus, RAM, etc.
I'm still waiting to find out how many CPUs on this DB server.
(please quote properly)
On 2015-03-15 19:55:23 -0400, mich...@sqlexec.com wrote:
> Why is 500 connections "insane". We got 32 CPU with 96GB and 3000 max
> connections, and we are doing fine, even when hitting our max concurrent
> connection peaks around 4500. At a previous site, we were using 20
On 16.3.2015 00:55, mich...@sqlexec.com wrote:
> Why is 500 connections "insane". We got 32 CPU with 96GB and 3000
> max connections, and we are doing fine, even when hitting our max
> concurrent connection peaks around 4500. At a previous site, we were
> using 2000 max connections on 24 CPU and 64
Why is 500 connections "insane". We got 32 CPU with 96GB and 3000 max
connections, and we are doing fine, even when hitting our max concurrent
connection peaks around 4500. At a previous site, we were using 2000
max connections on 24 CPU and 64GB RAM, with about 1500 max concurrent
connection
On 15.3.2015 23:47, Andres Freund wrote:
> On 2015-03-15 12:25:07 -0600, Scott Marlowe wrote:
>> Here's the problem with a large shared_buffers on a machine that's
>> getting pushed into swap. It starts to swap BUFFERs. Once buffers
>> start getting swapped you're not just losing performance, that
On 2015-03-15 20:54:51 +0300, Ilya Kosmodemiansky wrote:
> On Sun, Mar 15, 2015 at 8:46 PM, Andres Freund wrote:
> > That imo doesn't really have anything to do with it. The primary benefit
> > of a BBU with writeback caching is accelerating (near-)synchronous
> > writes. Like the WAL.
>
> My poi
On 15.3.2015 18:54, Ilya Kosmodemiansky wrote:
> On Sun, Mar 15, 2015 at 8:46 PM, Andres Freund wrote:
>> That imo doesn't really have anything to do with it. The primary
>> benefit of a BBU with writeback caching is accelerating
>> (near-)synchronous writes. Like the WAL.
>
> My point was, that
How many CPUs in play here on the PG Cluster Server,
cat /proc/cpuinfo | grep processor | wc -l
I see you got pg_stat_statements enabled, what are the SQL you
experience during this heavy load time? And does explain on them show a
lot of sorting activity that requires more work_mem.
Please
On 2015-03-15 12:25:07 -0600, Scott Marlowe wrote:
> Here's the problem with a large shared_buffers on a machine that's
> getting pushed into swap. It starts to swap BUFFERs. Once buffers
> start getting swapped you're not just losing performance, that huge
> shared_buffers is now working against y
On Sun, Mar 15, 2015 at 11:46 AM, Andres Freund wrote:
> On 2015-03-15 20:42:51 +0300, Ilya Kosmodemiansky wrote:
>> On Sun, Mar 15, 2015 at 8:20 PM, Andres Freund
>> wrote:
>> > On 2015-03-15 11:09:34 -0600, Scott Marlowe wrote:
>> >> shared_mem of 12G is almost always too large. I'd drop it do
On Sun, Mar 15, 2015 at 8:46 PM, Andres Freund wrote:
> That imo doesn't really have anything to do with it. The primary benefit
> of a BBU with writeback caching is accelerating (near-)synchronous
> writes. Like the WAL.
My point was, that having no proper raid controller (today bbu surely
neede
On 2015-03-15 20:42:51 +0300, Ilya Kosmodemiansky wrote:
> On Sun, Mar 15, 2015 at 8:20 PM, Andres Freund wrote:
> > On 2015-03-15 11:09:34 -0600, Scott Marlowe wrote:
> >> shared_mem of 12G is almost always too large. I'd drop it down to ~1G or
> >> so.
> >
> > I think that's a outdated wisdom,
On Sun, Mar 15, 2015 at 8:20 PM, Andres Freund wrote:
> On 2015-03-15 11:09:34 -0600, Scott Marlowe wrote:
>> shared_mem of 12G is almost always too large. I'd drop it down to ~1G or so.
>
> I think that's a outdated wisdom, i.e. not generally true.
Quite agreed. With note, that proper configured
On 2015-03-15 11:09:34 -0600, Scott Marlowe wrote:
> shared_mem of 12G is almost always too large. I'd drop it down to ~1G or so.
I think that's a outdated wisdom, i.e. not generally true. I've now seen
a significant number of systems where a larger shared_buffers can help
quite massively. The pr
On Sun, Mar 15, 2015 at 11:09 AM, Scott Marlowe wrote:
Clarification:
> 64MB work mem AND max_connections = 500 is a recipe for disaster. No
> db can actively process 500 queries at once without going kaboom, ad
> having 64MB work_mem means it will go kaboom long before it reaches
> 500 active c
On Sun, Mar 15, 2015 at 10:43 AM, Scott Marlowe wrote:
> On Sun, Mar 15, 2015 at 7:50 AM, Andres Freund wrote:
>> On 2015-03-15 13:07:25 +0100, Robert Kaye wrote:
>>>
>>> > On Mar 15, 2015, at 12:13 PM, Josh Krupka wrote:
>>> >
>>> > It sounds like you've hit the postgres basics, what about some
On 03/15/2015 09:43 AM, Scott Marlowe wrote:
* Consider installing perf (linux-utils-$something) and doing a
systemwide profile.
3.2 isn't the greatest kernel around, efficiency wise. At some point you
might want to upgrade to something newer. I've seen remarkable
differences around this.
On 03/15/2015 05:08 AM, Robert Kaye wrote:
On Mar 15, 2015, at 12:41 PM, Andreas Kretschmer
wrote:
just a wild guess: raid-controller BBU faulty
We don’t have a BBU in this server, but at least we have redundant power
supplies.
In any case, how would a fault batter possibly cause this?
On Sun, Mar 15, 2015 at 7:50 AM, Andres Freund wrote:
> On 2015-03-15 13:07:25 +0100, Robert Kaye wrote:
>>
>> > On Mar 15, 2015, at 12:13 PM, Josh Krupka wrote:
>> >
>> > It sounds like you've hit the postgres basics, what about some of the
>> > linux check list items?
>> >
>> > what does free
On 2015-03-15 13:07:25 +0100, Robert Kaye wrote:
>
> > On Mar 15, 2015, at 12:13 PM, Josh Krupka wrote:
> >
> > It sounds like you've hit the postgres basics, what about some of the linux
> > check list items?
> >
> > what does free -m show on your db server?
>
> total used
> On Mar 15, 2015, at 13:45, Josh Krupka wrote:
> Hmm that's definitely odd that it's swapping since it has plenty of free
> memory at the moment. Is it still under heavy load right now? Has the
> output of free consistently looked like that during your trouble times?
And it seems better t
On 15.3.2015 13:07, Robert Kaye wrote:
>
>> If the load problem really is being caused by swapping when things
>> really shouldn't be swapping, it could be a matter of adjusting your
>> swappiness - what does cat /proc/sys/vm/swappiness show on your server?
>
> 0
>
> We adjusted that too, but no
Hi!
What shows your pg_stat_bgwriter for one day?
> On Mar 15, 2015, at 11:54, Robert Kaye wrote:
>
> Hi!
>
> We at MusicBrainz have been having trouble with our Postgres install for the
> past few days. I’ve collected all the relevant information here:
>
> http://blog.musicbrainz.org/20
On Sun, Mar 15, 2015 at 8:07 AM, Robert Kaye wrote:
>
> what does free -m show on your db server?
>
>
> total used free sharedbuffers cached
> Mem: 48295 31673 16622 0 5 12670
> -/+ buffers/cache: 18997 29298
>
On 2015-03-15 13:08:13 +0100, Robert Kaye wrote:
> > On Mar 15, 2015, at 12:41 PM, Andreas Kretschmer
> > wrote:
> >
> > just a wild guess: raid-controller BBU faulty
>
> We don’t have a BBU in this server, but at least we have redundant power
> supplies.
>
> In any case, how would a fault ba
pls check this if it helps:
http://ubuntuforums.org/showthread.php?t=2258734
在 2015/3/15 18:54, Robert Kaye 写道:
Hi!
We at MusicBrainz have been having trouble with our
Postgres install for the past few days. I’ve colle
> On Mar 15, 2015, at 12:41 PM, Andreas Kretschmer
> wrote:
>
> just a wild guess: raid-controller BBU faulty
We don’t have a BBU in this server, but at least we have redundant power
supplies.
In any case, how would a fault batter possibly cause this?
--
--ruaok
Robert Kaye --
> On Mar 15, 2015, at 12:13 PM, Josh Krupka wrote:
>
> It sounds like you've hit the postgres basics, what about some of the linux
> check list items?
>
> what does free -m show on your db server?
total used free sharedbuffers cached
Mem: 48295
Robert Kaye wrote:
> Hi!
>
> We at MusicBrainz have been having trouble with our Postgres install for the
> past few days. I’ve collected all the relevant information here:
>
> http://blog.musicbrainz.org/2015/03/15/postgres-troubles/
>
> If anyone could provide tips, suggestions or other re
It sounds like you've hit the postgres basics, what about some of the linux
check list items?
what does free -m show on your db server?
If the load problem really is being caused by swapping when things really
shouldn't be swapping, it could be a matter of adjusting your swappiness -
what does ca
40 matches
Mail list logo
