[PERFORM] SOLVED: Re: HELP: How to tame the 8.3.x JDBC driver with a biq guery result set

2010-04-15 Thread Dave Crooke
When a connection is used for both reading and writing, a commit() also destroys any open cursors. Simple workaround - use two connections. See full discussion on JDBC list. Cheers Dave On Thu, Apr 15, 2010 at 3:01 PM, Dave Crooke wrote: > I have followed the instructions below to no avail ...

Re: [PERFORM] Autovaccum with cost_delay does not complete on one solaris 5.10 machine

2010-04-15 Thread Josh Berkus
> Josh, are you sure that both servers are identical in terms of both > GUC-related and per-table autovacuum settings? I should check per-table. GUC, yes, because the company has source management for config files. -- -- Josh Berkus

Re: [PERFORM] Autovaccum with cost_delay does not complete on one solaris 5.10 machine

2010-04-15 Thread Tom Lane
Josh Berkus writes: >> But it shouldn't >> be sleeping after each page with normal cost_delay parameters, should it? > Right, that's why I find this puzzling. If the problem was easier to > reproduce it would be easier to analyze. The behavior would be explained if VacuumCostLimit were getting

Re: [PERFORM] 8.3.9 - latency spikes with Linux (and tuning for consistently low latency)

2010-04-15 Thread Greg Smith
Marinos Yannikos wrote: vm.dirty_ratio = 80 This is tuned the opposite direction of what you want. The default tuning in the generation of kernels you're using is: /proc/sys/vm/dirty_ratio = 10 /proc/sys/vm/dirty_background_ratio = 5 And those should be considered upper limits if you want

Re: [PERFORM] stats collector suddenly causing lots of IO

2010-04-15 Thread Tom Lane
Chris writes: > I have a lot of centos servers which are running postgres. Postgres isn't > used > that heavily on any of them, but lately, the stats collector process keeps > causing tons of IO load. It seems to happen only on servers with centos 5. > The versions of postgres that are running

Re: [PERFORM] Autovaccum with cost_delay does not complete on one solaris 5.10 machine

2010-04-15 Thread Josh Berkus
>> pg_usleep calls select(), and some googling indicates that select() is >> implemented as pollsys() on recent Solaris versions. So Josh's >> assumption that those are delay calls seems plausible. It's certainly the behavior I'm seeing otherwise. In "normal operation", the number of pages read

Re: [PERFORM] Autovaccum with cost_delay does not complete on one solaris 5.10 machine

2010-04-15 Thread Alvaro Herrera
Tom Lane wrote: > Alvaro Herrera writes: > > We don't call pollsys anywhere. Something in Solaris must be doing it > > under the hood. > > pg_usleep calls select(), and some googling indicates that select() is > implemented as pollsys() on recent Solaris versions. So Josh's > assumption that th

Re: [PERFORM] Autovaccum with cost_delay does not complete on one solaris 5.10 machine

2010-04-15 Thread Tom Lane
Alvaro Herrera writes: > We don't call pollsys anywhere. Something in Solaris must be doing it > under the hood. pg_usleep calls select(), and some googling indicates that select() is implemented as pollsys() on recent Solaris versions. So Josh's assumption that those are delay calls seems plau

[PERFORM] Re: HELP: How to tame the 8.3.x JDBC driver with a biq guery result set

2010-04-15 Thread Dave Crooke
I have followed the instructions below to no avail any thoughts? http://jdbc.postgresql.org/documentation/83/query.html#query-with-cursor This is what happens when I reduce the fetch_size to 50 ... stops after about 950msec and 120 fetches (6k rows) 13:59:56,054 [PerfDataMigrator] ERRO

Re: [PERFORM] Autovaccum with cost_delay does not complete on one solaris 5.10 machine

2010-04-15 Thread Alvaro Herrera
Josh Berkus wrote: > Basically, vacuuming of a table which normally takes about 20 minutes > interactively with vacuum_cost_delay set to 20 had not completed after > 14 hours. When I trussed it, I saw activity which indicated to me that > autovacuum was doing a pollsys, presumably for cost_limit,

[PERFORM] Autovaccum with cost_delay does not complete on one solaris 5.10 machine

2010-04-15 Thread Josh Berkus
All, We're having a very strange problem where autovacuum does not complete on a Postgres 8.3.8/Solaris 5.10 system. The reason I say strange is: this is one of a twin pair of identical systems,and the other system does not have this issue. Basically, vacuuming of a table which normally takes ab

Re: [PERFORM] 8.3.9 - latency spikes with Linux (and tuning for consistently low latency)

2010-04-15 Thread Kevin Grittner
Tom Lane wrote: > Have you checked whether the spikes correlate with checkpoints? > Turn on log_checkpoints and watch for awhile. If so, fooling with > the checkpoint parameters might give some relief. If that by itself doesn't do it, I've found that making the background writer more aggress

Re: [PERFORM] 8.3.9 - latency spikes with Linux (and tuning for consistently low latency)

2010-04-15 Thread Tom Lane
Marinos Yannikos writes: > we are seeing latency spikes in the 2-3 second range (sometimes 8-10s) for > queries that usually take 3-4ms on our systems and I am running out of things > to > try to get rid of them. Have you checked whether the spikes correlate with checkpoints? Turn on log_chec

[PERFORM] 8.3.9 - latency spikes with Linux (and tuning for consistently low latency)

2010-04-15 Thread Marinos Yannikos
Hi, we are seeing latency spikes in the 2-3 second range (sometimes 8-10s) for queries that usually take 3-4ms on our systems and I am running out of things to try to get rid of them. Perhaps someone here has more ideas - here's a description of the systems and what I've tried with no impact a

Re: [PERFORM] JDBC question for PG 8.3.9

2010-04-15 Thread Dave Cramer
On Wed, Apr 14, 2010 at 7:10 PM, Craig Ringer wrote: > On 15/04/10 04:49, Dave Crooke wrote: >> >> Hi foilks >> >> I am using PG 8.3 from Java. I am considering a performance tweak which >> will involve holding about 150 java.sql.PreparedStatment objects open >> against a single PGSQL connection.