){
> sql = "INSERT INTO tblfoo(foo,bar) VALUES("+it.next()+","+CONST.BAR+");";
> }
You should try to wrap that into a single transaction. PostgreSQL
waits for I/O write completion for each INSERT as it's
implicitely in its own transaction. Maybe the added pe
ng decisions than
the kernel because it has additional information, but I am not
sure in which circumstances and the amount of better decisions it
can take.
--
Guillaume Cottenceau, MNC Mobile News Channel SA, an Alcatel-Lucent Company
Av. de la Gare 10, 1003 Lausanne, Switzerland - direct +41 21 3
in order to get the next timestamp smaller
> (or larger) than a given one?
Well, how can the planner know inside which partition the wanted
row is? There might be no data, say, inside a couple of
partitions in the past before finding the wanted row, in which
case 3 partitions in the pas
e which no satisfactory
data could be found by the CHECK constraint, but I think it's not
possible (too complicated) to infer that any found row in your
other partitions would not be in the final resultset because of
1. the query's resultset order 2. the limit 3. the actual
conditions in
untime-config-resource.html#RUNTIME-CONFIG-RESOURCE-FSM
Have you investigated this? It seems that you already know about
the FSM stuff, according to your question about FSM and 8.3.
You can also run VACUUM ANALYZE more frequently (after all, it
doesn't lock the table).
--
Guillaume Cottenceau
soleted tuples stored in the FSM and actually
the occupied space is reused before a VACUUM is performed, or is
something else happening? Maybe the FSM is only storing a
reference to diskspages containing only dead rows, and that's the
difference I've been missing?
--
Guillaume Cottencea
ssions'
AND c.oid = i.indrelid
AND c2.oid = i.indexrelid
ORDER BY c2.relname;
--
Guillaume Cottenceau, MNC Mobile News Channel SA, an Alcatel-Lucent Company
Av. de la Gare 10, 1003 Lausanne, Switzerland - direct +41 21 317 50 36
---(end of broadcast)---
TIP 2: Don't 'kill -9' the postmaster
Heikki Linnakangas writes:
> Guillaume Cottenceau wrote:
> > According to documentation[1], VACUUM FULL's only benefit is
> > returning unused disk space to the operating system; am I correct
> > in assuming there's also the benefit of optimizing the
> > p
Guillaume Cottenceau writes:
> With that in mind, I've tried to estimate how much benefit would
> be brought by running VACUUM FULL, with the output of VACUUM
> VERBOSE. However, it seems that for example the "removable rows"
> reported by each VACUUM VERBOSE run i
007-05/msg00112.php
--
Guillaume Cottenceau, MNC Mobile News Channel SA, an Alcatel-Lucent Company
Av. de la Gare 10, 1003 Lausanne, Switzerland - direct +41 21 317 50 36
---(end of broadcast)---
TIP 6: explain analyze is your friend
Michael Stone writes:
> On Tue, May 15, 2007 at 06:43:50PM +0200, Guillaume Cottenceau wrote:
> >patch - basically, I think the documentation under estimates (or
> >sometimes misses) the benefit of VACUUM FULL for scans, and the
> >needs of VACUUM FULL if the routine VA
"Jim C. Nasby" writes:
> On Wed, May 16, 2007 at 09:41:46AM +0200, Guillaume Cottenceau wrote:
[...]
> > Come on, I don't suggest to remove several bold warnings about
> > it, the best one being "Therefore, frequently using VACUUM FULL
> > can
tch to "postgresql can't shrink
relation" (almost) returns:
http://archives.postgresql.org/pgsql-novice/2002-12/msg00126.php
--
Guillaume Cottenceau, MNC Mobile News Channel SA, an Alcatel-Lucent Company
Av. de la Gare 10, 1003 Lausanne, Switzerland - direct +41 21 317 50 36
---
will end up with
the same results and be even faster than any use of PostgreSQL.
If anyone needs data, then just say you had data corruption, and
that since 100% dataloss is accepted, then all's well.
--
Guillaume Cottenceau
--
Sent via pgsql-performance mailing list (pgsql-performanc
Marti Raudsepp writes:
> On Fri, Nov 5, 2010 at 13:11, Guillaume Cottenceau wrote:
>> Don't use PostgreSQL, just drop your data, you will end up with
>> the same results and be even faster than any use of PostgreSQL.
>> If anyone needs data, then just say you had da
akes new suggestions worthwhile, while previous ones
are now seen as useless.
--
Guillaume Cottenceau
--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance
ly fast compared to your CPU. Even if some queries
will run faster from a side-effect of these settings, you're
likely to create other random problems...
--
Guillaume Cottenceau
--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscr
4.rpm | 1.6 MB 02:48
> (7/7): postgresql90-se (68%) 44% [= ] 7.0 kB/s | 2.2 MB
> 06:33 ETA
>
> 7 kilobytes per second??? That brings back the times of the good, old
> 9600 USR modems and floppy disks.
What's your point and in what is it related to that ML?
--
Guillau
#x27; experience on this matter.
I apologize for this long email but I wanted to be sure I gave
enough information on the data and things I have tried to fix the
problem myself. If anyone can see what I am doing wrong, I would
be very interested in pointers.
Thanks in advance!
Btw, I use postgres 7.4.5 with -B 1000 -N 500 and all
postgresql.conf default values except timezone = 'UTC', on an
ext3 partition with data=ordered, and run Linux 2.6.12.
--
Guillaume Cottenceau
---(end of broadcast)---
TIP 5: don't forget to increase your free space map settings
Guillaume,
Thanks for your answer.
> On 17 Mar 2006 11:09:50 +0100, Guillaume Cottenceau
> wrote:
> > Reading the documentation and postgresql list archives, I have
> > run ANALYZE right before my tests, I have increased the
> > statistics target to 50 for the conside
Hi Mark,
Thanks for your reply.
> Guillaume Cottenceau wrote:
[...]
> > Btw, I use postgres 7.4.5 with -B 1000 -N 500 and all
> > postgresql.conf default values except timezone = 'UTC', on an
> > ext3 partition with data=ordered, and run Linux 2.6.12.
>
>
la used to perform the interpolation. I
have absolutely no knowledge on pg internals so this is rather
new/fresh for me, I have no idea how smart that choice is (but
based on my general feeling about pg, I'm suspecting this is
actually smart but I am not smart enough to see why ;p).
--
Guillau
omments on the result of pmap
showing around 450M of "private memory" used by pg, if anyone can
share insight about it. Though most people seem freebsd-oriented,
and this might be very much linux-centric.
--
Guillaume Cottenceau
---(end of broadcast)---
TIP 5: don't forget to increase your free space map settings
but I'll wager that it
> will be substantially better than what's in there now. FYI, see also
> http://archives.postgresql.org/pgsql-performance/2005-04/msg00669.php
Sad that Tom didn't share his thoughts about your cost algorithm
question in this message.
--
ke tens of INSERTs per second into a small table, no more.
"iostat" reports very large figures in the "await" field compared
to other servers using raid1 controllers, that's my best guess,
but I was unable to find why and how to fix (and the vendor has
been very helpless
"Jim C. Nasby" writes:
> On Tue, Mar 21, 2006 at 02:03:19PM +0100, Guillaume Cottenceau wrote:
> > "Jim C. Nasby" writes:
> >
> > > On Tue, Mar 21, 2006 at 10:40:45PM +1200, Mark Kirkwood wrote:
> > > > I was going to recommend
Hi Scott,
Scott Marlowe writes:
> On Wed, 2006-03-22 at 02:04, Guillaume Cottenceau wrote:
[...]
> > Yes, we use 7.4.5 actually, because "it just works", so production
> > wants to first deal with all the things that don't work before
> > upgrading.
time about more than 2 minutes.
>
> If my query is:
> SELECT * FROM table ORDER BY id, name OFFSET 5 LIMIT 1
> It takes about 2 seconds.
First you should read the appropriate documentation.
http://www.postgresql.org/docs/8.1/interactive/performance-tips.html
--
Guillaume Co
g/pgsql-performance/2006-03/msg00407.php
--
Guillaume Cottenceau
Create your personal SMS or WAP Service - visit http://mobilefriends.ch/
---(end of broadcast)---
TIP 3: Have you checked our extensive FAQ?
http://www.postgresql.org/docs/faq
:
[1] Processes were always showing one/some postmaster on SELECT,
a constant load of 1, and vmstat always showing activity in
IO blocks out (application generate all sort of typical
statements, some SELECT, UPDATE, INSERT either "directly" or
through stored procedures)
Hi Markus,
Thanks for your message.
> Guillaume Cottenceau wrote:
>
> > We noticed a slowdown on our application while traffic was kinda
> > heavy. The logics after reading the docs commanded us to trim the
> > enlarged tables, run VACUUM ANALYZE and then expect fast
&g
Guillaume,
Thanks for your help.
> On 28 Aug 2006 11:43:16 +0200, Guillaume Cottenceau <[EMAIL PROTECTED]> wrote:
> > max_fsm_pages is 2
> > max_fsm_relations is 1000
> > Do they look low?
>
> Yes they are probably too low if you don't run VACUUM on a
very much insists on solving index data corruption
with REINDEX and doesn't talk much about removing old obsolete
data)
(also, is there any way to REINDEX all index of all tables
easily? as when we do just "VACUUM ANALYZE" for the whole
database)
> Now, when the query planne
s a contrib module at least since 7.4, and included
> in the server since 8.1). If you think that vacuum during working hours
> puts too much load on your server, there are options to tweak that, at
> least in 8.1.
Ok, thanks. Unfortunately production insists on sticking on 7.4.5
for the moment
y need realtime query statistics which I am
not sure PG does.
After all, memory added to shared buffers should be mecanically
removed from effective cache size (or others), so I cannot just
increase it until the OS cannot cache anymore :)
--
Guillaume Cottenceau
---(
assumption about the effective size of the
disk cache (that is, the portion of the kernel's disk cache
that will be used for PostgreSQL data files). This is
measured in disk pages, which are normally 8192 bytes each.
The default is 1000.
--
Guillaume Cottenceau
Create y
"Merlin Moncure" writes:
> On 01 Sep 2006 19:00:52 +0200, Guillaume Cottenceau <[EMAIL PROTECTED]> wrote:
> > Hi,
> >
> > I've been looking at the results from the pg_statio* tables, to
> > view the impact of increasing the shared buffers to in
"Phadnis" writes:
> plz unsubscribe me..
>
> i am sending mail to this id.. for unsubscribing.. is it correct..
> my mail box is gettin flooded..
you managed to subscribe, you'll probably manage to unsubcribe.
hint: the email headers contain the information for un
kernel to synchronize a write and waiting until it is
finished). Same can probably happen to the "sync" command.
--
Guillaume Cottenceau
Create your personal SMS or WAP Service - visit http://mobilefriends.ch/
---(end of broadcast)---
TIP 6: explain analyze is your friend
y temporarily disabling sequential
scans. Have a look at this chapter:
http://www.postgresql.org/docs/7.4/interactive/runtime-config.html#RUNTIME-CONFIG-QUERY
--
Guillaume Cottenceau
Create your personal SMS or WAP Service - visit http://mobilefriends.ch/
---(end of broadcas
;bar"
CREATE TABLE
foo=# insert into bar (baz) values ('');
INSERT 217426996 1
foo=# insert into bar (baz) values ('');
ERROR: duplicate key violates unique constraint "bar_pkey"
--
Guillaume Cottenceau
Create your personal SMS or WAP Service - visit http://
e as possible, albeit with low dev priority)
Ref:
[1] inspired by http://developer.postgresql.org/~wieck/vacuum_cost/
--
Guillaume Cottenceau, MNC Mobile News Channel SA, an Alcatel-Lucent Company
Av. de la Gare 10, 1003 Lausanne, Switzerland
---(end of bro
, it's like using dynamite to knock a whole in a wall
> for a window.
Thanks for opening a new kind of trol^Hargument against VACUUM
FULL, that one's more fresh (at least to me, who doesn't follow
the list too close anyway).
Just for the record, I inherited a poorly (actually, "
o large for current disks - at least for us, we've found that 2
is more correct)
[...]
> -> Seq Scan on _user (cost=0.00..205537.72 rows=806972 width=24)
--
Guillaume Cottenceau, MNC Mobile News Channel SA, an Alcatel-Lucent Company
Av. de la Gare 10, 1003 Lausanne, S
ble is mandatory
for your running application(s).
--
Guillaume Cottenceau, MNC Mobile News Channel SA
---(end of broadcast)---
TIP 5: don't forget to increase your free space map settings
|1
pg_toast_87582_index |1
(...)
[2] db=# SELECT sum(relpages)*8/1024 FROM pg_class, pg_namespace WHERE
pg_namespace.oid = pg_class.relnamespace AND relkind = 'i' AND nspname =
'public';
?column?
--
644
--
Guillaume Cottenceau
---(end of broadcast)---
TIP 2: Don't 'kill -9' the postmaster
Tom Lane writes:
> Guillaume Cottenceau <[EMAIL PROTECTED]> writes:
>> I have made a comparison restoring a production dump with default
>> and large maintenance_work_mem. The speedup improvement here is
>> only of 5% (12'30 => 11'50).
>
>> Appre
> (4395-628)*8/1024.0 MB of bloat
(IIRC, this VACUUM output is for 7.4, it has changed a bit
since then)
--
Guillaume Cottenceau
--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance
er won't be able to then restart if the filesystem is
still full (it needs some free disk space for its startup).
Or maybe this has been fixed in recent versions?
--
Guillaume Cottenceau
--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to y
Matthew Wakeling writes:
> On Thu, 15 May 2008, Guillaume Cottenceau wrote:
>> Also, IIRC when PG writes data up to a full filesystem,
>> postmaster won't be able to then restart if the filesystem is
>> still full (it needs some free disk space for its startup).
&g
"Joshua D. Drake" writes:
> Guillaume Cottenceau wrote:
>> Matthew Wakeling writes:
>
>> It is still relevant, as with 5% margin, you can afford changing
>> that to 0% with tune2fs, just the time for you to start PG and
>> remove some data by SQL, then sh
)
-> Seq Scan on foo (cost=0.00..164217.00 rows=1070009 width=4) (actual
time=2379.873..2379.888 rows=15 loops=1)
Filter: (bar = 8)
Total runtime: 2379.974 ms
(on 8.3.1)
--
Guillaume Cottenceau, MNC Mobile News Channel SA, an Alcatel-Lucent Company
Av. de la Gare 10
27;8';
QUERY PLAN
--
Index Scan using foobar on foo (cost=0.00..30398.66 rows=1079089 width=154)
Index Cond: (bar = 8)
Filter: (baz IS NULL)
(3 rows)
Thi
ll limit to 15
rows.
> postgres not be best to ignore the limit when deciding the best index to use
> - in this simple query wouldn't the best plan to use always be the same
> with or without a limit?
I am not too sure, but I'd say no: when PG considers the LIMIT,
then it knows
need only 3 disk pages, so it shouldn't be faster than with a
seqscan, theoretically; however, I am not sure then why on my
simple "foo" test it isn't using the same decision..
Btw, that should not solve your problem, but normally, to help PG
choose indexscan often enough, it
99[1], and you're
looking for less than 1% of rows, the expected rows may be at the
beginning or at the end of the heap?
Ref:
[1] or even 1, as ANALYZE doesn't sample all the rows?
--
Guillaume Cottenceau, MNC Mobile News Channel SA, an Alcatel-Lucent Company
Av. de la Gare 10, 1003
blue, is it just because when postgresql fsync's after
a write, on a normal system the write has to really happen on
disk and waiting for it to be complete, whereas with BBU cache
the fsync is almost immediate because the write cache actually
replaces the "really on disk" write?
-
Richard Huxton writes:
> I'm guessing what you've got is a table that's not being vacuumed
> because you've had a transaction that's been open for weeks.
Or because no vacuuming at all is performed on this table (no
autovacuum and no explicit VACUUM on dat
ke these figures at all)
Of course, these are good for us (bloat is very, very low and
performance impact is not experienced in production), not
necessarily for you. You should conduct your own tests.
Be sure to also consider http://developer.postgresql.org/~wieck/vacuum_cost/
--
Guillaume Co
uscule performance (the culprit was
shared with untuned FSM and friends).
--
Guillaume Cottenceau
--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance
estimate is different enough), I've
opted to tell the JDBC driver to use the protocol version 2, as
prepared statements were not so much prepared back then (IIRC
parameter interpolation is performed in driver and the whole SQL
query is passed each time, parsed, and planned) using
protocolVersion=2
Matthew Wakeling writes:
> It appears that I am being censored.
Do you seriously think that censorman would kill your previous
mails, but would let a "It appears that I am being censored" mail
go through?
--
Guillaume Cottenceau
--
Sent via pgsql-performance mailing list (pgsq
zéro explaining how to implement
producer-consumers? :) But that must really be thought before
implementing. It's not worth piling queries in memory because it
will create other problems if queries are produced faster than
consumed in the long run.
--
Guillaume Cottenceau
--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance
. Specifically,
> you are probably looking for "autovacuum" to be enabled.
autovacuum is enabled by default on PG 8.3 as well.
--
Guillaume Cottenceau
--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance
ew the currently running queries:
SELECT procpid, datname, current_query, query_start FROM pg_stat_activity WHERE
current_query <> ''
That may also be interesting.
--
Guillaume Cottenceau
--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make ch
Phoenix Kiula writes:
> Tasks: 568 total, 1 running, 537 sleeping, 6 stopped, 24 zombie
The stopped and zombie processes look odd. Any reason for these?
--
Guillaume Cottenceau
--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to y
ential benefit of running VACUUM
FULL (or CLUSTER) in production (once your DB is bloated, a
normal VACUUM doesn't remove the bloat).
db_production=# VACUUM VERBOSE table;
[...]
INFO: "table": found 408 removable, 64994 nonremovable row versions in 4395
pages
er2|
0.6 |0
...
A few investigations show that when tbloat is close to 1.0 then
it seems not reliable, otherwise it seems useful.
pg 8.4.7
--
Guillaume Cottenceau
--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscript
me = 'public' ORDER BY relpages DESC;
relkind = 'i' for indexes.
--
Guillaume Cottenceau
--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance
oring almost all "object x
account" combinations in object_perm) is optimal.
--
Guillaume Cottenceau
--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance
uming of database "" failed: ERROR: duplicate key value
> violates unique constraint "c"
> DETAIL: Key (indexrelid)=(2678) already exists.
>
> We are using Postgres 9.0.1
>
> Can you please help us out in understanding the cause of this error?
tern possibilities)
Any thoughts on what would be the best approach? Mine looks a bit
ugly.
Thanks,
--
Guillaume Cottenceau
--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance
tual time=0.015..192.520 rows=200 loops=1)
Index Cond: (ca = 1)
Heap Fetches: 0
Total runtime: 240.918 ms
(4 rows)
DROP TABLE ta;
DROP TABLE
--
Guillaume Cottenceau
--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your sub
, raise max_fsm_* on your 8.4 or
upgrade to 9.x).
--
Guillaume Cottenceau
--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance
ile
rewriting the table. Otherwise, VACUUM VERBOSE on both the
established DB and a backup/restore on a fresh DB also provide a
helpful comparison of how many pages are used for suspected
tables.
--
Guillaume Cottenceau
--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
T
ain counting.
> This performs quickly:
>
> SELECT reltuples AS count
> FROM pg_class
> WHERE relname = 'NewsArticle';
This is not the same. This one uses precomputed statistics, and
doesn't scan the actual table data.
> But I'd like to add conditions so I don
Benjamin Dugast writes:
> • fsync to off (that helped but we can't do this)
not exactly your question, but maybe synchronous_commit=off is a
nice enough intermediary solution for you (it may give better
performances at other places too for only an affordable cost)
--
Guillaume Co
PER SLO
>
> Something I can do ? Something I can check for ?
>
> //Bill
>
--
Guillaume Cottenceau
--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance
78 matches
Mail list logo