[GENERAL] how to truncate/purge the pg_xlog directory?

2007-10-12 Thread tfinneid
Hi As far as I understand pg_xlog is the transaction log i.e. the WAL. I have a pg 8.2 which has consumed about 113MB in pg_xlog/, while the cluster is virtually empty. There are a couple of tables in the postgres schema, but they contain no data at the moment. My disk is filling up and I want t

Re: [GENERAL] how to truncate/purge the pg_xlog directory?

2007-10-12 Thread tfinneid
> As checkpoints are issued, old WAL files will either be recycled > (renamed) or deleted. Change the setting, then try (as user postgres) > issuing a "vacuum full " followed by "SELECT pg_switch_xlog()" that > should recycle them quicker. That'll only get you down to 3 or 4 files > though - that's

Re: [GENERAL] how to truncate/purge the pg_xlog directory?

2007-10-12 Thread tfinneid
> [EMAIL PROTECTED] wrote: > > OK, so that's 7 files? > What is your checkpoint_segments setting in postgresql.conf? If it's 3 > then you can indeed expect a maximum of 7 WAL files. For a low-activity > system you can always turn the setting down. The setting was not set, so I expect that it used

[GENERAL] select count() out of memory

2007-10-25 Thread tfinneid
Hi I am volume testing a db model that consists of a paritioned tables. The db has been running for a week and a half now and has built up to contain approx 55000 partition tables of 18000 rows each. The root table therefore contains about 1 billion rows. When I try to do a "select count(*)" of th

Re: [GENERAL] select count() out of memory

2007-10-25 Thread tfinneid
Hi I have tried to answer to the best of my knowledge but its running on Soalris 10, and I am not that familiar with solaris ( Go Linux!!! :) > any more memory. Either you have a very low memory ulimit (look at ulimit > -a > in the same session as Postgres) or your machine is really low on memory

Re: [GENERAL] select count() out of memory

2007-10-25 Thread tfinneid
> <[EMAIL PROTECTED]> writes: > >> max_connections = 1000 > > Do you actually have anywhere near this number of processes? What is your > setting for work_mem? Keep in mind every process could use as much as > work_mem > and actually it's possible to use that much several times over. > > Also, what

Re: [GENERAL] select count() out of memory

2007-10-25 Thread tfinneid
> God morgen > > Please display these memory settings from your postgresql.conf file > sort_mem > shared_buffers I have shown the entire configuration. if its not in the configuration shown, I have changed its value. I have used the configuration example provided by Sun regarding running postgres

Re: [GENERAL] select count() out of memory

2007-10-25 Thread tfinneid
> [EMAIL PROTECTED] wrote: > >> > are a dump of Postgres's current memory allocations and could be >> useful in >> > showing if there's a memory leak causing this. >> >> The file is 20M, these are the last lines: (the first line continues >> unttill ff_26000) >> >> >> idx_attributes_g1_seq_1_ff_4_v

Re: [GENERAL] select count() out of memory

2007-10-25 Thread tfinneid
> I have shown the entire configuration. if its not in the configuration > shown, I have changed its value. I meant to say "I haven't changed its value" thomas ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings

Re: [GENERAL] select count() out of memory

2007-10-25 Thread tfinneid
> [EMAIL PROTECTED] wrote: >> > [EMAIL PROTECTED] wrote: >> > >> >> > are a dump of Postgres's current memory allocations and could be >> >> useful in >> >> > showing if there's a memory leak causing this. >> >> >> >> The file is 20M, these are the last lines: (the first line continues >> >> unttil

Re: [GENERAL] select count() out of memory

2007-10-25 Thread tfinneid
> Alvaro Herrera <[EMAIL PROTECTED]> writes: >> [EMAIL PROTECTED] wrote: >>> I did a test previously, where I created 1 million partitions (without >>> data) and I checked the limits of pg, so I think it should be ok. > >> Clearly it's not. > > You couldn't have tested it too much --- even planning

Re: [GENERAL] select count() out of memory

2007-10-25 Thread tfinneid
> [EMAIL PROTECTED] wrote: > It will work on a million partitions and more, provided you do > operations on single partitions. Thats good enough for me, thats exactly what I want. I just used the select count() on the root to get a feeling of how many rows it was in total. An then I thought that t

Re: [GENERAL] select count() out of memory

2007-10-25 Thread tfinneid
>> The db worked fine until it reached perhaps 30-40 thousand partitions. > > It depends on how you have the partitions set up and how you're > accessing them. Are all of these partitions under the same parent > table? If so, then trying run a SELECT COUNT(*) against the parent > table is simply

Re: [GENERAL] select count() out of memory

2007-10-25 Thread tfinneid
> Are you selecting directly from the child table, or from the parent > table with constraint_exclusion turned on? the problem was when selecting from the parent table, but selecting from child tables are no problem. As stated in other replies, I only wanted to know how many rows where in the tabl

Re: [GENERAL] select count() out of memory

2007-10-25 Thread tfinneid
> So, out of curiosity, I asked my Oracle DBA friend if she'd ever heard > of anyone having 60,000 or so partitions in a table, and she looked at > me like I had a third eye in my forehead and said in her sweet voice > "Well, that would certainly be an edge case". She sounded like she > was worrie

Re: [GENERAL] select count() out of memory

2007-10-25 Thread tfinneid
> [EMAIL PROTECTED] wrote: >> Thats good enough for me, thats exactly what I want. > > In that case, why use partitions at all? They are simple independent > tables. For two reasons, - the data logically belongs together. - because its more practical to create tables as childs of a parent table t

Re: [GENERAL] select count() out of memory

2007-10-25 Thread tfinneid
> Excellent, it sounds like you should be fine then. One thing to > note: if you want to get an "idea" of how many rows you have in your > partitions, you can run a SUM aggregate on reltuples in pg_class for > all of your partitions. The more recent the last ANALYZE for each > table, the more ac

Re: [GENERAL] select count() out of memory

2007-10-25 Thread tfinneid
> [EMAIL PROTECTED] wrote: >> > [EMAIL PROTECTED] wrote: >> >> Thats good enough for me, thats exactly what I want. >> > >> > In that case, why use partitions at all? They are simple independent >> > tables. >> >> For two reasons, >> - the data logically belongs together. >> - because its more pra

Re: [GENERAL] select count() out of memory

2007-10-25 Thread tfinneid
> [EMAIL PROTECTED] writes: >>> In that case, why use partitions at all? They are simple independent >>> tables. > >> For two reasons, >> - the data logically belongs together. >> - because its more practical to create tables as childs of a parent >> table >> than as independent tables. >>- ch

Re: [GENERAL] select count() out of memory

2007-10-25 Thread tfinneid
> this is my config > > checkpoint_segments = 96 > effective_cache_size = 128000 > shared_buffers = 43 > max_fsm_pages = 208000 > max_fsm_relations = 1 > > max_connections = 1000 > > autovacuum = off# enable autovacuum subprocess? > > fsync = on

Re: [GENERAL] select count() out of memory

2007-10-26 Thread tfinneid
> "Sam Mason" <[EMAIL PROTECTED]> writes: > >> On Fri, Oct 26, 2007 at 08:25:09AM +0200, Thomas Finneid wrote: >>> Gregory Stark wrote: >>> >Tom's point is that if you have 55k tables then just *finding* the >>> newest >>> >child table is fairly expensive. You're accessing a not >>> insignificant-s

[GENERAL] plpgsql returning resultset

2008-09-02 Thread tfinneid
Hi I know the subject has been discussed before, but I dont find what any information that helps me make it work, so please bear with me. In pg 8.2 I want to write a function that gathers data from different tables and joins it into a single resultset, similar to "select * from tableA", but the p

Re: [GENERAL] plpgsql returning resultset

2008-09-02 Thread tfinneid
> Hi there, > > You need to do it like this: > > with val_list in > select * from tableA do > loop > return next val_list; > end loop; > return; > > There's an example here: Does that work in 8.2, cause i get the same error message as I described above regards thoma