Re: [GENERAL] Memory usage per session

Hi >I think the "problem" that he is having is fixable only by changing how >PostgreSQL itself works. >His problem is a PL/pgSQL function which is 11K lines in length. >When invoked, this function is "compiled" into a large tokenized parse tree. >This parse tree is only usable in the session whic

Re: [GENERAL] Memory usage per session

Hi > On Jul 08, John McKown modulated: > ... >> I think the "problem" that he is having is fixable only by changing how >> PostgreSQL itself works. His problem is a PL/pgSQL function which is >> 11K lines in length. When invoked, this function is "compiled" into a >> large tokenized parse tree. Th

Re: [GENERAL] Memory usage per session

On 7/8/2016 12:00 PM, Karl Czajkowski wrote: 3. Rewrite or refactor such complex stored procedures in a different programming language such as C or Python, so your PL/pgsql stored procedures remain small glue around libraries of code. Postgres makes it very trivial to extend the syst

Re: [GENERAL] Memory usage per session

On Jul 08, John McKown modulated: ... > I think the "problem" that he is having is fixable only by changing how > PostgreSQL itself works. His problem is a PL/pgSQL function which is > 11K lines in length. When invoked, this function is "compiled" into a > large tokenized parse tree. This parse tre

Re: [GENERAL] Memory usage per session

2016-07-08 17:49 GMT+02:00 : > Hi > > > >> Oracle: about 5M > >> postgreSql: about 160М > > > > >The almost session memory is used for catalog caches. So you should to > have big catalog and long living sessions. > > >What do you do exactly? > > I've generate test code that emulates instruction t

Re: [GENERAL] Memory usage per session

On Fri, Jul 8, 2016 at 11:26 AM, Melvin Davidson wrote: > > > On Fri, Jul 8, 2016 at 11:49 AM, wrote: > >> Hi >> >> >> >> Oracle: about 5M >> >> postgreSql: about 160М >> >> >> >> >The almost session memory is used for catalog caches. So you should to >> have big catalog and long living sessions

Re: [GENERAL] Memory usage per session

On Fri, Jul 8, 2016 at 11:49 AM, wrote: > Hi > > > >> Oracle: about 5M > >> postgreSql: about 160М > > > > >The almost session memory is used for catalog caches. So you should to > have big catalog and long living sessions. > > >What do you do exactly? > > I've generate test code that emulates i

Re: [GENERAL] Memory usage per session

Title: Re: [GENERAL] Memory usage per session Hi >> Oracle: about 5M >> postgreSql: about 160М >The almost session memory is used for catalog caches. So you should to have big catalog and long living sessions.  >What do you do exactly? I've generate test code tha

Re: [GENERAL] Memory usage per session

Title: Re: [GENERAL] Memory usage per session Hi >> >> The test performs about 11K lines of code >> >> Oracle: about 5M >> >> postgreSql: about 160М >What is the actual O/S that PostgreSQL is installed on?  >How much total memory is on the server? 

Re: [GENERAL] Memory usage per session

On 08/07/2016 17:07, amatv...@bitec.ru wrote: Hi The test performs about 11K lines of code Oracle: about 5M postgreSql: about 160М Do you have 100 CPUs on this system which apparently doesn't have 16G of RAM available for PG to use? We can say at fact: We currently work at oracle. Our code

Re: [GENERAL] Memory usage per session

On Fri, Jul 8, 2016 at 10:07 AM, wrote: > Hi > > >> >> The test performs about 11K lines of code > >> >> Oracle: about 5M > >> >> postgreSql: about 160М > > > > Do you have 100 CPUs on this system which apparently doesn't have 16G > > of RAM available for PG to use? > We can say at fact: > We cur

Re: [GENERAL] Memory usage per session

Title: Re: [GENERAL] Memory usage per session Здравствуйте. >> Oracle: about 5M >> postgreSql: about 160М >​I'm admittedly ignorant of this type of testing. But if the memory usage for PostgreSQL is in the server, perhaps due to caching (how to test?), then it likely

Re: [GENERAL] Memory usage per session

Hi >> >> The test performs about 11K lines of code >> >> Oracle: about 5M >> >> postgreSql: about 160М > Do you have 100 CPUs on this system which apparently doesn't have 16G > of RAM available for PG to use? We can say at fact: We currently work at oracle. Our code base about 4000 k line of co

Re: [GENERAL] Memory usage per session

* amatv...@bitec.ru (amatv...@bitec.ru) wrote: > > On 08/07/2016 14:11, amatv...@bitec.ru wrote: > >> The test performs about 11K lines of code > >> Memory usage per session: > >> Oracle: about 5M > >> MSSqlServer: about 4M > >> postgreSql: about 160М > > > Visual C??? > > You will have to run Pos

Re: [GENERAL] Memory usage per session

On Fri, Jul 8, 2016 at 8:16 AM, wrote: > Hi > > > On 08/07/2016 14:11, amatv...@bitec.ru wrote: > >> Hi. > >> The test performs about 11K lines of code > >> Memory usage per session: > >> Oracle: about 5M > >> MSSqlServer: about 4M > >> postgreSql: about 160М > > > > Visual C??? > > You will have

Re: [GENERAL] Memory usage per session

Hi 2016-07-08 15:16 GMT+02:00 : > Hi > > > On 08/07/2016 14:11, amatv...@bitec.ru wrote: > >> Hi. > >> The test performs about 11K lines of code > >> Memory usage per session: > >> Oracle: about 5M > >> MSSqlServer: about 4M > >> postgreSql: about 160М > > > > Visual C??? > > You will have to run

Re: [GENERAL] Memory usage per session

Hi > On 08/07/2016 14:11, amatv...@bitec.ru wrote: >> Hi. >> The test performs about 11K lines of code >> Memory usage per session: >> Oracle: about 5M >> MSSqlServer: about 4M >> postgreSql: about 160М > Visual C??? > You will have to run PostgreSQL on a proper Unix system to test for > perfor

Re: [GENERAL] Memory usage per session

On 08/07/2016 14:11, amatv...@bitec.ru wrote: Hi. We have tested postgreSql,Oracle,MSSqlServer. The test performs about 11K lines of code Memory usage per session: Oracle: about 5M MSSqlServer: about 4M postgreSql: about 160М The result of postgreSql is very sad(Our typical business logic has abo

[GENERAL] Memory usage per session

Hi. We have tested postgreSql,Oracle,MSSqlServer. The test performs about 11K lines of code Memory usage per session: Oracle: about 5M MSSqlServer: about 4M postgreSql: about 160М The result of postgreSql is very sad(Our typical business logic has about 30K lines of code). How can I reduce memory

Re: [GENERAL] Memory usage per postmaster process

=?UTF-8?B?R3J6ZWdvcnogVGHFhGN6eWs=?= writes: > On 11/02/2013 08:47 PM, Tom Lane wrote: >> Author: Tom Lane >> Branch: master Release: REL9_1_BR [3e5f9412d] 2010-10-06 19:31:05 -0400 >> >> Reduce the memory requirement for large ispell dictionaries. > I checked this patch: > http://www.postgresql.

Re: [GENERAL] Memory usage per postmaster process

On 11/02/2013 08:47 PM, Tom Lane wrote: Author: Tom Lane Branch: master Release: REL9_1_BR [3e5f9412d] 2010-10-06 19:31:05 -0400 Reduce the memory requirement for large ispell dictionaries. I checked this patch: http://www.postgresql.org/message-id/AANLkTi=4fUi1zoFMpZ==yf14rjdv_g1xgakvqmdy

Re: [100% SPAM] Re: [GENERAL] Memory usage per postmaster process

=?UTF-8?B?R3J6ZWdvcnogVGHFhGN6eWs=?= writes: > Did tsearch2 dictionary caching implementation improve after 8.3 on this > matter? Well, there was this: Author: Tom Lane Branch: master Release: REL9_1_BR [3e5f9412d] 2010-10-06 19:31:05 -0400 Reduce the memory requirement for large ispell d

Re: [100% SPAM] Re: [GENERAL] Memory usage per postmaster process

On 11/02/2013 07:47 PM, John R Pierce wrote: On 11/2/2013 11:03 AM, Grzegorz Tańczyk wrote: Is there any way to limit total memory usage by postgres and keep maximum connections limit? Postgresql.conf settings are default for 8.3.23. I need to have 100 connections in pool. the size of your

Re: [GENERAL] Memory usage per postmaster process

On 11/2/2013 11:03 AM, Grzegorz Tańczyk wrote: Is there any way to limit total memory usage by postgres and keep maximum connections limit? Postgresql.conf settings are default for 8.3.23. I need to have 100 connections in pool. the size of your connection pool shouldn't be much more than 2

[GENERAL] Memory usage per postmaster process

Hello, Recently I had some problem with tsearch2 in postgres 8.3. Application was using JDBC connection pool and it was possible that each connection from the pool used tsearch2(with dictionary) in some query. This resulted in almost every postmaster process using >0.5g memory. Solution of thi

[GENERAL] Memory usage per postmaster process

Hello, Recently I had some problem with tsearch2 in postgres 8.3. Application was using JDBC connection pool and it was possible that each connection from the pool used tsearch2(with dictionary) in some query. This resulted in almost every postmaster process using >0.5g memory. Solution of thi

Re: [GENERAL] Memory usage after upgrade to 9.2.4

I found where the problem is: http://www.postgresql.org/docs/9.1/static/release-9-1-6.html It could be nice if something is added to the 9.2 release notes to warn the admins. 2013/4/24 Adrian Klaver > On 04/24/2013 11:20 AM, Daniel Cristian Cruz wrote: > >> I've done an explain analyze under

Re: [GENERAL] Memory usage after upgrade to 9.2.4

On 04/24/2013 11:20 AM, Daniel Cristian Cruz wrote: I've done an explain analyze under the test environment, and there is no aggressive memory usage. So I dropped the database in the new cluster and restored a fresh dump from production (in theory, that's the difference between the two environme

Re: [GENERAL] Memory usage after upgrade to 9.2.4

I've done an explain analyze under the test environment, and there is no aggressive memory usage. So I dropped the database in the new cluster and restored a fresh dump from production (in theory, that's the difference between the two environments). Some minutes after I got an answer: after a dum

Re: [GENERAL] Memory usage after upgrade to 9.2.4

2013/4/23 Adrian Klaver > On 04/23/2013 04:23 AM, Daniel Cristian Cruz wrote: > >> 2013/4/22 Daniel Cristian Cruz > > >> >> >> query1: >> EXPLAIN ANALYZE SELECT ced.id_evento, ced.inicio, ced.termino, >> ced.evento, ced.id_eventos IS NOT NULL AS aul

Re: [GENERAL] Memory usage after upgrade to 9.2.4

On 04/23/2013 04:23 AM, Daniel Cristian Cruz wrote: 2013/4/22 Daniel Cristian Cruz mailto:danielcrist...@gmail.com>> query1: EXPLAIN ANALYZE SELECT ced.id_evento, ced.inicio, ced.termino, ced.evento, ced.id_eventos IS NOT NULL AS aula_dividida, ac.titulo, ced.id_tipo_evento, ced.

Re: [GENERAL] Memory usage after upgrade to 9.2.4

2013/4/22 Daniel Cristian Cruz > query1: > EXPLAIN ANALYZE SELECT ced.id_evento, ced.inicio, ced.termino, ced.evento, > ced.id_eventos IS NOT NULL AS aula_dividida, ac.titulo, ced.id_tipo_evento, > ced.tipo_evento, ac.media_referencia, p.nome, ef.nome AS nomeEspacoFisico, > ( SELECT count ( pre2.

Re: [GENERAL] Memory usage after upgrade to 9.2.4

I'm running pgBadger over the log, and will get some queries to explain analyze them. The 9.2 cluster is running in the same server as the production, so I will try to compare some critical explains and publish on explain.depesz.com. 2013/4/21 Adrian Klaver > On 04/21/2013 09:29 AM, Daniel Cris

Re: [GENERAL] Memory usage after upgrade to 9.2.4

On 04/21/2013 09:29 AM, Daniel Cristian Cruz wrote: 2013/4/21 Adrian Klaver mailto:adrian.kla...@gmail.com>> 1) Major upgrade from 9.1.4 to 9.2.4. Used pg_upgrade Tested on VM with 9.2.4 and no problems. Same machine used for production server 9.1.4 and 9.2.4 When

Fwd: [GENERAL] Memory usage after upgrade to 9.2.4

And this one only to Adrian. Sorry to all. -- Forwarded message -- From: Daniel Cristian Cruz Date: 2013/4/20 Subject: Re: [GENERAL] Memory usage after upgrade to 9.2.4 To: Adrian Klaver 2013/4/20 Adrian Klaver > On 04/20/2013 04:30 PM, Daniel Cristian Cruz wr

Fwd: [GENERAL] Memory usage after upgrade to 9.2.4

Sorry, I answered to Tomas only... -- Forwarded message -- From: Daniel Cristian Cruz Date: 2013/4/21 Subject: Re: [GENERAL] Memory usage after upgrade to 9.2.4 To: Tomas Vondra I had the same environment, almost: 2013/4/21 Tomas Vondra > > 2) What are the hardware

Re: [GENERAL] Memory usage after upgrade to 9.2.4

2013/4/21 Adrian Klaver > On 04/21/2013 07:50 AM, Daniel Cristian Cruz wrote: > >> >> >> >> 2013/4/21 Adrian Klaver > > >> >> >> On 04/21/2013 06:37 AM, Daniel Cristian Cruz wrote: >> >> 2013/4/21 Tom Lane mailto:t...@sss.pgh.pa.us> >>

Re: [GENERAL] Memory usage after upgrade to 9.2.4

On 04/21/2013 07:50 AM, Daniel Cristian Cruz wrote: 2013/4/21 Adrian Klaver mailto:adrian.kla...@gmail.com>> On 04/21/2013 06:37 AM, Daniel Cristian Cruz wrote: 2013/4/21 Tom Lane mailto:t...@sss.pgh.pa.us> >>

Re: [GENERAL] Memory usage after upgrade to 9.2.4

2013/4/21 Adrian Klaver > On 04/21/2013 06:37 AM, Daniel Cristian Cruz wrote: > >> 2013/4/21 Tom Lane mailto:t...@sss.pgh.pa.us>> >> >> >> Tomas Vondra mailto:t...@fuzzy.cz>> writes: >> > I do have a log with the memory context info printed after the OOM >> > killed the session - se

Re: [GENERAL] Memory usage after upgrade to 9.2.4

On 21.4.2013 15:14, Tom Lane wrote: > Tomas Vondra writes: >> I do have a log with the memory context info printed after the OOM >> killed the session - see it attached. > > The only thing that seems rather bloated is the CacheMemoryContext, > which seems to be because the backend has cached info

Re: [GENERAL] Memory usage after upgrade to 9.2.4

On 04/21/2013 06:37 AM, Daniel Cristian Cruz wrote: 2013/4/21 Tom Lane mailto:t...@sss.pgh.pa.us>> Tomas Vondra mailto:t...@fuzzy.cz>> writes: > I do have a log with the memory context info printed after the OOM > killed the session - see it attached. The only thing that seems

Re: [GENERAL] Memory usage after upgrade to 9.2.4

On 04/20/2013 05:19 PM, Daniel Cristian Cruz wrote: Copying to list to fill in blanks. 2013/4/20 Adrian Klaver mailto:adrian.kla...@gmail.com>> On 04/20/2013 04:30 PM, Daniel Cristian Cruz wrote: 2013/4/20 Adrian Klaver mailto:adrian.kla...@gmail.com>

Re: [GENERAL] Memory usage after upgrade to 9.2.4

2013/4/21 Tom Lane > Tomas Vondra writes: > > I do have a log with the memory context info printed after the OOM > > killed the session - see it attached. > > The only thing that seems rather bloated is the CacheMemoryContext, > which seems to be because the backend has cached info about several

Re: [GENERAL] Memory usage after upgrade to 9.2.4

Tomas Vondra writes: > I do have a log with the memory context info printed after the OOM > killed the session - see it attached. The only thing that seems rather bloated is the CacheMemoryContext, which seems to be because the backend has cached info about several thousand tables and indexes. G

Re: [GENERAL] Memory usage after upgrade to 9.2.4

Hi, we got a report of (probably) the same issue on a local mailing list. Maybe it'll help in finding the root cause, so I'm resending the info here too. On 21.4.2013 01:19, Adrian Klaver wrote: > On 04/20/2013 04:08 PM, Daniel Cristian Cruz wrote: >> I think I didn't make it clear: the session m

Re: [GENERAL] Memory usage after upgrade to 9.2.4

On 04/20/2013 04:30 PM, Daniel Cristian Cruz wrote: 2013/4/20 Adrian Klaver mailto:adrian.kla...@gmail.com>> On 04/20/2013 04:08 PM, Daniel Cristian Cruz wrote: I think I didn't make it clear: the session memory usage is growing up too fast, until all server memor

Re: [GENERAL] Memory usage after upgrade to 9.2.4

2013/4/20 Adrian Klaver > On 04/20/2013 04:08 PM, Daniel Cristian Cruz wrote: > >> I think I didn't make it clear: the session memory usage is growing up >> too fast, until all server memory got used and swap occurs. >> >> Never saw something like that. The version is under a test enviroment >> f

Re: [GENERAL] Memory usage after upgrade to 9.2.4

On 04/20/2013 04:08 PM, Daniel Cristian Cruz wrote: I think I didn't make it clear: the session memory usage is growing up too fast, until all server memory got used and swap occurs. Never saw something like that. The version is under a test enviroment for a long time... Thanks if someone could

Re: [GENERAL] Memory usage after upgrade to 9.2.4

I think I didn't make it clear: the session memory usage is growing up too fast, until all server memory got used and swap occurs. Never saw something like that. The version is under a test enviroment for a long time... Thanks if someone could help me. 2013/4/20 Daniel Cristian Cruz > Hi all,

[GENERAL] Memory usage after upgrade to 9.2.4

Hi all, I've upgrade from 9.1.4 to 9.2.4, and got some very weird issue. My server got connections with RES (column from top utility) with too much memory: top - 19:50:58 up 384 days, 23:55, 2 users, load average: 4.28, 6.51, 7.68 Tasks: 417 total, 1 running, 416 sleeping, 0 stopped, 0 z

Re: [GENERAL] Memory usage and configuration settings

On Mon, Mar 5, 2012 at 6:37 AM, Mike C wrote: > Hi, > > I have been using table 17-2, Postgres Shared Memory Usage > (http://www.postgresql.org/docs/9.1/interactive/kernel-resources.html) > to calculate approximately how much memory the server will use. I'm > using Postgres 9.1 on a Linux 2.6 (RHE

Re: [GENERAL] Memory usage and configuration settings

Mike C writes: > Ok, that makes sense. With regards to work_mem, am I right in thinking > the child processes only allocate enough memory to meet the task at > hand, rather than the full 16M specified in the config file? They only allocate what's needed ... but you have to keep in mind that work_

Re: [GENERAL] Memory usage and configuration settings

On Mon, Mar 5, 2012 at 4:04 PM, Tom Lane wrote: > Mike C writes: >> I have been using table 17-2, Postgres Shared Memory Usage >> (http://www.postgresql.org/docs/9.1/interactive/kernel-resources.html) >> to calculate approximately how much memory the server will use. I'm >> using Postgres 9.1 on

Re: [GENERAL] Memory usage and configuration settings

Mike C writes: > I have been using table 17-2, Postgres Shared Memory Usage > (http://www.postgresql.org/docs/9.1/interactive/kernel-resources.html) > to calculate approximately how much memory the server will use. I'm > using Postgres 9.1 on a Linux 2.6 (RHEL 6) 64bit system, with 8GB RAM. > Data

[GENERAL] Memory usage and configuration settings

Hi, I have been using table 17-2, Postgres Shared Memory Usage (http://www.postgresql.org/docs/9.1/interactive/kernel-resources.html) to calculate approximately how much memory the server will use. I'm using Postgres 9.1 on a Linux 2.6 (RHEL 6) 64bit system, with 8GB RAM. Database is approximately

Re: [GENERAL] Memory Usage and OpenBSD

Jeff Ross wrote: Hopefully if I can get it to run well under pgbench the same setup will work well with drupal. The site I was worried about when I went to this bigger server has started a little slower than originally projected so the old server is handling the load. The standard TPC-B-like

Re: [GENERAL] Memory Usage and OpenBSD

Greg Smith wrote: Jeff Ross wrote: I think I'm doing it right. Here's the whole script. I run it from another server on the lan. That looks basically sane--your description was wrong, not your program, which is always better than the other way around. Note that everything your script is d

Re: [GENERAL] Memory Usage and OpenBSD

Jeff Ross wrote: I think I'm doing it right. Here's the whole script. I run it from another server on the lan. That looks basically sane--your description was wrong, not your program, which is always better than the other way around. Note that everything your script is doing and way more i

Re: [GENERAL] Memory Usage and OpenBSD

Greg Smith wrote: Jeff Ross wrote: pgbench is run with this: pgbench -h varley.openvistas.net -U _postgresql -t 2 -c $SCALE pgbench with scale starting at 10 and then incrementing by 10. I call it three times for each scale. I've turned on logging to 'all' to try and help figure out whe

Re: [GENERAL] Memory Usage and OpenBSD

Jeff Ross wrote: pgbench is run with this: pgbench -h varley.openvistas.net -U _postgresql -t 2 -c $SCALE pgbench with scale starting at 10 and then incrementing by 10. I call it three times for each scale. I've turned on logging to 'all' to try and help figure out where the system panics

Re: [GENERAL] Memory Usage and OpenBSD

Tom Lane wrote: Martijn van Oosterhout writes: On Tue, Feb 09, 2010 at 08:19:51PM +0500, Anton Maksimenkov wrote: Can anybody briefly explain me how one postgres process allocate memory for it needs? There's no real maximum, as it depends on the exact usage. However, in ge

Re: [GENERAL] Memory Usage and OpenBSD

Martijn van Oosterhout writes: > On Tue, Feb 09, 2010 at 08:19:51PM +0500, Anton Maksimenkov wrote: >> Can anybody briefly explain me how one postgres process allocate >> memory for it needs? > There's no real maximum, as it depends on the exact usage. However, in > general postgres tries to keep

Re: [GENERAL] Memory Usage and OpenBSD

2010/2/10 Martijn van Oosterhout : >> Can anybody briefly explain me how one postgres process allocate >> memory for it needs? > > There's no real maximum, as it depends on the exact usage. However, in > general postgres tries to keep below the values in work_mem and > maintainence_workmem. Most of

Re: [GENERAL] Memory Usage and OpenBSD

On Tue, Feb 09, 2010 at 08:19:51PM +0500, Anton Maksimenkov wrote: > It means that on openbsd i386 we have about 2,2G of virtual space for > malloc, shm*. So, postgres will use that space. > > But mmap() use random addresses. So when you get big chunk of memory > for shared buffers (say, 2G) then

Re: [GENERAL] Memory Usage and OpenBSD

2010/2/9 Scott Marlowe : > On Tue, Feb 9, 2010 at 3:18 AM, Anton Maksimenkov wrote: >>> Isn't the usual advice here is to log the ulimit setting from the pg >>> startup script so you can what it really is for the user at the moment >> I think that "su" is enough: > In previous discussions it was m

Re: [GENERAL] Memory Usage and OpenBSD

On Tue, Feb 9, 2010 at 3:18 AM, Anton Maksimenkov wrote: > 2010/1/28 Scott Marlowe : >>> related to maximum per-process data space.  I don't know BSD very well >>> so I can't say if datasize is the only such value for BSD, but it'd be >>> worth checking.  (Hmm, on OS X which is at least partly BSD

Re: [GENERAL] Memory Usage and OpenBSD

2010/1/28 Scott Marlowe : >> related to maximum per-process data space.  I don't know BSD very well >> so I can't say if datasize is the only such value for BSD, but it'd be >> worth checking.  (Hmm, on OS X which is at least partly BSDish, I see >> -m and -v in addition to -d, so I'm suspicious Op

Re: [GENERAL] Memory Usage and OpenBSD

On Wed, Jan 27, 2010 at 4:42 PM, Tom Lane wrote: > related to maximum per-process data space.  I don't know BSD very well > so I can't say if datasize is the only such value for BSD, but it'd be > worth checking.  (Hmm, on OS X which is at least partly BSDish, I see > -m and -v in addition to -d,

Re: [GENERAL] Memory Usage and OpenBSD

Jeff Ross writes: > Tom Lane wrote: >> Better look at the "ulimit" values the postmaster is started with; > OpenBSD makes a _postgresql user on install and it is in the daemon class > with > the following values: > daemon:\ > :ignorenologin:\ > :datasize=infinity:\ >

Re: [GENERAL] Memory Usage and OpenBSD

Tom Lane wrote: Jeff Ross writes: Running a simple select only pgbench test against it will fail with an out of memory error as it tries to vacuum --analyze the newly created database with 750 tuples. Better look at the "ulimit" values the postmaster is started with; you shouldn't be get

Re: [GENERAL] Memory Usage and OpenBSD

Jeff Ross writes: > Running a simple select only pgbench test against it will fail with an out of > memory error as it tries to vacuum --analyze the newly created database with > 750 tuples. Better look at the "ulimit" values the postmaster is started with; you shouldn't be getting that out-

[GENERAL] Memory Usage and OpenBSD

I'm not getting something about the best way to set up a server using PostgreSQL as a backend for a busy web server running drupal. The postgresql performance folks http://wiki.postgresql.org/wiki/Tuning_Your_PostgreSQL_Server say that in a server with more that 1GB of ram "a reasonable sta

Re: [GENERAL] memory usage of group by select

On Tue, Dec 29, 2009 at 4:09 PM, Alvaro Herrera wrote: > It's expecting 85k distinct groups. If that's not accurate, then > HashAggregate would use more memory than expected. Great diagnosis. There are actually about 76 million distinct groups. > See if you can make it work by setting enable

Re: [GENERAL] memory usage of group by select

Alvaro Herrera writes: > It's expecting 85k distinct groups. If that's not accurate, then > HashAggregate would use more memory than expected. See if you can make > it work by setting enable_hashagg = off. > If that works, good -- the real solution is different. Maybe you need > to ANALYZE mor

Re: [GENERAL] memory usage of group by select

Anthony wrote: > On Tue, Dec 29, 2009 at 3:41 PM, Anthony wrote: > > > I'm running a group by query on a table with over a billion rows and my > > memory usage is seemingly growing without bounds. Eventually the mem usage > > exceeds my physical memory and everything starts swapping. > > > > I

Re: [GENERAL] memory usage of group by select

On Tue, Dec 29, 2009 at 3:41 PM, Anthony wrote: > I'm running a group by query on a table with over a billion rows and my > memory usage is seemingly growing without bounds. Eventually the mem usage > exceeds my physical memory and everything starts swapping. > I guess I didn't ask my question.

[GENERAL] memory usage of group by select

Hi all, I'm running a group by query on a table with over a billion rows and my memory usage is seemingly growing without bounds. Eventually the mem usage exceeds my physical memory and everything starts swapping. Here is what I gather to be the relevant info: My machine has 768 megs of ram. s

[GENERAL] Memory usage of COPY command

When loading (inserting) data into a table with COPY I have read in the documentation that rows are appended to the end of the table instead of being added to existing table pages, so I'm wondering about memory utilization. Our application uses a number of COPY statements in parallel, so COPY perf

Re: [GENERAL] Memory usage during vacuum

I had thought that I had dropped and reloaded this table but apparently I hadn't and I had set the statistics target for one column to 500 while experimenting. Resetting it to -1 and running with a default of 300 gets ~ 70 megs memory footprint during the analyze now. Thanks Tom for indulging my

Re: [GENERAL] Memory usage during vacuum

Shelby Cain <[EMAIL PROTECTED]> writes: > It still decided to sample 15 rows. Am I missing > something obvious here? Shouldn't fewer rows be > sampled when I set the collection target to 1? The sample size is 300 rows times the largest per-column analysis target, where default_statistics_tar

Re: [GENERAL] Memory usage during vacuum

Currently my default is 300 (yes - very large I know) but overriding default_statistics_target with a value of 1 and re-running vacuum analyze on the same large table results in no change in maximum memory consumption during the process that I can see. It should be noted that I see this behavior o

Re: [GENERAL] Memory usage during vacuum

Shelby Cain <[EMAIL PROTECTED]> writes: > I apologize for my original post being unclear. I'm > running "vacuum analyze" and seeing the behavior > mentioned. Does specifying the analyze option imply > "vacuum full"? No; I just figured you were probably using FULL without saying so. However ...

Re: [GENERAL] Memory usage during vacuum

-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Thursday 25 March 2004 09:12 am, Shelby Cain wrote: > I agree in principle that the solution is to run on a > server with more memory instead of my local > development box. However, I'm not going to be able to > simply request that additional memor

Re: [GENERAL] Memory usage during vacuum

I agree in principle that the solution is to run on a server with more memory instead of my local development box. However, I'm not going to be able to simply request that additional memory be installed as these are "standard" boxes that IT distributes to employees. Regardless, I'm more curious a

Re: [GENERAL] Memory usage during vacuum

I apologize for my original post being unclear. I'm running "vacuum analyze" and seeing the behavior mentioned. Does specifying the analyze option imply "vacuum full"? On a hunch I just ran analyze and the backend's memory usage soared up to 100+ megs. I suspect that means it isn't the vacuum

Re: [GENERAL] Memory usage during vacuum

-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 How about plugging in more memory ? 40MB seems a bit low for a database server footprint - well, certainly depends on what you do. But if your machine starts swapping with an extra 40 MB of memory consumption I'd say the machine is undersized for th

Re: [GENERAL] Memory usage

On Wed, Feb 25, 2004 at 02:10:56PM -0700, Rick Gigger wrote: > I want to know how much memory I've got free on my system. > > The free command gives me something like this: > > total used free sharedbuffers cached > Mem: 20648322046196 18636

[GENERAL] Memory usage

I want to know how much memory I've got free on my system. The free command gives me something like this: total used free sharedbuffers cached Mem: 20648322046196 18636 0 1468921736968 -/+ buffers/cache: 1623361902496 Sw

[GENERAL] memory usage and cache query

My process in php submit about 1000 querys (in a while) like : "INSERT INTO table SELECT id from table2 where ..." This type of query for me does'nt require any memory in the php process (all to postgres process) is use pg_exec and pg_freeresult just after each query call. But when this script ru

Re: [GENERAL] Memory Usage

"Nathan Barnett" <[EMAIL PROTECTED]> writes: > I am having a small issue with PostgreSQL 7.0.3 on FreeBSD 4.2 Stable. When > I perform an update on a table with roughly 2 million rows, the postgres > process starts eating up memory until it eventually uses up all of the > memory and exits without

[GENERAL] Memory Usage

I am having a small issue with PostgreSQL 7.0.3 on FreeBSD 4.2 Stable. When I perform an update on a table with roughly 2 million rows, the postgres process starts eating up memory until it eventually uses up all of the memory and exits without finishing. I have also seen the same thing happen i