Re: [PERFORM] go for a script! / ex: PostgreSQL vs. MySQL

2003-10-14 Thread Sean Chittenden
> >> This would be parameters such as the block size and a few > >> other compile time parameters. If we can get to some of these > >> read-only parameters than that would make this step easier, > >> certainly for the new recruits amongst us. > > > > Actually, from my perspective, we shouldn't

Re: [PERFORM] go for a script! / ex: PostgreSQL vs. MySQL

2003-10-14 Thread Tom Lane
Josh Berkus <[EMAIL PROTECTED]> writes: > Unless there's a way to find it in the compiled source? See pg_controldata. regards, tom lane ---(end of broadcast)--- TIP 8: explain analyze is your friend

Re: [PERFORM] go for a script! / ex: PostgreSQL vs. MySQL

2003-10-14 Thread Josh Berkus
Ivan, > There was some talk, either on this list or freebsd-performance about > setting the default block size for PostgreSQL running on FreeBSD to be 16k > because of performance reasons. That is: *default* for the port, user is > not asked. So an automagical method to scale non-default block siz

Re: [PERFORM] go for a script! / ex: PostgreSQL vs. MySQL

2003-10-14 Thread Ivan Voras
> Date: Sun, 12 Oct 2003 13:30:45 -0700 > From: Josh Berkus <[EMAIL PROTECTED]> > To: Nick Barr <[EMAIL PROTECTED]> > Cc: [EMAIL PROTECTED] > Subject: Re: go for a script! / ex: PostgreSQL vs. MySQL > Message-ID: <[EMAIL PROTECTED]> > >> This would be parameters such as the block size and a f

Re: [PERFORM] go for a script! / ex: PostgreSQL vs. MySQL

2003-10-13 Thread Christopher Kings-Lynne
Yes, indeed, sysctl can tweak these values fairly adequately. Now, numbers of semaphors are not as readily tweaked; I wound up limited, the other day, when I tried setting values for... kern.ipc.semmns kern.ipc.semmni Same. Maybe that was the option I was thinking was read-only: houston# sy

Re: [PERFORM] go for a script! / ex: PostgreSQL vs. MySQL

2003-10-13 Thread Christopher Kings-Lynne
If shmmax and shmmall are too small, then: PostgreSQL requires some more shared memory to cache some tables, x Mb, do you want to increase your OS kernel parameters? Tweak shmmax and shmmall CK> Note that this still requires a kernel recompile on FreeBSD :( According to whom? sysctl is your fri

Re: [PERFORM] go for a script! / ex: PostgreSQL vs. MySQL

2003-10-13 Thread Christopher Browne
Centuries ago, Nostradamus foresaw when [EMAIL PROTECTED] (Vivek Khera) would write: >> "JB" == Josh Berkus <[EMAIL PROTECTED]> writes: > > JB> Chris, >>> > PostgreSQL requires some more shared memory to cache some tables, x Mb, >>> > do you want to increase your OS kernel parameters? >>> > >>>

Re: [PERFORM] go for a script! / ex: PostgreSQL vs. MySQL

2003-10-13 Thread Tom Lane
Sean Chittenden <[EMAIL PROTECTED]> writes: > Every few months one of the uber core BSD hackers threatens to rewrite > that part of PG because high up in the BSD camp, it's common belief > that shm* is a source of performance loss for PostgreSQL. They're full of it. RAM is RAM, no? Once you've

Re: [PERFORM] go for a script! / ex: PostgreSQL vs. MySQL

2003-10-13 Thread Vivek Khera
> "CK" == Christopher Kings-Lynne <[EMAIL PROTECTED]> writes: >> If shmmax and shmmall are too small, then: >> PostgreSQL requires some more shared memory to cache some tables, x >> Mb, do you want to increase your OS kernel parameters? >> Tweak shmmax and shmmall CK> Note that this still req

Re: [PERFORM] go for a script! / ex: PostgreSQL vs. MySQL

2003-10-13 Thread Vivek Khera
> "JB" == Josh Berkus <[EMAIL PROTECTED]> writes: JB> Chris, >> > PostgreSQL requires some more shared memory to cache some tables, x Mb, >> > do you want to increase your OS kernel parameters? >> > >> >Tweak shmmax and shmmall >> >> Note that this still requires a kernel recompile on Fre

Re: [PERFORM] go for a script! / ex: PostgreSQL vs. MySQL

2003-10-13 Thread Sean Chittenden
> > > PostgreSQL requires some more shared memory to cache some > > > tables, x Mb, do you want to increase your OS kernel parameters? > > > > > >Tweak shmmax and shmmall > > > > Note that this still requires a kernel recompile on FreeBSD :( > > Not our fault, now is it? This would mean that

Re: [PERFORM] go for a script! / ex: PostgreSQL vs. MySQL

2003-10-13 Thread Sean Chittenden
> >> echo "effective_cache_size = $((`sysctl -n vfs.hibufspace` / 8192))" > >> > >> I've used it for my dedicated servers. Is this calculation correct? > > SC> Yes, or it's real close at least. vfs.hibufspace is the amount > of SC> kernel space that's used for caching IO operations (minus the >

Re: [PERFORM] go for a script! / ex: PostgreSQL vs. MySQL

2003-10-13 Thread Shridhar Daithankar
On Monday 13 October 2003 19:34, Vivek Khera wrote: > > "SC" == Sean Chittenden <[EMAIL PROTECTED]> writes: > >> > >> echo "effective_cache_size = $((`sysctl -n vfs.hibufspace` / 8192))" > >> > >> I've used it for my dedicated servers. Is this calculation correct? > > SC> Yes, or it's real clo

Re: [PERFORM] go for a script! / ex: PostgreSQL vs. MySQL

2003-10-13 Thread Vivek Khera
> "SC" == Sean Chittenden <[EMAIL PROTECTED]> writes: >> echo "effective_cache_size = $((`sysctl -n vfs.hibufspace` / 8192))" >> >> I've used it for my dedicated servers. Is this calculation correct? SC> Yes, or it's real close at least. vfs.hibufspace is the amount of SC> kernel space tha

Re: [PERFORM] go for a script! / ex: PostgreSQL vs. MySQL

2003-10-12 Thread pginfo
Hi, Josh Berkus wrote: > Nick, > > > I reckon do a system scan first, and parse the current PostgreSQL conf > > file to figure out what the settings are. Also back it up with a date > > and time appended to the end to make sure there is a backup before > > overwriting the real conf file. Then a b

Re: [PERFORM] go for a script! / ex: PostgreSQL vs. MySQL

2003-10-12 Thread Josh Berkus
Chris, > > PostgreSQL requires some more shared memory to cache some tables, x Mb, > > do you want to increase your OS kernel parameters? > > > >Tweak shmmax and shmmall > > Note that this still requires a kernel recompile on FreeBSD :( Not our fault, now is it? This would mean that we woul

Re: [PERFORM] go for a script! / ex: PostgreSQL vs. MySQL

2003-10-12 Thread Josh Berkus
Nick, > I reckon do a system scan first, and parse the current PostgreSQL conf > file to figure out what the settings are. Also back it up with a date > and time appended to the end to make sure there is a backup before > overwriting the real conf file. Then a bunch of questions. What sort of > qu

Re: [PERFORM] go for a script! / ex: PostgreSQL vs. MySQL

2003-10-11 Thread Christopher Kings-Lynne
If shmmax and shmmall are too small, then: PostgreSQL requires some more shared memory to cache some tables, x Mb, do you want to increase your OS kernel parameters? Tweak shmmax and shmmall Note that this still requires a kernel recompile on FreeBSD :( Chris ---(en

Re: [PERFORM] go for a script! / ex: PostgreSQL vs. MySQL

2003-10-11 Thread Nick Barr
Josh Berkus wrote: shared_buffers = 1/16th of total memory effective_cache_size = 80% of the supposed kernel cache. But only if it's a dedicated DB machine. If it's not, all memory values should be cut in half. What I would prefer would be an interactive script which would, by asking th

Re: [PERFORM] go for a script! / ex: PostgreSQL vs. MySQL

2003-10-11 Thread Ron Johnson
On Sat, 2003-10-11 at 05:22, Harald Fuchs wrote: > In article <[EMAIL PROTECTED]>, > Rod Taylor <[EMAIL PROTECTED]> writes: > > >> Would anyone object to a patch that exports the blocksize via a > >> readonly GUC? Too many tunables are page dependant, which is > >> infuriating when copying config

Re: [PERFORM] go for a script! / ex: PostgreSQL vs. MySQL

2003-10-11 Thread Sean Chittenden
> >NB> shared_buffers = 1/16th of total memory > >NB> effective_cache_size = 80% of the supposed kernel cache. > > I think Sean(?) mentioned this one for FreeBSD (Bash code): sh, not bash. :) > echo "effective_cache_size = $((`sysctl -n vfs.hibufspace` / 8192))" > > I've used it for my dedicat

Re: [PERFORM] go for a script! / ex: PostgreSQL vs. MySQL

2003-10-10 Thread Christopher Kings-Lynne
NB> shared_buffers = 1/16th of total memory NB> effective_cache_size = 80% of the supposed kernel cache. I think Sean(?) mentioned this one for FreeBSD (Bash code): echo "effective_cache_size = $((`sysctl -n vfs.hibufspace` / 8192))" I've used it for my dedicated servers. Is this calculation cor

Re: [PERFORM] go for a script! / ex: PostgreSQL vs. MySQL

2003-10-10 Thread Rod Taylor
On Fri, 2003-10-10 at 18:59, Sean Chittenden wrote: > > NB> So far: > > > > NB> shared_buffers = 1/16th of total memory > > NB> effective_cache_size = 80% of the supposed kernel cache. > > > > Please take into account the blocksize compiled into PG, too... > > Would anyone object to a patch that

Re: [PERFORM] go for a script! / ex: PostgreSQL vs. MySQL

2003-10-10 Thread Bruce Momjian
Sean Chittenden wrote: > > NB> So far: > > > > NB> shared_buffers = 1/16th of total memory > > NB> effective_cache_size = 80% of the supposed kernel cache. > > > > Please take into account the blocksize compiled into PG, too... > > Would anyone object to a patch that exports the blocksize via a

Re: [PERFORM] go for a script! / ex: PostgreSQL vs. MySQL

2003-10-10 Thread Sean Chittenden
> NB> So far: > > NB> shared_buffers = 1/16th of total memory > NB> effective_cache_size = 80% of the supposed kernel cache. > > Please take into account the blocksize compiled into PG, too... Would anyone object to a patch that exports the blocksize via a readonly GUC? Too many tunables are pa

Re: [PERFORM] go for a script! / ex: PostgreSQL vs. MySQL

2003-10-10 Thread Josh Berkus
Vivek, > when you compute optimal shared buffers and effective cache size, > these are in terms of blocksize. so if I have 16k block size, you > can't compute based on default 8k blocksize. at worst, it would have > to be a parameter you pass to the tuning script. Oh, yes! Thank you. --

Re: [PERFORM] go for a script! / ex: PostgreSQL vs. MySQL

2003-10-10 Thread Vivek Khera
> "JB" == Josh Berkus <[EMAIL PROTECTED]> writes: JB> Vivek, NB> shared_buffers = 1/16th of total memory NB> effective_cache_size = 80% of the supposed kernel cache. >> >> Please take into account the blocksize compiled into PG, too... JB> We can;t change the blocksize in a script that only

Re: [PERFORM] go for a script! / ex: PostgreSQL vs. MySQL

2003-10-10 Thread Josh Berkus
Vivek, > NB> shared_buffers = 1/16th of total memory > NB> effective_cache_size = 80% of the supposed kernel cache. > > Please take into account the blocksize compiled into PG, too... We can;t change the blocksize in a script that only does the .conf file. Or are you suggesting something else?

Re: [PERFORM] go for a script! / ex: PostgreSQL vs. MySQL

2003-10-10 Thread Josh Berkus
Nick, > Sounds good to me. I will carry on working on it but I would definitely > need some help, or at least a list of parameters to tweak, and some > recomended values based on data about the puter in question. > shared_buffers = 1/16th of total memory > effective_cache_size = 80% of the suppos

Re: [PERFORM] go for a script! / ex: PostgreSQL vs. MySQL

2003-10-10 Thread Vivek Khera
> "NB" == Nick Barr <[EMAIL PROTECTED]> writes: NB> So far: NB> shared_buffers = 1/16th of total memory NB> effective_cache_size = 80% of the supposed kernel cache. Please take into account the blocksize compiled into PG, too... -- =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=

Re: [PERFORM] go for a script! / ex: PostgreSQL vs. MySQL

2003-10-10 Thread Vivek Khera
> "SC" == Sean Chittenden <[EMAIL PROTECTED]> writes: SC> patches to extract info from their OS so that initdb can make useful SC> decisions. Or, lastly, does anyone think that this should be in a SC> different, external program? -sc Well, there should definitely be a way to run a "get curr

Re: [PERFORM] go for a script! / ex: PostgreSQL vs. MySQL

2003-10-10 Thread Nick Barr
Josh Berkus wrote: Nick, Having been following the thread on "go for a script! / ex: PostgreSQL vs. MySQL". I thought I would throw something together in Perl. Cool! Would you be willing to work with me so that I can inject some of my knowledge of .conf tuning? Sounds good to me.

Re: [PERFORM] go for a script! / ex: PostgreSQL vs. MySQL

2003-10-10 Thread Josh Berkus
Nick, > Having been following the thread on "go for a script! / ex: PostgreSQL vs. > MySQL". I thought I would throw something together in Perl. Cool! Would you be willing to work with me so that I can inject some of my knowledge of .conf tuning? -- Josh Berkus Aglio Database Solutions San

Re: [PERFORM] go for a script! / ex: PostgreSQL vs. MySQL

2003-10-10 Thread Nick Barr
- Original Message - From: "Nick Barr" <[EMAIL PROTECTED]> To: <[EMAIL PROTECTED]> Sent: Friday, October 10, 2003 1:35 PM Subject: go for a script! / ex: PostgreSQL vs. MySQL > I will also post it on me website and as I develop it further new versions > will appear there > > http://www.c

[PERFORM] go for a script! / ex: PostgreSQL vs. MySQL

2003-10-10 Thread Nick Barr
Heya Guys n Gals, Having been following the thread on "go for a script! / ex: PostgreSQL vs. MySQL". I thought I would throw something together in Perl. My current issue is that I only have access to a RH Linux box and so cannot make it cross-platform on my own :-(. Anyhow please find it attached.

Re: [PERFORM] go for a script! / ex: PostgreSQL vs. MySQL

2003-10-09 Thread Sean Chittenden
> Yeah, I had similar thought to Oliver's and suspected that this > would be the answer. Also, while it's not too hard to do this for a > single platform, it gets complecated once you start looking at > different ones. > > Josh, let me know when you're ready to do this. I'll try to help, > althou

Re: [PERFORM] go for a script! / ex: PostgreSQL vs. MySQL

2003-10-09 Thread Dror Matalon
Yeah, I had similar thought to Oliver's and suspected that this would be the answer. Also, while it's not too hard to do this for a single platform, it gets complecated once you start looking at different ones. Josh, let me know when you're ready to do this. I'll try to help, although my perl's

Re: [PERFORM] go for a script! / ex: PostgreSQL vs. MySQL

2003-10-09 Thread Josh Berkus
Oliver, > I think instead of thinking about where to put the > information about tuning, someone should provide a > "pgsql-autotune". Maybe even a shell script would do the > trick. Well, you see, there's the issue. "I think someone." Lots of people have spoken in favor of an "auto-conf" scrip

Re: [PERFORM] go for a script! / ex: PostgreSQL vs. MySQL

2003-10-09 Thread Paul Thomas
On 09/10/2003 09:29 Oliver Scheit wrote: Hi guys, I followed the discussion and here are my 0.2$: I think instead of thinking about where to put the information about tuning, someone should provide a "pgsql-autotune". Maybe even a shell script would do the trick. It's not so hard to find out, how

[PERFORM] go for a script! / ex: PostgreSQL vs. MySQL

2003-10-09 Thread Oliver Scheit
Hi guys, I followed the discussion and here are my 0.2$: I think instead of thinking about where to put the information about tuning, someone should provide a "pgsql-autotune". Maybe even a shell script would do the trick. It's not so hard to find out, how much memory is installed, and IMHO SHAR