Re: [HACKERS] Stats collection on Windows

2006-04-05 Thread Peter Brant
I'm going to rip out the destroy code and see how it goes. Patch will be forthcoming if things turn out well. Pete >>> Tom Lane <[EMAIL PROTECTED]> 04/05/06 4:49 pm >>> So I'm sort of thinking that the destroy delay has outlived its usefulness. regards, tom lane --

Re: [HACKERS] Stats collection on Windows

2006-04-05 Thread mark
On Wed, Apr 05, 2006 at 10:30:36AM -0400, Tom Lane wrote: > "Magnus Hagander" <[EMAIL PROTECTED]> writes: > >> What happens if process Y goes away between the time you > >> obtain a handle for it and the time you try to run this > >> DuplicateHandle call? > > I can put together some quick test-co

Re: [HACKERS] Stats collection on Windows

2006-04-05 Thread mark
On Wed, Apr 05, 2006 at 10:05:56AM -0400, Tom Lane wrote: > What happens if process Y goes away between the time you obtain a > handle for it and the time you try to run this DuplicateHandle call? Think of Windows HANDLE like UNIX fd, but Windows HANDLE works for everything - not just sockets, fil

Re: [HACKERS] Stats collection on Windows

2006-04-05 Thread Tom Lane
"Magnus Hagander" <[EMAIL PROTECTED]> writes: > After reviewing my own patch I just sent in (stats write delay), I > realised I had just upped that from 10 seconds to 5 minutes. Which is > then even longer than forever :) > Which prompted me to consider suggesting just this - do we really need > th

Re: [HACKERS] Stats collection on Windows

2006-04-05 Thread Magnus Hagander
> > Search for "is known to be dead, ignoring" to find the > re-used process > > IDs. Things start out clean, but after a few cycles > anywhere between > > 1 and 5 backends are being missed. > > Looking at the pgstats code, I notice that once it makes an > entry in the dead-backends hashtabl

Re: [HACKERS] Stats collection on Windows

2006-04-05 Thread Tom Lane
"Peter Brant" <[EMAIL PROTECTED]> writes: > I added some strategic printfs to pgstat.c. Attached is the output when > a little program is run which, in a loop, makes 10 connections, sleeps 3 > seconds, closes them, sleeps another 3 seconds. My workstation (Windows > XP) was otherwise idle. > Sea

Re: [HACKERS] Stats collection on Windows

2006-04-05 Thread Magnus Hagander
> There's a disconnect here. handles aren't process > identifiers: they're reference counted "pointers" to the > kernel structures for the process. If you are holding a > handle (ie: from CreateProcess or OpenProcess) that handle > cannot and will not be reclaimed until you call CloseHandle >

Re: [HACKERS] Stats collection on Windows

2006-04-05 Thread Magnus Hagander
> >> What happens if process Y goes away between the time you obtain a > >> handle for it and the time you try to run this > DuplicateHandle call? > > > I can put together some quick test-code for this if you need me to? > > Nah, it was just a rhetorical question meant to poke a hole > in the

Re: [HACKERS] Stats collection on Windows

2006-04-05 Thread stephen joseph butler
2006/4/5, Tom Lane <[EMAIL PROTECTED]>: AFAICS, don't-reuse-PIDs-too-quick has exact analogs that Windows hasto solve by ensuring it doesn't reuse HANDLEs too quick.There's a disconnect here. handles aren't process identifiers: they're reference counted "pointers" to the kernel structures for the p

Re: [HACKERS] Stats collection on Windows

2006-04-05 Thread Tom Lane
"Magnus Hagander" <[EMAIL PROTECTED]> writes: >> What happens if process Y goes away between the time you >> obtain a handle for it and the time you try to run this >> DuplicateHandle call? > I can put together some quick test-code for this if you need me to? Nah, it was just a rhetorical quest

Re: [HACKERS] Stats collection on Windows

2006-04-05 Thread stephen joseph butler
2006/4/5, Tom Lane <[EMAIL PROTECTED]>: What happens if process Y goes away between the time you obtain ahandle for it and the time you try to run this DuplicateHandle call?The structures for a process should remain as long as there are any open HANDLEs, even if the process has ended. Threads behav

Re: [HACKERS] Stats collection on Windows

2006-04-05 Thread Magnus Hagander
> >> HANDLE is process local? That is worse then, because then > there's no > >> guarentee that each process will see a different identifier. > > > HANDLE is process local. What you need to do is run > DuplicateHandle() > > on it specifying it should "also be valid for process Y" (for which >

Re: [HACKERS] Stats collection on Windows

2006-04-05 Thread Tom Lane
"Magnus Hagander" <[EMAIL PROTECTED]> writes: >> HANDLE is process local? That is worse then, because then >> there's no guarentee that each process will see a different >> identifier. > HANDLE is process local. What you need to do is run DuplicateHandle() on > it specifying it should "also be v

Re: [HACKERS] Stats collection on Windows

2006-04-05 Thread Magnus Hagander
> This is a little bit of a distraction though, as any system > that requires the user to be able to kill broken backends, is > only indicative of a broken backend. We're talking about how > to deal with a broken process, after the process owner > (PostgreSQL) has forgotten about it. Don't con

Re: [HACKERS] Stats collection on Windows

2006-04-05 Thread Magnus Hagander
> > > Yes, they provide the "kill" primitive, but only as a > compatibility > > > measure. > > They do? Where is it? Certainly not in the documented SDK > that I can > > see. > > I believe it is called KillProcess(). No such function documented on my MSDN, other than one in SQL Server DMO. Ar

Re: [HACKERS] Stats collection on Windows

2006-04-05 Thread Magnus Hagander
> > It's no different from a file descriptor on UNIX. > > > > Neither UNIX nor Windows promise that a process identifier is valid > > beyond the life of the process. UNIX avoids it from > happening, as it > > is necessary to avoid races with system calls such as > kill(). Windows > > does not

Re: [HACKERS] Stats collection on Windows

2006-04-05 Thread mark
On Wed, Apr 05, 2006 at 12:20:49PM +0200, Martijn van Oosterhout wrote: > On Wed, Apr 05, 2006 at 06:03:31AM -0400, [EMAIL PROTECTED] wrote: > > It's no different from a file descriptor on UNIX. > > Neither UNIX nor Windows promise that a process identifier is valid > > beyond the life of the proce

Re: [HACKERS] Stats collection on Windows

2006-04-05 Thread Martijn van Oosterhout
On Wed, Apr 05, 2006 at 06:03:31AM -0400, [EMAIL PROTECTED] wrote: > It's no different from a file descriptor on UNIX. > > Neither UNIX nor Windows promise that a process identifier is valid > beyond the life of the process. UNIX avoids it from happening, as it > is necessary to avoid races with s

Re: [HACKERS] Stats collection on Windows

2006-04-05 Thread mark
On Wed, Apr 05, 2006 at 10:31:37AM +0200, Magnus Hagander wrote: > > TerminateProcess takes a HANDLE, not a process identifier. > Yes. You get the handle by doing OpenProcess() with PROCESS_TERMINATE > access. > The functions used to enumerate processes all return the process id, not > a HANDLE.(S

Re: [HACKERS] Stats collection on Windows

2006-04-05 Thread mark
On Wed, Apr 05, 2006 at 09:58:54AM +0200, Martijn van Oosterhout wrote: > On Wed, Apr 05, 2006 at 03:38:28AM -0400, [EMAIL PROTECTED] wrote: > > Once upon a time, when I played with this stuff (I mostly use UNIX, not > > Windows), I concluded to myself that HANDLE was process-local, and that > > it

Re: [HACKERS] Stats collection on Windows

2006-04-05 Thread Magnus Hagander
> > >> Redmond crowd should be able to figure out that > recycling process > > >> IDs instantly would be a stupid idea...) > > > Can you explain more of this? IMHO, if we rely on feature > like this, > > > the difference is unstable-every-day vs. unstable-every-year. > > The mere existence of t

Re: [HACKERS] Stats collection on Windows

2006-04-05 Thread Magnus Hagander
> > Once upon a time, when I played with this stuff (I mostly use UNIX, > > not Windows), I concluded to myself that HANDLE was > process-local, and > > that it was allocated. Meaning - it won't be re-used until > you CloseHandle(). > > It's best then, to think of HANDLE as a opaque object. >

Re: [HACKERS] Stats collection on Windows

2006-04-05 Thread Martijn van Oosterhout
On Wed, Apr 05, 2006 at 03:38:28AM -0400, [EMAIL PROTECTED] wrote: > Once upon a time, when I played with this stuff (I mostly use UNIX, not > Windows), I concluded to myself that HANDLE was process-local, and that > it was allocated. Meaning - it won't be re-used until you CloseHandle(). > It's be

Re: [HACKERS] Stats collection on Windows

2006-04-05 Thread mark
On Wed, Apr 05, 2006 at 09:30:06AM +0200, Martijn van Oosterhout wrote: > On Wed, Apr 05, 2006 at 03:20:47AM -0400, [EMAIL PROTECTED] wrote: > > TerminateProcess takes a HANDLE, not a process identifier. Yes, they > > provide the "kill" primitive, but only as a compatibility measure. A > > "good"

Re: [HACKERS] Stats collection on Windows

2006-04-05 Thread Martijn van Oosterhout
On Wed, Apr 05, 2006 at 03:20:47AM -0400, [EMAIL PROTECTED] wrote: > TerminateProcess takes a HANDLE, not a process identifier. Yes, they > provide the "kill" primitive, but only as a compatibility measure. A > "good" Windows process, should maintain a HANDLE to the process, and > kill the process

Re: [HACKERS] Stats collection on Windows

2006-04-05 Thread mark
On Tue, Apr 04, 2006 at 11:17:49PM -0400, Tom Lane wrote: > "Qingqing Zhou" <[EMAIL PROTECTED]> writes: > > "Tom Lane" <[EMAIL PROTECTED]> wrote > >> Redmond crowd should be able to figure out that recycling process IDs > >> instantly would be a stupid idea...) > > Can you explain more of this? IMH

Re: [HACKERS] Stats collection on Windows

2006-04-05 Thread mark
On Tue, Apr 04, 2006 at 11:02:11PM -0400, Tom Lane wrote: > "Peter Brant" <[EMAIL PROTECTED]> writes: > > I think I've found the cause (or one of the causes) why stats > > collection is unreliable on Windows and I'm wondering about the best way > > to go about fixing it. > > The problem is that pro

Re: [HACKERS] Stats collection on Windows

2006-04-04 Thread Magnus Hagander
> >> Redmond crowd should be able to figure out that recycling > process IDs > >> instantly would be a stupid idea...) > > > Can you explain more of this? IMHO, if we rely on feature > like this, > > the difference is unstable-every-day vs. unstable-every-year. > > The mere existence of the k

Re: [HACKERS] Stats collection on Windows

2006-04-04 Thread Tom Lane
"Qingqing Zhou" <[EMAIL PROTECTED]> writes: > "Tom Lane" <[EMAIL PROTECTED]> wrote >> Redmond crowd should be able to figure out that recycling process IDs >> instantly would be a stupid idea...) > Can you explain more of this? IMHO, if we rely on feature like this, the > difference is unstable-e

Re: [HACKERS] Stats collection on Windows

2006-04-04 Thread Qingqing Zhou
"Tom Lane" <[EMAIL PROTECTED]> wrote > > Redmond crowd should be able to figure out that recycling process IDs > instantly would be a stupid idea...) > Can you explain more of this? IMHO, if we rely on feature like this, the difference is unstable-every-day vs. unstable-every-year. Regards, Qin

Re: [HACKERS] Stats collection on Windows

2006-04-04 Thread Qingqing Zhou
"Tom Lane" <[EMAIL PROTECTED]> wrote > >> The problem is that process IDs on Windows seem to be assigned without >> much rhyme or reason and it seems to happen relatively frequently that a >> new process will be assigned the same process ID as a process which >> recently died. > > That's an intere

Re: [HACKERS] Stats collection on Windows

2006-04-04 Thread Tom Lane
"Peter Brant" <[EMAIL PROTECTED]> writes: > I think I've found the cause (or one of the causes) why stats > collection is unreliable on Windows and I'm wondering about the best way > to go about fixing it. > The problem is that process IDs on Windows seem to be assigned without > much rhyme or rea

[HACKERS] Stats collection on Windows

2006-04-04 Thread Peter Brant
Hi all, I think I've found the cause (or one of the causes) why stats collection is unreliable on Windows and I'm wondering about the best way to go about fixing it. The problem is that process IDs on Windows seem to be assigned without much rhyme or reason and it seems to happen relatively frequ