Re: [HACKERS] Stats Collector Error 7.4beta1 and 7.4beta2

2003-09-10 Thread Bruno Wolff III
On Wed, Sep 10, 2003 at 12:49:31 -0400, Tom Lane <[EMAIL PROTECTED]> wrote: > > The question is whether any of this is worth worrying about in PG. > ISTM the correct solution to such a risk is to tighten your kernel's > packet filtering, not harden one piece of one application. On linux at leas

Re: [HACKERS] Stats Collector Error 7.4beta1 and 7.4beta2

2003-09-10 Thread Tom Lane
Alvaro Herrera <[EMAIL PROTECTED]> writes: > On Wed, Sep 10, 2003 at 07:27:02AM -0400, Andrew Dunstan wrote: >> If someone can spoof the packet address isn't there also a possibility >> that they can read your packets and see your random signature? > Spoofing the packet source address is not quite

Re: [HACKERS] Stats Collector Error 7.4beta1 and 7.4beta2

2003-09-10 Thread Bruce Momjian
Jan Wieck wrote: > > For IPv6 you could do it with a memcmp on the sin6_addr part, and > > put it inside an #ifdef HAVE_IPV6. > > > > If you want to write code to compare 2 addresses, please make it > > a general function and place it in ip.c. > > > > Anyway, I'm happy with the current use of rec

Re: [HACKERS] Stats Collector Error 7.4beta1 and 7.4beta2

2003-09-10 Thread Alvaro Herrera
On Wed, Sep 10, 2003 at 07:27:02AM -0400, Andrew Dunstan wrote: > If someone can spoof the packet address isn't there also a possibility > that they can read your packets and see your random signature? Spoofing the packet source address is not quite the same as sniffing a connection, which should

Re: [HACKERS] Stats Collector Error 7.4beta1 and 7.4beta2

2003-09-10 Thread Andrew Dunstan
Tom Lane said: > Jan Wieck <[EMAIL PROTECTED]> writes: >> So either we do the random signature thing, which I would favor as a >> one time be all, end all solution - or you do the actual from-address >> based implementation by restoring the old IPV4 behaviour and adding >> correct IPV6 behaviour

Re: [HACKERS] Stats Collector Error 7.4beta1 and 7.4beta2

2003-09-09 Thread Tom Lane
Jan Wieck <[EMAIL PROTECTED]> writes: > So either we do the random signature thing, which I would favor as a one > time be all, end all solution - or you do the actual from-address based > implementation by restoring the old IPV4 behaviour and adding correct > IPV6 behaviour. My feeling at this

Re: [HACKERS] Stats Collector Error 7.4beta1 and 7.4beta2

2003-09-09 Thread Jan Wieck
Kurt Roeckx wrote: On Tue, Sep 09, 2003 at 02:10:20AM -0700, Kevin Brown wrote: > I could go for Jan's idea of putting a random key into the messages, > if anyone feels that we should not trust to the kernel to enforce the > packet source address restriction. But the memcmp() test seems a clear

Re: [HACKERS] Stats Collector Error 7.4beta1 and 7.4beta2

2003-09-09 Thread Kurt Roeckx
On Tue, Sep 09, 2003 at 02:10:20AM -0700, Kevin Brown wrote: > > I could go for Jan's idea of putting a random key into the messages, > > if anyone feels that we should not trust to the kernel to enforce the > > packet source address restriction. But the memcmp() test seems a clear > > loser given

Re: [HACKERS] Stats Collector Error 7.4beta1 and 7.4beta2

2003-09-09 Thread Kevin Brown
Tom Lane wrote: > <[EMAIL PROTECTED]> writes: > > This analysis makes sense - I think using memcmp is clearly wrong here. > > Yeah, now that I think about it, we're betting on the kernel to > faithfully zero all unused bits in addrinfo structures. In an ideal > world, all kernels would do that, b

Re: [HACKERS] Stats Collector Error 7.4beta1 and 7.4beta2

2003-09-05 Thread Tom Lane
<[EMAIL PROTECTED]> writes: > This analysis makes sense - I think using memcmp is clearly wrong here. Yeah, now that I think about it, we're betting on the kernel to faithfully zero all unused bits in addrinfo structures. In an ideal world, all kernels would do that, but in the real world it seem

Re: [HACKERS] Stats Collector Error 7.4beta1 and 7.4beta2

2003-09-05 Thread andrew
This analysis makes sense - I think using memcmp is clearly wrong here. cheers andrew Jan Wieck said: > On a second thought, > > I still believe that this is just garbage in the padding bytes after > the IPV4 address. The code currently bind()'s and connect()'s > explicitly to an AF_INET addr

Re: [HACKERS] Stats Collector Error 7.4beta1 and 7.4beta2

2003-09-05 Thread Jan Wieck
Tom Lane wrote: I was about to say "I give up, let's just take out the comparison". Your point is interesting but easily avoided; if we aren't going to check fromaddr anymore then there's no need to use recvfrom(), it could as well be recv() and save the kernel a few cycles. Which then get's us

Re: [HACKERS] Stats Collector Error 7.4beta1 and 7.4beta2

2003-09-05 Thread Jan Wieck
On a second thought, I still believe that this is just garbage in the padding bytes after the IPV4 address. The code currently bind()'s and connect()'s explicitly to an AF_INET address. So all we ever should see is something from and AF_INET address. Everything else in the sin_family has to be

Re: [HACKERS] Stats Collector Error 7.4beta1 and 7.4beta2

2003-09-05 Thread Tom Lane
Jan Wieck <[EMAIL PROTECTED]> writes: > I still believe that this is just garbage in the padding bytes after the > IPV4 address. Yes. I have been looking at the behavior on my own Linux box, on which it turns out stats are broken too in CVS tip. It is very clear that getsockname() is returning

Re: [HACKERS] Stats Collector Error 7.4beta1 and 7.4beta2

2003-09-05 Thread Tom Lane
Jan Wieck <[EMAIL PROTECTED]> writes: > Tom Lane wrote: >> I was about to say "I give up, let's just take out the comparison". > Which then get's us back to your concern about assuming that HPUX and > Linux manpages can be taken as "every platform will" and hope all > kernels will limit the send

Re: [HACKERS] Stats Collector Error 7.4beta1 and 7.4beta2

2003-09-05 Thread Bruce Momjian
Tom Lane wrote: > Jan Wieck <[EMAIL PROTECTED]> writes: > > Tom Lane wrote: > >> What I'm wondering about is whether we are comparing the right number of > >> bytes ... have both address structs been reported to have the same > >> length? Maybe we need a min(). > > > I disagree. If getsockname(),

Re: [HACKERS] Stats Collector Error 7.4beta1 and 7.4beta2

2003-09-05 Thread Tom Lane
Jan Wieck <[EMAIL PROTECTED]> writes: > Tom Lane wrote: >> What I'm wondering about is whether we are comparing the right number of >> bytes ... have both address structs been reported to have the same >> length? Maybe we need a min(). > I disagree. If getsockname(), getpeername() or recvfrom() r

Re: [HACKERS] Stats Collector Error 7.4beta1 and 7.4beta2

2003-09-05 Thread Kurt Roeckx
On Fri, Sep 05, 2003 at 09:35:11AM -0400, Jan Wieck wrote: > Redhat 7.1 says > >The file descriptor sockfd must refer to a socket. If the >socket is of type SOCK_DGRAM then the serv_addr address is >the address to which datagrams are sent by default, and >the o

Re: [HACKERS] Stats Collector Error 7.4beta1 and 7.4beta2

2003-09-05 Thread Jan Wieck
Tom Lane wrote: Jan Wieck <[EMAIL PROTECTED]> writes: Redhat 7.1 says The file descriptor sockfd must refer to a socket. If the socket is of type SOCK_DGRAM then the serv_addr address is the address to which datagrams are sent by default, and the only addres

Re: [HACKERS] Stats Collector Error 7.4beta1 and 7.4beta2

2003-09-05 Thread Tom Lane
Jan Wieck <[EMAIL PROTECTED]> writes: > Redhat 7.1 says > The file descriptor sockfd must refer to a socket. If the > socket is of type SOCK_DGRAM then the serv_addr address is > the address to which datagrams are sent by default, and > the only address from w

Re: [HACKERS] Stats Collector Error 7.4beta1 and 7.4beta2

2003-09-05 Thread Bruce Momjian
Jan Wieck wrote: > Redhat 7.1 says > > The file descriptor sockfd must refer to a socket. If the > socket is of type SOCK_DGRAM then the serv_addr address is > the address to which datagrams are sent by default, and > the only address from which datagrams are

Re: [HACKERS] Stats Collector Error 7.4beta1 and 7.4beta2

2003-09-05 Thread Jan Wieck
Redhat 7.1 says The file descriptor sockfd must refer to a socket. If the socket is of type SOCK_DGRAM then the serv_addr address is the address to which datagrams are sent by default, and the only address from which datagrams are received. If Looks like the tes

Re: [HACKERS] Stats Collector Error 7.4beta1 and 7.4beta2

2003-09-04 Thread Tom Lane
Kurt Roeckx <[EMAIL PROTECTED]> writes: > Then it connects to itself. I don't get the logic behind that > howver. At least on HPUX, the connect(2) man page saith If the socket is of type SOCK_DGRAM, connect() specifies the peer address to which messages are to be sent, and the call ret

Re: [HACKERS] Stats Collector Error 7.4beta1 and 7.4beta2

2003-09-04 Thread Tom Lane
Adam Kavan <[EMAIL PROTECTED]> writes: > I don't really know the layout of these structures so I dumped them to a > file and attached them. The first 16 bytes is from fromaddr and the second > is from pgStatAddr. More legibly: 000 0200 8016 7f00 0001 010 0200 8016 7

Re: [HACKERS] Stats Collector Error 7.4beta1 and 7.4beta2

2003-09-04 Thread Jan Wieck
They are both structures of type sockaddr_in (sin_family 2 is AF_INET whereas sin_family 10 would've been AF_INET6), and all relevant fields of the structure look the same to me. The problem lies in the padding bytes that make sockaddr_in the same size as sockaddr. Since the static structure pg

Re: [HACKERS] Stats Collector Error 7.4beta1 and 7.4beta2

2003-09-04 Thread Jan Wieck
Kurt Roeckx wrote: On Thu, Sep 04, 2003 at 05:01:54PM -0400, Jan Wieck wrote: Kurt Roeckx wrote: >It could be useful to have a warning at the following line: > >if (memcmp(&fromaddr, &pgStatAddr, fromlen)) >continue; > >That way you can r

Re: [HACKERS] Stats Collector Error 7.4beta1 and 7.4beta2

2003-09-04 Thread Adam Kavan
At 06:49 PM 9/4/03 -0400, Tom Lane wrote: Hmm. Could you look and see what the actual values are in each address? regards, tom lane I don't really know the layout of these structures so I dumped them to a file and attached them. The first 16 bytes is from fromaddr and th

Re: [HACKERS] Stats Collector Error 7.4beta1 and 7.4beta2

2003-09-04 Thread Tom Lane
Adam Kavan <[EMAIL PROTECTED]> writes: >> if (memcmp(&fromaddr, &pgStatAddr, fromlen)) >> continue; > This is the very line that is giving me problems. I commented it out and > recompiled and now the stats system works. Of course I have to assume that > its bad to go around with out that check

Re: [HACKERS] Stats Collector Error 7.4beta1 and 7.4beta2

2003-09-04 Thread Adam Kavan
It could be useful to have a warning at the following line: if (memcmp(&fromaddr, &pgStatAddr, fromlen)) continue; That way you can rule out that that is a problem. Anyway, I still didn't see the error message he got in the first place. Ma

Re: [HACKERS] Stats Collector Error 7.4beta1 and 7.4beta2

2003-09-04 Thread Kurt Roeckx
On Thu, Sep 04, 2003 at 04:04:38PM -0400, Jan Wieck wrote: > > And I agree with Tom that it is very likely that the IPV4/IPV6 stuff is > the reason. IIRC the postmaster creates the socket and noone ever does > bind(2) on it - so it uses it's dynamically assigned port number. Both, > the collect

Re: [HACKERS] Stats Collector Error 7.4beta1 and 7.4beta2

2003-09-04 Thread Kurt Roeckx
On Thu, Sep 04, 2003 at 05:01:54PM -0400, Jan Wieck wrote: > > > Kurt Roeckx wrote: > > > >It could be useful to have a warning at the following line: > > > >if (memcmp(&fromaddr, &pgStatAddr, fromlen)) > >continue; > > > >That way you can

Re: [HACKERS] Stats Collector Error 7.4beta1 and 7.4beta2

2003-09-04 Thread Jan Wieck
Kurt Roeckx wrote: It could be useful to have a warning at the following line: if (memcmp(&fromaddr, &pgStatAddr, fromlen)) continue; That way you can rule out that that is a problem. Anyway, I still didn't see the error message he got i

Re: [HACKERS] Stats Collector Error 7.4beta1 and 7.4beta2

2003-09-04 Thread Jan Wieck
Kurt Roeckx wrote: On Thu, Sep 04, 2003 at 01:39:04AM -0400, Tom Lane wrote: Bruce Momjian <[EMAIL PROTECTED]> writes: > Doesn't the stats collector use unix domain sockets, not IP? No. IIRC, we deliberately chose IP/UDP because it had buffering behavior we liked. Once you said it was because n

Re: [HACKERS] Stats Collector Error 7.4beta1 and 7.4beta2

2003-09-04 Thread Kurt Roeckx
On Thu, Sep 04, 2003 at 01:39:04AM -0400, Tom Lane wrote: > Bruce Momjian <[EMAIL PROTECTED]> writes: > > Doesn't the stats collector use unix domain sockets, not IP? > > No. IIRC, we deliberately chose IP/UDP because it had buffering > behavior we liked. Once you said it was because not all pla

Re: [HACKERS] Stats Collector Error 7.4beta1 and 7.4beta2

2003-09-04 Thread Adam Kavan
I thing I haven't seen asked: is there a packet filter blocking local<->local UDP traffic by any chance? Iptables is set to accept everything. If it would help I can give you all log in information to poke around yourselves. I appreciate your help. --- Adam Kavan --- [EMAIL PROTECTED] --

Re: [HACKERS] Stats Collector Error 7.4beta1 and 7.4beta2

2003-09-04 Thread Gavin Sherry
On Thu, 4 Sep 2003, Bruce Momjian wrote: > Tom Lane wrote: > > "Matthew T. O'Connor" <[EMAIL PROTECTED]> writes: > > > ... Initially I saw an error in the logs about an IPv6 address > > > error but after I recompiled everthing with a simple ./configure > > > --prefix=/home/user/somethingelse/ I di

Re: [HACKERS] Stats Collector Error 7.4beta1 and 7.4beta2

2003-09-03 Thread Tom Lane
Bruce Momjian <[EMAIL PROTECTED]> writes: > Doesn't the stats collector use unix domain sockets, not IP? No. IIRC, we deliberately chose IP/UDP because it had buffering behavior we liked. There are pipes involved in the stats stuff too, but the weak link in my mind is the backend-to-stats-buffer

Re: [HACKERS] Stats Collector Error 7.4beta1 and 7.4beta2

2003-09-03 Thread Bruce Momjian
Tom Lane wrote: > "Matthew T. O'Connor" <[EMAIL PROTECTED]> writes: > > ... Initially I saw an error in the logs about an IPv6 address > > error but after I recompiled everthing with a simple ./configure > > --prefix=/home/user/somethingelse/ I didn't get the IPv6 error in the > > logs anymore. >

Re: [HACKERS] Stats Collector Error 7.4beta1 and 7.4beta2

2003-09-03 Thread Matthew T. O'Connor
On Thu, 2003-09-04 at 01:23, Tom Lane wrote: > Hm. Could it be an IPv6 issue --- that is, the stats collector is alive > and faithfully listening on some UDP port, but it's not the same port > the backends try to send to? Given the discussion over the past couple > of days about bizarre interpret

Re: [HACKERS] Stats Collector Error 7.4beta1 and 7.4beta2

2003-09-03 Thread Tom Lane
"Matthew T. O'Connor" <[EMAIL PROTECTED]> writes: > ... Initially I saw an error in the logs about an IPv6 address > error but after I recompiled everthing with a simple ./configure > --prefix=/home/user/somethingelse/ I didn't get the IPv6 error in the > logs anymore. Hm. Could it be an IPv6 iss

Re: [HACKERS] Stats Collector Error 7.4beta1 and 7.4beta2

2003-09-03 Thread Matthew T. O'Connor
On Wed, 2003-09-03 at 23:50, Tom Lane wrote: > Does 'ps' show that the stats collector and stats buffer postmaster > child processes are alive? Are there any suggestive complaints in > the postmaster's log? As Adam mentioned, I took a look at his system since the initial report was about a proble

Re: [HACKERS] Stats Collector Error 7.4beta1 and 7.4beta2

2003-09-03 Thread Tom Lane
Adam Kavan <[EMAIL PROTECTED]> writes: > I was attempting to get pg_autovacuum to work on my database and after > much hammering at it I discovered that the stats system was not > working. Does 'ps' show that the stats collector and stats buffer postmaster child processes are alive? Are there any