Re: [GENERAL] GSS Authentication

2010-06-12 Thread Bryan Montgomery
I've been trying this as well off and on. In my case I'm not convinced the
AD configuration is correct (And someone else manages that).

Can you use kinit with the key tab options to get a good response from the
server? I think I should be able to do this ..
$ kinit -V -k -t poe3b.keytab HTTP/poe3b.lab2k.net
kinit(v5): Preauthentication failed while getting initial credentials

I'd be interested to know if you get something different - and the steps you
went through on the AD side.

Bryan.

On Fri, Jun 11, 2010 at 5:51 PM,  wrote:

>  I'm trying to get my PostgreSQL server on Linux configured so that I can
> connect from a Windows client using GSS Authentication against Active
> Directory.  I found some helpful references on how to do this, but I'm still
> coming up short.  To summarize what I've done so far by way of
> configuration:
>
> 1) On the Linux server, setup my krb5.conf file such that I can get a
> ticket from AD using kinit and confirm using klist.
> 2) Setup a new account in AD and used ktpass to create a keytab file for
> the SPN.
> 3) Copied the keytab file onto my postgres server and updated my
> postgresql.conf file appropriately (set the krb_server_keyfile to point to
> the file I just created.)
>
> Then I wrote a little test Perl program to connect to my postgres
> database.
>
> use DBI;
> use strict;
>
> my $dbh = 
> DBI->connect('DBI:Pg:dbname=postgres;host=host.domain.com;krbsrvname=POSTGRES')
> or die DBI->errstr;
>
> When I try to run the Perl program I get this error:
>
> DBI connect('dbname=postgres;host=host.domain.com;krbsrvname=POSTGRES')
> failed: FATAL:  accepting GSS security context failed
> DETAIL:  Miscellaneous failure: Unknown code ggss 3 at g.pl line 4
> FATAL:  accepting GSS security context failed
> DETAIL:  Miscellaneous failure: Unknown code ggss 3 at g.pl line 4
>
> I then ramped up the debug logging on the postgres side and get this off
> the server:
>
> 2010-06-11 17:23:49 EDTDEBUG:  0: Processing received GSS token of
> length 2119
> 2010-06-11 17:23:49 EDTLOCATION:  pg_GSS_recvauth, auth.c:965
> 2010-06-11 17:23:49 EDTDEBUG:  0: gss_accept_sec_context major: 851968,
> minor: -2045022973, outlen: 0, outflags: 7f
> 2010-06-11 17:23:49 EDTLOCATION:  pg_GSS_recvauth, auth.c:984
> 2010-06-11 17:23:49 EDTFATAL:  XX000: accepting GSS security context failed
> 2010-06-11 17:23:49 EDTDETAIL:  Miscellaneous failure: Unknown code ggss 3
> 2010-06-11 17:23:49 EDTLOCATION:  pg_GSS_error, auth.c:866
>
> I'm using PostgreSQL 8.4.4 on Enterprise Linux 4.
>
> Can anyone offer any suggestions?
>
> Thanks in advance.
> Greig
>


Re: [GENERAL] Cognitive dissonance

2010-06-12 Thread John Gage

A one file html version would be a godsend.


On Jun 12, 2010, at 3:20 AM, Bruce Momjian wrote:


Robert Gravsjö wrote:
I am for #1, not so much for #2, mainly on the grounds of size.   
But
given #1 it would be possible for packagers to make their own  
choices

about whether to include plain-text docs.


Wouldn't it suffice to make it downloadable, like the pdf doc?


And/or make the HTML version downloadable side by side with the PDF.


That might be easy to do.  We already build the HTML, and requiring
people to recursively use wget is not user-friendly.

--
 Bruce Momjian  http://momjian.us
 EnterpriseDB http://enterprisedb.com

 + None of us is going to be here forever. +

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general



--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] Cognitive dissonance

2010-06-12 Thread Peter Eisentraut
On lör, 2010-06-12 at 11:18 +0200, John Gage wrote:
> A one file html version would be a godsend.

I've committed a build target for that now.  Use 'make postgres.html' in
doc/src/sgml/.



-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] GSS Authentication

2010-06-12 Thread Stephen Frost
Bryan,

* Bryan Montgomery (mo...@english.net) wrote:
> On Sat, Jun 12, 2010 at 12:58 AM, Stephen Frost  wrote:
> Can you elaborate on the DNS requirements? How would I check the reverse
> DNS? I assume just pinging both server by hostname?

Kerberos depends on reverse DNS.  Reverse DNS is IP Address -> DNS Name
(rather than forward/regular DNS, which is Name -> IP).  Specifically,
when a Kerberos client connects to a server, it will take the IP address
of the host it connected to and try to find the name, it will then use
*that* name to determine what ticket to request from the KDC.

Realm: EXAMPLE.COM
Client system: client.example.com
Client IP 10.10.10.1

Server system: server.example.com
Server IP: 10.10.10.20

Client connects to server and looks up "10.10.10.20" to find out the
server's name is "server.example.com", it will then ask the KDC for
a "postgres/server.example@example.com" ticket.  This allows the
server to have other aliases (eg: database.example.com) and for the
client to use that alias to connect to, but then only need 1 principal
(the server.example.com) in the KDC.

Thanks,

Stephen


signature.asc
Description: Digital signature


Re: [GENERAL] GSS Authentication

2010-06-12 Thread Stephen Frost
* Bryan Montgomery (mo...@english.net) wrote:
> I've been trying this as well off and on. In my case I'm not convinced the
> AD configuration is correct (And someone else manages that).

Yeah, that can be a challenge..  but it's *definitely* possible to get
it set up and working correctly.

> Can you use kinit with the key tab options to get a good response from the
> server? I think I should be able to do this ..
> $ kinit -V -k -t poe3b.keytab HTTP/poe3b.lab2k.net
> kinit(v5): Preauthentication failed while getting initial credentials

err, I'm not sure that should be expected to work.

What does klist -ek  return?  Also, you should be able to
kinit to *your* princ in the AD, and if you can do that, you should be
able to use your princ to request the service princ ticket from the KDC
by doing kinit -S HTTP/poe3b.lab2k.net your.princ

Also, provided your *client* is set up/configured correctly, you should
be able to see that it acquires the ticket (by using klist) when you try
to connect to the server, even if the server is misconfigured.

> I'd be interested to know if you get something different - and the steps you
> went through on the AD side.

You have to create an account in Active Directory for the PG service and
then use:

ktpass /princ POSTGRES/myserver.mydomain@mydomain.com /mapuser
postg...@mydomain.com /pass mypass /crypto AES256-SHA1 /ptype
KRB5_NT_PRINCIPAL /out krb5.keytab

Then copy that krb5.keytab to the server.  Note that you then have to
adjust the server config to have service name set to POSTGRES, and
adjust clients using the environment variables to indiciate they should
ask for POSTGRES (instead of the postgres default).

Thanks,

Stephen


signature.asc
Description: Digital signature


Re: [GENERAL] Cognitive dissonance

2010-06-12 Thread Tom Lane
Peter Eisentraut  writes:
> On lör, 2010-06-12 at 11:18 +0200, John Gage wrote:
>> A one file html version would be a godsend.

> I've committed a build target for that now.  Use 'make postgres.html' in
> doc/src/sgml/.

Huh, is that actually worth anything?  How many browsers will open it
without crashing, or will navigate the page with decent performance
if they do manage to open it?

(Not that I object to providing this Make target.  But I thought the
discussion was about plain-text output.)

regards, tom lane

-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] Cognitive dissonance

2010-06-12 Thread John Gage
It was.  But if the compromise is single file html, that is a vast  
improvement over the current system imho.


What I want is the thing that is maximally amenable to being searched  
conveniently using all the tools at our disposal especially regular  
expressons.  The point has been made that Google is the best system to  
search for Postgres documentation/knowledge/etc.  Frankly, I don't  
necessarily agree with that, particularly for the novice.  The  
documentation is where it is at, and it is the documentation that is  
referenced the most in these posts.


But there are no dichotomies here.  It is not either or.  It is a  
balance between what is easiest to produce and maintain and what is  
most productive to use.


And the background for my request is my respect for the extraordinary  
power and elegance of postgres.


John


On Jun 12, 2010, at 3:10 PM, Tom Lane wrote:


Peter Eisentraut  writes:

On lör, 2010-06-12 at 11:18 +0200, John Gage wrote:

A one file html version would be a godsend.


I've committed a build target for that now.  Use 'make  
postgres.html' in

doc/src/sgml/.


Huh, is that actually worth anything?  How many browsers will open it
without crashing, or will navigate the page with decent performance
if they do manage to open it?

(Not that I object to providing this Make target.  But I thought the
discussion was about plain-text output.)

regards, tom lane

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general



--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] Cognitive dissonance

2010-06-12 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote:
> Peter Eisentraut  writes:
> > I've committed a build target for that now.  Use 'make postgres.html' in
> > doc/src/sgml/.
> 
> Huh, is that actually worth anything?  How many browsers will open it
> without crashing, or will navigate the page with decent performance
> if they do manage to open it?

If it works with links, that'd probably work well for the use case
described...

Stephen


signature.asc
Description: Digital signature


Re: [GENERAL] GSS Authentication

2010-06-12 Thread Bryan Montgomery
Hi Steven,
Thanks for the info here. In particular,

On Sat, Jun 12, 2010 at 12:58 AM, Stephen Frost  wrote:

> You may also need to make sure that your default realm is set correctly
> and that your reverse DNS is working.  Also, can you look in the PG
> server-side logs and see what errors are being reported there?  There
> may be some during startup or when the client tries to connect that
> would be useful.
>
>Thanks,
>
>Stephen
>
> Can you elaborate on the DNS requirements? How would I check the reverse
DNS? I assume just pinging both server by hostname?

Thanks - Bryan.


Re: [GENERAL] Cognitive dissonance

2010-06-12 Thread Peter Eisentraut
On lör, 2010-06-12 at 09:10 -0400, Tom Lane wrote:
> Peter Eisentraut  writes:
> > On lör, 2010-06-12 at 11:18 +0200, John Gage wrote:
> >> A one file html version would be a godsend.
> 
> > I've committed a build target for that now.  Use 'make postgres.html' in
> > doc/src/sgml/.
> 
> Huh, is that actually worth anything?  How many browsers will open it
> without crashing, or will navigate the page with decent performance
> if they do manage to open it?

Text output is generated by going through HTML.  I haven't figured out
the best way to do the second step yet.  We use lynx for INSTALL and
HISTORY, but the results for this big file aren't very clean.

Browsers seem to handle the file OK, btw.


-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] Cognitive dissonance

2010-06-12 Thread Bruce Momjian
Peter Eisentraut wrote:
> On l?r, 2010-06-12 at 09:10 -0400, Tom Lane wrote:
> > Peter Eisentraut  writes:
> > > On l?r, 2010-06-12 at 11:18 +0200, John Gage wrote:
> > >> A one file html version would be a godsend.
> > 
> > > I've committed a build target for that now.  Use 'make postgres.html' in
> > > doc/src/sgml/.
> > 
> > Huh, is that actually worth anything?  How many browsers will open it
> > without crashing, or will navigate the page with decent performance
> > if they do manage to open it?
> 
> Text output is generated by going through HTML.  I haven't figured out
> the best way to do the second step yet.  We use lynx for INSTALL and
> HISTORY, but the results for this big file aren't very clean.
> 
> Browsers seem to handle the file OK, btw.

Well, I tried lynx and the output looked fine to me, so I applied the
attached patch to allow single-page text output.  You can see the HTML
and text file results here:

http://momjian.us/expire/

The new rule name is postgres.txt.  The file size are:

7,789,730  postgres.html
5,155,672  postgres.txt

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

  + None of us is going to be here forever. +
Index: doc/src/sgml/Makefile
===
RCS file: /cvsroot/pgsql/doc/src/sgml/Makefile,v
retrieving revision 1.145
diff -c -c -r1.145 Makefile
*** doc/src/sgml/Makefile	12 Jun 2010 15:42:44 -	1.145
--- doc/src/sgml/Makefile	12 Jun 2010 15:57:31 -
***
*** 108,113 
--- 108,117 
  postgres.html: postgres.sgml $(ALLSGML) stylesheet.dsl
  	$(JADE.html.call) -V nochunks -V rootchunk -V '(define %root-filename% #f)' -V '(define use-output-dir #f)' -i include-index $<
  
+ # single-page text
+ postgres.txt: postgres.html
+ 	$(LYNX) -force_html -dump -nolist -stdin $< > $@
+ 
  HTML.index: postgres.sgml $(ALMOSTALLSGML) stylesheet.dsl
  	@$(MKDIR_P) html
  	$(JADE.html.call) -V html-index $<

-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


[GENERAL] 2010 eclipse community survey

2010-06-12 Thread Merlin Moncure
Primary Database:
Postgres 11%
Mysql 31.8%
Oracle 21.6%

I found the 2009 results:
Postgres 9.9%
Mysql 27.7%
Oracle 27.3%

Postgres beat out sql server.  Of course, eclipse is heavily skewed
towards java which is going to be very lightweight on the ms stack for
obvious reasons.

I think the main takeaways are:
*) open source solutions continuing to gain strength in the marketplace
*) oracle is the big loser (the word 'hemorrhaging' comes to mind)
*) mysql/postgres are the only open source players  ('ot...@5.4%)
*) mysql license issues/development issues have apparently not
impacted it in terms of developer mindshare
*) postgres is growing. 9.0 hs/sr I suspect will have a dramatic impact.

merlin

-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] Cognitive dissonance

2010-06-12 Thread John Gage

UFB!  This was definitely worth the visit from the Nebula.

Thanks very, very much.

Sensational.

Thanks again,

John Gage


On Jun 12, 2010, at 6:01 PM, Bruce Momjian wrote:


http://momjian.us/expire/

The new rule name is postgres.txt.  The file size are:

7,789,730  postgres.html
5,155,672  postgres.txt

--
 Bruce Momjian  http://momjian.us
 EnterpriseDB http://enterprisedb.com





Re: [GENERAL] Cognitive dissonance

2010-06-12 Thread Bruce Momjian
John Gage wrote:
> UFB!  This was definitely worth the visit from the Nebula.
> 
> Thanks very, very much.
> 
> Sensational.
> 
> Thanks again,
> 
> John Gage

We still have to decide how to make these accessible from our web site.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

  + None of us is going to be here forever. +

-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] Cognitive dissonance

2010-06-12 Thread Tim Landscheidt
Bruce Momjian  wrote:

> [...]
> + # single-page text
> + postgres.txt: postgres.html
> + $(LYNX) -force_html -dump -nolist -stdin $< > $@
  ^^
> +
> [...]

Isn't that unnecessary/wrong as the filename is supplied on
the command line?

Tim


-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] Cognitive dissonance

2010-06-12 Thread Bruce Momjian
Tim Landscheidt wrote:
> Bruce Momjian  wrote:
> 
> > [...]
> > + # single-page text
> > + postgres.txt: postgres.html
> > +   $(LYNX) -force_html -dump -nolist -stdin $< > $@
>   ^^
> > +
> > [...]
> 
> Isn't that unnecessary/wrong as the filename is supplied on
> the command line?

Ah, good catch. I wasn't able to test that because my lynx version
doesn't support -stdin.  Thanks, and updated.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

  + None of us is going to be here forever. +

-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] Re: Error on Windows server could not open relation base/xxx/xxx Permission denied

2010-06-12 Thread John T. Dow
I posted this two days ago and nobody has commented. I'm reposting the message 
because I really need advice. Background info: My client got the "permission 
denied" error on his original server and we removed the AV software without 
solving the problem. Since the computer is running Windows 2000 Server, SP4, it 
was suggested that a newer OS might help. We moved the data and the problem 
persists even on the new computer. What follows is my status report about the 
new computer as well as the process of copying the data. Note that backing up 
the database on the original server also gets the "permission denied" error.

Posting from 6/10 follows.

Here's the current status.

We installed postgres on an XP machine, 2002 SP3. (Same as my computer, which 
never has a problem.)

We tried to do a backup of the database on the old computer, to copy the data 
to the new computer. The backup failed with the same problem mentioned in the 
subject line.

So we zipped up the data directory and unzipped it on the XP computer.

We then attempted to paste in the "large" block of text (200 lines of plain 
ascii, 49000 bytes) and got the same problem as before.

Note that the load on the server and on postgres is very low, and that the 
problem can be recreated with 100% certainty when we paste certain text into 
certain fields.

This computer is running "Symantec Endpoint Protection", with the proactive 
threat feature turned off.

Question: Is it possible that there's corruption in the database which is being 
incorrectly reported as "Permission denied"?

Perhaps the original problem on the other computer created the corruption? Or 
the corruption came from another source and on both computers creates the 
incorrect message?

We could of course recreate pretty much the same database. We're in development 
mode now: it was loaded with data from the legacy system extracted a few months 
ago and since then there has been additional data entered and changed as people 
have played with and tested the application.

Is this a random event? A bug? Advice please on what to do next.

John





On Tue, 08 Jun 2010 08:37:02 -0400, John T. Dow wrote:

>On Tue, 08 Jun 2010 10:25:49 +0800, Craig Ringer wrote:
>
>>On 8/06/2010 9:11 AM, John T. Dow wrote:
>>> OP here
>>>
>>> We removed AVG from the computer and rebooted.
>>>
>>> Same problem.
>>
>>OK, good to know. Thanks very much for testing that, and my apologies 
>>for recommending something that didn't work out. Of course, it would 
>>have been hard to progress without eliminating that possible factor.
>>
>>> Could it be 2000 Server? SP4? I've seen reports of other problems that went 
>>> away depending on the version of Windows.
>>
>>Well, certainly I'd expect that Pg on Windows 2000 server gets about 
>>zero regular testing. Why would you deploy a server OS that's already 10 
>>years out of date, went EOL five years ago, and lost even the option of 
>>paid extended support this year?
>>
>>http://support.microsoft.com/lifecycle/?LN=en-au&x=14&y=11&p1=7274
>>
>
>
>
>
>
>Good point.
>
>It's not my server, it's my client's server, and I don't know the history of 
>it.
>
>They have mentioned another computer which runs XP I believe. It's dedicated 
>to a single task and could double as the database server, although I don't 
>think it has any RAID. I will suggest that we try installing Postgres on that 
>computer and see if the problem goes away. If so, they might choose to make 
>that their solution (perhaps adding another hard drive and a RAID controller). 
>The application, daily backups, and WAL files could all live on the original 
>server.
>
>If they go that route, we'd never know for certain what the original problem 
>was.
>
>I'll post back after anything is done.
>
>JOhn
>
>
>
>
>
>
>
>
>
>>It'd be interesting to investigate this issue ... but win2k server isn't 
>>exactly easy to come by. Anyone on the list got a win2k server (or 
>>license) around they can do some experimenting on? All I have here is 
>>NT4 (not kidding - legacy system) and Win2k8 plus the usual desktop 
>>suspects.
>>
>>--
>>Craig Ringer
>>
>>
>>-- 
>>Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
>>To make changes to your subscription:
>>http://www.postgresql.org/mailpref/pgsql-general
>
>
>
>-- 
>Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
>To make changes to your subscription:
>http://www.postgresql.org/mailpref/pgsql-general





-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] Re: Error on Windows server could not open relation base/xxx/xxx Permission denied

2010-06-12 Thread Adrian Klaver
On Saturday 12 June 2010 11:07:32 am John T. Dow wrote:
> I posted this two days ago and nobody has commented. I'm reposting the
> message because I really need advice. Background info: My client got the
> "permission denied" error on his original server and we removed the AV
> software without solving the problem. Since the computer is running Windows
> 2000 Server, SP4, it was suggested that a newer OS might help. We moved the
> data and the problem persists even on the new computer. What follows is my
> status report about the new computer as well as the process of copying the
> data. Note that backing up the database on the original server also gets
> the "permission denied" error.
>
> Posting from 6/10 follows.
>
> Here's the current status.
>
> We installed postgres on an XP machine, 2002 SP3. (Same as my computer,
> which never has a problem.)
>
> We tried to do a backup of the database on the old computer, to copy the
> data to the new computer. The backup failed with the same problem mentioned
> in the subject line.
>
> So we zipped up the data directory and unzipped it on the XP computer.
>
> We then attempted to paste in the "large" block of text (200 lines of plain
> ascii, 49000 bytes) and got the same problem as before.
>
> Note that the load on the server and on postgres is very low, and that the
> problem can be recreated with 100% certainty when we paste certain text
> into certain fields.
>
> This computer is running "Symantec Endpoint Protection", with the proactive
> threat feature turned off.
>
> Question: Is it possible that there's corruption in the database which is
> being incorrectly reported as "Permission denied"?
>
> Perhaps the original problem on the other computer created the corruption?
> Or the corruption came from another source and on both computers creates
> the incorrect message?
>
> We could of course recreate pretty much the same database. We're in
> development mode now: it was loaded with data from the legacy system
> extracted a few months ago and since then there has been additional data
> entered and changed as people have played with and tested the application.
>
> Is this a random event? A bug? Advice please on what to do next.
>
> John
>

Some more questions.
What is the relation that is having the permissions issue?
Are the permissions on that file different from the others in the base 
directory 
tree?
What is the 'certain text'?
What are the 'certain fields' and do they have any functions running on them?



-- 
Adrian Klaver
adrian.kla...@gmail.com

-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] Best way to store case-insensitive data?

2010-06-12 Thread Adrian von Bidder
On Friday 11 June 2010 09.27:15 Michal Politowski wrote:

[email address local part is case sensitive]

> In practice I've yet to see a system having both smith and Smith
> and them being different, but still it is theoretically posible.

I routinely modify email addresses I store to my addressbook to all 
lowercase.  I have yet to have a single case where this gets me into 
problems; I think it's probably quite exotic to find a system that actually 
is case sensitive.

cheers
-- vbi

-- 
featured product: ClamAV Antivirus - http://www.clamav.net/


signature.asc
Description: This is a digitally signed message part.


Re: [GENERAL] Best way to store case-insensitive data?

2010-06-12 Thread Scott Marlowe
2010/6/11 Mike Christensen :
> Yea this is a valid point.  It's very possible my design won't work
> for the long term, and at some point I'll have to store the email name
> exactly as it was entered, and allow the lookup logic to be case
> insensitive with a lowercase index.  However, I think the way I have
> it now should not break any known email server heh.

Instead of mangling data when you store it, mangle it later when you
retrieve it.  with a functional index on the column, you get the
comparison data stored in an index, ready to go.

Performance test the index:

create test_index on table (lower(fieldname));

versus storing the emails in lower case.

-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] Best way to store case-insensitive data?

2010-06-12 Thread Scott Marlowe
n Sat, Jun 12, 2010 at 3:21 PM, Scott Marlowe  wrote:
> Performance test the index:
>
> create test_index on table (lower(fieldname));
>
> versus storing the emails in lower case.

Some quick testing on that tells me that storing in lower case will be
about twice as fast at retrieval.  But we're talking things like 1ms
versus 2ms.

-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] Re: Error on Windows server could not open relation base/xxx/xxx Permission denied

2010-06-12 Thread Adrian Klaver
On Saturday 12 June 2010 12:59:18 pm John T. Dow wrote:

I am CC'ing the list so more eyes can follow this.

> >
> >Some more questions.
> >What is the relation that is having the permissions issue?
> >Are the permissions on that file different from the others in the base
> > directory tree?
> >What is the 'certain text'?
> >What are the 'certain fields' and do they have any functions running on
> > them?
> >
> >
> >
> >--
> >Adrian Klaver
>
> The actual message is: could not open relation base/16384/16642: Permission
> denied.
>
> I presume that the actual file names are not important as they change from
> time to time?



They may or may not depending on what type of relation they are and the type of 
operation done to them. I should have been more specific. What is 16642? 
SELECT relname,relkind from pg_class where relfilenode=16642 should provide an 
answer.

>
> No, we have not diddled with the permissions. All files inherit their
> permissions from "data".

The question is really not whether you changed the permissions, but whether 
they 
have been changed? Hate to be anal about this, but have you actually looked at 
the permissions for that file or are you assuming inheritance of permissions?

>
> The text I've been using for test purposes is a plain ascii version of the
> US Constitution. It's about 50K, nothing special about it. Just a plain
> text file I had on hand.

But large enough to invoke TOASTing the value. See here for more info:
http://www.postgresql.org/docs/8.4/interactive/storage-toast.html

>
> The fields where we noticed it at a couple jTextArea fields in a Java
> application. They are defined as varchar. There is nothing special about
> these fields. No functions, etc. They are fields to hold plain English
> text. Data is normally entered by someone typing it in. (One example is a
> police officer typing in his report of an arrest he's made. After he types
> it in, it's printed and signed and becomes a legal document.) There is a
> similar field in each of three tables: two of the three produced this error
> when we pasted in the text, but the third one had no problem. We also
> pasted the text into some other fields and got the problem. However, except
> for these fields and this text, we have load many fields with data from a
> legacy system and have typed in information into many fields and have had
> no problems. That is to say, the problem is rare, although it has happened
> every time we've pasted that text into those two fields.

Might be worth showing us the complete schema for those tables. Just to be 
clear, which describes the problem:
A)100% reproducible by putting the above text into either of the two fields
OR
B) Rare but when it happens it involves the above text and these two fields.

>
> John



-- 
Adrian Klaver
adrian.kla...@gmail.com

-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] Re: Error on Windows server could not open relation base/xxx/xxx Permission denied

2010-06-12 Thread John T. Dow
I will provide some answers tomorrow, but I don't have access now. It's a law 
enforcement agency in another state and the officer I work with will be in 
Sunday. The original computer's postgres server is stopped, and port forwarding 
wasn't changed yet so I don't have access to the new computer where the server 
is running. I have no access to the files themselves unless he's there and 
let's me in.

Thanks. Keep tuned.

John

On Sat, 12 Jun 2010 15:59:06 -0700, Adrian Klaver wrote:

>On Saturday 12 June 2010 12:59:18 pm John T. Dow wrote:
>
>I am CC'ing the list so more eyes can follow this.
>
>> >
>> >Some more questions.
>> >What is the relation that is having the permissions issue?
>> >Are the permissions on that file different from the others in the base
>> > directory tree?
>> >What is the 'certain text'?
>> >What are the 'certain fields' and do they have any functions running on
>> > them?
>> >
>> >
>> >
>> >--
>> >Adrian Klaver
>>
>> The actual message is: could not open relation base/16384/16642: Permission
>> denied.
>>
>> I presume that the actual file names are not important as they change from
>> time to time?
>
>
>
>They may or may not depending on what type of relation they are and the type 
>of 
>operation done to them. I should have been more specific. What is 16642? 
>SELECT relname,relkind from pg_class where relfilenode=16642 should provide an 
>answer.
>
>>
>> No, we have not diddled with the permissions. All files inherit their
>> permissions from "data".
>
>The question is really not whether you changed the permissions, but whether 
>they 
>have been changed? Hate to be anal about this, but have you actually looked at 
>the permissions for that file or are you assuming inheritance of permissions?
>
>>
>> The text I've been using for test purposes is a plain ascii version of the
>> US Constitution. It's about 50K, nothing special about it. Just a plain
>> text file I had on hand.
>
>But large enough to invoke TOASTing the value. See here for more info:
>http://www.postgresql.org/docs/8.4/interactive/storage-toast.html
>
>>
>> The fields where we noticed it at a couple jTextArea fields in a Java
>> application. They are defined as varchar. There is nothing special about
>> these fields. No functions, etc. They are fields to hold plain English
>> text. Data is normally entered by someone typing it in. (One example is a
>> police officer typing in his report of an arrest he's made. After he types
>> it in, it's printed and signed and becomes a legal document.) There is a
>> similar field in each of three tables: two of the three produced this error
>> when we pasted in the text, but the third one had no problem. We also
>> pasted the text into some other fields and got the problem. However, except
>> for these fields and this text, we have load many fields with data from a
>> legacy system and have typed in information into many fields and have had
>> no problems. That is to say, the problem is rare, although it has happened
>> every time we've pasted that text into those two fields.
>
>Might be worth showing us the complete schema for those tables. Just to be 
>clear, which describes the problem:
>A)100% reproducible by putting the above text into either of the two fields
>OR
>B) Rare but when it happens it involves the above text and these two fields.
>
>>
>> John
>
>
>
>-- 
>Adrian Klaver
>adrian.kla...@gmail.com



-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


Re: [GENERAL] Re: Error on Windows server could not open relation base/xxx/xxx Permission denied

2010-06-12 Thread Craig Ringer
On 13/06/10 02:34, Adrian Klaver wrote:

>> Question: Is it possible that there's corruption in the database which is
>> being incorrectly reported as "Permission denied"?

It's certainly not impossible. It'd really help if Pg would print more
details from Windows' error reporting - GetLastError() etc - in cases
like this. In fact, some searching reveals complaints about just that as
far back as mid-2008 related to the exact error you're encountering.


Anyway: When you moved the data dir over, did you reset all the
permissions on it so that it is owned by the "postgres" user on the new
machine? Applying those permissions recursively?

Does the file that PostgreSQL is complaining about actually exist?

Is it always the same 'xxx/xxx'?


Is it an index or a relation? You can find out using the Pg catalogs:

 http://www.postgresql.org/docs/current/static/storage-file-layout.html
 http://www.postgresql.org/docs/current/static/catalog-pg-class.html

... from which you'll see that of:

   base/xxx/yyy

'base/xxx' is the prefix for your database, and within that 'yyy' is the
oid of the table, so you can  find out some details about it with the
following SQL:


  \x
  select * from pg_class where oid = yyy;


Does the table/index name reported by that query match one that is
actually used in the problem query? What is it? Please post the full
output of the above query.


If it's an index, does REINDEXing your database help?

If it's a relation, does CLUSTERing that relation succeed? Help?

>> Is this a random event? A bug? Advice please on what to do next.

It's really, really hard to know, especially with the involvement of
elderly OSes and antivirus software. Could it be a Pg bug causing this?
Of course. But it's really, really hard to know what, when, and how,
especially with no access to the machines and data in question.

Please keep a copy of this damaged cluster around, even if you decide to
go ahead and rebuild the cluster. Now that its on a known-working
platform and the issue has been shown not to be proximately* caused by
antivirus software, it'd be preferable to find out what's actually going
on here. That will be impossible without the damaged cluster.

(* ie if the AV software was involved, it was to damage something that
stays damaged after the AV is taken out of the picture)

--
Craig Ringer

-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general