Re: [HACKERS] WIP patch: Collation support

2008-09-23 Thread Magnus Hagander
Petr Jelinek wrote: > Magnus Hagander wrote: >>> exec("locale -a") ... >>> >>> I suppose we'd need something else for Windows, but I'm sure there's >>> a way. >> >> IIRC, the data is in the registry. Should be enumerable somehow - we'll >> have to do it platform specific of course, but it's not the

Re: [HACKERS] WIP patch: Collation support

2008-09-23 Thread Petr Jelinek
Magnus Hagander wrote: exec("locale -a") ... I suppose we'd need something else for Windows, but I'm sure there's a way. IIRC, the data is in the registry. Should be enumerable somehow - we'll have to do it platform specific of course, but it's not the first time we'd do that for windows...

Re: [HACKERS] WIP patch: Collation support

2008-09-23 Thread Zdenek Kotala
Heikki Linnakangas napsal(a): Zdenek Kotala wrote: pg_collation catalog is also important for pg_dump, because system collation names are not compatible over OS and pg_dump output should be portable. pg_collation adds abstract layer which solve this problem. That's a valid point. We'll still

Re: [HACKERS] WIP patch: Collation support

2008-09-23 Thread Magnus Hagander
Tom Lane wrote: > Martijn van Oosterhout <[EMAIL PROTECTED]> writes: >> On Tue, Sep 23, 2008 at 01:32:38PM +0300, Heikki Linnakangas wrote: >>> "locale -a" manages to do it somehow... > >> Sure, by (on glibc) opening the binary archive and parsing it and then >> trying to reverse lookup the alias

Re: [HACKERS] WIP patch: Collation support

2008-09-23 Thread Tom Lane
Martijn van Oosterhout <[EMAIL PROTECTED]> writes: > On Tue, Sep 23, 2008 at 01:32:38PM +0300, Heikki Linnakangas wrote: >> "locale -a" manages to do it somehow... > Sure, by (on glibc) opening the binary archive and parsing it and then > trying to reverse lookup the alias list. We could ofcourse

Re: [HACKERS] WIP patch: Collation support

2008-09-23 Thread Heikki Linnakangas
Zdenek Kotala wrote: pg_collation catalog is also important for pg_dump, because system collation names are not compatible over OS and pg_dump output should be portable. pg_collation adds abstract layer which solve this problem. That's a valid point. We'll still need a way to map OS locale to

Re: [HACKERS] WIP patch: Collation support

2008-09-23 Thread Zdenek Kotala
Martijn van Oosterhout napsal(a): On Mon, Sep 22, 2008 at 06:11:04PM +0300, Heikki Linnakangas wrote: This patch should allow to use both system catalog and ICU. Not without another patch that actually introduces ICU support. What that would look like, how that would be stored in the catalogs,

Re: [HACKERS] WIP patch: Collation support

2008-09-23 Thread Martijn van Oosterhout
On Tue, Sep 23, 2008 at 01:32:38PM +0300, Heikki Linnakangas wrote: > Martijn van Oosterhout wrote: > >You're assuming collations are denumerable. They're not. There is no way > >to find the list of available collations/locales. You may be able to > >guess a few but certainly not all of them. > >

Re: [HACKERS] WIP patch: Collation support

2008-09-23 Thread Heikki Linnakangas
Martijn van Oosterhout wrote: On Mon, Sep 22, 2008 at 10:22:35AM +0300, Heikki Linnakangas wrote: BTW, the original patch didn't have any provision for creating rows in pg_collation reflecting the locales available in the OS, but I think we'd need that. Otherwise the DBA would need to manually

Re: [HACKERS] WIP patch: Collation support

2008-09-23 Thread Martijn van Oosterhout
On Mon, Sep 22, 2008 at 06:11:04PM +0300, Heikki Linnakangas wrote: > >This patch should allow to use both system catalog and ICU. > > Not without another patch that actually introduces ICU support. What > that would look like, how that would be stored in the catalogs, and > whether we want tha

Re: [HACKERS] WIP patch: Collation support

2008-09-23 Thread Martijn van Oosterhout
On Mon, Sep 22, 2008 at 10:22:35AM +0300, Heikki Linnakangas wrote: > BTW, the original patch didn't have any provision for creating rows in > pg_collation reflecting the locales available in the OS, but I think > we'd need that. Otherwise the DBA would need to manually run CREATE > COLLATION fo

Re: [HACKERS] WIP patch: Collation support

2008-09-23 Thread Dave Page
On Tue, Sep 23, 2008 at 10:20 AM, Heikki Linnakangas <[EMAIL PROTECTED]> wrote: > Committed. *adds yet another item to his pgAdmin todo list* :-( > Tom Lane wrote: >> >> * You should try to get rid of LOCALE_NAME_BUFLEN altogether. Definitely >> the comment about it in pg_control.h is now obsole

Re: [HACKERS] WIP patch: Collation support

2008-09-23 Thread Heikki Linnakangas
Committed. Tom Lane wrote: * You should try to get rid of LOCALE_NAME_BUFLEN altogether. Definitely the comment about it in pg_control.h is now obsolete. Yep. I removed LOCALE_NAME_BUFLEN. The real max length of a locale name is now NAMEDATALEN, because it's stored in a name field in pg_data

Re: [HACKERS] WIP patch: Collation support

2008-09-22 Thread Heikki Linnakangas
Zdenek Kotala wrote: Heikki Linnakangas napsal(a): For anyone counting, Firebird added support for ICU more than three years ago. ICU is orthogonal to this patch. This patch didn't provide ICU support, and we could start using ICU without the catalog changes. This patch should allow to use

Re: [HACKERS] WIP patch: Collation support

2008-09-22 Thread Zdenek Kotala
Heikki Linnakangas napsal(a): For anyone counting, Firebird added support for ICU more than three years ago. ICU is orthogonal to this patch. This patch didn't provide ICU support, and we could start using ICU without the catalog changes. This patch should allow to use both system catalog a

Re: [HACKERS] WIP patch: Collation support

2008-09-22 Thread Heikki Linnakangas
Tom Lane wrote: What might be sensible to ask is whether it is ever actually reasonable for LC_COLLATE and LC_CTYPE to have different settings. If we were willing to enforce that they be the same, we could reduce this to just the standard syntax COLLATE=something and be done with it. Not being

Re: [HACKERS] WIP patch: Collation support

2008-09-22 Thread Tom Lane
Heikki Linnakangas <[EMAIL PROTECTED]> writes: > The first thing that we see is that the COLLATE keyword means different > things, so it's probably best to change that into: > CREATE DATABASE WITH LC_COLLATE= LC_CTYPE= > in the stripped-down version. Then we need a way to map the > stripped-do

Re: [HACKERS] WIP patch: Collation support

2008-09-22 Thread Heikki Linnakangas
Martijn van Oosterhout wrote: On Fri, Sep 19, 2008 at 10:13:43AM +0300, Heikki Linnakangas wrote: It's not like the patch is going to disappear from planet Earth if it doesn't get committed for 8.4. It's still valuable and available when the new catalogs are needed. I just prefer it as it was

Re: [HACKERS] WIP patch: Collation support

2008-09-19 Thread Martijn van Oosterhout
On Fri, Sep 19, 2008 at 10:13:43AM +0300, Heikki Linnakangas wrote: > In addition to catalog changes, for finer grained collation you need > changes in the planner and executor as well. In planner, to provide > support for the concept of a sort order using a specific collation, and > track that

Re: [HACKERS] WIP patch: Collation support

2008-09-19 Thread Heikki Linnakangas
Heikki Linnakangas wrote: Here's an updated version of the stripped-down patch, now with documentation changes, plus a couple of minor bug fixes. Another update, marching towards committing. Now with pg_dump/pg_dumpall support, and collation/ctype is also shown in psql \l output. I wonder if

Re: [HACKERS] WIP patch: Collation support

2008-09-19 Thread Tom Lane
Heikki Linnakangas <[EMAIL PROTECTED]> writes: > Here's an updated version of the stripped-down patch, now with > documentation changes, plus a couple of minor bug fixes. > I think this is looking pretty good now, and I would appreciate review > from others before I go ahead committing this. I

Re: [HACKERS] WIP patch: Collation support

2008-09-19 Thread Heikki Linnakangas
Here's an updated version of the stripped-down patch, now with documentation changes, plus a couple of minor bug fixes. I think this is looking pretty good now, and I would appreciate review from others before I go ahead committing this. Is there any more places in the documentation that needs

Re: [HACKERS] WIP patch: Collation support

2008-09-19 Thread Heikki Linnakangas
Martijn van Oosterhout wrote: FWIW, I disagree with the stripping down and think we should aim for the whole patch as submitted. Historically, the hardest part of getting collation support into postgres has been the catalog changes and easier the support for something other than OS locales. I sup

Re: [HACKERS] WIP patch: Collation support

2008-09-18 Thread Martijn van Oosterhout
On Thu, Sep 18, 2008 at 05:41:16PM +0300, Heikki Linnakangas wrote: > Attached is an updated version of the stripped-down patch. I've cleaned > it up a bit, and added more sanity checks. Documentation is still > missing and I haven't test it much. FWIW, I disagree with the stripping down and thi

Re: [HACKERS] WIP patch: Collation support

2008-09-18 Thread Heikki Linnakangas
Gregory Stark wrote: Heikki Linnakangas <[EMAIL PROTECTED]> writes: Well, I proposed disallowing using a different collation than the source database, except for using template0 as the source. That's pretty limited, but is trivial to implement and still let's you have databases with different c

Re: [HACKERS] WIP patch: Collation support

2008-09-16 Thread Alvaro Herrera
Zdenek Kotala escribió: > Gregory Stark napsal(a): >> AFAIK we can't easily connect to the new database and do some fiddling with >> it, can we? If we could we could check if there are any non-empty indexes >> which depend on the collation and only print the warning if we find any (and >> even mar

Re: [HACKERS] WIP patch: Collation support

2008-09-16 Thread Zdenek Kotala
Heikki Linnakangas napsal(a): Martijn van Oosterhout wrote: On Wed, Sep 10, 2008 at 12:51:02PM +0300, Heikki Linnakangas wrote: Since the set of collations isn't exactly denumerable, we need some way to allow the user to specify the collation they want. The only collation PostgreSQL knows about

Re: [HACKERS] WIP patch: Collation support

2008-09-16 Thread Zdenek Kotala
Gregory Stark napsal(a): Heikki Linnakangas <[EMAIL PROTECTED]> writes: AFAIK we can't easily connect to the new database and do some fiddling with it, can we? If we could we could check if there are any non-empty indexes which depend on the collation and only print the warning if we find a

Re: [HACKERS] WIP patch: Collation support

2008-09-16 Thread Gregory Stark
Heikki Linnakangas <[EMAIL PROTECTED]> writes: > Well, I proposed disallowing using a different collation than the source > database, except for using template0 as the source. That's pretty limited, but > is trivial to implement and still let's you have databases with different > collations in the

Re: [HACKERS] WIP patch: Collation support

2008-09-16 Thread Heikki Linnakangas
Martijn van Oosterhout wrote: On Wed, Sep 10, 2008 at 12:51:02PM +0300, Heikki Linnakangas wrote: Since the set of collations isn't exactly denumerable, we need some way to allow the user to specify the collation they want. The only collation PostgreSQL knows about is the C collation. Anything e

Re: [HACKERS] WIP patch: Collation support

2008-09-10 Thread Martijn van Oosterhout
On Wed, Sep 10, 2008 at 12:51:02PM +0300, Heikki Linnakangas wrote: > >Since the set of collations isn't exactly denumerable, we need some way > >to allow the user to specify the collation they want. The only > >collation PostgreSQL knows about is the C collation. Anything else is > >user-defined.

Re: [HACKERS] WIP patch: Collation support

2008-09-10 Thread Zdenek Kotala
Heikki Linnakangas napsal(a): Design & functionality changes left: - move retrieveing collation from pg_database to pg_type The problem there is that pg_collation is local catalog, but pg_database is global catalog. IIRC, It was discussed during last commitfest. I think it is bad idea to m

Re: [HACKERS] WIP patch: Collation support

2008-09-10 Thread Heikki Linnakangas
Martijn van Oosterhout wrote: On Wed, Sep 10, 2008 at 11:29:14AM +0300, Heikki Linnakangas wrote: Radek Strnad wrote: - because of pg_collation and pg_charset are catalogs individual for each database, if you want to create a database with collation other than specified, create it in template1

Re: [HACKERS] WIP patch: Collation support

2008-09-10 Thread Martijn van Oosterhout
On Wed, Sep 10, 2008 at 11:29:14AM +0300, Heikki Linnakangas wrote: > Radek Strnad wrote: > >- because of pg_collation and pg_charset are catalogs individual for each > >database, if you want to create a database with collation other than > >specified, create it in template1 and then create databas

Re: [HACKERS] WIP patch: Collation support

2008-09-10 Thread Heikki Linnakangas
Radek Strnad wrote: Progress so far: - created catalogs pg_collation a pg_charset which are filled with three standard collations - initdb changes rows called "DEFAULT" in both catalogs during the bki bootstrap phase with current system LC_COLLATE and LC_CTYPE or those set by command line. - new

Re: [HACKERS] WIP patch: Collation support

2008-09-02 Thread Tom Lane
Martijn van Oosterhout <[EMAIL PROTECTED]> writes: > And I think that's backwards. Why can we only use a feature once every > OS out there implements it? We still run on systems that don't have SSL > support. LC_TYPE settings are not portable between systems, yet that > doesn't bother anyone. Why s

Re: [HACKERS] WIP patch: Collation support

2008-09-02 Thread Alvaro Herrera
Radek Strnad escribió: > Ok, so do you suggest to leave it with a notice "reindex database" or start > to solve it somehow? I don't know. If there are two tasks that need the same treatment, it seems a safe conclusion that they need a common solution. -- Alvaro Herrera

Re: [HACKERS] WIP patch: Collation support

2008-09-02 Thread Martijn van Oosterhout
On Tue, Sep 02, 2008 at 05:42:13PM +0100, Gregory Stark wrote: > Martijn van Oosterhout <[EMAIL PROTECTED]> writes: > > On Tue, Sep 02, 2008 at 04:46:16PM +0300, Peter Eisentraut wrote: > >> >While it's true POSIX locales don't handle this, other collation > >> >libraries do and we should support t

Re: [HACKERS] WIP patch: Collation support

2008-09-02 Thread Gregory Stark
Martijn van Oosterhout <[EMAIL PROTECTED]> writes: > On Tue, Sep 02, 2008 at 04:46:16PM +0300, Peter Eisentraut wrote: >> >While it's true POSIX locales don't handle this, other collation >> >libraries do and we should support them if the user wants. I think that's backwards. We have to go with

Re: [HACKERS] WIP patch: Collation support

2008-09-02 Thread Martijn van Oosterhout
On Tue, Sep 02, 2008 at 04:46:16PM +0300, Peter Eisentraut wrote: > >While it's true POSIX locales don't handle this, other collation > >libraries do and we should support them if the user wants. > > Do they handle exactly those two attributes specifically? Can you point > out references? Or do

Re: [HACKERS] WIP patch: Collation support

2008-09-02 Thread Peter Eisentraut
Martijn van Oosterhout wrote: On Tue, Sep 02, 2008 at 02:50:47PM +0300, Peter Eisentraut wrote: Radek Strnad wrote: - new collations can be defined with command CREATE COLLATION name> FOR FROM [STRCOLFN ] [ ] [ ] [ LCCOLLATE ] [ LCCTYPE ] How do you plan to make a collation case sensiti

Re: [HACKERS] WIP patch: Collation support

2008-09-02 Thread Radek Strnad
I think at least case sensitivity can be done by comparing two strings converted to upper case with toupper() function. Regards Radek Strnad > > On Tue, Sep 2, 2008 at 2:00 PM, Martijn van Oosterhout <[EMAIL > PROTECTED]>wrote: > >> On Tue, Sep 02, 2008 at 02:50:47PM +0300, Peter Eisentraut wro

Re: [HACKERS] WIP patch: Collation support

2008-09-02 Thread Martijn van Oosterhout
On Tue, Sep 02, 2008 at 02:50:47PM +0300, Peter Eisentraut wrote: > Radek Strnad wrote: > >- new collations can be defined with command CREATE COLLATION >name> FOR FROM > >[STRCOLFN ] > >[ ] [ ] [ LCCOLLATE ] > >[ LCCTYPE ] > > How do you plan to make a collation case sensitive or accent

Re: [HACKERS] WIP patch: Collation support

2008-09-02 Thread Peter Eisentraut
Radek Strnad wrote: - new collations can be defined with command CREATE COLLATION name> FOR FROM [STRCOLFN ] [ ] [ ] [ LCCOLLATE ] [ LCCTYPE ] How do you plan to make a collation case sensitive or accent sensitive? I have previously commented that this is not a realistic view on how

Re: [HACKERS] WIP patch: Collation support

2008-09-01 Thread Radek Strnad
Ok, so do you suggest to leave it with a notice "reindex database" or start to solve it somehow? Regards Radek Strnad On Mon, Sep 1, 2008 at 12:08 AM, Alvaro Herrera <[EMAIL PROTECTED]>wrote: > Radek Strnad escribió: > > > - when creating database with different collation than database cluster,

Re: [HACKERS] WIP patch: Collation support

2008-08-31 Thread Alvaro Herrera
Radek Strnad escribió: > - when creating database with different collation than database cluster, the > database has to be reindexed. Any idea how to do it? Function > ReindexDatabase works only when database is opened. We have this Todo item: Set proper permissions on non-system schemas durin