Re: [HACKERS] unaccent module - two params function should be immutable

2013-11-18 Thread Bruce Momjian
On Fri, Nov 8, 2013 at 06:00:53PM -0500, Tom Lane wrote: > Bruce Momjian writes: > > [ mark unaccent functions immutable ] > > > Applied. > > This patch is flat out wrong and needs to be reverted. > > The functions were correctly marked (by you!) in commit > c0577c92a84cc477a88fe6868c16c4a7e33

Re: [HACKERS] unaccent module - two params function should be immutable

2013-11-08 Thread Tom Lane
Bruce Momjian writes: > [ mark unaccent functions immutable ] > Applied. This patch is flat out wrong and needs to be reverted. The functions were correctly marked (by you!) in commit c0577c92a84cc477a88fe6868c16c4a7e3348b11 on the basis of the discussion of bug #5781, http://www.postgresql.org

Re: [HACKERS] unaccent module - two params function should be immutable

2013-10-08 Thread Alvaro Herrera
Bruce Momjian escribió: > On Tue, Oct 8, 2013 at 02:25:25PM -0300, Alvaro Herrera wrote: > > Bruce Momjian escribió: > > > > > Do we need to update any version or anything? I didn't think so. > > > > I think there should be an 1.1 version here. That way, if somebody is > > using the existing d

Re: [HACKERS] unaccent module - two params function should be immutable

2013-10-08 Thread Bruce Momjian
On Tue, Oct 8, 2013 at 02:25:25PM -0300, Alvaro Herrera wrote: > Bruce Momjian escribió: > > > Do we need to update any version or anything? I didn't think so. > > I think there should be an 1.1 version here. That way, if somebody is > using the existing definition from the 1.0 module, they ca

Re: [HACKERS] unaccent module - two params function should be immutable

2013-10-08 Thread Alvaro Herrera
Bruce Momjian escribió: > Do we need to update any version or anything? I didn't think so. I think there should be an 1.1 version here. That way, if somebody is using the existing definition from the 1.0 module, they can get the new definition by doing an extension upgrade. -- Álvaro Herrera

Re: [HACKERS] unaccent module - two params function should be immutable

2013-10-08 Thread Pavel Stehule
2013/10/8 Bruce Momjian > On Tue, Oct 8, 2013 at 06:38:30PM +0200, Pavel Stehule wrote: > > I am not sure - does pg_upgrade change of flag after upgrade without > increasing > > version number? > > What happens in pg_upgrade is that the CREATE EXTENSION command is > pg_dump'ed, and run by pg_upr

Re: [HACKERS] unaccent module - two params function should be immutable

2013-10-08 Thread Bruce Momjian
On Tue, Oct 8, 2013 at 06:38:30PM +0200, Pavel Stehule wrote: > I am not sure - does pg_upgrade change of flag after upgrade without > increasing > version number? What happens in pg_upgrade is that the CREATE EXTENSION command is pg_dump'ed, and run by pg_uprade, and it then pulls from the SQL

Re: [HACKERS] unaccent module - two params function should be immutable

2013-10-08 Thread Pavel Stehule
2013/10/8 Bruce Momjian > On Tue, Oct 8, 2013 at 06:31:03PM +0200, Pavel Stehule wrote: > > > > > > > > 2013/10/8 Bruce Momjian > > > > On Tue, Sep 24, 2013 at 05:36:58PM -0400, Bruce Momjian wrote: > > > On Tue, Sep 17, 2013 at 10:15:47AM -0400, Robert Haas wrote: > > > > On Sat, S

Re: [HACKERS] unaccent module - two params function should be immutable

2013-10-08 Thread Bruce Momjian
On Tue, Oct 8, 2013 at 06:31:03PM +0200, Pavel Stehule wrote: > > > > 2013/10/8 Bruce Momjian > > On Tue, Sep 24, 2013 at 05:36:58PM -0400, Bruce Momjian wrote: > > On Tue, Sep 17, 2013 at 10:15:47AM -0400, Robert Haas wrote: > > > On Sat, Sep 14, 2013 at 9:42 AM, Pavel Stehule >

Re: [HACKERS] unaccent module - two params function should be immutable

2013-10-08 Thread Pavel Stehule
2013/10/8 Bruce Momjian > On Tue, Sep 24, 2013 at 05:36:58PM -0400, Bruce Momjian wrote: > > On Tue, Sep 17, 2013 at 10:15:47AM -0400, Robert Haas wrote: > > > On Sat, Sep 14, 2013 at 9:42 AM, Pavel Stehule < > pavel.steh...@gmail.com> wrote: > > > >> I have developed the attached patch based on

Re: [HACKERS] unaccent module - two params function should be immutable

2013-10-08 Thread Bruce Momjian
On Tue, Sep 24, 2013 at 05:36:58PM -0400, Bruce Momjian wrote: > On Tue, Sep 17, 2013 at 10:15:47AM -0400, Robert Haas wrote: > > On Sat, Sep 14, 2013 at 9:42 AM, Pavel Stehule > > wrote: > > >> I have developed the attached patch based on your suggestion. I did not > > >> see anything in the co

Re: [HACKERS] unaccent module - two params function should be immutable

2013-09-24 Thread Bruce Momjian
On Tue, Sep 17, 2013 at 10:15:47AM -0400, Robert Haas wrote: > On Sat, Sep 14, 2013 at 9:42 AM, Pavel Stehule > wrote: > >> I have developed the attached patch based on your suggestion. I did not > >> see anything in the code that would make it STABLE, except a lookup of a > >> dictionary librar

Re: [HACKERS] unaccent module - two params function should be immutable

2013-09-17 Thread Robert Haas
On Sat, Sep 14, 2013 at 9:42 AM, Pavel Stehule wrote: >> I have developed the attached patch based on your suggestion. I did not >> see anything in the code that would make it STABLE, except a lookup of a >> dictionary library: >> >> dictOid = get_ts_dict_oid(stringToQualifiedNameList("un

Re: [HACKERS] unaccent module - two params function should be immutable

2013-09-14 Thread Pavel Stehule
2013/9/11 Bruce Momjian > On Tue, Feb 19, 2013 at 08:30:29AM +0100, Pavel Stehule wrote: > > Hello > > > > There was a proposal to change flag of function to immutable - should > > be used in indexes > > > > CREATE FUNCTION unaccent(regdictionary, text) > > RETURNS text > > AS 'MO

Re: [HACKERS] unaccent module - two params function should be immutable

2013-09-10 Thread Bruce Momjian
On Tue, Feb 19, 2013 at 08:30:29AM +0100, Pavel Stehule wrote: > Hello > > There was a proposal to change flag of function to immutable - should > be used in indexes > > CREATE FUNCTION unaccent(regdictionary, text) > RETURNS text > AS 'MODULE_PATHNAME', 'unaccent_dict' >

Re: [HACKERS] Unaccent performance

2013-06-22 Thread Andres Freund
On 2013-06-21 22:52:04 +0100, Thom Brown wrote: > > CREATE OR REPLACE FUNCTION public.myunaccent(sometext text) > > RETURNS text > > LANGUAGE sql > > IMMUTABLE > > AS $function$ > > SELECT > > replace(replace(replace(replace(replace(replace(replace(replace(replace(replace(replace(replace(replace

Re: [HACKERS] Unaccent performance

2013-06-21 Thread Thom Brown
On 21 June 2013 19:04, Thom Brown wrote: > Hi, > > The unaccent extension is great, especially with its customisability, but > it's not always easy to recommend. I witnessed a customer using no less > than 56 nested replace functions in an SQL function. I looked to see how > much this can be mi

[HACKERS] Unaccent performance

2013-06-21 Thread Thom Brown
Hi, The unaccent extension is great, especially with its customisability, but it's not always easy to recommend. I witnessed a customer using no less than 56 nested replace functions in an SQL function. I looked to see how much this can be mitigated by unaccent. It turns out that not all the ch

[HACKERS] unaccent module - two params function should be immutable

2013-02-18 Thread Pavel Stehule
Hello There was a proposal to change flag of function to immutable - should be used in indexes CREATE FUNCTION unaccent(regdictionary, text) RETURNS text AS 'MODULE_PATHNAME', 'unaccent_dict' LANGUAGE C STABLE STRICT; is there any progress? Regards Pavel Stehule --

Re: [HACKERS] unaccent extension missing some accents

2011-11-10 Thread Tom Lane
Bruce Momjian writes: > Tom Lane wrote: >> However, the bigger picture is that OS X's UTF8 locales are broken >> through-and-through, and most of their other problems are not feasible >> to work around. > If Apple's low-level code came from FreeBSD and NetBSD, how did they get > so broken? AFAIK

Re: [HACKERS] unaccent extension missing some accents

2011-11-10 Thread Bruce Momjian
Tom Lane wrote: > J Smith writes: > > I've attached a patch against master for unaccent.c that uses swscanf > > along with char2wchar and wchar2char instead of sscanf directly to > > initialize the unaccent extension and it appears to fix the problem in > > both the master and 9.1 branches. > > s

Re: [HACKERS] unaccent extension missing some accents

2011-11-07 Thread J Smith
On Mon, Nov 7, 2011 at 11:53 AM, Florian Pflug wrote: > > Various issues with OSX and UTF-8 locales seems to come up quite often, yet > we're not really in a position to do anything about them. > > Thus, I think we should warn about these issues and save people the trouble > of finding out about t

Re: [HACKERS] unaccent extension missing some accents

2011-11-07 Thread Tom Lane
J Smith writes: > Would it even really be worth it to look into any of the other locale > issues on OSX, given that PostgreSQL is now included in their default > installs starting with 10.7, or would this really be more of a case of > hoping Apple some day fixes the issue upstream? To my mind, th

Re: [HACKERS] unaccent extension missing some accents

2011-11-07 Thread J Smith
On Mon, Nov 7, 2011 at 11:59 AM, Tom Lane wrote: > > If you have time to check that the patch I just committed fixes your > problem, it'd be worth doing.  I did not test it on OS X ... Looks good to me, thanks. Would it even really be worth it to look into any of the other locale issues on OSX,

Re: [HACKERS] unaccent extension missing some accents

2011-11-07 Thread Tom Lane
J Smith writes: > Anyways, lemme know if there's anything else I could help with or > could test and whatnot. Cheers. If you have time to check that the patch I just committed fixes your problem, it'd be worth doing. I did not test it on OS X ... regards, tom lane -- S

Re: [HACKERS] unaccent extension missing some accents

2011-11-07 Thread J Smith
On Mon, Nov 7, 2011 at 11:12 AM, Tom Lane wrote: > I looked at this a bit and realized that sscanf is actually doing a > couple of critical things for us, which are lost in translation when > doing it like this: > > 1. It ignores whitespace other than the dividing tab.  If we don't > continue to d

Re: [HACKERS] unaccent extension missing some accents

2011-11-07 Thread Florian Pflug
On Nov7, 2011, at 17:46 , J Smith wrote: > On Mon, Nov 7, 2011 at 11:12 AM, Tom Lane wrote: >> If OS X's UTF8 locales weren't so thoroughly broken (eg sorting does not >> work), I might be tempted to try to do it that way, but I still fail >> to see the point. After reviewing the code I feel that

Re: [HACKERS] unaccent extension missing some accents

2011-11-07 Thread Tom Lane
J Smith writes: > Alright, I wrote up another patch that uses strchr to parse out the > lines of the unaccent.rules file, foregoing sscanf completely. > Hopefully this looks a bit better than using swscanf. I looked at this a bit and realized that sscanf is actually doing a couple of critical thi

Re: [HACKERS] unaccent extension missing some accents

2011-11-06 Thread J Smith
Alright, I wrote up another patch that uses strchr to parse out the lines of the unaccent.rules file, foregoing sscanf completely. Hopefully this looks a bit better than using swscanf. As for the other problems with isspace and such on OSX, it might be worth looking at the python portability fixes

Re: [HACKERS] unaccent extension missing some accents

2011-11-06 Thread J Smith
On 2011-11-06, at 7:15 PM, Tom Lane wrote: > > swscanf doesn't seem like an acceptable approach: it's a function that > is relied on nowhere else in PG, so it adds new portability risks of its > own. It doesn't exist on some platforms that we support (like the one > I'm typing this message on) an

Re: [HACKERS] unaccent extension missing some accents

2011-11-06 Thread Tom Lane
J Smith writes: > I've attached a patch against master for unaccent.c that uses swscanf > along with char2wchar and wchar2char instead of sscanf directly to > initialize the unaccent extension and it appears to fix the problem in > both the master and 9.1 branches. swscanf doesn't seem like an ac

Re: [HACKERS] unaccent extension missing some accents

2011-11-06 Thread J Smith
On Sun, Nov 6, 2011 at 1:18 PM, Florian Pflug wrote: > > What's the locale of the database you're seeing this in, and which charset > does it use? > > I think scanf() uses isspace() and friends, and last time I looked the > locale definitions where all pretty bogus on OSX. So maybe scanf() somehow

Re: [HACKERS] unaccent extension missing some accents

2011-11-06 Thread Florian Pflug
On Nov6, 2011, at 18:43 , J Smith wrote: > I put some elog debugging lines into unaccent.c and found that sscanf > sometimes reads the scanned line by finding only one byte for the for > the source character rather than the two required for the complete > UTF-8 code point. It appears that the follo

Re: [HACKERS] unaccent extension missing some accents

2011-11-06 Thread J Smith
Gah! Accidentally hit Send. Let me finish that last message before sending this time! G'day list. I've been messing around with the unaccent extension and I've noticed that some of the characters listed in the unaccent.rules file aren't actually being unaccented on my system. Here are the syste

[HACKERS] unaccent extension missing some accents

2011-11-06 Thread J Smith
G'day list. I've been messing around with the unaccent extension and I've noticed that some of the characters listed in the unaccent.rules file aren't actually being unaccented on my system. Here are the system details and whatnot. - OSX 10.7.2 - the server is compiled via macports. Tried using

Re: [HACKERS] unaccent contrib

2011-09-26 Thread Oleg Bartunov
On Wed, 21 Sep 2011, Tom Lane wrote: Euler Taveira de Oliveira writes: On 21-09-2011 13:28, Daniel VАzquez wrote: "unaccent" is compatible with postgresql 8.4 (but not is in their contrib version distribution) No, it is not. AFAICS it is necessary to add some backend code that is not in 8

Re: [HACKERS] unaccent contrib

2011-09-22 Thread Robert Haas
2011/9/22 Daniel Vázquez : > Before 9.x, how do unaccent full text searches ? It seems that Oleg has published something on his web site that supposedly works with 8.4: http://www.sai.msu.su/~megera/wiki/unaccent But I'm not really sure how it works, or even where the source code is. I would su

Re: [HACKERS] unaccent contrib

2011-09-22 Thread Euler Taveira de Oliveira
On 22-09-2011 12:39, Daniel Vázquez wrote: Before 9.x, how do unaccent full text searches ? Perform pre-processing (normalization) of the string *before* inserting and *before* searching. -- Euler Taveira de Oliveira - Timbira http://www.timbira.com.br/ PostgreSQL: Consultoria, D

Re: [HACKERS] unaccent contrib

2011-09-22 Thread Daniel Vázquez
Before 9.x, how do unaccent full text searches ? i 2011/9/21 Tom Lane > Euler Taveira de Oliveira writes: > > On 21-09-2011 13:28, Daniel Vázquez wrote: > >> "unaccent" is compatible with postgresql 8.4 (but not is in their > contrib > >> version distribution) > > > No, it is not. AFAICS it is

Re: [HACKERS] unaccent contrib

2011-09-21 Thread Tom Lane
Euler Taveira de Oliveira writes: > On 21-09-2011 13:28, Daniel Vázquez wrote: >> "unaccent" is compatible with postgresql 8.4 (but not is in their contrib >> version distribution) > No, it is not. AFAICS it is necessary to add some backend code that is not in > 8.4. [ pokes at it ] Yeah, you

Re: [HACKERS] unaccent contrib

2011-09-21 Thread Devrim GÜNDÜZ
On Wed, 2011-09-21 at 18:28 +0200, Daniel Vázquez wrote: > Can Global Development Group, make some acumulative rpm for contrib > modules that are backward compatible??? No (as the RPM maintainer). -- Devrim GÜNDÜZ Principal Systems Engineer @ EnterpriseDB: http://www.enterprisedb.com PostgreSQL

Re: [HACKERS] unaccent contrib

2011-09-21 Thread Euler Taveira de Oliveira
On 21-09-2011 15:23, Daniel Vázquez wrote: No alternatives for unaccent on 8.4? Not that I know of. -- Euler Taveira de Oliveira - Timbira http://www.timbira.com.br/ PostgreSQL: Consultoria, Desenvolvimento, Suporte 24x7 e Treinamento -- Sent via pgsql-hackers mailing list (pgsql

Re: [HACKERS] unaccent contrib

2011-09-21 Thread Daniel Vázquez
... ok No alternatives for unaccent on 8.4? 2011/9/21 Euler Taveira de Oliveira > On 21-09-2011 13:28, Daniel Vázquez wrote: > >> "unaccent" is compatible with postgresql 8.4 (but not is in their contrib >> version distribution) >> >> No, it is not. AFAICS it is necessary to add some backend co

Re: [HACKERS] unaccent contrib

2011-09-21 Thread Euler Taveira de Oliveira
On 21-09-2011 13:28, Daniel Vázquez wrote: "unaccent" is compatible with postgresql 8.4 (but not is in their contrib version distribution) No, it is not. AFAICS it is necessary to add some backend code that is not in 8.4. -- Euler Taveira de Oliveira - Timbira http://www.timbira.com

[HACKERS] unaccent contrib

2011-09-21 Thread Daniel Vázquez
Hi guys! I know Postgresql 9.x includes "unaccent" contrib on their deliver package. "unaccent" is compatible with postgresql 8.4 (but not is in their contrib version distribution) what's better way to setup "unaccent" module on Postgresql 8.4 production server. Copy contrib/unaccent from 9.x t

Re: [HACKERS] unaccent

2002-09-19 Thread Peter Eisentraut
[EMAIL PROTECTED] writes: > Not "to_ascii", since there are so many extended UNICODE characters that > doesn't have any accent and should not be converted to an ASCII character. Really, the accent conversion should be part of the character set conversion routines. At least my local iconv does t

Re: [HACKERS] unaccent

2002-09-18 Thread nngodinh
t;Date: Wed, 18 Sep 2002 17:04:56 +0300 (GMT) >From: Oleg Bartunov <[EMAIL PROTECTED]> >To: [EMAIL PROTECTED] >Cc: [EMAIL PROTECTED] >Subject: Re: [HACKERS] unaccent > > >On Wed, 18 Sep 2002 [EMAIL PROTECTED] wrote: > >> The best way to use it is quite simple.

Re: [HACKERS] unaccent

2002-09-18 Thread Oleg Bartunov
itleidx = txt2txtidx(unac(title)); works fine. Perhaps, you have a problem with query ? > > Bye. > > >-- Messaggio Originale -- > >Date: Wed, 18 Sep 2002 15:08:59 +0300 (GMT) > >From: Oleg Bartunov <[EMAIL PROTECTED]> > >To: [EMAIL PROTECTED] > >Cc: [EMAIL PR

Re: [HACKERS] unaccent

2002-09-18 Thread Oleg Bartunov
On Wed, 18 Sep 2002, Karel Zak wrote: > On Wed, Sep 18, 2002 at 03:08:59PM +0300, Oleg Bartunov wrote: > > On Wed, 18 Sep 2002 [EMAIL PROTECTED] wrote: > > > > > > Get libunac from: http://www.nongnu.org/unac/ (it uses iconv) > > > > > > Extract the archive, compile it (make). Move pg_unac.so to

Re: [HACKERS] unaccent

2002-09-18 Thread nngodinh
Bartunov <[EMAIL PROTECTED]> >Cc: [EMAIL PROTECTED], [EMAIL PROTECTED] >Subject: Re: [HACKERS] unaccent > > >On Wed, Sep 18, 2002 at 03:08:59PM +0300, Oleg Bartunov wrote: >> On Wed, 18 Sep 2002 [EMAIL PROTECTED] wrote: >> > >> > Get libunac from: http://

Re: [HACKERS] unaccent

2002-09-18 Thread nngodinh
ecause (of course) it doesn't allow functions as parameters. So my first idea was to integrate unac in tsearch. Bye. >-- Messaggio Originale -- >Date: Wed, 18 Sep 2002 15:08:59 +0300 (GMT) >From: Oleg Bartunov <[EMAIL PROTECTED]> >To: [EMAIL PROTECTED] >Cc: [EMAIL PROTECT

Re: [HACKERS] unaccent

2002-09-18 Thread Karel Zak
On Wed, Sep 18, 2002 at 03:08:59PM +0300, Oleg Bartunov wrote: > On Wed, 18 Sep 2002 [EMAIL PROTECTED] wrote: > > > > Get libunac from: http://www.nongnu.org/unac/ (it uses iconv) > > > > Extract the archive, compile it (make). Move pg_unac.so to your postgresql > > shared libraries dir. > > > I t

Re: [HACKERS] unaccent

2002-09-18 Thread Oleg Bartunov
On Wed, 18 Sep 2002 [EMAIL PROTECTED] wrote: > Greetings, > > As far as I use the txtidx data structure in conjunction with gist indexing > to make a word indexing of a very large UNICODE db, I've implemented a PostgreSQL > function that uses libunac to unaccent TEXT fileds. > > The resulting tex

[HACKERS] unaccent

2002-09-18 Thread nngodinh
Greetings, As far as I use the txtidx data structure in conjunction with gist indexing to make a word indexing of a very large UNICODE db, I've implemented a PostgreSQL function that uses libunac to unaccent TEXT fileds. The resulting text is in UTF-8, but you can modify it in the sources with a