Re: [GENERAL] full text search to_tsquery performance with ispell dictionary

2011-05-12 Thread Stanislav Raskin
On 11.05.11 17:31, "Tom Lane" wrote: >You really, really, really need to fix whatever is preventing you from >using pooling. Opening a database connection to run one query is just >horridly inefficient. Very true. I did not mean that anything actually prevents us from using pooling. We just

Re: [GENERAL] full text search to_tsquery performance with ispell dictionary

2011-05-12 Thread Stanislav Raskin
On 11.05.11 17:04, "t...@fuzzy.cz" wrote: >We had exactly the same problem and persistent connection solved it. First testing with persistent connections seems to work like a charm. Will do some thorough testing and watch the memory load. Hopefully, I will not trip over some sort of pitfall. Go

Re: [GENERAL] full text search to_tsquery performance with ispell dictionary

2011-05-11 Thread Oleg Bartunov
On Wed, 11 May 2011, Stanislav Raskin wrote: Yes, loading a large dictionary is known to be a fairly expensive operation. There's been discussions about how to make it cheaper, but nothing's been done yet. regards, tom lane Hi Tom, thanks for the quick response. Bad news for m

Re: [GENERAL] full text search to_tsquery performance with ispell dictionary

2011-05-11 Thread Stanislav Raskin
On 11.05.11 16:42, "Pavel Stehule" wrote: >I wrote a >patch that stores loaded dictionary in shared memory. Hi Pavel, very interesting. I will give it a closer look. What do you think about using ispell to create, store and index tsvectors, but at the same time to use the stemmer to create ts

Re: [GENERAL] full text search to_tsquery performance with ispell dictionary

2011-05-11 Thread Tom Lane
Stanislav Raskin writes: > Is there any way of hack or compromise to achieve good performance without > losing fts ability? > I am thinking, for example, of a way to permanently keep a loaded > dictionary in memory instead of loading it for every connection. As I > wrote in response to Pavel Stehu

Re: [GENERAL] full text search to_tsquery performance with ispell dictionary

2011-05-11 Thread Pavel Stehule
2011/5/11 Stanislav Raskin : > On 11.05.11 16:42, "Pavel Stehule" wrote: > > >>I wrote a >>patch that stores loaded dictionary in shared memory. > > Hi Pavel, > > very interesting. I will give it a closer look. > > What do you think about using ispell to create, store and index tsvectors, > but at

Re: [GENERAL] full text search to_tsquery performance with ispell dictionary

2011-05-11 Thread tv
>> >> >> >>Yes, loading a large dictionary is known to be a fairly expensive >>operation. There's been discussions about how to make it cheaper, but >>nothing's been done yet. >> >>regards, tom lane > > Hi Tom, > > thanks for the quick response. Bad news for me ;( > We develop ajax-dri

Re: [GENERAL] full text search to_tsquery performance with ispell dictionary

2011-05-11 Thread Stanislav Raskin
On 11.05.11 15:45, "Pavel Stehule" wrote: >it is expected behave :( . A loading of ispell dictionary is very slow. > >Use a german snowball instead. > >You can you a some pooling connection software too. Thank you for the response. Is the dictionary german_stem supplied with postgresql a snowb

Re: [GENERAL] full text search to_tsquery performance with ispell dictionary

2011-05-11 Thread Stanislav Raskin
> > > >Yes, loading a large dictionary is known to be a fairly expensive >operation. There's been discussions about how to make it cheaper, but >nothing's been done yet. > >regards, tom lane Hi Tom, thanks for the quick response. Bad news for me ;( We develop ajax-driven web apps, wh

Re: [GENERAL] full text search to_tsquery performance with ispell dictionary

2011-05-11 Thread Pavel Stehule
Hello 2011/5/11 Stanislav Raskin : > > On 11.05.11 15:45, "Pavel Stehule" wrote: > >>it is expected behave :( . A loading of ispell dictionary is very slow. >> >>Use a german snowball instead. >> >>You can you a some pooling connection software too. > > > Thank you for the response. > Is the dict

Re: [GENERAL] full text search to_tsquery performance with ispell dictionary

2011-05-11 Thread Tom Lane
Stanislav Raskin writes: > The problem is, that if I open a new connection to the database and do > something like this > SELECT to_tsquery('german_de', 'abcd'); > it takes A LOT of time for the query to complete for the first time. About > 1-1,5s. If I submit the same query for a second, third, f

Re: [GENERAL] full text search to_tsquery performance with ispell dictionary

2011-05-11 Thread Pavel Stehule
Hello 2011/5/11 Stanislav Raskin : > Hello everybody, > I was experimenting with the FTS feature on postgres 8.3.4 lately and > encountered a weird performance issue when using a custom FTS configuration. > I use this german ispell dictionary, re-encoded to utf8: > http://www.sai.msu.su/~megera/po

[GENERAL] full text search to_tsquery performance with ispell dictionary

2011-05-11 Thread Stanislav Raskin
Hello everybody, I was experimenting with the FTS feature on postgres 8.3.4 lately and encountered a weird performance issue when using a custom FTS configuration. I use this german ispell dictionary, re-encoded to utf8: http://www.sai.msu.su/~megera/postgres/gist/tsearch/V2/dicts/ispell/ispell-