Re: [PERFORM] Writting a "search engine" for a pgsql DB

2007-02-28 Thread Dave Page
Oleg Bartunov wrote: > Guys, current tsearch2 should works with millions of documents. ... > Search itself is incredibly fast ! Oh, I know - you and Teodor have done a wonderful job. Regards, Dave. ---(end of broadcast)--- TIP 3: Have you check

Re: [PERFORM] Writting a "search engine" for a pgsql DB

2007-02-28 Thread Oleg Bartunov
On Wed, 28 Feb 2007, Dave Page wrote: Steinar H. Gunderson wrote: On Tue, Feb 27, 2007 at 01:33:47PM +, Dave Page wrote: When we outgrow PostgreSQL & Tsearch2, then, well, we'll need to stop pretending to be Google... Just for the record: Google has been known to sponsor sites in need wi

Re: [PERFORM] Writting a "search engine" for a pgsql DB

2007-02-28 Thread Dave Page
Steinar H. Gunderson wrote: > On Tue, Feb 27, 2007 at 01:33:47PM +, Dave Page wrote: >> When we outgrow PostgreSQL & Tsearch2, then, well, we'll need to stop >> pretending to be Google... > > Just for the record: Google has been known to sponsor sites in need with > Google Minis and such earli

Re: [PERFORM] Writting a "search engine" for a pgsql DB

2007-02-28 Thread Steinar H. Gunderson
On Tue, Feb 27, 2007 at 01:33:47PM +, Dave Page wrote: > When we outgrow PostgreSQL & Tsearch2, then, well, we'll need to stop > pretending to be Google... Just for the record: Google has been known to sponsor sites in need with Google Minis and such earlier -- I don't know what their[1] polic

Re: [PERFORM] Writting a "search engine" for a pgsql DB

2007-02-28 Thread Dave Page
Charles Sprickman wrote: > On Tue, 27 Feb 2007, Dave Page wrote: > >> Magnus Hagander wrote: >>> >>> Just as a datapoint, we did try to use mnogosearch for the >>> postgresql.org website+archives search, and it fell over completely. >>> Indexing took way too long, and we had search times several t

Re: [PERFORM] Writting a "search engine" for a pgsql DB

2007-02-27 Thread Magnus Hagander
On Tue, Feb 27, 2007 at 06:36:11PM -0500, Charles Sprickman wrote: > On Tue, 27 Feb 2007, Dave Page wrote: > > >Magnus Hagander wrote: > >> > >>Just as a datapoint, we did try to use mnogosearch for the > >>postgresql.org website+archives search, and it fell over completely. > >>Indexing took way

Re: [PERFORM] Writting a "search engine" for a pgsql DB

2007-02-27 Thread Charles Sprickman
On Tue, 27 Feb 2007, Dave Page wrote: Magnus Hagander wrote: Just as a datapoint, we did try to use mnogosearch for the postgresql.org website+archives search, and it fell over completely. Indexing took way too long, and we had search times several thousand times longer than with tsearch2. Th

Re: [PERFORM] Writting a "search engine" for a pgsql DB

2007-02-27 Thread Chris
Madison Kelly wrote: Hi all, I am asking in this list because, at the end of the day, this is a performance question. I am looking at writing a search engine of sorts for my database. I have only ever written very simple search engines before which amounted to not much more that the que

Re: [PERFORM] Writting a "search engine" for a pgsql DB

2007-02-27 Thread Dave Page
Magnus Hagander wrote: > > Just as a datapoint, we did try to use mnogosearch for the > postgresql.org website+archives search, and it fell over completely. > Indexing took way too long, and we had search times several thousand > times longer than with tsearch2. > > That said, I'm sure there are

Re: [PERFORM] Writting a "search engine" for a pgsql DB

2007-02-27 Thread Magnus Hagander
On Mon, Feb 26, 2007 at 04:24:12PM -0500, Charles Sprickman wrote: > On Mon, 26 Feb 2007, Madison Kelly wrote: > > >Hi all, > > > >I'd really like to come up with a more intelligent search engine that > >doesn't take two minutes to return results. :) I know, in the end good > >indexes and underl

Re: [PERFORM] Writting a "search engine" for a pgsql DB

2007-02-26 Thread Charles Sprickman
On Mon, 26 Feb 2007, Madison Kelly wrote: Hi all, I'd really like to come up with a more intelligent search engine that doesn't take two minutes to return results. :) I know, in the end good indexes and underlying hardware will be important, but a sane as possible query structure helps to st

Re: [PERFORM] Writting a "search engine" for a pgsql DB

2007-02-26 Thread Jeff Davis
On Mon, 2007-02-26 at 11:29 -0500, Madison Kelly wrote: >I am looking at writing a search engine of sorts for my database. I > have only ever written very simple search engines before which amounted > to not much more that the query string being used with ILIKE on a pile > of columns. This w

Re: [PERFORM] Writting a "search engine" for a pgsql DB

2007-02-26 Thread Mark Stosberg
> > Now see, this is exactly the kind of sagely advice I was hoping for! :) > > I'll look into tsearch2, and failing that for some reason, I love the > keyword table idea. For example keyword search code, you can try this package: http://downloads.sourceforge.net/cascade/cascade-devel-pieces-1.1

Re: [PERFORM] Writting a "search engine" for a pgsql DB

2007-02-26 Thread Madison Kelly
Mark Stosberg wrote: Madison Kelly wrote: I think the more direct question I was trying to get at is "How do you build a 'relavence' search engine? One where results are returned/sorted by relevance of some sort?". At this point, the best I can think of, would be to perform multiple queries; f

Re: [PERFORM] Writting a "search engine" for a pgsql DB

2007-02-26 Thread Mark Stosberg
Madison Kelly wrote: > > I think the more direct question I was trying to get at is "How do you > build a 'relavence' search engine? One where results are returned/sorted > by relevance of some sort?". At this point, the best I can think of, > would be to perform multiple queries; first matching

Re: [PERFORM] Writting a "search engine" for a pgsql DB

2007-02-26 Thread Magnus Hagander
> Joshua, I've been digging around the CVS (web) looking for the search > engine code but so far have only found the reference (www.search) in > 'general.php' but can't locate the file. You wouldn't happen to have a > direct link would you? It's all in module "portal". You will find the indexing

Re: [PERFORM] Writting a "search engine" for a pgsql DB

2007-02-26 Thread Madison Kelly
Mark Stosberg wrote: Joshua D. Drake wrote: Madison Kelly wrote: Hi all, I am asking in this list because, at the end of the day, this is a performance question. I am looking at writing a search engine of sorts for my database. I have only ever written very simple search engines before wh

Re: [PERFORM] Writting a "search engine" for a pgsql DB

2007-02-26 Thread Joshua D. Drake
>>> So I am hoping some of you guys and gals might be able to point me >>> towards some resources or offer some tips or gotcha's before I get >>> started on this. I'd really like to come up with a more intelligent >>> search engine that doesn't take two minutes to return results. :) I >>> know,

Re: [PERFORM] Writting a "search engine" for a pgsql DB

2007-02-26 Thread Mark Stosberg
Joshua D. Drake wrote: > Madison Kelly wrote: >> Hi all, >> >> I am asking in this list because, at the end of the day, this is a >> performance question. >> >> I am looking at writing a search engine of sorts for my database. I >> have only ever written very simple search engines before which

Re: [PERFORM] Writting a "search engine" for a pgsql DB

2007-02-26 Thread Joshua D. Drake
Madison Kelly wrote: > Hi all, > > I am asking in this list because, at the end of the day, this is a > performance question. > > I am looking at writing a search engine of sorts for my database. I > have only ever written very simple search engines before which amounted > to not much more th

[PERFORM] Writting a "search engine" for a pgsql DB

2007-02-26 Thread Madison Kelly
Hi all, I am asking in this list because, at the end of the day, this is a performance question. I am looking at writing a search engine of sorts for my database. I have only ever written very simple search engines before which amounted to not much more that the query string being used w