Actually, I am planning to use the exact kinds of queries Nutch is using, except that I am using memoryindex. So I am match a bunch of queries to a single page, so it's almost exactly like Nutch except doing it in the opposite way. So I think that it's more intense than TermQuery. What do you think?
Erik Hatcher <[EMAIL PROTECTED]> wrote: Speculating about a possible future performance issue is like predicting the weather. You need to measure and find out given your data and how you want to architect your index, and of course it is hardware and software (in how you use Lucene) dependent too. Doing a search with Lucene is different than a SQL SELECT... you don't actually get the data back immediately from a search, so you'd fetch the actual documents, which very likely is only the top 20 or so at a time. If I was a betting man, I'd put my money on Lucene beating MySQL in the TermQuery scenario you described (e=hello) ;) But you'd be wise to step out of design mode and get some real-world tests going. And even if there is a performance difference, we're talking milliseconds most likely. Erik On 24 Nov 2005, at 22:48, Victor Lee wrote: > No, I am in the middle of development, actually more like design > phase. So I don't know the speed of my searches. The final > product will have heavy load on SELECT query if using database with > possibility of joining 2 tables. I don't worry about speed of > searches during light traffic. I am worry about what happens > during heavy load. > > For a simple example, if I am doing SELECT a, b, c, d FROM mytable > WHERE e = hello in a mysql database. And if I am doing the same > thing in Lucene by search "hello" in index and make it return > a,b,c,d. And I do all these continuously to simulate heavy load. > Which one is more likely to have more queries per second? I think > it would be mysql based on what I read on their benchmarks on the > web. what do you think? > > Erik Hatcher wrote: How fast are your > searches currently? With just a pure > searcher.search(Query) using a basic query type like TermQuery, I > very seriously doubt you'd beat MySQL performance. What kind of > Query are you using for your searches? > > Erik > > > On 24 Nov 2005, at 17:54, Victor Lee wrote: > >> Sorry, actually I meant all search results, not just frequent >> results. And there is only one search term per search, it's the >> stuff that belongs to the search terms change often. >> >> Victor Lee wrote: Hi, >> I use Lucene to index stuff that are changed very often but don't >> need to be real-time to searchers. e.g. the search result can be >> changed couple times per minute, but I only need to show the change >> every 5 minutes or so. Is it a good idea to save the search result >> to a database like mysql db, so that searches will load from db >> directly rather than doing the same search over and over again? >> Then after 5 minutes, the system will do the actual search again >> and then save search result to the db for next 3-5 minutes' searches? >> >> Will it make the server handle heavy load better? Since most of >> the access goes into the database server rather than the webserver >> with java in backend. Of course, there will be more to implement >> and maintain. >> >> Many thanks. >> >> >> >> __________________________________________________ >> Do You Yahoo!? >> Tired of spam? Yahoo! Mail has the best spam protection around >> http://mail.yahoo.com >> >> >> __________________________________________________ >> Do You Yahoo!? >> Tired of spam? Yahoo! Mail has the best spam protection around >> http://mail.yahoo.com > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > > > > __________________________________________________ > Do You Yahoo!? > Tired of spam? Yahoo! Mail has the best spam protection around > http://mail.yahoo.com --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] --------------------------------- Yahoo! Music Unlimited - Access over 1 million songs. Try it free.
