Re: TermDoc to TermDocsEnum

2011-03-23 Thread nitinhardeniya
I have changed the code according to MIGRATE.txt but now i am getting an error at public long getCorpCount(Vector clauses) { long count=0; try { SpanQuery [] clause= new SpanQuery[clauses.size()]; clause= clauses.toArray(clause); //SpanNear

Re: Grouping...

2011-03-23 Thread Grant Ingersoll
On Mar 22, 2011, at 6:43 AM, Dawn Zoë Raison wrote: > Hi Folks, > > Before I run off and reinvent the wheel here - has anyone done any form of > result grouping with lucene? > > My use case looks something like this: > Newspaper pages are stored as documents in the lucene index. > I need to li

RE: TermDoc to TermDocsEnum

2011-03-23 Thread Burton-West, Tom
Hi, If I understand correctly what you are trying to do as far as getting corpusTF, you might want to look at the implementation of the "-t" flag in org.apache.lucene.misc/HighFreqTerms.java in contib. Take a look at the getTotalTermFreq method in trunk. http://svn.apache.org/viewvc/lucene

Re: Sorting by multiple dependent fields

2011-03-23 Thread Ahmet Arslan
> I'm searching for things near your location (as specified > by longitude and latitude).  I've got the search > working correctly (with the help of NumericField), but now I > need to sort the results by distance from you.  The > closer things appear at the top of the list.  There is a contrib pac

Sorting by multiple dependent fields

2011-03-23 Thread Tim Koop
I'm trying to sort on two dependent fields. I realize that if they were independent I could use new Sort(SortField...) with lots of SortFields, but I need to compute something based on two fields. Let me explain: I'm searching for things near your location (as specified by longitude and lati

Re: TermDoc to TermDocsEnum

2011-03-23 Thread Michael McCandless
MIGRATE.txt is here: https://svn.apache.org/repos/asf/lucene/dev/trunk/lucene/MIGRATE.txt DocsEnum doesn't have a "getSpans()", so you mean you're hitting a compilation error? Maybe step back a bit and describe what you're trying to do...? Mike http://blog.mikemccandless.com On Tue, Mar 2

Re: Too many open files error

2011-03-23 Thread Ian Lea
No. Certainly not for lucene 3.0.3 on linux anyway. As someone has already suggested, run lsof to see which files your process has open. Maybe you aren't closing the source documents. -- Ian. On Wed, Mar 23, 2011 at 10:46 AM, Vo Nhu Tuan wrote: > I have already used compound file and closed

Re: Too many open files error

2011-03-23 Thread Vo Nhu Tuan
I have already used compound file and closed everything I can. Would this line cause problem when called many times? IndexWriter writer = new IndexWriter(FSDirectory.open(indexFile), new StandardAnalyzer(Version.LUCENE_30), true, IndexWriter.MaxFieldLength.LIMITED); This FSDirectory.open() is th

Re: Too many open files error

2011-03-23 Thread Li Li
and also try using compound files (cfs) 2011/3/23 Vo Nhu Tuan : > Hi, > > Can someone help me with this problem please? I got these when running my > program: > > java.io.FileNotFoundException: > /Users/vonhutuan/Documents/workspace/InformationExtractor/index_wordlist/_i82.frq > (Too many open

Re: Too many open files error

2011-03-23 Thread Mindaugas Žakšauskas
Hi, Have a look at this: http://wiki.apache.org/lucene-java/LuceneFAQ#Why_am_I_getting_an_IOException_that_says_.22Too_many_open_files.22.3F m. On Wed, Mar 23, 2011 at 9:48 AM, Vo Nhu Tuan wrote: > Hi, > > Can someone help me with this problem please? I got these when running my > program: >

Re: Too many open files error

2011-03-23 Thread Li Li
use lsof to count the number of opened files ulimit to modify it. maybe u need ask adminstrator to modify limit.conf 2011/3/23 Vo Nhu Tuan : > Hi, > > Can someone help me with this problem please? I got these when running my > program: > > java.io.FileNotFoundException: > /Users/vonhutuan/Docume

Too many open files error

2011-03-23 Thread Vo Nhu Tuan
Hi, Can someone help me with this problem please? I got these when running my program: java.io.FileNotFoundException: /Users/vonhutuan/Documents/workspace/InformationExtractor/index_wordlist/_i82.frq (Too many open files) at java.io.RandomAccessFile.open(Native Method) at java.

Re: how to get all documents in the results ?

2011-03-23 Thread Patrick Diviacco
ok, yes, I've solved. Thanks for help. On 23 March 2011 09:15, Anshum wrote: > So functionally I am assuming you've achieved what you'd been aiming for. > About the scores, the matchalldocs does score docs based on norm factors > etc. > therefore the score wouldn't be 0. > -- > Anshum Gupta > ht

Re: how to get all documents in the results ?

2011-03-23 Thread Anshum
So functionally I am assuming you've achieved what you'd been aiming for. About the scores, the matchalldocs does score docs based on norm factors etc. therefore the score wouldn't be 0. -- Anshum Gupta http://ai-cafe.blogspot.com On Wed, Mar 23, 2011 at 1:38 PM, Patrick Diviacco < patrick.divia.

Re: how to get all documents in the results ?

2011-03-23 Thread Patrick Diviacco
yeah it is clear. However I don't just want all documents, I still want to perform my specific query and on the bottom of the relevant docs, to list all not relevant docs as well. (I need this for successive steps). However now it seems to work. I've added MatchAllDocsQuery to my BooleanQuery, and

RE: Lucene, Luke: unknown format version: -12

2011-03-23 Thread Uwe Schindler
Hi, Why do you want to use Lucene trunk? You code is quite simple and does not use any of the special new features of Lucene trunk, so why no use stable versions for now? - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de > -Original Mess

Re: Lucene, Luke: unknown format version: -12

2011-03-23 Thread Patrick Diviacco
argh, yeah exactly. There is no way to inspect the index without luke ? Or to use luke with Lucene 4.0 ? thanks On 23 March 2011 09:01, Uwe Schindler wrote: > Hi, > > You are using a Lucene version that is not yet released (maybe 3.x branch > or > trunk). This seems to be not supported by th

Re: how to get all documents in the results ?

2011-03-23 Thread Patrick Diviacco
Sorry for spam. I've actually added the following line: booleanQuery.add(new MatchAllDocsQuery(), BooleanClause.Occur.SHOULD); and now I get all collection back. So it seems to work... I'm a bit confused about the score of the not relevant docs: i.e. the last result is 298736393 107419924 All-te

Re: how to get all documents in the results ?

2011-03-23 Thread Anshum
Hi Patrick, You *don't* need to add a MatchAllDocs query to anything. If you just want all docs, just pass it to the searcher.search function and you'd get all results. MatchAllDocs query is the same as BooleanQuery , just that MADQ matches all docs in the index. You wouldn't need to specify anythi

RE: Lucene, Luke: unknown format version: -12

2011-03-23 Thread Uwe Schindler
Hi, You are using a Lucene version that is not yet released (maybe 3.x branch or trunk). This seems to be not supported by the used Luke version. - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de > -Original Message- > From: Patrick D

Re: Distributing a Lucene application?

2011-03-23 Thread sol myr
Thanks :) Thankfully we don't delete from the database - just mark items as "inactive" (actual delete occurs only in a yearly cleanup process). We can live with inaccurate results, including deleted/inactive items. Have you used DBSight? Would you mind sharing your opinion - did you like it better

Re: how to get all documents in the results ?

2011-03-23 Thread Patrick Diviacco
The issue with My confusion about MatchAllDocsQuery is that I cannot specify which terms in which fields to search with it. I'm probably wrong. I currently have a BooleanQuery, that I use to build the query with several fields and several terms. Can I just pass MatchAllDocsQuery to BooleanQuery

Re: Am I correctly parsing the strings ? Terms or Phrases ?

2011-03-23 Thread Patrick Diviacco
ups, I've just used toString as you suggested and this is what I got: *.*:*.* title:Flickrmeetup_01 description:Michael description:R. description:Ross tags:rochester tags:ny tags:usa tags:flickrmeetup tags:king76 tags:eos350d tags:canon50mmf14 tags:mikros tags:canon tags:ef tags:50mm tags:f14 tag