RE: null results
hi all, even i am getting similar error. i found that it is because of unsorted index..anybody has idea about it? Thanks in advance Regards Nitin -Original Message- From: Gasi [mailto:[EMAIL PROTECTED] Sent: Friday, September 02, 2005 2:45 AM To: java-user@lucene.apache.org Subject: null results Hallo everybody, I am beginning with Lucene. I tested the Webdemo described in the Book on my HomePC. When I upload it on a webserver, this example works but he doesn't find any keyword, the result of every search is null results. I have uploaded the same index consists of 3files deletable,segments and _9m.cfs. On my homepc i have as an example for the jeyword "java" many results , on the webhost account null results. Perhaps anybody had the same problem than me and can give my a hint what I do wrong. Thank you Greetings Gaston - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Highlighting does not work with PayloadTermQueries
Hi,
I'm using SOLR and Lucene in my application for search.
I'm facing an issue of highlighting using FastVectorHighlighter not working
when I use PayloadTermQueries as clauses of a BooleanQuery.
After Debugging I found that In DefaultSolrHighlighter.Java,
fvh.getFieldQuery does not return any term in the termMap.
FastVectorHighlighter fvh = new FastVectorHighlighter(
// FVH cannot process hl.usePhraseHighlighter parameter per-field
basis
params.getBool( HighlightParams.USE_PHRASE_HIGHLIGHTER, true ),
// FVH cannot process hl.requireFieldMatch parameter per-field basis
params.getBool( HighlightParams.FIELD_MATCH, false ) );
*FieldQuery fieldQuery = fvh.getFieldQuery( query );*
The reason of empty termmap is, PayloadTermQuery is discarded while
constructing the FieldQuery.
void flatten( Query sourceQuery, Collection flatQueries ){
if( sourceQuery instanceof BooleanQuery ){
BooleanQuery bq = (BooleanQuery)sourceQuery;
for( BooleanClause clause : bq.getClauses() ){
if( !clause.isProhibited() )
flatten( clause.getQuery(), flatQueries );
}
}
else if( sourceQuery instanceof DisjunctionMaxQuery ){
DisjunctionMaxQuery dmq = (DisjunctionMaxQuery)sourceQuery;
for( Query query : dmq ){
flatten( query, flatQueries );
}
}
else if( sourceQuery instanceof TermQuery ){
if( !flatQueries.contains( sourceQuery ) )
flatQueries.add( sourceQuery );
}
else if( sourceQuery instanceof PhraseQuery ){
if( !flatQueries.contains( sourceQuery ) ){
PhraseQuery pq = (PhraseQuery)sourceQuery;
if( pq.getTerms().length > 1 )
flatQueries.add( pq );
else if( pq.getTerms().length == 1 ){
flatQueries.add( new TermQuery( pq.getTerms()[0] ) );
}
}
}
// else discard queries
}
What is the best way to get highlighting working with Payload Term Queries?
Thanks
Nitin
--
View this message in context:
http://lucene.472066.n3.nabble.com/Highlighting-does-not-work-with-PayloadTermQueries-tp3762128p3762128.html
Sent from the Lucene - Java Users mailing list archive at Nabble.com.
-
To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-user-h...@lucene.apache.org
Help for Implementing Most relevance Search algorithm in lucene for my project
Hi, I wanted to implement "most relevant search" in Lucene for my project. I am currently using the Lucene provided by Hybris 5.3 i.e. Lucene 4.6.1. The scenario is that I have type ahead functionality (autosuggest) implemented already in the project so when user starts typing in the input box, after typing three characters he/she starts to see a drop down with suitable options. Now I want those options to be based on Most relevant search .How do I move ahead around similarities, scorer, score-boosting etc. to get this done. How do I enable getting score of the document from scoredoc now as searcher.setDefaultFieldSortScoring(true, false); doesn't seem to work in Lucene 4.6.1 Also which similarity is best to build this up . The default is good enough for me or BM25 or any other ? Regards, Nitin Chauhan Developer
Re: Boolean Search Query is not workng
Hi Priyanka Construct three different termqueries for house, home and flat. Now for first two termqueries you can set boost on them by setBoost(float b) method of TermQuery class. Now above three termqueries will be clauses of your boolean query, with OCCUR.SHOULD and search in normal way. This may help you. -Nitin On Friday 23 January 2015 03:57 PM, Priyanka Tufchi wrote: Hi Ian I tried with what u sent Query-java~5 jakarta~5 apache tomcat Document : "1", "java jakarta tomcat" "2", java jakarata"" "3", "java jakarta apache" Score : 1 :0.27094576 3 :0.27094576 2 :0.010494952 If we go by query it is giving same score ..It is not working. Thanks Priyanka On Fri, Jan 23, 2015 at 3:19 PM, Ian Lea wrote: How about "home~10 house~10 flat". See http://lucene.apache.org/core/4_10_3/queryparser/index.html -- Ian. On Fri, Jan 23, 2015 at 7:17 AM, Priyanka Tufchi wrote: Hi ALL I am working on a project which uses lucene for searching . I am struggling with boolean based Query : Actual Scenario is e.g In Query, if I give house home flat then inside It should search house or home or flat but I want to give them with weightage like house and home should get high weigh and flat should get less then rest. If document contain "Home" .Lucene search should not go for house and flat. I searched on Internet for some good stuff but not able to find any code sample or proper syntax for reference . Thanks Priyanka -- Launchship Technology respects your privacy. This email is intended only for the use of the party to which it is addressed and may contain information that is privileged, confidential, or protected by law. If you have received this message in error, or do not want to receive any further emails from us, please notify us immediately by replying to the message and deleting it from your computer. - To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org - To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
Re: combine to MultiTermQuery with OR
Hi sascha, You can do with boolean query, Take your three queries and OR them with boolean clause Occur.should. -Nitin On Tuesday 10 February 2015 08:58 PM, Sascha Janz wrote: Hi, i want to combine two MultiTermQueries. One searches over FieldA, one over FieldB. Both queries should be combined with "OR" operator. so in lucene Syntax i want to search FieldA:Term1 OR FieldB:Term1, FieldA:Term2 OR FieldB:Term2, FieldA:Term3 OR FieldB:Term3... how can i do this? greetings sascha - To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org - To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
first time using lucene
Hello , I have recently started downloaded lucene. My project is to add LSI(Latent Semantic Indexing) to the indexing method of the lucene, to improve the indexing of documents. I am totally new into this field. Please help me in this matter and guide me how to proceed in the right direction. I first want to input some documents and see the indexed document. Please guide me step by step how to do that. Thanking You Nitin
problem in running lucene
Hello , I have recently started downloaded lucene. This is the first time i am using lucene.My project is to add LSI(Latent Semantic Indexing) to the indexing method of the lucene, to improve the indexing of documents. I first want to index some webpages and see how does search work in lucene.The problem I am facing is that whenver i run lucene jar file through command prompt, i get error as "failed to load main-class manifest attribute from lucene-core-2.4.0.jar .I m using java 1.6.0_05. Please help me with this. Thanking You Nitin
Re: problem in running lucene
Hello Sir, i downloaded lucene, then i went into the directory of jar file lucene-core-2.4.0.jar . I typed the command java -jar lucene-core-2.4.0.jar to run the jar file from command prompt. then the following error came "failed to load main-class manifest attribute from lucene-core-2.4.0.jar" . I want to index a web document and see the result after searching. Regards Nitin On Sun, Jan 25, 2009 at 5:47 AM, Grant Ingersoll wrote: > Can you share the steps you have taken? The actual commands, that is. > > -Grant > > > On Jan 24, 2009, at 2:33 PM, nitin gopi wrote: > > Hello , I have recently started downloaded lucene. This is the first time >> i >> am using lucene.My project is to add LSI(Latent Semantic Indexing) to the >> indexing method of the lucene, to improve the indexing of documents. >> I first want to index some webpages and see how does search work >> in lucene.The problem I am facing is that whenver i run lucene jar file >> through command prompt, i get error as "failed to load main-class manifest >> attribute from lucene-core-2.4.0.jar .I m using java 1.6.0_05. Please help >> me with this. >> >> Thanking You >> Nitin >> > > -- > Grant Ingersoll > > Lucene Helpful Hints: > http://wiki.apache.org/lucene-java/BasicsOfPerformance > http://wiki.apache.org/lucene-java/LuceneFAQ > > > > > > > > > > > > - > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > For additional commands, e-mail: java-user-h...@lucene.apache.org > >
query regarding indexing method
Hello all, I want to know what algorithm lucene uses for indexing documents. Can I use lucene in my application with my own algorithm for indexing? regards, Nitin Gopi
class used to create term document matrix in lucene
Hi all Can anybody tell me which class and its methods are used to create term document matrix in lucene? Regards, Nitin
problem with indexing
hi all, I want to modify the indexing process lucene to suit my application. I partially read the book LUCENE IN ACTION but it contains no information to extract the term document matrix. I want to apply LSI to the indexed document. Please somebody help me out. regards nitin
error in code
Hi all,
I am getting error in running this code. Can somebody please tell me what
is the problem? The code is given below. The bold lines were giving error as
*cannot find symbol *
import java.io.File;
import java.io.FileReader;
import java.io.Reader;
import java.util.Date;
import org.apache.lucene.analysis.Analyzer;
import org.apache.lucene.analysis.standard.StandardAnalyzer;
import org.apache.lucene.document.Document;
import org.apache.lucene.document.Field;
import org.apache.lucene.index.IndexWriter;
/**
* This class demonstrates the process of creating an index with Lucene
* for text files in a directory.
*/
public class TextFileIndexer {
public static void main(String[] args) throws Exception{
//fileDir is the directory that contains the text files to be indexed
File fileDir = new File("C:\\files_to_index ");
//indexDir is the directory that hosts Lucene's index files
File indexDir = new File("C:\\luceneIndex");
Analyzer luceneAnalyzer = new StandardAnalyzer();
IndexWriter indexWriter = new IndexWriter(indexDir,luceneAnalyzer,true);
File[] textFiles = fileDir.listFiles();
long startTime = new Date().getTime();
//Add documents to the index
for(int i = 0; i < textFiles.length; i++){
if(textFiles[i].isFile() > textFiles[i].getName().endsWith(".txt")){
System.out.println("File " + textFiles[i].getCanonicalPath()
+ " is being indexed");
Reader textReader = new FileReader(textFiles[i]);
Document document = new Document();
*document.add(Field.Text("content",textReader));
document.add(Field.Text("path",textFiles[i].getPath()));*
indexWriter.addDocument(document);
}
}
indexWriter.optimize();
indexWriter.close();
long endTime = new Date().getTime();
System.out.println("It took " + (endTime - startTime)
+ " milliseconds to create an index for the files in the
directory "
+ fileDir.getPath());
}
}
Regards ,
Nitin Gopi
Re: error in code
hi Ganesh,
the program still gives error in the second line . it says that *cannot
find symbol * .I think that we are initializing the object of Field class
two times, that is why we are getting error.
document.add(new Field("content",textReader));
document.add(new Field("path",textFiles[i].getPath()));
Thanks,
Nitin Gopi
On Fri, Mar 6, 2009 at 11:01 AM, Ganesh wrote:
> Hello gopi,
>
> My comments.
>
> if(textFiles[i].isFile() > textFiles[i].getName().endsWith(".txt")){
>>
> && should be used.
>
> *document.add(Field.Text("content",textReader));
>>
>document.add(new Field("content", textReader);
>
> document.add(Field.Text("path",textFiles[i].getPath()));*
>>
> document.add(new Field("path", textFiles[i].getPath());
>
> Regards
> Ganesh
>
> - Original Message - From: "nitin gopi"
> To:
> Sent: Friday, March 06, 2009 8:24 AM
> Subject: error in code
>
>
>
> Hi all,
>>
>> I am getting error in running this code. Can somebody please tell me what
>> is the problem? The code is given below. The bold lines were giving error
>> as
>> *cannot find symbol *
>>
>>
>>
>> import java.io.File;
>> import java.io.FileReader;
>> import java.io.Reader;
>> import java.util.Date;
>>
>> import org.apache.lucene.analysis.Analyzer;
>> import org.apache.lucene.analysis.standard.StandardAnalyzer;
>> import org.apache.lucene.document.Document;
>> import org.apache.lucene.document.Field;
>> import org.apache.lucene.index.IndexWriter;
>>
>> /**
>> * This class demonstrates the process of creating an index with Lucene
>> * for text files in a directory.
>> */
>> public class TextFileIndexer {
>> public static void main(String[] args) throws Exception{
>> //fileDir is the directory that contains the text files to be indexed
>> File fileDir = new File("C:\\files_to_index ");
>>
>> //indexDir is the directory that hosts Lucene's index files
>> File indexDir = new File("C:\\luceneIndex");
>> Analyzer luceneAnalyzer = new StandardAnalyzer();
>> IndexWriter indexWriter = new IndexWriter(indexDir,luceneAnalyzer,true);
>> File[] textFiles = fileDir.listFiles();
>> long startTime = new Date().getTime();
>>
>> //Add documents to the index
>> for(int i = 0; i < textFiles.length; i++){
>>if(textFiles[i].isFile() > textFiles[i].getName().endsWith(".txt")){
>> System.out.println("File " + textFiles[i].getCanonicalPath()
>> + " is being indexed");
>> Reader textReader = new FileReader(textFiles[i]);
>> Document document = new Document();
>> *document.add(Field.Text("content",textReader));
>> document.add(Field.Text("path",textFiles[i].getPath()));*
>> indexWriter.addDocument(document);
>>}
>> }
>>
>> indexWriter.optimize();
>> indexWriter.close();
>> long endTime = new Date().getTime();
>>
>> System.out.println("It took " + (endTime - startTime)
>> + " milliseconds to create an index for the files in the
>> directory "
>> + fileDir.getPath());
>> }
>> }
>>
>> Regards ,
>> Nitin Gopi
>>
>>
> Send instant messages to your online friends http://in.messenger.yahoo.com
> -
> To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
> For additional commands, e-mail: java-user-h...@lucene.apache.org
>
>
doubt in adding a field in document
hi all, i am having error in my code. the line giving error is bold in the
code.the error is cannot find symbol.
thank you
nitin
import java.io.File;
import java.io.FileReader;
import java.io.Reader;
import java.util.Date;
// import org.apache.lucene;
import org.apache.lucene.analysis.Analyzer;
import org.apache.lucene.analysis.standard.StandardAnalyzer;
import org.apache.lucene.document.Document;
import org.apache.lucene.document.Field;
import org.apache.lucene.index.IndexWriter;
/**
* This class demonstrates the process of creating an index with Lucene
* for text files in a directory.
*/
public class TextFileIndexer {
public static void main(String[] args) throws Exception{
//fileDir is the directory that contains the text files to be indexed
File fileDir = new File("C:\\files_to_index ");
//indexDir is the directory that hosts Lucene's index files
File indexDir = new File("C:\\luceneIndex");
Analyzer luceneAnalyzer = new StandardAnalyzer();
IndexWriter indexWriter = new IndexWriter(indexDir,luceneAnalyzer,true);
File[] textFiles = fileDir.listFiles();
long startTime = new Date().getTime();
//Add documents to the index
for(int i = 0; i < textFiles.length; i++){
if(textFiles[i].isFile() && textFiles[i].getName().endsWith(".txt")){
System.out.println("File " + textFiles[i].getCanonicalPath()
+ " is being indexed");
Reader textReader = new FileReader(textFiles[i]);
Document document = new Document();
Field field1 = new Field("content",textReader);
document.add(field1);
* Field field2 = new Field("path",textFiles[i].getPath());*
document.add(field2);
// document.add(new Field("path",textFiles[i].getPath()));
indexWriter.addDocument(document);
}
}
indexWriter.optimize();
indexWriter.close();
long endTime = new Date().getTime();
System.out.println("It took " + (endTime - startTime)
+ " milliseconds to create an index for the files in the
directory "
+ fileDir.getPath());
}
}
lsi as indexing algorithm with lucene
hi all , has any body tried to use LSI(latent semantic indexing) for indexing in lucene? - To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
Re: lsi as indexing algorithm with lucene
hi Paul, I am new to this field of search engine. My aim is to develop a semantic search engine. Initially I was trying to develop that by using LSI. But since it is patented that is why there are no many implementation attempts. I want to ask is it possible to create a search engine using lucene and semantic vector which is semantically better than lucene? On 3/18/09, Paul Libbrecht wrote: > Nitin, > > LSI is patented so it's not been a flurry of implementation attempts. > However, SemanticVectors is a library that does similar approaches to > LSA/LSI for indexing and is based on Lucene's term-vectors. > > paul > > > Le 18-mars-09 à 07:09, nitin gopi a écrit : > >> hi all , has any body tried to use LSI(latent semantic indexing) for >> indexing in lucene? > > - To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
Re: Getting started with Lucene
hi you should add classpath of both lucene demo jar file and lucene core jar file . then run the command to build the index final step is to run the command to search files nitin On Fri, Mar 20, 2009 at 6:05 PM, Uwe Schindler wrote: > As I know, the demos are not compiled by default in the release (because > they show how you use Lucene and are so included as .java source files in > the binary distribution). You have to build the demos using ANT. > > Uwe > > - > Uwe Schindler > H.-H.-Meier-Allee 63, D-28213 Bremen > http://www.thetaphi.de > eMail: u...@thetaphi.de > > > > -Original Message- > > From: nga pham [mailto:nga.p...@gmail.com] > > Sent: Wednesday, March 18, 2009 9:37 PM > > To: java-user@lucene.apache.org > > Subject: Getting started with Lucene > > > > Hi, I have a project that involve Lucene. > > Currently I, > > 1) downloaded Lucene-2.4.1. into my CentOS 4.7 box. > > 2) succesfullly downloaded java, version 6. > > 3) successfully completed setting CLASSPATH > > > > as I ran the command : java org.apache.lucene.demo.SearchFiles > > > > I get an error saying: > > Exception in thread :main: java.lang.NoClassDefFoundError: > > org/apache/lucene/demo/SearchFiles > > Caused by: java.lang.ClassNotFoundException: > > org.apache.lucene.demo.SearchFiles > > at java.net.URLClassLoader$1.run(Unknown Source) > > at java.security.AccessController.doPrivileged(Native > > Method) > > ect. > > Could not find the main class. org.apache.lucene.demo.SearchFiles. > > Program > > will exit. > > > > > > Thank you, > > Nga P. > > > > note: I think I am in your mailing list > > > - > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > For additional commands, e-mail: java-user-h...@lucene.apache.org > >
semantic vectors
hi all, I want to know everything about semantic vectors. I want to know how does it indexes the documents such that the results produced are semantically better than normal search. I also want to know how it is different from semantic web, which uses the concept of ontologies and metadata. It would be very helpful if somebody mail me all the study material related to it? Thanking You Nitin
lsa on term document vector matrix
hi all, Can somebody help me to figure out how can I use "jama svd package" to apply LSA to the "term document vector" produced by lucene? Yours Sincerely, Nitin
problem with the output of using SVD with lucene
Hi all, I had implemented something, which I am going to describe in following steps 1. I took the input as 2 text files 2. I removed stop words from them 3. I did stemming over them 4. I formed the term document matrix using lucene. In the matrix values were the number of times the term has appeared in the document. 5. I calculated the term frequency and the inverse document frequency. I then multiplied them and formed the weight of each term. 6. I then calculated the SVD of the resultant matrix.As a result I got 3 matrices U(term vector matrix), S(singular values) and V(right singular values) My question is how are the 3 matrices formed in step 6 are going to be useful for me. How they prove that they have solved the problem of synonemy and polysemy.Last time you have mentioned that if we take only the first k values of the matrix then it proves to be useful. But how?? Please reply as soon as possible. Thanking You, Yours Sincerely Nitin Gopi
