Re: Lucene Query Metrics

Hello, There's nothing like that. On top of my head is a profile collector in Elasticsearch. On Wed, Dec 4, 2024 at 11:46 PM ashwini singh wrote: > Does lucene provide extensions (utilities)to extract metrics from Lucene > during the request execution? Or applications can only track execution >

Re: Lucene Query Metrics

Does lucene provide extensions (utilities)to extract metrics from Lucene during the request execution? Or applications can only track execution stats on top of Lucene. On Tue, 3 Dec 2024 at 23:20, Adrien Grand wrote: > Lucene doesn't expose query metrics, it's up to the application that > integr

Re: Lucene Query Metrics

Lucene doesn't expose query metrics, it's up to the application that integrates Lucene to compute and expose metrics that are relevant to them. Le mer. 4 déc. 2024, 00:31, ashwini singh a écrit : > Hey everyone, > > Does lucene provide any query metrics (perf) ? I am looking for something > very

Re: Lucene Query Parser Special Characters

The documentation I'm referencing is here: https://lucene.apache.org/core/4_5_1/queryparser/org/apache/lucene/queryparser/classic/package-summary.html#package_description, with a note: org.apache.lucene.queryparser.flexible.standard

Re: Lucene Query Parser Special Characters

On Fri, Oct 14, 2016 at 2:47 AM, Ashley Ryan wrote: > Obviously, our work around of escaping the angle brackets works as we need > it to, but it seems to me that your documentation is incorrect. Am I > misunderstanding the documentation or conflating the issue I'm seeing with > the topic of specia

Re: lucene query complexity

Sigh. Yeah, I agree that a simple big-O won't work for Lucene. But nonetheless, we really should have some sort of performance characterization. When people ask me about how to characterize Lucene/Solr performance I always tell them that it is highly non-linear, with lots of optimizations and optio

Re: lucene query complexity

What if we have some assumptions. For example, we assume that we have only one segment and the entire segment is in memory ? thanks, Zong On Fri, Nov 20, 2015 at 4:38 AM, Adrien Grand wrote: > I don't think the big-O notation is appropriate to measure the cost of > Lucene queries. > > Le mer. 1

Re: lucene query complexity

I don't think the big-O notation is appropriate to measure the cost of Lucene queries. Le mer. 11 nov. 2015 à 20:31, search engine a écrit : > Hi, > > I've been thinking how to use big O annotation to show complexity for > different types of queries, like term query, prefix query, phrase query,

Re: Lucene Query to String

Hey Alessandro, thanks for your reply. A few minutes ago I found an issue in my code and now everything is working as expected. Sorry, for wasting for time :/ 2015-11-10 16:25 GMT+01:00 Alessandro Benedetti <[abenede...@apache.org](mailto:abenede...@apache.org)>: \-- Mit fr

Re: Lucene Query to String

Hi Markus, what is the logic behind your query parser? How the query is expected to be rewritten ? I've never seen that kind of rewritten query, but if you tell us what you are expecting to rewrite, maybe would be easier to help! Cheers On 10 November 2015 at 14:31, Markus Boese wrote: > Hi, >

Re: Lucene query behavior using NOT

On Sun, Feb 8, 2015 at 9:04 PM, Uwe Schindler wrote: > Hi, > > Lucene does not use algebraic / boolean logic! Maybe review this blog > post: https://lucidworks.com/blog/why-not-and-or-and-not/ This article is an old classic. The "plus, minus, nothing" operators aren't without their issues either

RE: Lucene query behavior using NOT

Hi, Lucene does not use algebraic / boolean logic! Maybe review this blog post: https://lucidworks.com/blog/why-not-and-or-and-not/ As you see in your examples and how the wqueries are parsed, AND, OR, and NOT are mapped to something else: Lucene only knows if a term MUST be in results, or SHO

RE: lucene query with additional clause field not null

Hi, Use FieldValueFilter for that: http://lucene.apache.org/core/4_10_2/core/org/apache/lucene/search/FieldValueFilter.html If you need a query instead of a Filter, wrap it with ConstantScoreQuery. This is also much faster than a RangeQuery like suggested by Ahmed. Uwe - Uwe Schindler H.-H

Re: lucene query with additional clause field not null

Hi Sascha, Generally RangeQuery is used for that, e.g. fieldName:[* TO *] Ahmet On Monday, December 1, 2014 9:44 PM, Sascha Janz wrote: Hi, is there a chance to add a additional clause to a query for a field that should not be null ? greetings sascha -

Re: Lucene Query

Thanks so much, that works. Jin On Tue, Aug 19, 2014 at 4:13 PM, Uwe Schindler wrote: > Hi, > Look at his docs. He has only 2 docs, the second one 3 keywords. > > I would use a simple phrase query with a slop value < Analyzers > positionIncrementGap. This is the gap between fields with same na

Re: Lucene Query

Oh sorry guys, ignore what I said. I am going to get myself a coffee. Uwe is absolutely correct here. On Aug 19, 2014, at 01:13 PM, Uwe Schindler wrote: Hi, Look at his docs. He has only 2 docs, the second one 3 keywords. I would use a simple phrase query with a slop value < Analyzers positi

Re: Lucene Query

Hi, Look at his docs. He has only 2 docs, the second one 3 keywords. I would use a simple phrase query with a slop value < Analyzers positionIncrementGap. This is the gap between fields with same name. Span or phrase cannot cross the gap, if slop if small enough, but large enough to find the te

Re: Lucene Query

Whoops, the constraint should be MUST to force all terms present: http://lucene.apache.org/core/4_6_0/core/org/apache/lucene/search/BooleanClause.Occur.html#MUST On Aug 19, 2014, at 01:05 PM, "Tri Cao" wrote: OR operator does that, AND only returns docs with ALL terms present. Note that you h

Re: Lucene Query

OR operator does that, AND only returns docs with ALL terms present. Note that you have two options here 1. Create a BooleanQuery object (see the Java doc I linked below) and programatically add the term queries with the following constraint: http://lucene.apache.org/core/4_6_0/core/org/apache/l

Re: Lucene Query

Thanks for reply, but won't BooleanQuery return both doc1 and doc2 with query: label:States AND label:America AND label:United Best, Jin On Tue, Aug 19, 2014 at 2:07 PM, Tri Cao wrote: > given that example, the easy way is a boolean AND query of all the terms: > > > http://lucene.apache.org/c

Re: Lucene Query

given that example, the easy way is a boolean AND query of all the terms: http://lucene.apache.org/core/4_6_0/core/org/apache/lucene/search/BooleanQuery.html However, if your corpus is more sophisticated you'll find that relevance ranking is not always that trivial :) On Aug 19, 2014, at 11:00

Re: Lucene Query Wrong Result for phrase.

The standard tokenizer will strip off those escaped quotes at query time. Ditto for the hyphen at index time. Try constructing your own analyzer using the white space tokenizer instead of the standard tokenizer. -- Jack Krupansky -Original Message- From: itisismail Sent: Friday, Ju

Re: Lucene Query Wrong Result for phrase.

Thanks IanI am using same analyzer for both indexing & searching. Problem is either I search hello world or hello-world or hello:world or hello,world in all cases I am getting same result. All of the queries are wrapped in double quotes. Actuall message indexed is hello - world.Also I am using phr

Re: Lucene Query Wrong Result for phrase.

Probably because something in the analysis chain is removing the hyphen. Check out the javadocs. Generally you should also make sure you use the same analyzer at index and search time. -- Ian. On Fri, Jul 18, 2014 at 6:52 AM, itisismail wrote: > Hi I have created index with 1 field with simp

Re: Lucene Query Syntax with analyzed and unanalyzed text

org.apache.lucene.analysis.miscellaneous.PerFieldAnalyzerWrapper in analyzers-common is what you need. There's an example in the javadocs. Build and use the wrapper instance in place of StandardAnalyzer or whatever you are using now. -- Ian. On Mon, Sep 16, 2013 at 5:36 PM, Scott Smith wrote

Re: Lucene Query About Sorting

On Jun 26, 2012, at 5:32 AM, Apostolis Xekoukoulotakis wrote: > I am just new here. > > When you make a query, you create an ordering of the documents based on > this query. > If you have a second ordering, you have to decide what to do with those 2 > orderings. You have to decide how to join th

Re: Lucene Query About Sorting

Cant he synchronously iterate over both fields postingLists and use one priorityQueue that picks the docs that contain the query and have the best order according to the second field. It requires more work but this should be feasible. 2012/6/27 Ian Lea > I think he wants 1, sort all matched doc

Re: Lucene Query About Sorting

I think he wants 1, sort all matched docs by field A. If lucene sorting doesn't work for you you can always sort the hits yourself using whatever technique you want. Sorting large numbers of docs is always going to be expensive. -- Ian. On Wed, Jun 27, 2012 at 8:54 AM, Li Li wrote: > what do

Re: Lucene Query About Sorting

what do you want to do? 1. sort all matched docs by field A. 2. sort all matched docs by relevant score, selecting top 100 docs and then sort by field A On Wed, Jun 27, 2012 at 1:44 PM, Yogesh patel wrote: > Thanks for reply Ian , > > But i just gave suppose document number..i have 2-3 GB index

Re: Lucene Query About Sorting

Thanks for reply Ian , But i just gave suppose document number..i have 2-3 GB index and every day , it goes higher. so i cant use searcher.maxdoc(). So i need this solution. Can you please help me out? On Tue, Jun 26, 2012 at 10:42 PM, Ian Lea wrote: > Do you mean you want all hits that match

Re: Lucene Query About Sorting

Do you mean you want all hits that match B:abc, sorted by field A? As opposed to the top 100 hits sorted by field A? Just pass a higher value in the search(query, ... 100, ...) call. It will be slower and potentially use more memory but with only 10K docs you probably won't notice. -- Ian.

Re: Lucene Query About Sorting

I am just new here. When you make a query, you create an ordering of the documents based on this query. If you have a second ordering, you have to decide what to do with those 2 orderings. You have to decide how to join those two. The default search orders your results by the query and picks the

Re: Lucene Query Parser

Just use one of the search() methods that does sorting and specify an array of sort fields with SortField.SCORE first, then your name fields. But be aware that complex real world textual queries and docs rarely produce identical scores. You could post-process the results and group them into "good

Re: lucene query type

Lucene will generally do the right thing without much help - docs with more matches will score better and thus be presented first. But you can give it a helping hand, perhaps with some query boosting: build a boolean query and add an AND query with a positive boost, then an OR query with smaller o

Re: Lucene query with long strings

On Mar 24, 2010, at 9:20 AM, Shashi Kant wrote: > Add the common terms such as "University", "School", "Medicine", > "Institute" etc. to stopwords list, so you are left with Stanford, > "Palo Alto" etc. I don't know if I would remove them, but you might consider using the CommonGram or n-gram a

Re: Lucene query with long strings

Add the common terms such as "University", "School", "Medicine", "Institute" etc. to stopwords list, so you are left with Stanford, "Palo Alto" etc. Then use Ahmet's suggestion of using a booleanquery .setMinimumNumberShouldMatch() to (say) 75% of the query string length. Finally, if you wish to

RE: Lucene query with long strings

Hi Aaron, Your "false positives" comments point to a mismatch between what you're currently asking Lucene for (any document matching any one of the terms in the query) and what you want (only fully "correct" matches). You need to identify the terms of the query that MUST match and tell Lucene

Re: Lucene query with long strings

> hi all, I have been playing > with Lucene for a while now, but stuck on a perplexing > issue. > > I have an index, with a field "Affiliation", some example > values are: > > - "Stanford University School of Medicine, Palo Alto, CA > USA", > - "Institute of Neurobiology, School of Medicine, Sta

Re: Lucene query syntax using grouping, MUST, MUST_NOT and SHOULD

--- On Tue, 8/25/09, Henric Müller wrote: > From: Henric Müller > Subject: Lucene query syntax using grouping, MUST, MUST_NOT and SHOULD > To: java-user@lucene.apache.org > Date: Tuesday, August 25, 2009, 12:20 PM > Hi, > > I have experience some strange things when dealing with > Lucene quer

Re: Lucene Query returns always the first ID

your problem is about new one Document for all sourcefiles i think you have 2 solution 1.new document in first step of loop(like below code) [code] for (int i = 0; i < sourcefiles.size(); i++) { for (int j = 0; j < sourcefiles.elementAt(i).getNumberOfRevisions(

Re: lucene query parser for double-worded term query

Erick, Thanks! It's the analyzer problem. I should have used the same analyzer, KeywordAnalyzer, to create the query parser. Thanks a lot! -- Chris Lu - Instant Scalable Full-Text Search On Any Database/Application site: http://www.dbsight.net demo: http://search.dbsight

Re: lucene query parser for double-worded term query

What analyzers are you using for both indexing and querying? Have you looked at your index with Luke to see what's actually in the index? The reason I'm asking is I'm wondering whether you are having capitalization issues. That is, your index analyzer lower cases the tokens and your query analyzer

Re: lucene query parser for double-worded term query

Yonik, Thanks for your quick reply! But I found after backslash escape spaces, both tags:San\ Francisco tags:"San\ Francisco" turns into PhraseQuery, just like tags:"San Francisco", still no results returned. Maybe Lucene Query Parser does not handle this case? -- Chris Lu -

Re: lucene query parser for double-worded term query

You can backslash escape spaces, so these should both work: tags:San\ Francisco tags:"San\ Francisco" -Yonik On Tue, Jun 24, 2008 at 8:14 PM, Chris Lu <[EMAIL PROTECTED]> wrote: > I have a tags field. And each tag can have multiple words, like "San > Francisco". Each tag is analyzed into Keyword

Re: Lucene Query

The problem with your code snippets are that they aren't plain Lucene API calls. I'm assuming that you've got your own classes that actually compile . There's nothing I can say about "what's going on" without knowing what your custom classes are doing. We need to know what analyzers you are

RE: Lucene query question

[mailto:[EMAIL PROTECTED] Sent: Wednesday, May 10, 2006 5:59 PM To: java-user@lucene.apache.org Subject: Re: Lucene query question I'll take a quick stab at it. What analyzer are you using with the query? In the search page of Luke, near the upper right there's the "Analyzer t

Re: Lucene query question

Hi Erick, I appreciate the help. I am using the "StandardAnalyzer" for both the query and for indexing. --Mike On 5/10/06, Erick Erickson <[EMAIL PROTECTED]> wrote: I'll take a quick stab at it. What analyzer are you using with the query? In the search page of Luke, near the upper right ther

Re: Lucene query question

I'll take a quick stab at it. What analyzer are you using with the query? In the search page of Luke, near the upper right there's the "Analyzer to use for query parsing:" box. You might try the WhitespaceAnalyzer, since that shouldn't do anything "interesting". Also, below the search box on the s

Re: Lucene query question

Mr. Gospodnetic, Thanks for the quick response. You make a good point about the field being tokenized, and I initially had the e-mail field UN_TOKENIZED but it did not change the result of the query (my example search was still failing). Do you have any ideas on what could be causing that? Th

Re: Lucene query question

Mike, Do you really want to tokenize your emails? StandardAnalyzer may in fact recognize email addresses and leave them as one token, but it would probably be better practice to make that email field UN_TOKENIZED. Most of the time when people have trouble finding a Document they _know_ is in

Re: lucene query analysis

"Raghavendra Prabhu" <[EMAIL PROTECTED]> wrote on 15/03/2006 08:37:25 AM: > Hi > > The problem which i am facing is that the query is Case Sensitive > > If i type in BIG letters i am not able to see answers and if i type in > small letters i am able to see results > > Is there anything by which i

Re: Lucene Query ... understanding

: Am just trying to see if i understand the lucene query below correctly. : : +(+contentNew:radio +contentNew:mp3) +entity:product +(name:radio : mp3^4.0 (contentNew:radio contentNew:mp3) contentNew:radio mp3^2.0) : : Let me see if can understand the above query correctly: your interpretation isn

Re: Lucene Query Writing question

On Tuesday 17 January 2006 20:52, Dan Katz wrote: ... > Question 1) Is there a way in Lucene to have some sort of limit based > on term count. For example, "atleast5 Apple" to find items with the > word apple only when it has at least 5 mentions. This can be done, but you'll need to write your

Re: lucene query

'not treating xml file as text file' is quite fuzzy it doesn't say how you treat the files that you wish to perform retrieval on. It may be of interest to you that I am in the process of allowing search for mathematical formulae based on lucene-index built from OMDoc encoded documents (an XML sy

Re: lucene query

Thanks for the reply. -Eshwari --- Erik Hatcher <[EMAIL PROTECTED]> wrote: > > On Jun 29, 2005, at 1:28 PM, eshwari pss wrote: > > Does Lucene support XML searching? - I mean not > > treating the xml file as text file. > > The short answer is yes. > http://www.lucenebook.com/search?query=xml

Re: lucene query

On Jun 29, 2005, at 1:28 PM, eshwari pss wrote: Does Lucene support XML searching? - I mean not treating the xml file as text file. The short answer is yes. http://www.lucenebook.com/search?query=xml The longer response is more involved - what are your needs? I built a search engine for the