Re: How exclude empty fields?

2016-11-16 Thread Chris Hostetter
: The issue I have is that some promotions are permanent so they don't have : an endDate set. : : I tried doing: : : ( +Promotion.endDate:[210100TOvariable containing yesterday's date] : || -Promotion.endDate:* ) 1) mixing prefix ops with "||" like this is most certainly not doing what

Re: How exclude empty fields?

2016-11-11 Thread Ahmet Arslan
Hi, Match all docs query minus Promotion.endDate:[* TO *] +*:* -Promotion.endDate:[* TO *] Ahmet On Friday, November 11, 2016 5:59 PM, voidmind wrote: Hi, I have indexed content about Promotions with effectiveDate and endDate fields for when the promotions start and end. I want to query for

How exclude empty fields?

2016-11-11 Thread voidmind
Hi, I have indexed content about Promotions with effectiveDate and endDate fields for when the promotions start and end. I want to query for expired promotions so I do have this criteria, which works fine: +Promotion.endDate:[210100TOvariable containing yesterday's date] The issue I hav

RE: Lucene Empty Non-empty Fields

2013-11-04 Thread Vitaly Funstein
Or FieldValueFilter - that's probably easier to use. > -Original Message- > From: Michael McCandless [mailto:luc...@mikemccandless.com] > Sent: Monday, November 04, 2013 4:37 AM > To: Lucene Users > Subject: Re: Lucene Empty Non-empty Fields &

Re: Lucene Empty Non-empty Fields

2013-11-04 Thread Michael McCandless
You can also use FieldCache.getDocsWithField? Mike McCandless http://blog.mikemccandless.com On Mon, Nov 4, 2013 at 7:33 AM, manoj raj wrote: > I did some experiments for finding empty fields, But i want to know whether > there is any other better method. Have to reduce hard disk

Lucene Empty Non-empty Fields

2013-11-04 Thread manoj raj
I did some experiments for finding empty fields, But i want to know whether there is any other better method. Have to reduce hard disk space. Method 1: Add "NULL String" in empty fields We can search with null string for empty column & non empty column Observations: - Index

Re: Searching for null (empty) fields, how to use -field:[* TO *]

2010-09-29 Thread Frank Geary
in CentOS // rather than with the difference in the OS itself. Term[] fieldTerms = new Term[ 1 ]; fieldTerms[0] = new Term( theFieldName, "" ); indexWriter.deleteDocuments( fieldTerms ); // End of code that did NOT always work for me. Frank Geary -- View this message in context: http://lucene.

Re: Searching for null (empty) fields, how to use -field:[* TO *]

2008-03-12 Thread thogau
Thanks Erick, I ended up by following your second suggestion. It has been a bit tricky since I had to plug into a MapConverter but it works as expected. Thanks to all. --thogau You could also think about making a filter, probably when you open your searcher. You can use TermDocs/TermEnum to fi

Re: Searching for null (empty) fields, how to use -field:[* TO *]

2008-03-12 Thread thogau
Thanks for your suggestion markmiller. When I try this query, I get both documents as hits. The one with the field having a value and also the one with the field not set... Any idea why? markrmiller wrote: > > You cannot have a purely negative query like you can in Solr. > > Try: *:* -MY_FIELD

Re: Searching for null (empty) fields, how to use -field:[* TO *]

2008-03-11 Thread Erick Erickson
You could also think about making a filter, probably when you open your searcher. You can use TermDocs/TermEnum to find all of the documents that *do* have entries for your field, assemble those into a filter, then invert that filter. Keep the filter around and use it whenever you need to. Perhaps

Re: Searching for null (empty) fields, how to use -field:[* TO *]

2008-03-11 Thread eks dev
g Sent: Tuesday, 11 March, 2008 3:59:38 PM Subject: Re: Searching for null (empty) fields, how to use -field:[* TO *] Yes, my index is a "full-snapshot" created every "n" hours, there are no incremental updates, so I decided to make another MatchAllDocsQuery taking advantage that

Re: Searching for null (empty) fields, how to use -field:[* TO *]

2008-03-11 Thread German Kondolf
Yes, my index is a "full-snapshot" created every "n" hours, there are no incremental updates, so I decided to make another MatchAllDocsQuery taking advantage that my index is read-only and basically removing this checks. Regards Ger [EMAIL PROTECTED] On Tue, Mar 11, 2008 at 11:54 AM, Yonik Seele

Re: Searching for null (empty) fields, how to use -field:[* TO *]

2008-03-11 Thread Yonik Seeley
On Tue, Mar 11, 2008 at 10:41 AM, German Kondolf <[EMAIL PROTECTED]> wrote: > *:* is parsed as a MatchAllDocsQuery? > > I've got some preformance issues in Lucene 2.2 because > MatchAllDocsQuery ask for a "isDeleted()" for every document, I didn't > tried it in 2.3. That will still be the case

Re: Searching for null (empty) fields, how to use -field:[* TO *]

2008-03-11 Thread German Kondolf
*:* is parsed as a MatchAllDocsQuery? I've got some preformance issues in Lucene 2.2 because MatchAllDocsQuery ask for a "isDeleted()" for every document, I didn't tried it in 2.3. On Tue, Mar 11, 2008 at 11:34 AM, Mark Miller <[EMAIL PROTECTED]> wrote: > You cannot have a purely negative query l

Re: Searching for null (empty) fields, how to use -field:[* TO *]

2008-03-11 Thread German Kondolf
Hi, I was looking for the same functionality, after a short googling didn't find a solution, I assume it must exist but I finally decided to "fill" those empty fields with a representative "null value", "__null__", this is possible only if you know previously A

Re: Searching for null (empty) fields, how to use -field:[* TO *]

2008-03-11 Thread Mark Miller
You cannot have a purely negative query like you can in Solr. Try: *:* -MY_FIELD_NAME:[* TO *] thogau wrote: Hi, I browsed the forum searching for a way to make a query that retrieves document that do not have any value for a given field (say MY_FIELD_NAME). I read several posts advising to

Searching for null (empty) fields, how to use -field:[* TO *]

2008-03-11 Thread thogau
Hi, I browsed the forum searching for a way to make a query that retrieves document that do not have any value for a given field (say MY_FIELD_NAME). I read several posts advising to use this syntax : -MY_FIELD_NAME:[* TO *] However, I am not able to have it working... I have 2 documents, t

Re: Does lucene performance suffer with a lot of empty fields ?

2006-08-01 Thread Chris Hostetter
: >From what I gather, I can go ahead & create an Index & for each Document & : only add the relevant fields. Is this correct? : I should still be able to search with queries like "mel Movies:braveheart". : Right ? : : Would this impact the search performance ? : Any other words of caution for me ?

Re: Does lucene performance suffer with a lot of empty fields ?

2006-08-01 Thread Erick Erickson
I can't speak to performance, but there's no problem having different fields for different documents. Stated differently, you don't need to have all fields in all documents. It took me a while to get my head out of database tables and accept this I doubt there's a problem with speed, but as

Does lucene performance suffer with a lot of empty fields ?

2006-08-01 Thread Mek
I have 1 generic index, but am Indexing a lot of different things, like actors, politicians, scientists, sportsmen. And as you can see that though there are some common fields, like name & DOB, there are also fields for each of these types of people that are different. e.g. Actors will have "Movi

Re: Index empty fields

2006-07-31 Thread Yonik Seeley
On 7/31/06, Simon Willnauer <[EMAIL PROTECTED]> wrote: Hello, I do have a question about fields with empty content should be added to the document / index or not. I do have a index schema, which defines all field a document can have. if one of the real documents has no content for a certain fiel

Re: Index empty fields

2006-07-31 Thread Otis Gospodnetic
eld during indexing, and put a dummy value it in instead of am empty string. Otis - Original Message From: Simon Willnauer <[EMAIL PROTECTED]> To: java-user@lucene.apache.org Sent: Monday, July 31, 2006 1:11:22 PM Subject: Re: Index empty fields Hi Otis, well if i have to such

Re: Index empty fields

2006-07-31 Thread Simon Willnauer
;-type attribute with that dummy value. Perhaps Solr already has something like this, I'm not sure. Otis - Original Message From: Simon Willnauer <[EMAIL PROTECTED]> To: java-user@lucene.apache.org Sent: Monday, July 31, 2006 8:22:57 AM Subject: Index empty fields Hello, I do ha

Re: Index empty fields

2006-07-31 Thread Otis Gospodnetic
aultValue'-type attribute with that dummy value. Perhaps Solr already has something like this, I'm not sure. Otis - Original Message From: Simon Willnauer <[EMAIL PROTECTED]> To: java-user@lucene.apache.org Sent: Monday, July 31, 2006 8:22:57 AM Subject: Index empty fie

Index empty fields

2006-07-31 Thread Simon Willnauer
Hello, I do have a question about fields with empty content should be added to the document / index or not. I do have a index schema, which defines all field a document can have. if one of the real documents has no content for a certain field. should that field be added to the index or not. Would

Re: Empty fields ...

2006-07-20 Thread Erick Erickson
What? You actually want me to put forth some effort? That's crazy talk .. Thanks, I think I've got it now. Best Erick

Re: Empty fields ...

2006-07-20 Thread Chris Hostetter
: Thanks much for that clarification, it helps a lot. The original request was : to find docs wthat were NOT NULL, so I'm glad I'm not the only one who : But with your RangeFilter comment, that seems unnecessary. You can use a : RangeFilter with null, null as bounds, then just flip the bits in t

Re: Empty fields ...

2006-07-19 Thread Erick Erickson
Chris: Thanks much for that clarification, it helps a lot. The original request was to find docs wthat were NOT NULL, so I'm glad I'm not the only one who remembers things...er...incongruently with reality About empty values for a field. That'll teach me to try to remember without looking

Re: Empty fields ...

2006-07-19 Thread Chris Hostetter
: Zip IS NOT NULL : : why invert the bitset? i think the orriginal request was to find all docs where the field did *not* have any value ... or in your vernacular: where Zip IS NULL : a token containing the empty string matches documents that : > contain that token : > : : Isn't this exactly what

Re: Empty fields ...

2006-07-19 Thread Erick Erickson
Ok, I'm confused again, not unusual To create a bitset for the following condition Zip IS NOT NULL why invert the bitset? a token containing the empty string matches documents that contain that token Isn't this exactly what he wants? Or am I mis-reading this? I'm reading it as "any do

Re: Empty fields ...

2006-07-19 Thread Chris Hostetter
: TermDocs termDocs = reader.termDocs(); : termDocs.seek(new Term("", "")); : while (termDocs.next()) { : bits.set(termDocs.doc()); : } : : I *think* (and I'm remembering things folks wrote, haven't done this myself) : that the empty string for the Term matches all terms. If not, you m

Re: Empty fields ...

2006-07-19 Thread Dragon Fly
Thank you very much. From: "Erick Erickson" <[EMAIL PROTECTED]> Reply-To: java-user@lucene.apache.org To: java-user@lucene.apache.org Subject: Re: Empty fields ... Date: Wed, 19 Jul 2006 09:48:04 -0400 Try something like TermDocs termDocs = reader.termDocs(); termDo

Re: Empty fields ...

2006-07-19 Thread Erick Erickson
Try something like TermDocs termDocs = reader.termDocs(); termDocs.seek(new Term("", "")); while (termDocs.next()) { bits.set(termDocs.doc()); } I *think* (and I'm remembering things folks wrote, haven't done this myself) that the empty string for the Term matches all terms. If not, y

Re: Empty fields ...

2006-07-19 Thread Dragon Fly
My index gets rebuilt every night so I probably can afford to construct the filters right after the index is rebuilt. How do I check each document (for empty fields) though? Would I use an IndexReader to loop through the documents? If so, which method(s) in the IndexReader class should I use

Re: Empty fields ...

2006-07-18 Thread Erick Erickson
Quoting the guys "it depends" ... At root, a filter is a bitset. So size-wise, you are using 1 bit/doc (plus some small overhead). Both the storage required and the time to construct are dependent on the characteristics of your corpus. I guess the only way you can answer that for your particular

Re: Empty fields ...

2006-07-18 Thread Dragon Fly
AIL PROTECTED]> Reply-To: java-user@lucene.apache.org To: java-user@lucene.apache.org Subject: Re: Empty fields ... Date: Tue, 18 Jul 2006 10:41:28 -0400 You could use a filter. You'd just construct a filter with the bits on for all docs that had that term. You could cache these to make futur

Re: Empty fields ...

2006-07-18 Thread Erick Erickson
You could use a filter. You'd just construct a filter with the bits on for all docs that had that term. You could cache these to make future searchers easier. I envision, say, caching one filter for each field and keeping them around so you could re-use them whenever you wanted. See ConstantScore

Empty fields ...

2006-07-18 Thread Dragon Fly
Hi, I have 10 fields in my index and some of the fields can be empty. I'd like to be able do something like "IS NOT NULL" in SQL. For example: Name:Jane AND Addr IS NOT NULL AND Zip IS NOT NULL Zip:90210 AND Name IS NOT NULL Is there an easy way to do this? Thank you. _