Re: Negative Filtering (such as for profanity)

2007-03-12 Thread Daniel Noll
I _think_ Lucene 2.1 (or is it trunk?, I lose track) has the ability to delete all documents containing a term. Actually it's been in IndexReader for longer than I can remember. We're still on 1.4.3 and it's in there. (Only difference in 2.1 is that it's now on IndexWriter as well.) Dan

Re: Negative Filtering (such as for profanity)

2007-03-08 Thread Grant Ingersoll
8 PM Subject: Re: Negative Filtering (such as for profanity) Not sure if this helpful given your proposed solution, but could you do something on the indexing side, such as: 1. Remove the profanity from the token stream, much like a stopword. This would also mean stripping it from the displ

Re: Negative Filtering (such as for profanity)

2007-03-07 Thread Doron Cohen
Also, QueryParser would generate that combination with: *:* -naughty1 -naughty2 > Thanks! I was not aware of that class, for some reason. >> http://lucene.apache. >> org/java/docs/api/org/apache/lucene/search/MatchAllDocsQuery.html >> >> You can use that Query in front of a NOT query clause.

Re: Negative Filtering (such as for profanity)

2007-03-07 Thread Greg Gershman
ser@lucene.apache.org Sent: Wednesday, March 7, 2007 2:07:38 PM Subject: Re: Negative Filtering (such as for profanity) Not sure if this helpful given your proposed solution, but could you do something on the indexing side, such as: 1. Remove the profanity from the token stream, much like a

Re: Negative Filtering (such as for profanity)

2007-03-07 Thread Paul Elschot
On Wednesday 07 March 2007 16:07, Greg Gershman wrote: > I'm attempting to create a profanity filter. I thought to use a QueryFilter created with a Query of (-$#!+ AND [EMAIL PROTECTED] AND etc). The problem I have run into is that, as a pure negative query is not supported (a query for (-term

Re: Negative Filtering (such as for profanity)

2007-03-07 Thread Grant Ingersoll
Not sure if this helpful given your proposed solution, but could you do something on the indexing side, such as: 1. Remove the profanity from the token stream, much like a stopword. This would also mean stripping it from the display text 2. If your TokenFilter comes across a profanity, some

Re: Negative Filtering (such as for profanity)

2007-03-07 Thread Greg Gershman
: Wednesday, March 7, 2007 11:02:51 AM Subject: Re: Negative Filtering (such as for profanity) Thanks! I was not aware of that class, for some reason. I tried creating a NegativeQueryFilter, it works just fine. Can you think of any reason why one approach would be better than the other? If there&

Re: Negative Filtering (such as for profanity)

2007-03-07 Thread mark harwood
harwood <[EMAIL PROTECTED]> To: java-user@lucene.apache.org Sent: Wednesday, 7 March, 2007 4:05:56 PM Subject: Re: Negative Filtering (such as for profanity) Sounds like the sort of filter that could be usefully cached. You can do all this in Java code or the XML query parser (in contrib) mi

Re: Negative Filtering (such as for profanity)

2007-03-07 Thread mark harwood
)); //run query as normal Cheers Mark - Original Message From: Greg Gershman <[EMAIL PROTECTED]> To: java-user@lucene.apache.org Sent: Wednesday, 7 March, 2007 3:07:45 PM Subject: Negative Filtering (such as for profanity) I'm attempting to create a profanity filt

Re: Negative Filtering (such as for profanity)

2007-03-07 Thread Greg Gershman
al Message From: Mark Miller <[EMAIL PROTECTED]> To: java-user@lucene.apache.org Sent: Wednesday, March 7, 2007 10:19:32 AM Subject: Re: Negative Filtering (such as for profanity) http://lucene.apache.org/java/docs/api/org/apache/lucene/search/MatchAllDocsQuery.html You can use that Query i

Re: Negative Filtering (such as for profanity)

2007-03-07 Thread Mark Miller
http://lucene.apache.org/java/docs/api/org/apache/lucene/search/MatchAllDocsQuery.html You can use that Query in front of a NOT query clause. Greg Gershman wrote: I'm attempting to create a profanity filter. I thought to use a QueryFilter created with a Query of (-$#!+ AND [EMAIL PROTECTED] A

Negative Filtering (such as for profanity)

2007-03-07 Thread Greg Gershman
I'm attempting to create a profanity filter. I thought to use a QueryFilter created with a Query of (-$#!+ AND [EMAIL PROTECTED] AND etc). The problem I have run into is that, as a pure negative query is not supported (a query for (-term) DOES NOT return the inverse of a query for (term)), I b