RE: DIH Strange Problem

2011-11-23 Thread Husain, Yavar
Hi Thanks for your replies. I carried out these 2 steps (it did not solve my problem): 1. I tried setting responseBuffering to adaptive. Did not work. 2. For checking Database connection I wrote a simple java program to connect to database and fetch some results with the same driver that I use

Re: need a way so that solr return result for misspelled terms

2011-11-23 Thread meghana
this seems to be good. if it is possible , i want to make it from solr features / configuration changes. but can go for it, if it is not possible or not much compatible. Thanks. iorixxx wrote > >> I have configured spellchecker component in my solr. it >> works with custom >> request handler (h

Re: need a way so that solr return result for misspelled terms

2011-11-23 Thread meghana
We are using solr query parser... just need some schema and / or solrconfig configuration to do the misspell search and find results. -- View this message in context: http://lucene.472066.n3.nabble.com/need-a-way-so-that-solr-return-result-for-misspelled-terms-tp3530584p3532979.html Sent from th

Re: trouble with CollationKeyFilter

2011-11-23 Thread Michael Sokolov
Thanks for confirming that, and laying out the options, Robert. -Mike On 11/23/2011 9:03 PM, Robert Muir wrote: hi, locale sensitive range queries don't work with these filters, only sort, although erick erickson has a patch that will enable this (the lowercasing wildcards patch, then you coul

Re: Solr real time update

2011-11-23 Thread yu shen
Thanks for the information. I will play with it. Spark 2011/11/23 Nagendra Nagarajayya > Spark: > > Solr with RankingAlgorithm is not a plugin but a change of search library > from Lucene to RankingAlgorithm. Here is more info on the changes you will > need to make to your solrconfig.xml: > > h

Re: Separate ACL and document index

2011-11-23 Thread Floyd Wu
Thank you for your sharing, My current solution is similar to 2). But my problem is ACL is early-binding (which means I build index and embedded ACL with document index) I don't want to rebuild full index(a lucene/solr Document with PDF content and ACL) when front end change only permission setting

Re: Autocomplete(terms) performance problem

2011-11-23 Thread solr-ra
I have now enabled the infix search. So you will be able to do both edge as well as infix search. Type "francisco peak" in the edge field, and in the below infix input field, try, "cisco peak", both will get you to the same selections. Please give it a try now: http://solr-ra.tgels.org/solr-ra-au

Re: trouble with CollationKeyFilter

2011-11-23 Thread Robert Muir
hi, locale sensitive range queries don't work with these filters, only sort, although erick erickson has a patch that will enable this (the lowercasing wildcards patch, then you could add this filter to your multiterm chain). separately locale range queries and sort both work easily on trunk (wit

Re: Dismax, pf and qf

2011-11-23 Thread Chris Hostetter
: Now there are some scenario when I want just the pf active (without : qf). Othen then surrounding my query with double quotes, is there : another way to do that? I mean, i would like to do the following : : _query:"{!dismax pf=author^100}vincent kwner" ...nope ... the pf is just a boosting fac

Re: WordDelimiterFilter MultiPhraseQuery case insesitive Issue

2011-11-23 Thread Shawn Heisey
On 11/23/2011 2:54 PM, Uomesh wrote: Hi, case insesitive search is not working if I use WordDelimiterFilter splitOnCaseChange="1" I am searching for word norton and here is result norton: returns result Norton: returns result but nOrton: no results I want nOrton should results. Please help. b

Synonyms 1 fetching 2001, how to avoid

2011-11-23 Thread RaviWhy
Hi, I am searching on movie titles. with synonyms text file mapped to 1,one. With this, when I am searching for '1' I am expecting '1 in kind' but I am getting results which have titles like "2001: My year" . I am using query time analyser with I am going to try with expand=false. But anyt

WordDelimiterFilter MultiPhraseQuery case insesitive Issue

2011-11-23 Thread Uomesh
Hi, case insesitive search is not working if I use WordDelimiterFilter splitOnCaseChange="1" I am searching for word norton and here is result norton: returns result Norton: returns result but nOrton: no results I want nOrton should results. Please help. below is my field type.

Re: FunctionQuery score=0

2011-11-23 Thread John
Thanks Hoss, I will give those a try and let you know. Cheers. On Wed, Nov 23, 2011 at 8:35 PM, Chris Hostetter wrote: > > : Which answers my query needs. BUT, my boost function actually changes > some > : of the results to be of score 0, which I want to be excluded from the > : result set. > >

trouble with CollationKeyFilter

2011-11-23 Thread Michael Sokolov
I'm using CollectionKeyFilter to sort my documents using the Unicode root collation, and my documents do appear to be getting sorted correctly, but I'm getting weird results when performing range filtering using the sort key field. For example: ifp_sortkey_ls:["youth culture" TO "youth cultur

Re: Separate ACL and document index

2011-11-23 Thread Robert Stewart
I have used two different ways: 1) Store mapping from users to documents in some external database such as MySQL. At search time, lookup mapping for user to some unique doc ID or some group ID, and then build query or doc set which you can cache in SOLR process for some period. Then use that as

Re: strange behavior of scores and term proximity use

2011-11-23 Thread Ariel Zerbib
I tested with the version 4.0-2011-11-04_09-29-42. Ariel 2011/11/17 Erick Erickson > Hmmm, I'm not seeing similar behavior on a trunk from today, when did > you get your copy? > > Erick > > On Wed, Nov 16, 2011 at 2:06 PM, Ariel Zerbib > wrote: > > Hi, > > > > For this term proximity query: a

Re: Solr dismax scoring and weight

2011-11-23 Thread darul
Thanks a lot Erick for this explanation. Do you mean words are stored in bytes, that's it ? -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-dismax-scoring-and-weight-tp3490096p3531917.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: FunctionQuery score=0

2011-11-23 Thread Chris Hostetter
: Which answers my query needs. BUT, my boost function actually changes some : of the results to be of score 0, which I want to be excluded from the : result set. Ok .. so the crux of the issue is that your boost function results in a value of 0 for some documents, and you would like those docum

Re: Problems with AutoSuggest feature(Terms Components)

2011-11-23 Thread Erick Erickson
I'll have to defer that to one of the sharding experts. Best Erick On Tue, Nov 22, 2011 at 1:28 PM, mechravi25 wrote: > Hi Erick, > > Thanks for your reply. I would know all the options that can be given under > the defaults section and how they can be overridden. is there any > documentatio

Re: Re : AW: How to select all docs of 'today' ?

2011-11-23 Thread Erick Erickson
One subtlety to note is that caching is messed up by this form since NOW evaluates to the second, and submitting two successive queries exactly like this won't re-use the cache. On a query like this it may not matter unless you're paging But on filter queries, its a good habit to cultivate to

Re: Problem with pdf files indexing

2011-11-23 Thread Erick Erickson
The first thing I'd do is go over to the server and try using the admin interface to query on *:*. If that returns nothing, look at the admin/schema browser page and see what's in your fields, if anything. Then go back to SolrJ and work on the query part sans the indexing part once you're sure you

Re: Architecture and Capacity planning for large Solr index

2011-11-23 Thread Erick Erickson
Whether three shards will give you adequate throughput is not an answerable question. Here's what I suggest. Get a single box of the size you expect your servers to be and index 1/3 of your documents on it. Run stress tests. That's really the only way to be fairly sure your hardware is adequate. A

Highlighting too much, indexing not seeing commas?

2011-11-23 Thread Robert Brown
Solr 3.3.0 I have a field/type indexed as below. For a particular document the content of this field is 'FreeBSD,Perl,Linux,Unix,SQL,MySQL,Exim,Postgresql,Apache,Exim' Using eDismax, mm=1 When I query for... +perl +(apache sql) +(linux unix) Strangely, the highlighting is being returned as

Problem with Solr logging under Jetty

2011-11-23 Thread Shawn Heisey
I am having a problem with jdk logging with Solr, using the jetty included with Solr. In jetty.xml, I have the following defined: java.util.logging.config.file etc/logging.properties Contents of etc/logging.properties: == # Logging level .level=WARNING # Write to a file handlers = java.

Re: Huge Performance: Solr distributed search

2011-11-23 Thread Robert Stewart
If you request 1000 docs from each shard, then aggregator is really fetching 30,000 total documents, which then it must merge (re-sort results, and take top 1000 to return to client). Its possible that SOLR merging implementation needs optimized, but it does not seem like it could be that slow. H

Re: Integrating Surround Query Parser

2011-11-23 Thread Ahmet Arslan
> is this is the trunk of solr 4.0 , > can't i implement in solr 3.1 .? Author of the patch would know answer to this. But why not use trunk?

Re: Huge Performance: Solr distributed search

2011-11-23 Thread Artem Lokotosh
> If the response time from each shard shows decent figures, then aggregator> > seems to be a bottleneck. Do you btw have a lot of concurrent users?For now > is not a problem, but we expect from 1K to 10K of concurrent users and maybe > more On Wed, Nov 23, 2011 at 4:43 PM, Dmitry Kan wrote: >

Re: how to : multicore setup with same config files

2011-11-23 Thread Vadim Kisselmann
Hi, yes, see http://wiki.apache.org/solr/DistributedSearch Regards Vadim 2011/11/2 Val Minyaylo > Have you tried to query multiple cores at same time? > > > On 10/31/2011 8:30 AM, Vadim Kisselmann wrote: > >> it works. >> it was one wrong placed backslash in my config;) >> sharing the config/sc

Re: Huge Performance: Solr distributed search

2011-11-23 Thread Dmitry Kan
If the response time from each shard shows decent figures, then aggregator seems to be a bottleneck. Do you btw have a lot of concurrent users? On Wed, Nov 23, 2011 at 4:38 PM, Artem Lokotosh wrote: > > Is this log from the frontend SOLR (aggregator) or from a shard? > from aggregator > > > Can

Re: Autocomplete(terms) performance problem

2011-11-23 Thread roySolr
Thanks for your answer Nagendra, The problem is i want to do some infix searches. When i search for "sisco" i want the autocomplete with "san fran*sisco*". In the example you gave me it's also not possible. Roy -- View this message in context: http://lucene.472066.n3.nabble.com/Autocomplete-ter

Re: need a way so that solr return result for misspelled terms

2011-11-23 Thread Erik Hatcher
Sure... if you're using the "lucene" query parser and put a ~ after every term in the query :) But that would mean that either the users or your application do this. Erik On Nov 23, 2011, at 09:03 , meghana wrote: > Hi Erik, > > Thanks for your reply. i come to know that Lucene pro

Re: Huge Performance: Solr distributed search

2011-11-23 Thread Artem Lokotosh
> Is this log from the frontend SOLR (aggregator) or from a shard? from aggregator > Can you merge, e.g. 3 shards together or is it much effort for your team? Yes, we can merge. We'll try to do this and review how it will works Thanks, Dmitry Any another ideas? On Wed, Nov 23, 2011 at 4:01 PM, D

Re: Collection Distribution vs Replication in Solr

2011-11-23 Thread Alireza Salimi
Yeah, and actually later I've found someone mentioned that they had done some benchmarks and found that replication is faster than collection distribution. Thanks On Wed, Nov 23, 2011 at 9:02 AM, Mark Miller wrote: > > On Oct 27, 2011, at 2:57 PM, Alireza Salimi wrote: > > > Hi guys, > > > > If

Re: Integrating Surround Query Parser

2011-11-23 Thread Rahul Mehta
is this is the trunk of solr 4.0 , can't i implement in solr 3.1 .? On Wed, Nov 23, 2011 at 7:23 PM, Ahmet Arslan wrote: > > After this i tried with solr3.1-src. > > Please suggest what should i do ? > > Please use solr-trunk. > svn checkout http://svn.apache.org/repos/asf/lucene/dev/trunk >

Re: need a way so that solr return result for misspelled terms

2011-11-23 Thread Ahmet Arslan
> I have configured spellchecker component in my solr. it > works with custom > request handler (however its not working with standard > request handler , but > this is not concern at now) . but its returning suggestions > for the matching > spells, instead of it we want that we can directly get >

UpdateRequestProcessor - processCommit

2011-11-23 Thread Matthew Parker
TWIMC: I creating a custom UpdateRequestProcessor chain, where I need to commit records to a database once the import process has completed. I'm assuming the processCommit method is called for each UpdateRequestProcessor chain class when the records are being commited to the Lucene index. I'm de

Re: DIH Strange Problem

2011-11-23 Thread Shawn Heisey
On 11/23/2011 5:21 AM, Chantal Ackermann wrote: Hi Yavar, my experience with similar problems was that there was something wrong with the database connection or the database. Chantal It's also possible that your JDBC driver might be trying to buffer the entire result set. There's a link on

Re: To push the terms.limit parameter from the master core to all the shard cores.

2011-11-23 Thread Mark Miller
On Nov 22, 2011, at 1:31 PM, mechravi25 wrote: > Can you please suggest the definition of the terms component for the > underlying shard cores. If you look at my earlier email, you will see the limit is set in "invariants" rather than defaults. This makes it so the param cannot be dynamically

Re: need a way so that solr return result for misspelled terms

2011-11-23 Thread meghana
Hi Erik, Thanks for your reply. i come to know that Lucene provides the fuzzy search by applying tilde("~") symbol at the end of search with like delll~0.8 can we apply such fuzzy logic in solr in any way? Thanks Meghana Erik Hatcher-4 wrote > > Meghana - > > There's currently no facility

Re: Collection Distribution vs Replication in Solr

2011-11-23 Thread Mark Miller
On Oct 27, 2011, at 2:57 PM, Alireza Salimi wrote: > Hi guys, > > If we ignore the features that Replication provides ( > http://wiki.apache.org/solr/SolrReplication#Features), > which approach is better? > Is there any performance problems with Replication? > > Replications seems quite easier

Re: Huge Performance: Solr distributed search

2011-11-23 Thread Dmitry Kan
Hello, Is this log from the frontend SOLR (aggregator) or from a shard? Can you merge, e.g. 3 shards together or is it much effort for your team? In our setup we currently have 16 shards with ~30GB each, but we rarely search in all of them at once. Best, Dmitry On Wed, Nov 23, 2011 at 3:12 PM,

Re: Solr Performance/Architecture

2011-11-23 Thread Shawn Heisey
On 11/22/2011 11:52 PM, Husain, Yavar wrote: Hi Shawn That was so great of you to explain the architecture in such a detail. I enjoyed reading it multiple times. I have a question here: You mentioned that we can use crc32(DocumentId)% NumServers. Now actually I am using that in my data-confi

Re: Integrating Surround Query Parser

2011-11-23 Thread Ahmet Arslan
> After this i tried with solr3.1-src. > Please suggest what should i do ? Please use solr-trunk. svn checkout http://svn.apache.org/repos/asf/lucene/dev/trunk

Re: wild card search and lower-casing

2011-11-23 Thread Dmitry Kan
Yes, it should be ok, as currently we are on the English side. If that's beneficial for the effort, I could do a field test on 3.4 after you close the jira. Best, Dmitry On Wed, Nov 23, 2011 at 2:52 PM, Erick Erickson wrote: > Ah, I see what you're doing, go for it. > > I intend to commit it tod

Re: how to make effective search with fq and q params

2011-11-23 Thread Erik Hatcher
Meghana - Some important points about q/fq - * q is used for scoring. fq is for filtering, no scoring. * fq and q are cached independently You may want to combine the user entered terms (search term, title, and desc) in the q parameter. It's complicated/advanced, but you can use nested q

Re: how to make effective search with fq and q params

2011-11-23 Thread Erik Hatcher
Jeff, Just to clarify - with edismax, q=*:* is fine and matches all documents. With dismax (and also edismax), q.alt with no q is needed to match all documents. Erik On Nov 23, 2011, at 08:20 , Jeff Schmidt wrote: > Thanks, Erik. I'm moving on to edismax, and will set q.alt=*:* an

Re: need a way so that solr return result for misspelled terms

2011-11-23 Thread Erik Hatcher
Meghana - There's currently no facility in Solr to return results for suggestions automatically. You'll have to code this into your client to make another request to Solr for the suggestions returned from the first request. Erik On Nov 23, 2011, at 07:58 , meghana wrote: > Hi, > > I

Re: how to make effective search with fq and q params

2011-11-23 Thread Jeff Schmidt
Thanks, Erik. I'm moving on to edismax, and will set q.alt=*:* and not specify q if no user provided terms. Take it easy, Jeff On Nov 22, 2011, at 11:53 AM, Erik Hatcher wrote: > I think you're using dismax, not edismax. edismax will take q=*:* just fine > as it handles all Lucene syntax que

Huge Performance: Solr distributed search

2011-11-23 Thread Artem Lokotosh
Hi! * Data: - Solr 3.4; - 30 shards ~ 13GB, 27-29M docs each shard. * Machine parameters (Ubuntu 10.04 LTS): user@Solr:~$ uname -a Linux Solr 2.6.32-31-server #61-Ubuntu SMP Fri Apr 8 19:44:42 UTC 2011 x86_64 GNU/Linux user@Solr:~$ cat /proc/cpuinfo processor : 0 - 3 vendor_id : Genui

need a way so that solr return result for misspelled terms

2011-11-23 Thread meghana
Hi, I have configured spellchecker component in my solr. it works with custom request handler (however its not working with standard request handler , but this is not concern at now) . but its returning suggestions for the matching spells, instead of it we want that we can directly get result for

Re: wild card search and lower-casing

2011-11-23 Thread Erick Erickson
Ah, I see what you're doing, go for it. I intend to commit it today, but things happen. About changing the setLowerCaseExpandedTerms(true), yes that'll take care of this issue, although it has some locale-specific assumptions (i.e. string.toLowerCase() uses the default locale). That may not m

Re: how to make effective search with fq and q params

2011-11-23 Thread meghana
Hi Erik , Actually right now we can say that almost is done in filtering and passing q as *:* , but we need to find out a better way if there is any. So according to pravesh , i m thinking of to pass user entered text in query and date and other fields in filter query? or as per you q=*:* is fast

Re: Solr Search for misspelled search term

2011-11-23 Thread meghana
I have configured specllchecker component in my solrconfig below is the configuration false false 1 spellcheck using above configuration it works with below url http://192.168.1.59:8080/solr/core0/spellcheck?q=sc:directry&spel

Re: DIH Strange Problem

2011-11-23 Thread Chantal Ackermann
Hi Yavar, my experience with similar problems was that there was something wrong with the database connection or the database. Chantal On Wed, 2011-11-23 at 11:57 +0100, Husain, Yavar wrote: > I am using Solr 1.4.1 on Windows/MS SQL Server and am using DIH for importing > data. Indexing and al

Re: Solr Search for misspelled search term

2011-11-23 Thread neuron005
Do you mean stemming? For misspelled words you will have to edit your dictionary (stopwords.txt) i think where you can set solution for misspelled words! Hope So :) -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-Search-for-misspelled-search-term-tp3529961p3530504.html S

How to configure /select handler ?

2011-11-23 Thread neuron005
Another newbie question here Browse handler works perfect. Now I want to configure my "/select" handler so that I perform ajax-solr on it. How to perform it. The website https://github.com/evolvingweb/ajax-solr https://github.com/evolvingweb/ajax-solr explains how to perform it. I

Re: date range in solr 3.1

2011-11-23 Thread do3do3
what i got is the number of this period but i want to get this result only, what is the query to can get that like fq=source:"news" -- View this message in context: http://lucene.472066.n3.nabble.com/date-range-in-solr-3-1-tp3527498p3530424.html Sent from the Solr - User mailing list archive at

Re: Search on multiple fields is not working

2011-11-23 Thread Dmitry Kan
you probably wanted to query this: q=profileId:"99964" OR profileId:"10076" OR tagName:("MUSIC" AND "DESIGNER") otherwise SOLR matches "DESIGNER" against you default field (whatever it is) and ANDs it with tagName:"MUSIC" On Wed, Nov 23, 2011 at 11:07 AM, sivaprasad wrote: > Hi, > > I have two

DIH Strange Problem

2011-11-23 Thread Husain, Yavar
I am using Solr 1.4.1 on Windows/MS SQL Server and am using DIH for importing data. Indexing and all was working perfectly fine. However today when I started full indexing again, Solr halts/stucks at the line "Creating a connection for entity." There are no further messages after that.

Re: Integrating Surround Query Parser

2011-11-23 Thread Rahul Mehta
After this i tried with solr3.1-src. - and this time i got the core folder in the previous installation ,when this folder get created - /home/reach121/basf/*apache-solr-3.1.0/core/src/test/org/apache/solr/search/TestSurroundQueryParser.java * - and i have putted * * - but when

Search on multiple fields is not working

2011-11-23 Thread sivaprasad
Hi, I have two indexed fields called profileId and tagName.When i issue a query like q=profileId:"99964" OR profileId:"10076" OR tagName:"MUSIC" AND "DESIGNER", i am getting only the results for tagName:"MUSIC" AND "DESIGNER".The results are not containing profileId 99964 and 10076. Can anybody t

Re: Collection Distribution vs Replication in Solr

2011-11-23 Thread Andre Bois-Crettez
Indeed, I can not see any of the 3 images here : http://wiki.apache.org/solr/SolrReplication#Admin_Page_for_Replication It just displays the name of image file, as the img url seem to point to a logged-only link such as this one : http://wiki.apache.org/solr/SolrReplication?action=AttachFile&do=ge

Sort question

2011-11-23 Thread vraa
Hi I have a query where i sort by a column "price". This field can contain the following values 10 75000 15 1 225000 50 40 I want to sort these values so that always between 0 and 100 always comes last. Eg sorting by price asc should look like this: 75000 10 15 225000 1 40 50 I

Re: date range in solr 3.1

2011-11-23 Thread do3do3
i works now great thanks for you :) -- View this message in context: http://lucene.472066.n3.nabble.com/date-range-in-solr-3-1-tp3527498p3530038.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: Separate ACL and document index

2011-11-23 Thread Floyd Wu
I've been read much about Document Level Security https://issues.apache.org/jira/browse/SOLR-1895 https://issues.apache.org/jira/browse/SOLR-1872 https://issues.apache.org/jira/browse/SOLR-1834 But I not fully sure that these patch solved my problem? It seems to that change the original document A