Storing Json field in Lucene

2020-04-21 Thread ganesh m
Hi I am currently storing indexed field and stored field in separate database. In stored field database, Document Id, Type and Json string of metadata will be stored. Basically i am using it as key-value pair database. For every document to be indexed, we have three different metadata structure

Re: Storing and retrieving Java objects in Lucene

2018-02-19 Thread ganesh m
easily convert back to object. Regards Ganesh On 20-02-2018 08:34, Kumar, Santosh wrote: Hi, I have a requirement to store a Java object with multiple fields into the Lucene index. Basically, at the application startup I run a select query on entities ( there are 5 of them as of now and may in

Pagination using searchAfter

2015-09-04 Thread Ganesh
Ganesh - To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org

Pagination using searchAfter

2015-09-04 Thread Ganesh
the index. Every minute the IndexReader will be reopened. public TopDocs searchAfter(ScoreDoc after, Query query, int numHits) throws IOException Please let me know how to handle this. Regards Ganesh - To unsubscribe, e-mail

Fetching stored data takes more time

2014-07-30 Thread Ganesh
resolve this issue. Regards Ganesh - To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org

Re: deleteDocument with NRT

2014-07-14 Thread Ganesh
How Solr handles this scenario... Is it reopening reader after every delete OR it maintains the list of delete documents in cache? Regards Ganesh On 7/11/2014 4:00 AM, Tri Cao wrote: You need to reopen your searcher after deleting. From Java doc for SearcherManager: In addition you should

Re: Having problem with indexing/ searching with _ or -

2014-07-06 Thread Ganesh
Hi Smitha, You need to have your own custom analyzer which breaks the word by - or _. Use the same analyzer for indexing and searching. Regards Aditya www.findbestopensource.com On 7/4/2014 11:41 AM, Smitha Kuldeep (smtt) wrote: Hello team, We are using lucen-core-2.9.1.jar for indexing and

Re: Concurrent Execution Exception

2013-02-14 Thread Ganesh M
type. This is working fine for most of the customers and till now i didn't faced any issue.   While creating SortField, do i need to pass Default Int parser? I guess, currently it is using encoded int parser.   Regards Ganesh From: Uwe Schindler To: java-user@lucene.apache.org; &#

Concurrent Execution Exception

2013-02-14 Thread Ganesh M
really an INT?) ID: 256566961   org.apache.lucene.search.ParallelMultiSearcher$ExecutionHelper.next(ParallelMultiSearcher.java:225)   org.apache.lucene.search.ParallelMultiSearcher.search(ParallelMultiSearcher.java:127)  org.apache.lucene.search.Searcher.search(Searcher.java:49)   Regards Ganesh

Re: Multiple sort field

2012-07-17 Thread Ganesh
Any thoughts on this? Regards Ganesh - Original Message - From: "Ganesh" To: Sent: Tuesday, July 17, 2012 4:23 PM Subject: [Bulk] Multiple sort field Hello all, I have more than one record having same time stamp. When i sort by date time in decending order, the set

Multiple sort field

2012-07-17 Thread Ganesh
again sort on docid in reverse order. record_4 201207170102 record_3 201207170102 record_2 201207170101 record_1 201207170101 Is it good to use multiple sort fields? Using sort on docid will consume any memory? Is there any other way out to acheive this. Regards Ganesh

Re: Upgrade to 3.6 OR wait for 4.0

2012-07-09 Thread Ganesh
Thanks for the reply. Any idea how much time it would take to go for 4.0 stable release? I want to go for v4.0 but i have to use only the stable version. Regards Ganesh - Original Message - From: "Shai Erera" To: Sent: Tuesday, July 10, 2012 10:50 AM Subject: Re: Upgr

Upgrade to 3.6 OR wait for 4.0

2012-07-09 Thread Ganesh
Hello all, I am currently using v3.0.3 and planning to upgrade to v3.6. Shall i go ahead with the upgrade OR wait for 4.0? Regards Ganesh - To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional

Taking backup of Lucene DB

2012-05-25 Thread Ganesh
another system. When i started my application in another system, I could see my search database whiped out. Only write lock remains. The behaviour is unpredictable. I am not using SnapshotDeletionPolicy. Is that any other way to acheive this using SAN replicator? Regards Ganesh

Re: QueryParser strange behavior

2012-03-01 Thread Ganesh
It is fixed now. I need to enable qp.setAutoGeneratePhraseQueries(true); Regards Ganesh - Original Message - From: "Ganesh" To: Sent: Friday, March 02, 2012 10:46 AM Subject: Re: QueryParser strange behavior Hello Damerian, Please let me know, How you fixed the pro

Re: QueryParser strange behavior

2012-03-01 Thread Ganesh
her of the token matches, it returns the results. Regards Ganesh - Original Message - From: "Damerian" To: Sent: Thursday, March 01, 2012 7:57 PM Subject: Re: QueryParser strange behavior > Στις 1/3/2012 3:08 μμ, ο/η Ian Lea έγραψε: >> Not a clue. I suggest you po

Re: Lucene performance in 64 Bit

2012-03-01 Thread Ganesh
Thanks Li Li. Please share your experience in 64 bit. How big your indexes are? Regards Ganesh - Original Message - From: "Li Li" To: Sent: Thursday, March 01, 2012 3:03 PM Subject: Re: Lucene performance in 64 Bit >I think many users of lucene use large memory

Re: [Bulk] RE: RE: Date time as String or Numeric field

2012-02-28 Thread Ganesh
Thanks. I use this field for Rangequery and sort. I think it is best to use Int to gain some heap. Regards Ganesh - Original Message - From: "Uwe Schindler" To: Sent: Tuesday, February 28, 2012 5:08 PM Subject: [Bulk] RE: RE: Date time as String or Numeric field > Hi,

Re: RE: Date time as String or Numeric field

2012-02-28 Thread Ganesh
I tried NumericField with Integer value and Long value. There is no difference in space and heap utilization. Will it be? Are both are same? Regards Ganesh - Original Message - From: "Uwe Schindler" To: Sent: Tuesday, February 28, 2012 3:52 PM Subject: [Bulk] RE: Date time

Date time as String or Numeric field

2012-02-28 Thread Ganesh
Hello all, I was using DateTime as String and now i am using NumericField. Using NumericField takes more heap and storage space then the earlier String version. Is it good to move to NumericField or stick with String. I am using this field for search and sort. Regards Ganesh

Re: Multiple index vs Single Index

2012-02-23 Thread Ganesh
index. Regards Ganesh - Original Message - From: "Ian Lea" To: Sent: Thursday, February 23, 2012 3:40 PM Subject: Re: Multiple index vs Single Index Millions of docs in a single index is definitely OK. If it was my system I'd willingly trade slightly slower indexing for

Multiple index vs Single Index

2012-02-22 Thread Ganesh
. I thought to break in to multiple small index with same schema. Now optimization is discourged and the name has been changed to forceMerge to discourge its use. I just want to confirm, Is Single Index would be good for millions of records? Regards Ganesh

When deletes will be removed?

2012-02-22 Thread Ganesh
37 Feb 22 19:52 _203_1.del 332 Feb 22 20:42 _a3_k.del 487 Feb 22 20:42 _ig_j.del 511 Feb 22 20:42 _rc_j.del Regards Ganesh - To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e

Re: [Bulk] can I make incremental index/search more efficient?

2012-02-21 Thread Ganesh
, or document indexed after certain date. Regards Ganesh - Original Message - From: "Ilya Zavorin" To: Sent: Wednesday, February 22, 2012 2:39 AM Subject: [Bulk] can I make incremental index/search more efficient? >I have a fairly straightforward task: I have a co

Multiple CFS files are generated

2012-02-20 Thread Ganesh
q.tis _4xq.tvd _4xq.tvf _4xq.tvx _4xq_6.del _5o4.cfs _5wh.cfs _5wi.cfs _5wj.cfs _5wk.cfs segments.gen segments_3d8 Regards Ganesh - To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-u

IndexWriter in 3.5

2012-02-09 Thread Ganesh
documents I need to do this. When the application starts, I have my writer and reader ready. I have a thread which crawls the filesytem and adds it to index and another one performs search. Is this intentional? Regards Ganesh --

Re: upgrading from 3.0.3 to 3.5.0

2012-02-01 Thread Ganesh
tter in optimized indexes. What about 3.5.0? Regards Ganesh - Original Message - From: "Ian Lea" To: Sent: Wednesday, February 01, 2012 3:07 PM Subject: Re: upgrading from 3.0.3 to 3.5.0 > I am upgrading from 3.0.3 to 3.5.0. > > 1) NumberTools is deprecated. I am con

upgrading from 3.0.3 to 3.5.0

2012-02-01 Thread Ganesh
ent search performance has improved over time, and the default TieredMergePolicy now targets segments with deletions." This means usage of this method is not justified any more. I am doing optimization once per day. Is that not required now.

Re: [Bulk] RE: any tips for upgrading Lucene 3.0.3 -> 3.5.0?

2012-01-19 Thread Ganesh
I am also in the way to upgrade from 3.0.3 to 3.5. Any other API changes we need to care about? I use ParallelMultiSearcher. Regards Ganesh - Original Message - From: "Uwe Schindler" To: Sent: Friday, January 20, 2012 5:18 AM Subject: [Bulk] RE: any tips for upgrading Lu

Re: Lucene bangalore chapter

2011-12-30 Thread Ganesh
Count me in for Bangalore. This mail thread indicates, 3 from Bangalore and 1 from Mumbai. If we get more people then we could do some meetup. Regards Ganesh - Original Message - From: To: Sent: Tuesday, December 13, 2011 4:23 PM Subject: Re: Lucene bangalore chapter Do we have a

Re: Index size and performance degradation

2011-06-15 Thread Ganesh
less number of concurrent users going to search at a time. Regards Ganesh - Original Message - From: "Ganesh" To: ; Sent: Tuesday, June 14, 2011 3:28 PM Subject: Re: Index size and performance degradation Is it a bad idea to keep multiple shards in a single system? Rega

Re: Index size and performance degradation

2011-06-14 Thread Ganesh
Is it a bad idea to keep multiple shards in a single system? Regards Ganesh - Original Message - From: "Toke Eskildsen" To: Sent: Tuesday, June 14, 2011 12:58 PM Subject: Re: Index size and performance degradation > On Sun, 2011-06-12 at 10:10 +0200, Itamar Syn-Hershko

Re: Index size and performance degradation

2011-06-14 Thread Ganesh
Ganesh - Original Message - From: "Shai Erera" To: Sent: Sunday, June 12, 2011 9:13 AM Subject: Re: Index size and performance degradation >I agree w/ Erick, there is no cutoff point (index size for that matter) > above which you start sharding. > > What

Re: Sharding Techniques

2011-05-10 Thread Ganesh
in GB's. Small addition or deletion to the file will not cause more IO as it has to skip those bytes and write it at the end of file. Regards Ganesh - Original Message - From: "Burton-West, Tom" To: Sent: Tuesday, May 10, 2011 9:46 PM Subject: RE: Sharding Techni

Re: Sharding Techniques

2011-05-09 Thread Ganesh
? 2. Is your index optimized? I have a doubt, If we keep the indexes size to 30 GB then each file size (fdt, fdx etc) would in GB's. Small addition or deletion to the file will not cause more IO as it has to skip those bytes and write it at the end of file. Regards Ganesh - Ori

Re: RE: ParallelMultisearcher

2011-03-17 Thread Ganesh
There is no ParallelMultiSearcher.close() in 2.9.1 and its been added in 3.0. Thanks for pointing out. I am not aware of. Regards Ganesh - Original Message - From: "Uwe Schindler" To: Sent: Thursday, March 17, 2011 11:27 PM Subject: [Bulk] RE: ParallelMultisearcher &

Re: ParallelMultisearcher

2011-03-17 Thread Ganesh
shing frequently. Regards Ganesh - Original Message - From: "Devon H. O'Dell" To: Sent: Thursday, March 17, 2011 10:35 PM Subject: Re: ParallelMultisearcher > 2011/3/17 Ganesh : >> Is this bug https://issues.apache.org/jira/browse/LUCENE-2249 got fixed in >

Re: ParallelMultisearcher

2011-03-17 Thread Ganesh
Is this bug https://issues.apache.org/jira/browse/LUCENE-2249 got fixed in 3.0.3? Regards Ganesh - Original Message - From: "Ganesh" To: Sent: Thursday, March 17, 2011 7:03 PM Subject: Re: ParallelMultisearcher I tested the same with Lucene 2.9.1 and found very less th

Re: ParallelMultisearcher

2011-03-17 Thread Ganesh
I tested the same with Lucene 2.9.1 and found very less thread count but with 3.0.2 it is very high. Do i need to revert back to Lucene 2.9.1. Regards Ganesh - Original Message - From: "Ganesh" To: Sent: Thursday, March 17, 2011 5:17 PM Subject: ParallelMultisearcher

ParallelMultisearcher

2011-03-17 Thread Ganesh
Regards Ganesh - To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org

Re: Backup or replication option with lucene

2011-03-02 Thread Ganesh
Any suggestions. We are planning to move towords cloud and its become a mandatory requirement to have backup or replication of search db. Regards Ganesh - Original Message - From: "Ganesh" To: Sent: Tuesday, March 01, 2011 12:06 PM Subject: Backup or replication option w

Backup or replication option with lucene

2011-02-28 Thread Ganesh
Hello all, Could any one guide me how to backup or do replication with Lucene. Regards Ganesh Send free SMS to your Friends on Mobile from your Yahoo! Messenger. Download Now! http://messenger.yahoo.com/download.php - To

Re: Re: Facet search

2011-02-23 Thread Ganesh
) /Articles/aaa (10) Regards Ganesh - Original Message - From: "Chris Hostetter" To: "Lucene Users" Sent: Thursday, February 24, 2011 7:29 AM Subject: [Bulk] Re: Facet search > > : This is another indicator that we should re

Facet search

2011-02-23 Thread Ganesh
US(50) /Articles/xyz (25) /Articles/aaa (10) Regards Ganesh Send free SMS to your Friends on Mobile from your Yahoo! Messenger. Download Now! http://messenger.yahoo.com/downloa

Re: Keyword Analyzer

2011-02-18 Thread Ganesh
Exactly. QueryParser is splitting it in to two words. >>Just build the query directly. TermQuery or PhraseQuery? Regards Ganesh - Original Message - From: "Ian Lea" To: Sent: Friday, February 18, 2011 4:34 PM Subject: Re: Keyword Analyzer Presumably because wi

Keyword Analyzer

2011-02-18 Thread Ganesh
world"); My question is why i need to give the text within codes (basically as a phrase). While indexing i have not given and it is indexing as single term and while searching also it should consider the complete text as single term. Regards Ganesh Send free SMS to your Friends on Mobile

Re: Scale out design patterns

2011-02-03 Thread Ganesh
about the old users, their data will be still added to the server1. How to address this issue. Is rebuilding the index the only way. Could any one share their experience, How they solved scale out problems? Regards Ganesh - Original Message - From: "Anshum" To: Se

Scale out design patterns

2011-01-20 Thread Ganesh
. Regards Ganesh Send free SMS to your Friends on Mobile from your Yahoo! Messenger. Download Now! http://messenger.yahoo.com/download.php - To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e

Re: Re: Scale up design

2010-12-21 Thread Ganesh
Thanks. I going to try in 64 bit. I will post some update in a day or two. Do I need to compile the Lucene and analyzer code in 64 bit JVM? Do I need to use MMAPDirectory in 64 bit? Any other tips targeting 64 bit? Regards Ganesh - Original Message - From: "Danil ŢORIN"

Re: Re: Scale up design

2010-12-21 Thread Ganesh
Hello Simon, I don't hesitate to move to 64 bit. I require a suggestion whether to move to 64 bit (Scale up) or scale out with multiple system. I have started investigating 64 bit, i want to know about its performance and if anyone in this group has already tried using it. Regards G

Re: Re: Scale up design

2010-12-19 Thread Ganesh
I have done some benchmarking and based on that my estimate of RAM requirement would be 3 - 4 GB. My question is to go for 64 bit or scale out with 3 systems? Regards Ganesh - Original Message - From: "Toke Eskildsen" To: Sent: Thursday, December 16, 2010 4:20 PM Subje

Re: Re: Scale up design

2010-12-15 Thread Ganesh
Ganesh - Original Message - From: "Toke Eskildsen" To: Sent: Wednesday, December 15, 2010 4:36 PM Subject: [Bulk] Re: Scale up design > On Wed, 2010-12-15 at 09:42 +0100, Ganesh wrote: >> What is the advantage of going for 64 Bit. > > Larger maximum

Re: Scale up design

2010-12-15 Thread Ganesh
more ideas. We need to design whether to scale out or scale up. Regards Ganesh - Original Message - From: "Erick Erickson" To: Sent: Monday, December 13, 2010 7:00 PM Subject: Re: Scale up design Here's a great intro to the garbage collectio

Re: Scale up design

2010-12-12 Thread Ganesh
Have you tried using Lucene in 64 Bit with more than 8 GB RAM. Regards Ganesh - Original Message - From: "Pradeep Singh" To: Sent: Monday, December 13, 2010 12:16 PM Subject: Re: Scale up design > 8GB is used on laptops. For servers you need more. > > On Sun, De

Scale up design

2010-12-12 Thread Ganesh
their thoughts on this. Regards Ganesh Send free SMS to your Friends on Mobile from your Yahoo! Messenger. Download Now! http://messenger.yahoo.com/download.php - To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For

Re: PDF text extracted without spaces

2010-12-06 Thread Ganesh
Thanks for the link. I downloaded the tool and pdftotext extracts correctly. I also dropped a mail to the tika user group. It is a regression in latest release https://issues.apache.org/jira/browse/TIKA-548. Hopefully coming release will have a fix. Regards Ganesh - Original Message

Re: PDF text extracted without spaces

2010-12-03 Thread Ganesh
I first extract the contents from documents using tika and latter index it with Lucene. The problem is the extracted text from PDF using tika has no whitespaces. Regards Ganesh - Original Message - From: "McGibbney, Lewis John" To: Sent: Friday, December 03, 2010 4:40

Re: PDF text extracted without spaces

2010-12-03 Thread Ganesh
The main problem is i am not getting whitespace and newline char. This is happening only for PDF documents. Sample outoput: Someofthedifferencesare but it should be Some of the differences are Regards Ganesh - Original Message - From: "Alexander Aristov" To: Sent: Friday

PDF text extracted without spaces

2010-12-02 Thread Ganesh
. Could any one in this group could help me? I am using tika directly to extract the contents, which later gets indexed. Regards Ganesh Send free SMS to your Friends on Mobile from your Yahoo! Messenger. Download Now! http://messenger.yahoo.com/download.php

Re: best practice: 1.4 billions documents

2010-11-29 Thread Ganesh
I am using ParallelMultiSearcher and querying more than 50 shards. I get wrong results for query A NOT B. I am getting correct results if i query A -B. I am also getting correct results for wildcard and fuzzy. What is the solution? I need to use IndexSearcher with MultiReader? Regards Ganesh

Re: best practice: 1.4 billions documents

2010-11-25 Thread Ganesh
Thanks for the input. My results are sorted by date and i am not much bothered about score. Will i still be in trouble? Regards Ganesh - Original Message - From: "Robert Muir" To: Sent: Thursday, November 25, 2010 1:45 PM Subject: Re: best practice: 1.4 billions document

Re: best practice: 1.4 billions documents

2010-11-24 Thread Ganesh
now i didn't faced any issue. I used Lucene 2.9 and recently upgraded to 3.0.2. Do i need to switch to MultiReader? Regards Ganesh - Original Message - From: "Luca Rondanini" To: Sent: Monday, November 22, 2010 11:29 PM Subject: Re: best practice: 1.4 billions docu

Re: read past EOF

2010-08-11 Thread Ganesh
I am using Lucene 2.9.1 and there was no exception in the past. Regards Ganesh - Original Message - From: "Michael McCandless" To: Sent: Wednesday, August 11, 2010 3:28 PM Subject: Re: read past EOF It looks like it may be this issue: https://issues.apache.org/j

read past EOF

2010-08-10 Thread Ganesh
(DirectoryReader.java:69) org.apache.lucene.index.IndexReader.open(IndexReader.java:476) org.apache.lucene.index.IndexReader.open(IndexReader.java:403) Regards Ganesh Send free SMS to your Friends on Mobile from your Yahoo! Messenger. Download Now! http://messenger.yahoo.com

Roadmap for next major release

2010-06-24 Thread Ganesh
Hello all, What is the road map of next major release? Few days back, many have posted their expectation / ideas for the next release. What is the plan and what all the things we could expect from the next release Regards Ganesh Send free SMS to your Friends on Mobile from your Yahoo

Re: If you could have one feature in Lucene...

2010-03-01 Thread Ganesh
Replication support as like in Solr. Regards Ganesh - Original Message - From: "Grant Ingersoll" To: Sent: Wednesday, February 24, 2010 7:12 PM Subject: If you could have one feature in Lucene... > Wha

Re: If you could have one feature in Lucene...

2010-02-24 Thread Ganesh
the sort. This will certanily reduce the memory consumption. We cannot create one index for each table, which will be difficult to maintain. Regards Ganesh - Original Message - From: "Grant Ingersoll" To: Sent: Wednesday, February 24, 2010 7:12 PM Subject: If you could h

Scale Out

2010-02-08 Thread Ganesh
nodes, then i may need to sort / combine the results. Any thoughts on scaling / clustering? Whether i need to use Hadoop / Carrot etc... Regards Ganesh Send instant messages to your online friends http://in.messenger.yahoo.com

Re: [Bulk] RE: [Bulk] RE: Exception while adding document in 3.0

2010-02-02 Thread Ganesh
Yes. I am using the objects across threads. Thanks for pointing out. But still i didn't see much increase in performance by reusing documents and field objects. Regards Ganesh - Original Message - From: "Uwe Schindler" To: Sent: Tuesday, February 02, 2010 2:02 PM Sub

Re: [Bulk] RE: Exception while adding document in 3.0

2010-02-02 Thread Ganesh
this approach. Documents cannot be re-used in v3.0? Regards Ganesh - Original Message - From: "Uwe Schindler" To: Sent: Tuesday, February 02, 2010 1:41 PM Subject: [Bulk] RE: Exception while adding document in 3.0 > They can. > > - > Uwe Schindler > H.-H

Exception while adding document in 3.0

2010-02-01 Thread Ganesh
) org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:2472) org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:2446) Regards Ganesh Send instant messages to your online friends http://in.messenger.yahoo.com - To unsubscribe, e

Re: Roadmap for next release

2010-01-31 Thread Ganesh
Any reply to this thread? - Original Message - From: "Ganesh" To: Sent: Thursday, January 28, 2010 2:35 PM Subject: Roadmap for next release Hello all, Please provide me the information related to road map for the next release. This information will be really helpful t

Roadmap for next release

2010-01-28 Thread Ganesh
consumption by caching / offload it to disk 2. If all records are not part of sorting, Is there any way to create the custom field cache array based on some filter criteria. Regards Ganesh Send instant messages to your online friends http://in.messenger.yahoo.com

Re: Lucene as a primary datastore

2010-01-19 Thread Ganesh
Thanks Otis. The download link sent via email has file called cemail. There is no extn. I tried with html,pdf but it is not opening properly. Regards Ganesh - Original Message - From: "Otis Gospodnetic" To: Sent: Wednesday, January 20, 2010 11:54 AM Subject: Re: Lucene as

Re: Lucene as a primary datastore

2010-01-19 Thread Ganesh
ehave with millions of records but i guess their sorting memory consumption would be less. It would be great, If Lucene has the ability to do backups / replication. I don't know how to modify/use the solr script. Regards Ganesh - Original Message - From: "Otis Gospodneti

Reusing Document and Field objects

2010-01-18 Thread Ganesh
Hello all, I am indexing millions of documents. The app is single threaded. I need to create Document and Fields objects repeatedly. I have a thought to create it once and reuse by setting the field values to null. Is this advisable, Will it give any performance improvement? Regards Ganesh

Re: External sort

2009-12-18 Thread Ganesh
merging. Assign docid by sorting the selected field. This way we could achieve the sorting by zero RAM utilization. Mostly the sorted field is fixed for all application. Just some interest to know these things Regards Ganesh - Original Message - From: "Toke Eskildsen" T

Re: External sort

2009-12-17 Thread Ganesh
Thanks Toke. I worried to use long[] inverted = new long[reader.maxDoc]; as the memory consumption will be high for millions of document. Any idea of building external sort cache? Regards Ganesh - Original Message - From: "Toke Eskildsen" To: Sent: Thursday, December

External sort

2009-12-16 Thread Ganesh
external sorting? Is there any way to index the records in sorted order. Sort and Merge the records at indexing time and pull the records sorted by doc id? Any other ideas I need to do sort with less memory. Performance is not the concern. Regards Ganesh Send instant messages to your online

Re: Lucene Analyzer that can handle C++ vs C#

2009-12-15 Thread Ganesh
How about KeywordAnalyzer? It will treat C++ and C# as single term. Regards Ganesh - Original Message - From: "Chris Lu" To: Sent: Saturday, December 12, 2009 5:27 AM Subject: Re: Lucene Analyzer that can handle C++ vs C# > What we did in DBSight is to provide a res

Re: heap memory issues when sorting by a string field

2009-12-09 Thread Ganesh
externally with 2.9.1 ? If we reopen the DB multiple times whether these objects will be accumulated / All readers are referenced to single cache? Regards Ganesh - Original Message - From: "Toke Eskildsen" To: Sent: Thursday, December 10, 2009 5:16 AM Subject: RE: h

Re: IndexDivisor

2009-12-08 Thread Ganesh
then perform warming using TermQuery. I feel there should be some difference when opening with different value. But it didn't. I will try to perform the same using profiler. Regards Ganesh - Original Message - From: "Michael McCandless" To: Sent: Sunday, December 0

Re: IndexDivisor

2009-12-04 Thread Ganesh
mes memory. There is no deletions, no norms and i am not doing sorting Regards Ganesh - Original Message - From: "Michael McCandless" To: Sent: Friday, December 04, 2009 7:01 PM Subject: Re: IndexDivisor I'm confused -- what are these attachments? Output from a memor

Re: IndexDivisor

2009-12-03 Thread Ganesh
close all the searcher and reader. There is no deletions, no norms and i am not doing sorting. Regards Ganesh - Original Message - From: "Michael McCandless" To: Sent: Thursday, December 03, 2009 9:35 PM Subject: Re: IndexDivisor Can you run w/ a memory profiler? I d

Re: IndexDivisor

2009-12-03 Thread Ganesh
I am doing GC before calculating the memory. Even i set my indexdivisor to 1 but there seems to be no change. Below are my stats IndexDivisor Memory -1 7 MB 1 486 MB 100 180 MB 1000 176 MB. 1176MB Regards Ganesh - Original Message - From

Re: IndexDivisor

2009-12-03 Thread Ganesh
Thanks mike. I am opening the reader and warming it up and then calculating the memory consumed. long usedMemory = runtime.totalMemory() - runtime.freeMemory(); Regards Ganesh - Original Message - From: "Michael McCandless" To: Sent: Thursday, December 03, 2009 6:22 PM S

Re: IndexDivisor

2009-12-03 Thread Ganesh
emory will be allocated / increased. Regards Ganesh - Original Message - From: "Michael McCandless" To: Sent: Thursday, December 03, 2009 4:47 PM Subject: Re: IndexDivisor How do you measure memory consumption? If you pass -1 for the divisor do you still see no difference? C

Re: IndexDivisor

2009-12-03 Thread Ganesh
loaded all the database with index divisor value 1000, 5000, and 1 and the memory consumption seems to be same. Regards Ganesh - Original Message - From: "Michael McCandless" To: Sent: Thursday, December 03, 2009 4:09 PM Subject: Re: IndexDivisor That's indeed ho

Re: IndexDivisor

2009-12-03 Thread Ganesh
to IndexDivisor. Regards Ganesh - Original Message - From: "Danil ŢORIN" To: Sent: Friday, November 27, 2009 4:00 PM Subject: Re: IndexDivisor Try to open with very large value (MAX_INT) it will load only first term, and look up the rest from disk. On Fri, Nov 27, 2009 at 1

Sorting issues resolved in 3.0?

2009-12-03 Thread Ganesh
cache? Is there any functionality to load this fieldcache from disk rather completly keeping in to memory. Something similar to ehcache. Is any optimization done in sorting. Regards Ganesh Send instant messages to your online friends http://in.messenger.yahoo.com

Re: PerFieldAnalyzerWrapper

2009-11-29 Thread Ganesh
when the index is upgraded from 2.4.1 to 2.9.1 Regards Ganesh - Original Message - From: "Ganesh" To: Sent: Monday, November 30, 2009 10:14 AM Subject: PerFieldAnalyzerWrapper Hello all, I am having an index build using v2.4.1. I upgraded to 2,9.1. The index was sea

PerFieldAnalyzerWrapper

2009-11-29 Thread Ganesh
overcome this issue. Regards Ganesh Send instant messages to your online friends http://in.messenger.yahoo.com - To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org

Re: IndexDivisor

2009-11-27 Thread Ganesh
Thanks, May i know the purpose of using negative value? Regards Ganesh - Original Message - From: "Michael McCandless" To: Sent: Friday, November 27, 2009 3:17 PM Subject: Re: IndexDivisor > This is the expected behavior. > > If you intend to use the reader for

IndexDivisor

2009-11-26 Thread Ganesh
tting this value to -1, No memory will be used and the terms will be fetched from the disk. Search might be slow but with very less memory consumption. Regards Ganesh Send instant messages to your online friends http://in.messenger

Re: API changes in 2.9

2009-11-26 Thread Ganesh
to modify\search the index created by 2.4.1? Is backward compatibility maintained? Regards Ganesh - Original Message - From: "Uwe Schindler" To: Sent: Thursday, November 26, 2009 2:26 PM Subject: RE: API changes in 2.9 > You can use these APIs. They are only marked as

API changes in 2.9

2009-11-26 Thread Ganesh
;, but in both the class there is a note "NOTE: This API is experimental and might change in incompatible ways in the next release. " Shall I go ahead and update the API or use the older one till 3.0? This red note is added to couple of other placeses. Shall we go ahead and use the

Custom sorting

2009-11-17 Thread Ganesh
Hello all, I am having millions of records in the database and in that 75% of the records required to be sorted. Does 2.9 provides facility to do custom sorting (Avoid loading all records) ? Regards Ganesh Send instant messages to your online friends http://in.messenger.yahoo.com

Re: Max number of open IndexWriters

2009-11-15 Thread Ganesh
You could keep multiple writers opened and it will do no harm. I am doing this. Its good to reopen (close and open) the writer at certain interval. This will release the memory it holds. Whenever i create a new database, I reopen writers belong to all database. Regards Ganesh

Re: UpdateDocument failed

2009-11-06 Thread Ganesh
the User updates the same record in the same interval, then i am loosing one update. Is there any way to resolve this issue using Lucene. I don't think 2.9 supports payload per document, which could be updated independently to the document. Regards Ganesh - Original Message -

Re: Index files not deleted after optimization

2009-11-05 Thread Ganesh
Thanks Michael. Below order helped to solve the issue, 1. DB optimize 2. DB commit 3. Reopen new reader 4. Close old reader 5. Close writer 6. Open writer Regards Ganesh - Original Message - From: "Michael McCandless" To: Sent: Tuesday, November 03, 2009 9:05 PM S

  1   2   3   >