Hi, Random data was indexed. I wanted to see the worst case where little data is same across documents and which in most of my cases is. So, i guess in these scenarios compression becomes an overhead.
Arun On Thu, Jan 31, 2013 at 8:00 PM, Robert Muir <rcm...@gmail.com> wrote: > The top method here is your random string generation. > > are you indexing random data? > > On Thu, Jan 31, 2013 at 12:46 AM, arun k <arunk...@gmail.com> wrote: > > Hi, > > > > Please find the snapshots here. > > http://picpaste.com/Lucene3.0.2-G00Z5FfX.png > > http://picpaste.com/Lucene4.1-LsxpcQk0.png > > > > Arun > > > > > > On Wed, Jan 30, 2013 at 5:30 PM, Uwe Schindler <u...@thetaphi.de> wrote: > > > >> Hi, > >> > >> > >> > >> there is nothing attached to your mail; maybe the mailing list software > >> removed it. Can you place it somewhere on the web (e.g. pastebin,…)? > >> > >> > >> > >> Uwe > >> > >> > >> > >> ----- > >> > >> Uwe Schindler > >> > >> H.-H.-Meier-Allee 63, D-28213 Bremen > >> > >> <http://www.thetaphi.de/> http://www.thetaphi.de > >> > >> eMail: u...@thetaphi.de > >> > >> > >> > >> From: arun k [mailto:arunk...@gmail.com] > >> Sent: Wednesday, January 30, 2013 12:04 PM > >> To: java-user > >> Subject: Re: CompressingStoredFieldsFormat doesn't show improvement > >> > >> > >> > >> Adrein, > >> Please find the attached profilers report. > >> > >> > >> > >> On Wed, Jan 30, 2013 at 3:35 PM, Adrien Grand <jpou...@gmail.com> > wrote: > >> > >> On Wed, Jan 30, 2013 at 8:08 AM, arun k <arunk...@gmail.com> wrote: > >> > Adrein, > >> > > >> > I have created an index of size 370M of 1 million docs of 40 fields > of 40 > >> > chars and did the profiling. > >> > I see that the indexing and in particular the addDocument & > >> > ConcurrentMergeScheduler in 4.1 takes double the time compared to > 3.0.2. > >> > >> Can you provide me with the detailed profiles? > >> > >> > >> > Looks like CompressionStoredFieldsFormat is of little use in my > scenario. > >> > >> You can can disable stored fields compression and use another > >> StoredFieldsFormat by defining a custom codec > >> ( > >> > http://lucene.apache.org/core/4_1_0/core/org/apache/lucene/codecs/package-summary.html#package_description > >> ). > >> > >> > >> -- > >> Adrien > >> > >> --------------------------------------------------------------------- > >> To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > >> For additional commands, e-mail: java-user-h...@lucene.apache.org > >> > >> > >> > >> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > For additional commands, e-mail: java-user-h...@lucene.apache.org > >
