If you index from the outside (i.e. not using DIH) you have more control:
* how many threads you use
* how you batch documents
* how much you wait between indexing batches
...

Otis
--
Solr & ElasticSearch Support
http://sematext.com/





On Fri, Jan 4, 2013 at 6:25 PM, Marcin Rzewucki <mrzewu...@gmail.com> wrote:

> Thanks. I guess you're right - it's normal behaviour. Are there some
> guidelines how to use ramBufferSizeMB or only by testing ? Do you know if
> DIH is "gentler" than indexing via REST or solrj API ?
> Kind regards.
>
> On 4 January 2013 23:14, Otis Gospodnetic <otis.gospodne...@gmail.com
> >wrote:
>
> > Hi,
> >
> > I think what you are seeing is a general thing.  Regular search is slower
> > while there is indexing, too, of course.
> > So maybe it's best to mentally decouple indexing part here and simply
> make
> > your calls as fast as possible without indexing.  Then you can add
> indexing
> > and play with things like ramBufferSizeMB and anything else that has the
> > potential of making indexing "gentler" on resources, be that CPU or disk
> > or...
> >
> > Otis
> > --
> > Solr & ElasticSearch Support
> > http://sematext.com/
> >
> >
> >
> >
> >
> > On Fri, Jan 4, 2013 at 4:44 PM, Marcin Rzewucki <mrzewu...@gmail.com>
> > wrote:
> >
> > > Hi all,
> > >
> > > I'm using SolrCloud4x. I've experienced some problem with
> StatsComponent.
> > > It looks like query time increases during indexing. For example the
> > > following query:
> > >
> > >
> > >
> >
> http://host:8983/solr/core/select?q=*:*&wt=xml&shards.tolerant=true&stats=true&stats.field=total_assets_stl&stats.field=eur_total_revenues_stl&stats.field=eur_total_assets_stl&stats.field=total_revenues_stl&stats.field=usd_total_revenues_stl&stats.field=usd_total_assets_stl&stats.field=eur_total_assets_stl&rows=0
> > >
> > > takes less than 1s when there's no indexing in background and more than
> > 1s
> > > to couple of seconds while indexing. I'm using Trie fields with
> > > precisionStep set to -1 (it was precisionStep="8" before, but query
> times
> > > were much worse, so I changed it). My SolrCloud uses m1.large nodes in
> > AWS
> > > (7.5 GiB), mmap for index reading and 2GB for JVM, default settings for
> > > cache. I wonder what is the reason that StatsComponent is much slower
> > > during indexing ? Or is it normal behaviour ? Is it possible to improve
> > it
> > > ? Any ideas are welcome.
> > >
> > > Thanks!
> > >
> >
>

Reply via email to