Hi Dominique
Heavy query means high query rate on solr.
Honestly for 2-3 days we have stopped the queries on solr only we are doing
indexing !!!
As you see we have enabled Hard commit also to decrease tlog log size or
more and the same is written on solr docs. Still the same behaviour is
occurring on replicas. In some replicas, tlog size is of 300 GB.
We may increase softcommit timeout but as per my understanding it only
helps in searching availability prior to commit to indexes.
On Sun, 21 Mar 2021 at 4:29 AM, Dominique Bejean <dominique.bej...@eolya.fr>
wrote:
> Hi,
>
> By heavy query, do you mean you have a high query rate and/or you need
> index update be available fast after few seconds (NRT search) ?
>
> Do you see the hard commits in logs ?
> Can you try to increase autoSoftCommit to 30 seconds or more ?
>
> Regards
>
> Dominique
>
> Le sam. 20 mars 2021 à 18:53, Ritvik Sharma <ritvik.s...@gmail.com> a
> écrit :
>
> > Hi Guys
> >
> > We have faced an issue where tlog size is increasing unnecessarily. We
> are
> > using a "heavy indexing, heavy query" approach. We enabled hard commit
> > also,
> >
> > solr cloud: 6.6
> > zk: 3.4.10
> > shards: 2, replication factor= 2
> >
> >
> > solrconfig,
> >
> > <autoCommit>
> > <maxTime>${solr.autoCommit.maxTime:15000}</maxTime>
> > <maxDocs>10000</maxDocs>
> > <openSearcher>false</openSearcher>
> >
> >
> > </autoCommit>
> >
> > <!-- softAutoCommit is like autoCommit except it causes a
> > 'soft' commit which only ensures that changes are visible
> > but does not ensure that data is synced to disk. This is
> > faster and more near-realtime friendly than a hard commit.
> > -->
> >
> > <autoSoftCommit>
> > <maxTime>${solr.autoSoftCommit.maxTime:15000}</maxTime> -->
> > </autoSoftCommit>
> >
> > In every replica, tlog is increasing for more than 200GB which exhaust
> disk
> > space. Please suggest someting
> >
> > On Fri, 19 Mar 2021 at 16:36, Ritvik Sharma <ritvik.s...@gmail.com>
> wrote:
> >
> > > Hi Guys
> > >
> > > We have faced an issue where tlog size is increasing unnecessarily. We
> > are
> > > using a "heavy indexing, heavy query" approach. We enabled hard commit
> > > also,
> > >
> > > solr cloud: 6.6
> > > zk: 3.4.10
> > > shards: 2, replication factor= 2
> > >
> > >
> > > solrconfig,
> > >
> > > <autoCommit>
> > > <maxTime>${solr.autoCommit.maxTime:15000}</maxTime>
> > > <maxDocs>10000</maxDocs>
> > > <openSearcher>false</openSearcher>
> > >
> > >
> > > </autoCommit>
> > >
> > > <!-- softAutoCommit is like autoCommit except it causes a
> > > 'soft' commit which only ensures that changes are visible
> > > but does not ensure that data is synced to disk. This is
> > > faster and more near-realtime friendly than a hard commit.
> > > -->
> > >
> > > <autoSoftCommit>
> > > <maxTime>${solr.autoSoftCommit.maxTime:15000}</maxTime> -->
> > > </autoSoftCommit>
> > >
> > >
> >
>
