The documents are pretty large yes, 650 fields, circa 20kb/document so at peak
(300/sec) that's circa 6meg/sec. ramBufferSizeMB is 512 so we'd be averaging 1
segment every 90 seconds (ish)?
> This means you never explicitly commits from the client? But You
> autoCommit openSearcher=false every minute to flush transLog, and then
> autoSoftCommit every 15min to make changes visible?
Yes, this is correct
> Have you considered using PULL replicas for reading? Then you could tailor
> the HW on those servers to only serve reads, and they would replicate index
> from leader. You'd have e.g. 2xNRT + 7xPULL.
I did, but we unfortunately rely on RealTimeGet so need NRT.
On 08/04/2021, 15:10, "Jan Høydahl" <jan....@cominvent.com> wrote:
> - Update rate, and how you do commits?
>
> Update rates very throughout the day, but range from 20ops/sec to
300ops/sec. Commits are done using autoCommit on 1 min interval, softCommit on
15min interval.
This means you never explicitly commits from the client? But You autoCommit
openSearcher=false every minute to flush transLog, and then autoSoftCommit
every 15min to make changes visible?
I cannot see why these seeings would cause constant merging, unless you
commit more frequently? Your documents are large so the RAM-buffer will fill up
quickly and cause a flush, perhaps try increasing ramBufferSizeMb will lower
number of flushes and merges.
Have you considered using PULL replicas for reading? Then you could tailor
the HW on those servers to only serve reads, and they would replicate index
from leader. You'd have e.g. 2xNRT + 7xPULL.
Jan
Unless expressly stated otherwise in this email, this e-mail is sent on behalf
of Auto Trader Limited Registered Office: 1 Tony Wilson Place, Manchester,
Lancashire, M15 4FN (Registered in England No. 03909628). Auto Trader Limited
is part of the Auto Trader Group Plc group. This email and any files
transmitted with it are confidential and may be legally privileged, and
intended solely for the use of the individual or entity to whom they are
addressed. If you have received this email in error please notify the sender.
This email message has been swept for the presence of computer viruses.
