> @Sarthak - I see the term pre-copy all over LuceneServer & nrtSearch but
I
> haven't been able to distinguish the term from just "copy". Does the "pre"
> simply refer to the fact that the transfer of bits is happening before the
> replica starts to serve queries from that segment? I feel like I
On Sun, Mar 2, 2025 at 7:21 AM Marc Davenport
wrote:
>
> @Michael - That second simpler architecture is very similar to what we are
> considering; With the exception of a queue for announcing new
> segments rather than a polling process. It is good to know that it's a
> reasonable outline. You
On Sun, Mar 2, 2025 at 7:21 AM Marc Davenport
wrote:
>
> Thank you for the great replies everyone!
> I'm going to be mulling this over for a bit.
>
> @Steven - So in your system it sounds like you still transferred bits
> directly between the primary and the replicas? If you don't mind me asking,
Thank you for the great replies everyone!
I'm going to be mulling this over for a bit.
@Steven - So in your system it sounds like you still transferred bits
directly between the primary and the replicas? If you don't mind me asking,
how many replicas did you have?
@Michael - That second simpler a
> I'm still debating if there should be some direct knowledge of the
> replicas in the primary node. Or if the primary node can just churn away
> creating base indexes and updates and publish to a queue when it produces
a
> new set of segments. Then the replicas are then free to pick up the lates
Hi there,
I'm happy to share some details about how Amazon Product Search does its
segment replication. I haven't worked on Product Search in over three
years, so anything that I remember is not particularly novel. Also, it's
not really secret sauce -- I would have happily talked about it more in
> On Feb 26, 2025, at 2:53 PM, Marc Davenport
> wrote:
>
> Hello,
> Our current search solution is a pretty big monolith running on pretty
> beefy EC2 instances. Every node is responsible for indexing and serving
> queries. We want to start decomposing our service and are starting with
> se
Hello,
Our current search solution is a pretty big monolith running on pretty
beefy EC2 instances. Every node is responsible for indexing and serving
queries. We want to start decomposing our service and are starting with
separating the indexing and query handling responsibilities.
I'm in the re
