Re: Atomic indexing without whole document getting indexed again

2022-04-05 Thread Shawn Heisey
On 4/5/2022 12:38 PM, gnandre wrote: I conducted the test that you mentioned. Here is the diff - https://www.diffchecker.com/sdsMiGW5 Left hand side is the state before the in-place update. Right hand side is the state after the in-place update. That looks very strange to me.  If it were doin

Re: Atomic indexing without whole document getting indexed again

2022-04-05 Thread Matthew Lapointe
Are there any update request processors defined that could be adding default values? On Tue, Apr 5, 2022 at 4:53 PM gnandre wrote: > It is configured as a unique field. > > id > > > > On Tue, Apr 5, 2022 at 4:10 PM Matthew Lapointe > > wrote: > > > That's odd! The only other thing I can think

Re: Atomic indexing without whole document getting indexed again

2022-04-05 Thread gnandre
It is configured as a unique field. id On Tue, Apr 5, 2022 at 4:10 PM Matthew Lapointe wrote: > That's odd! The only other thing I can think to check would be to verify > that the "id" field is configured as the unique key field for the > collection. > > Matthew > > On Tue, Apr 5, 2022 at 3:4

Re: Atomic indexing without whole document getting indexed again

2022-04-05 Thread Matthew Lapointe
That's odd! The only other thing I can think to check would be to verify that the "id" field is configured as the unique key field for the collection. Matthew On Tue, Apr 5, 2022 at 3:43 PM gnandre wrote: > Thanks, Matthew. > > I tried debugging as you suggested. It seems that it is still doing

Re: Atomic indexing without whole document getting indexed again

2022-04-05 Thread gnandre
Thanks, Matthew. I tried debugging as you suggested. It seems that it is still doing atomic update instead of in-place update. I am not using SolrCloud, so I don't think that SOLR-13081 is applicable in my situation. I am using Solr 8.5.2 in standalone mode. I am not sure why in-place updates are

Re: Atomic indexing without whole document getting indexed again

2022-04-05 Thread Matthew Lapointe
Hi, I encountered a similar issue recently trying to differentiate between atomic and in-place updates. I ended up enabling debug logging for the DirectUpdateHandler2 class via Solr UI → Logging → Level options. Then the logs should print something like "DirectUpdateHandler2 updateDocValues" for a

Re: Atomic indexing without whole document getting indexed again

2022-04-05 Thread gnandre
Thanks, Shawn. I conducted the test that you mentioned. Here is the diff - https://www.diffchecker.com/sdsMiGW5 Left hand side is the state before the in-place update. Right hand side is the state after the in-place update. On Tue, Apr 5, 2022 at 1:05 PM Shawn Heisey wrote: > On 4/5/22 10:53,

Re: Atomic indexing without whole document getting indexed again

2022-04-05 Thread Shawn Heisey
On 4/5/22 10:53, gnandre wrote: Hi, here are the relevant fields from the schema. There are no copyfields for views_count. Here are the corresponding atomic indexing and commit requests: curl http://solr:8983/solr/answers/update -d '[{"id" : "answers:question:8029","views_count" : {"set":1

Re: Atomic indexing without whole document getting indexed again

2022-04-05 Thread gnandre
Hi, here are the relevant fields from the schema. There are no copyfields for views_count. Here are the corresponding atomic indexing and commit requests: curl http://solr:8983/solr/answers/update -d '[{"id" : "answers:question:8029","views_count" : {"set":111}}]' curl "http://solr:8983/solr

Re: Atomic indexing without whole document getting indexed again

2022-04-01 Thread Shawn Heisey
On 3/31/22 13:36, gnandre wrote: Here is what I tried to confirm if it is still doing atomic indexing and not in-place indexing. I changed one other unrelated field's name and reloaded the schema. Now, when I performed the indexing just for the field that I wanted to update in-place, it should no

Re: Atomic indexing without whole document getting indexed again

2022-03-31 Thread gnandre
Here is what I tried to confirm if it is still doing atomic indexing and not in-place indexing. I changed one other unrelated field's name and reloaded the schema. Now, when I performed the indexing just for the field that I wanted to update in-place, it should not have complained about this other

Re: Atomic indexing without whole document getting indexed again

2022-03-31 Thread gnandre
Thanks, this is what I was looking for. Although, when I am experimenting with them now, I see no performance improvement. I suspect that it is still doing atomic updates and not in-place updates. How do I confirm whether in-place updates are happening vs atomic updates? I can't tell it simply by l

Re: Atomic indexing without whole document getting indexed again

2022-03-30 Thread Walter Underwood
Integer view counts probably do meet those requirements, but you might need to update all 25 million documents every day, which is not going to be fast. wunder Walter Underwood wun...@wunderwood.org http://observer.wunderwood.org/ (my blog) > On Mar 30, 2022, at 9:34 AM, Shawn Heisey wrote: >

Re: Atomic indexing without whole document getting indexed again

2022-03-30 Thread Shawn Heisey
On 3/30/22 10:27, gnandre wrote: IIRC, under the hood, atomic indexing indexes the whole document again even if you might be updating just one field of that document. This costs hugely in terms of indexing performance because the other fields might be requiring some significant heavy tokenization

Atomic indexing without whole document getting indexed again

2022-03-30 Thread gnandre
IIRC, under the hood, atomic indexing indexes the whole document again even if you might be updating just one field of that document. This costs hugely in terms of indexing performance because the other fields might be requiring some significant heavy tokenization. Is there any way around this?