[
https://issues.apache.org/jira/browse/SOLR-17120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17812471#comment-17812471
]
ASF subversion and git services commented on SOLR-17120:
--------------------------------------------------------
Commit 571c8871278bc14aea683420aea58ef64e38bbae in solr's branch
refs/heads/jira/SOLR-16858 from Calvin Smith
[ https://gitbox.apache.org/repos/asf?p=solr.git;h=571c8871278 ]
SOLR-17120: handle null value when merging partials (#2214)
* SOLR-17120 handle null value when merging partials
- this change avoids a `NullPointerException` that can occur
under some circumstances when performing multiple partial
updates of the same document
> NullPointerException in UpdateLog.applyOlderUpdates in solr 6.6-9.4 involving
> partial updates
> ---------------------------------------------------------------------------------------------
>
> Key: SOLR-17120
> URL: https://issues.apache.org/jira/browse/SOLR-17120
> Project: Solr
> Issue Type: Bug
> Components: update
> Affects Versions: 6.6.2, 8.11.2, 9.4
> Environment: The issue occurred on Linux, CentOS 7.9, with the
> following JDK version:
> {noformat}
> openjdk version "11.0.20" 2023-07-18 LTS
> OpenJDK Runtime Environment (Red_Hat-11.0.20.0.8-1.el7_9) (build
> 11.0.20+8-LTS)
> OpenJDK 64-Bit Server VM (Red_Hat-11.0.20.0.8-1.el7_9) (build 11.0.20+8-LTS,
> mixed mode, sharing){noformat}
> Reporter: Calvin Smith
> Assignee: Christine Poerschke
> Priority: Major
> Fix For: 8.11.3, 9.5
>
> Attachments: demo.sh
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> I mailed the solr-users mailing list about this issue, but didn't get any
> responses there, so am creating this issue. The subject of the email thread
> for additional context was "NullPointerException in
> UpdateLog.applyOlderUpdates under solr 8&9 involving partial updates and high
> update load" - link:
> [https://lists.apache.org/thread/n9zm4gocl7cf073syy1159dy6ojjrywl]
> I'm seeing a Solr HTTP 500 error when performing a partial update of a
> document that turns out to triggered by there having been a recent update of
> the same document that included a partial update that set a field to
> {{{}null{}}}. I've observed the behavior in versions 6.6.2, 8.11.2, and
> 9.4.0, which are the only 3 versions I've tried.
> To give an example, an update doc like
>
> {code:java}
> {
> "id": "123",
> "camera_unit": {"set": null}
> }{code}
>
> followed shortly thereafter (not sure of exact timing, but I was using a
> {{commitWithin}} of 600s and the subsequent updates were less than 20 seconds
> later), after some other updates had happened for different documents, there
> was another update of the same document, like
>
> {code:java}
> {
> "id": "123",
> "playlist": {
> "set": [
> 12345
> ]
> },
> "playlist_index_321": {
> "set": 0
> }
> }{code}
>
> This later update may, but doesn't always, cause the
> {{{}NullPointerException{}}}, so there is some other factor such as the state
> of the {{tlog}} that also has to be satisfied for the error to occur.
> The exception is thrown by the following code in {{UpdateLog.java}}
> ({{{}org.apache.solr.update.UpdateLog{}}}):
>
> {code:java}
> /** Add all fields from olderDoc into newerDoc if not already present in
> newerDoc */
> private void applyOlderUpdates(
> SolrDocumentBase<?, ?> newerDoc, SolrInputDocument olderDoc,
> Set<String> mergeFields) {
> for (String fieldName : olderDoc.getFieldNames()) {
> // if the newerDoc has this field, then this field from olderDoc can be
> ignored
> if (!newerDoc.containsKey(fieldName)
> && (mergeFields == null || mergeFields.contains(fieldName))) {
> for (Object val : olderDoc.getFieldValues(fieldName)) {
> newerDoc.addField(fieldName, val);
> }
> }
> }
> }{code}
>
> The exception is due to the inner for statement trying to iterate over the
> {{null}} value being returned by {{{}olderDoc.getFieldValues(fieldName){}}}.
> When I change that method to the following:
>
> {code:java}
> /** Add all fields from olderDoc into newerDoc if not already present in
> newerDoc */
> private void applyOlderUpdates(
> SolrDocumentBase<?, ?> newerDoc, SolrInputDocument olderDoc,
> Set<String> mergeFields) {
> for (String fieldName : olderDoc.getFieldNames()) {
> // if the newerDoc has this field, then this field from olderDoc can be
> ignored
> if (!newerDoc.containsKey(fieldName)
> && (mergeFields == null || mergeFields.contains(fieldName))) {
> Collection<Object> values = olderDoc.getFieldValues(fieldName);
> if (values == null) {
> newerDoc.addField(fieldName, null);
> } else {
> for (Object val : values) {
> newerDoc.addField(fieldName, val);
> }
> }
> }
> }
> }{code}
>
> Then after rebuilding the solr-core JAR with {{./gradlew devFull}} and
> restarting Solr with that custom jar file, I can no longer reproduce the
> error.
> I'm not familiar with the Solr codebase though and am not at all sure that
> {{newerDoc.addField(fieldName, null)}} is what should be done there.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
