[
https://issues.apache.org/jira/browse/SOLR-5944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15414049#comment-15414049
]
Hoss Man commented on SOLR-5944:
--------------------------------
It seems like solutions for dealing with LUCENE-7344 in Solr could fall into 3
broad categories:
{panel:title=IndexWriter changes}
These are solutions that might involve changes to IndexWriter
* The ideal solution would be to "fix" LUCENE-7344 at the IndexWriter level, so
that it magically does hte correct thing -- but I certainly have no idea how to
do that, and McCandless didn't think it was viable. So this is an unlikely
solution unless/untill Shai has more time to work through his ideas.
* An improvement on the overall situation might be if
{{IndexWriter.deleteDocuments}} could at least *detect* when the Query involves
a DocValues field which has non-committed updates, and in that case throw a
distinct Exception (ie: {{CanNotDeleteOnUpdatedDocValues extends
InvalidStateException}}
** Solr could then, at a minimum, propogate that exception up to the user, and
documentation could make it clera that if you want to mix atomic updates of
docvalue with DBQs on those docvalues, you need to commit or you will get this
exception
{panel}
{panel:title=Workaround's in Solr: Invisible To Users}
This would be any type of solution that let's us maintain the current approach
in the existing patches of making docvalue updates be a complete implementation
detail that happens automatically behind the scenes when the field types &
updates support it.
* One possibility would be to track somewhere (at the solr level) if/when
inplace updates have been used since the last commit, and do an implicit commit
if/when a DBQ comes in.
** to prevent excessive unexpected commits, we could completley disable all
inplace updates unless autocommit was enabled, since that should be the common
case for most "real time" applications of solr anywhere, where inplace updates
will be the most helpful
* The previous idea could be combined with the idea of
{{IndexWriter.deleteDocuments}} throwing a {{CanNotDeleteOnUpdatedDocValues}}
exception;
** if/when Solr encounters a {{CanNotDeleteOnUpdatedDocValues}}, we could do an
implicit commit and retry the DBQ
{panel}
{panel:title=Workaround's in Solr: involving API Changes/Additions to current
patch}
These are solutions that would involved explicit changes to the options offered
to users so they can explicitly indicate "I want an inplace update" with the
understanding that tha may prevent DBQs from working properly
* As ishan suggested, one possibility would be to require fields have explicit
{{inplaceUpdates="true"}} configuration in schema.xml to indicate that they may
be updated inplace
** at a later date, we could always increase the default schema {{version}} and
change the default value of this new field property to automatically be true
based on the {{docValues}} and other field attributes
** the biggest downside I see to this approach is that it doesn't really do
anything to prevent the underlying problem:
*** it doesn't help "enforce" that DBQs won't be used against these fields, and
it doesn't give the user any clear indication of a problem if they try --
Solr/Lucene will continue to silently delete the wrong docs.
* Alternatively, we could make this notion of requiring that inplaceupdates be
explicitly requested be something that can be specified on a per request basis,
along the lines of...{code}
{ "id": "12345",
"price": { "inc": 42,
"inplace": true }}
{code}
** again, the default could be "false" and at a later date we could always
change the default to be based on the {{docValues}} and other field attributes
** this approach seems _slightly_ better to me then a schema setting because:
*** the {{"inplace"}} directive is front and center for clients sending
updates, so they are more likeley to be aware of the caveats around using it
with DBQ (even if they haven't read the schema first hand)
*** clients can mix and match inplace atomic updates with "regular" atomic
updates on the same field (or diff fields definied by the same
{{<dynamicField/>}}, if/when they do/don't care about being able to reliably do
a DBQ on that field.
** But ultimately this still has the same scary behavior: Solr can silently
delete teh wrong docs w/o immediate/obvious reason why if the user doing the
delete doesn't know/care if/when/why a doc may have had an inplace atomic
update.
{panel}
My current feeling is that a hybrid solution may be the best bet:
* Require users to explicitly configure/request "inplace" updates in some way
* Document that when inplace updates are configured/requested, hard commits may
happen implicitly if/when a DBQ is executed to ensure correct documents are
deleted
* Only do the hard commit if neccessary because IndexWriter threw
{{CanNotDeleteOnUpdatedDocValues}} the first time we tried the DBQ, then retry.
> Support updates of numeric DocValues
> ------------------------------------
>
> Key: SOLR-5944
> URL: https://issues.apache.org/jira/browse/SOLR-5944
> Project: Solr
> Issue Type: New Feature
> Reporter: Ishan Chattopadhyaya
> Assignee: Shalin Shekhar Mangar
> Attachments: DUP.patch, SOLR-5944.patch, SOLR-5944.patch,
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch,
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch,
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch,
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch,
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch,
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch,
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch,
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch,
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch,
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch,
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch,
> TestStressInPlaceUpdates.eb044ac71.beast-167-failure.stdout.txt,
> TestStressInPlaceUpdates.eb044ac71.beast-587-failure.stdout.txt,
> TestStressInPlaceUpdates.eb044ac71.failures.tar.gz,
> hoss.62D328FA1DEA57FD.fail.txt, hoss.62D328FA1DEA57FD.fail2.txt,
> hoss.62D328FA1DEA57FD.fail3.txt, hoss.D768DD9443A98DC.fail.txt,
> hoss.D768DD9443A98DC.pass.txt
>
>
> LUCENE-5189 introduced support for updates to numeric docvalues. It would be
> really nice to have Solr support this.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
