[
https://issues.apache.org/jira/browse/SOLR-6469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14233501#comment-14233501
]
Michael Ryan edited comment on SOLR-6469 at 12/3/14 8:52 PM:
-------------------------------------------------------------
We ran into this exception today on Solr 4.9.0, though for a slightly different
reason. Simplified request looks like this:
q=\*:\*&fl=articleId&rows=1&hl=true&shards=foo1,foo2
The workaround in this case is to either request additional fields besides the
unique key field, or to not request highlighting.
This hits the same NPE at HighlightComponent.java:189. It throws an NPE because
more highlighting data is returned from the shards than are in resultIds. This
happens because onePassDistributedQuery is set to true in QueryComponent, and
so the highlight data is requested from the shards on the first (single) pass.
The NPE is easy enough to fix for my issue - just add a null check before sdoc
is used, and skip it if null. But there is still an underlying inefficiency -
many more docs are highlighted on the shards than are actually necessary. If
you have a lot of shards, this highlighting becomes non-trivial (e.g.,
highlighting thousands of docs instead of hundreds). I don't see any way to fix
this without doing a second pass. Maybe ignore onePassDistributedQuery if
highlights are requested? (That'll make the distributed logic in
QueryComponent/HighlightComponent even more fun.)
Should I create another Jira for my issue since it doesn't involve grouping?
was (Author: michaelryan):
We ran into this exception today, though for a slightly different reason.
Simplified request looks like this:
q=\*:\*&fl=articleId&rows=1&hl=true&shards=foo1,foo2
The workaround in this case is to either request additional fields besides the
unique key field, or to not request highlighting.
This hits the same NPE at HighlightComponent.java:189. It throws an NPE because
more highlighting data is returned from the shards than are in resultIds. This
happens because onePassDistributedQuery is set to true in QueryComponent, and
so the highlight data is requested from the shards on the first (single) pass.
The NPE is easy enough to fix for my issue - just add a null check before sdoc
is used, and skip it if null. But there is still an underlying inefficiency -
many more docs are highlighted on the shards than are actually necessary. If
you have a lot of shards, this highlighting becomes non-trivial (e.g.,
highlighting thousands of docs instead of hundreds). I don't see any way to fix
this without doing a second pass. Maybe ignore onePassDistributedQuery if
highlights are requested? (That'll make the distributed logic in
QueryComponent/HighlightComponent even more fun.)
Should I create another Jira for my issue since it doesn't involve grouping?
> Solr search with multicore + grouping + highlighting cause NPE
> --------------------------------------------------------------
>
> Key: SOLR-6469
> URL: https://issues.apache.org/jira/browse/SOLR-6469
> Project: Solr
> Issue Type: Bug
> Components: highlighter, multicore, SearchComponents - other
> Affects Versions: 4.8.1
> Environment: Windows 7, Intellij
> Reporter: Shay Sofer
> Assignee: Noble Paul
> Labels: patch
> Attachments: SOLR-6469.patch
>
>
> Integration of Grouping + shards + highlighting cause NullPointerException.
> Query:
> localhost:8983/solr/Global_A/select?q=%2Btext%3A%28shay*%29+&rows=100&fl=id%2CobjId%2Cnull&shards=http%3A%2F%2F127.0.0.1%3A8983%2Fsolr%2F0_A%2Chttp%3A%2F%2F127.0.0.1%3A8983%2Fsolr%2FGlobal_A&group=true&group.query=name__s%3Ashay&sort=name__s_sort+asc&hl=true
> results:
> java.lang.NullPointerException
> at
> org.apache.solr.handler.component.HighlightComponent.finishStage(HighlightComponent.java:189)
> at
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:330)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1952)
> at
> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:774)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:418)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:207)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
> at
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
> at
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075)
> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
> at
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
> at
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
> at
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
> at org.eclipse.jetty.server.Server.handle(Server.java:368)
> at
> org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
> at
> org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
> at
> org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:942)
> at
> org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1004)
> at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:640)
> at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
> at
> org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
> at
> org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
> at java.lang.Thread.run(Thread.java:722)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
