[
https://issues.apache.org/jira/browse/SOLR-6398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joel Bernstein updated SOLR-6398:
---------------------------------
Description:
This ticket builds on the existing AnalyticsQuery / MergeStrategy framework by
adding the abstract class IterativeMergeStrategy, which has built-in support
for call-backs to the shards. The IterativeMergeStrategy is designed to support
the execution of Parallel iterative Algorithms, such as Gradient Descent,
inside of Solr.
To use the IterativeMergeStrategy you extend it and implement process(). This
gives you access to the callback() method which makes it easy to make repeated
calls to all the shards and run algorithms that require iteration.
Below is an example of a class that extends IterativeMergeStrategy. In this
example it collects the *count* from the shards and then calls back to shards
executing the *!count* AnalyticsQuery and sending it merged counts from all the
shards.
{code}
class TestIterative extends IterativeMergeStrategy {
public void process(ResponseBuilder rb, ShardRequest sreq) throws Exception
{
int count = 0;
for(ShardResponse shardResponse : sreq.responses) {
NamedList response = shardResponse.getSolrResponse().getResponse();
NamedList analytics = (NamedList)response.get("analytics");
Integer c = (Integer)analytics.get("mycount");
count += c.intValue();
}
ModifiableSolrParams params = new ModifiableSolrParams();
params.add("distrib", "false");
params.add("fq","{!count base="+count+"}");
params.add("q","*:*");
/*
* Call back to all the shards in the response and process the result.
*/
QueryRequest request = new QueryRequest(params);
List<Future<CallBack>> futures = callBack(sreq.responses, request);
int nextCount = 0;
for(Future<CallBack> future : futures) {
QueryResponse response = future.get().getResponse();
NamedList analytics =
(NamedList)response.getResponse().get("analytics");
Integer c = (Integer)analytics.get("mycount");
nextCount += c.intValue();
}
NamedList merged = new NamedList();
merged.add("mycount", nextCount);
rb.rsp.add("analytics", merged);
}
}
{code}
was:
This ticket builds on the existing AnalyticsQuery / MergeStrategy framework by
adding the abstract class IterativeMergeStrategy, which has built-in support
for call-backs to the shards. The IterativeMergeStrategy is designed to support
the execution of Parallel iterative Algorithms, such as Gradient Descent,
inside of Solr.
To use the IterativeMergeStrategy you extend it and implement process(). This
gives you access to the callback() method which makes it easy to make repeated
calls to all the shards and run algorithms that require iteration.
Below is an example of a class that extends IterativeMergeStrategy. In this
example it collects the *count* from the shards and then calls back to shards
executing *!count* sending it merged counts from all the shards.
{code}
class TestIterative extends IterativeMergeStrategy {
public void process(ResponseBuilder rb, ShardRequest sreq) throws Exception
{
int count = 0;
for(ShardResponse shardResponse : sreq.responses) {
NamedList response = shardResponse.getSolrResponse().getResponse();
NamedList analytics = (NamedList)response.get("analytics");
Integer c = (Integer)analytics.get("mycount");
count += c.intValue();
}
ModifiableSolrParams params = new ModifiableSolrParams();
params.add("distrib", "false");
params.add("fq","{!count base="+count+"}");
params.add("q","*:*");
/*
* Call back to all the shards in the response and process the result.
*/
QueryRequest request = new QueryRequest(params);
List<Future<CallBack>> futures = callBack(sreq.responses, request);
int nextCount = 0;
for(Future<CallBack> future : futures) {
QueryResponse response = future.get().getResponse();
NamedList analytics =
(NamedList)response.getResponse().get("analytics");
Integer c = (Integer)analytics.get("mycount");
nextCount += c.intValue();
}
NamedList merged = new NamedList();
merged.add("mycount", nextCount);
rb.rsp.add("analytics", merged);
}
}
{code}
> Add IterativeMergeStrategy to support Parallel Iterative Algorithms
> -------------------------------------------------------------------
>
> Key: SOLR-6398
> URL: https://issues.apache.org/jira/browse/SOLR-6398
> Project: Solr
> Issue Type: Improvement
> Reporter: Joel Bernstein
> Priority: Minor
> Fix For: 5.2, Trunk
>
> Attachments: SOLR-6398.patch, SOLR-6398.patch, SOLR-6398.patch,
> SOLR-6398.patch
>
>
> This ticket builds on the existing AnalyticsQuery / MergeStrategy framework
> by adding the abstract class IterativeMergeStrategy, which has built-in
> support for call-backs to the shards. The IterativeMergeStrategy is designed
> to support the execution of Parallel iterative Algorithms, such as Gradient
> Descent, inside of Solr.
> To use the IterativeMergeStrategy you extend it and implement process(). This
> gives you access to the callback() method which makes it easy to make
> repeated calls to all the shards and run algorithms that require iteration.
> Below is an example of a class that extends IterativeMergeStrategy. In this
> example it collects the *count* from the shards and then calls back to shards
> executing the *!count* AnalyticsQuery and sending it merged counts from all
> the shards.
> {code}
> class TestIterative extends IterativeMergeStrategy {
> public void process(ResponseBuilder rb, ShardRequest sreq) throws
> Exception {
> int count = 0;
> for(ShardResponse shardResponse : sreq.responses) {
> NamedList response = shardResponse.getSolrResponse().getResponse();
> NamedList analytics = (NamedList)response.get("analytics");
> Integer c = (Integer)analytics.get("mycount");
> count += c.intValue();
> }
> ModifiableSolrParams params = new ModifiableSolrParams();
> params.add("distrib", "false");
> params.add("fq","{!count base="+count+"}");
> params.add("q","*:*");
> /*
> * Call back to all the shards in the response and process the result.
> */
> QueryRequest request = new QueryRequest(params);
> List<Future<CallBack>> futures = callBack(sreq.responses, request);
> int nextCount = 0;
> for(Future<CallBack> future : futures) {
> QueryResponse response = future.get().getResponse();
> NamedList analytics =
> (NamedList)response.getResponse().get("analytics");
> Integer c = (Integer)analytics.get("mycount");
> nextCount += c.intValue();
> }
> NamedList merged = new NamedList();
> merged.add("mycount", nextCount);
> rb.rsp.add("analytics", merged);
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
