[
https://issues.apache.org/jira/browse/SOLR-8588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joel Bernstein updated SOLR-8588:
---------------------------------
Description:
The TopicStream is a *publish/subscribe messaging service* built on top of
SolrCloud. The TopicStream returns all *new* documents for a specific query.
Version numbers will be used as checkpoints for Topics to ensure single
delivery of each document. When combined with the DaemonStream (SOLR-8550),
Topics can provide continuous streaming. Sample syntax:
{code}
topic(checkpointCollection, dataCollection, id="topicA", q="awesome stuff"
checkpointEvery="1000")
{code}
The checkpoint collection will be used to persist the topic checkpoints.
Example combined with the DaemonStream:
{code}
daemon(topic(...)...)
{code}
When combined with SOLR-7739 this allows for messaging based on *machine
learned* classifications.
The TopicStream supports 3 models of publish/subscribe messaging:
1) *Request & response*: In this model a topic(...) expression can be saved and
executed at any time. In this scenario the TopicStream will always retrieve
it's checkpoints and start from where it left off.
2) *Continuous pull streaming*: In this model you would wrap the TopicStream in
a DaemonStream and call read() in a loop inside a java program. This would
provide a continuous stream of new content as it arrives in the index.
3) *Continuous push streaming*: In this model you would send an expression like
this to the /stream handler: *daemon(update(topic(...)...)...)*. This daemon
process would run inside Solr and continuously stream new documents from the
topic and push them to another SolrCloud collection. Other pushing expressions
can be created to push documents in different ways or take other types of
actions.
was:
The TopicStream is a *publish/subscribe messaging service* built on top of
SolrCloud. The TopicStream returns all *new* documents for a specific query.
Version numbers will be used as checkpoints for Topics to ensure single
delivery of each document. When combined with the DaemonStream (SOLR-8550),
Topics can provide continuous streaming. Sample syntax:
{code}
topic(checkpointCollection, dataCollection, id="topicA", q="awesome stuff"
checkpointEvery="1000")
{code}
The checkpoint collection will be used to persist the topic checkpoints.
Example combined with the DaemonStream:
{code}
daemon(topic(...)...)
{code}
When combined with SOLR-7739 this allows for messaging based on *machine
learned* classifications.
The TopicStream supports 3 models of publish/subscribe messaging:
1) *Request & response*: In this model the a topic(...) expression can be saved
and executed at any time. In this scenario the TopicStream will always retrieve
it's checkpoints and start from where it left off.
2) *Continuous pull streaming*: In this model you would wrap the TopicStream in
a DaemonStream and call read() in a loop inside a java program. This would
provide a continuous stream of new content as it arrives in the index.
3) *Continuous push streaming*: In this model you would send an expression like
this to the /stream handler: *daemon(update(topic(...)...)...)*. This daemon
process would run inside Solr and continuously stream new documents from the
topic and push them to another SolrCloud collection. Other pushing expressions
can be created to push documents in different ways or take other types of
actions.
> Add TopicStream to the streaming API
> ------------------------------------
>
> Key: SOLR-8588
> URL: https://issues.apache.org/jira/browse/SOLR-8588
> Project: Solr
> Issue Type: New Feature
> Reporter: Joel Bernstein
> Assignee: Joel Bernstein
> Attachments: SOLR-8588.patch, SOLR-8588.patch, SOLR-8588.patch,
> SOLR-8588.patch
>
>
> The TopicStream is a *publish/subscribe messaging service* built on top of
> SolrCloud. The TopicStream returns all *new* documents for a specific query.
> Version numbers will be used as checkpoints for Topics to ensure single
> delivery of each document. When combined with the DaemonStream (SOLR-8550),
> Topics can provide continuous streaming. Sample syntax:
> {code}
> topic(checkpointCollection, dataCollection, id="topicA", q="awesome stuff"
> checkpointEvery="1000")
> {code}
> The checkpoint collection will be used to persist the topic checkpoints.
> Example combined with the DaemonStream:
> {code}
> daemon(topic(...)...)
> {code}
> When combined with SOLR-7739 this allows for messaging based on *machine
> learned* classifications.
> The TopicStream supports 3 models of publish/subscribe messaging:
> 1) *Request & response*: In this model a topic(...) expression can be saved
> and executed at any time. In this scenario the TopicStream will always
> retrieve it's checkpoints and start from where it left off.
> 2) *Continuous pull streaming*: In this model you would wrap the TopicStream
> in a DaemonStream and call read() in a loop inside a java program. This
> would provide a continuous stream of new content as it arrives in the index.
> 3) *Continuous push streaming*: In this model you would send an expression
> like this to the /stream handler: *daemon(update(topic(...)...)...)*. This
> daemon process would run inside Solr and continuously stream new documents
> from the topic and push them to another SolrCloud collection. Other pushing
> expressions can be created to push documents in different ways or take other
> types of actions.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
