[
https://issues.apache.org/jira/browse/SOLR-17065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17783571#comment-17783571
]
Jan Høydahl commented on SOLR-17065:
------------------------------------
Given the small footprint of the feature I'm not negative to including it in
Solr's main tarball (not slim). Unfortunately the community has not managed to
mature packages into something that is GA. How would you e.g. be able to use
CrossDC from SolrOperator if this was an independent package, without requiring
internet access from each Solr node to download package jars, and Operator to
orchestrate the dance? I suspect users would end up building their custom Solr
docker image with everything embedded... But how do you embed a pkg-mgr package
into a Docker image etc?
Also the additional infrastructure needed to build, test, release, add
Docker-container for the tiny Kafka-consumer sounds like a lot to ask,
especially since we don't have any prior history of maintaining and testing
such independent components.
Well, I of course wish we had a brighter story on "lean". My above comments are
mostly being pragmatic, trying to avoid puting the entire weight of all our
modularization wishes on CrossDC.
Perhaps we should revive the 1st party packages / module repository / releasing
more separate artifacts discussion on the dev list and not block this JIRA.
Once we have a unified plan we can start moving things out of main repo in
10.0, 11.0 etc.
> Migrate Sandbox CrossDC functionality into Solr
> -----------------------------------------------
>
> Key: SOLR-17065
> URL: https://issues.apache.org/jira/browse/SOLR-17065
> Project: Solr
> Issue Type: Task
> Security Level: Public(Default Security Level. Issues are Public)
> Components: CrossDC, module - crossDC
> Reporter: Houston Putman
> Assignee: Houston Putman
> Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> As a part of [SIP-13: Cross Data Center
> Replication|https://cwiki.apache.org/confluence/display/SOLR/SIP-13%3A+Cross+Data+Center+Replication],
> the [solr-sandbox|https://github.com/apache/solr-sandbox] repository was
> created and a new Cross DC implementation has been developed.
> This CrossDC implementation relies on Kafka (for now, maybe other queues in
> the future), and has two parts.
> * A producer, which is a Solr plugin (An updateRequestProcessor), that sends
> documents to kafka after successfully indexing them.
> * A consumer, which is a standalone application, that reads documents for
> kafka and sends them to the mirrored SolrClouds.
> This has been in development for a year now and is used at multiple companies
> (even in production).
> We believe that the project has reached a level of maturity that it can be
> "graduated" from the sandbox module and be included in future Solr releases.
>
> As a part of the move I propose the following:
> * The Producer will be branded as the "cross-dc" Solr module, and be
> available just as any other module is.
> * The Consumer will be branded as the "cross-dc-manager" and be packaged
> just as the prometheus-exporter is, a standalone application that comes with
> Solr and uses many of the same Jars. The reason for the name change is that
> we might expand the role of this application to something beyond just
> "consuming" from the Kafka queue. In that case its much easier to rename it
> now than later.
>
> We will let this bake in the main branch for a while before backporting to
> 9.x.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
