[
https://issues.apache.org/jira/browse/SOLR-17065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17825730#comment-17825730
]
Houston Putman commented on SOLR-17065:
---------------------------------------
Hey Jan, there has been a dev list discussion about this:
[https://lists.apache.org/thread/ld5j8qtvh2wvnc6fqbkt94ot5f3h8k39]
{quote}How would you e.g. be able to use CrossDC from SolrOperator if this was
an independent package, without requiring internet access from each Solr node
to download package jars, and Operator to orchestrate the dance? I suspect
users would end up building their custom Solr docker image with everything
embedded... But how do you embed a pkg-mgr package into a Docker image etc?'
{quote}
Yeah, users would have to build their own images with the packages that they
want.
{quote}Perhaps we should revive the 1st party packages / module repository /
releasing more separate artifacts discussion on the dev list and not block this
JIRA. Once we have a unified plan we can start moving things out of main repo
in 10.0, 11.0 etc.
{quote}
I agree with this. If we want to make modules separate from the Solr release,
we can do that, but it shouldn't hold up improving Solr. For users that want a
slim Solr distribution, we already provide that.
{quote}So I'm not sure where to put this comment, but I am not sure this is a
good idea to bring into Solr main. I'd prefer we actually move some of the
existing modules out of Solr main. Each module brings in relatively heavy
dependencies that we need to keep upgrading and making sure they are all
compatible.
Is there a reason this can't stay outside of main? There are multiple companies
using it without it being in the Solr main repo. Could it instead go to a
solr-modules or solr-plugins repo if the name "solr-sandbox" is not suitable?
{quote}
I completely empathize with your problems around updating dependencies Kevin,
and adding this would just make that work harder. However, I'm not sure that
having every module in a separate repo is really the way forward. If they are
official modules, then Solr would still need to be compatible with them, it
would just need a lot more work to keep them up-to-date when they are out of
sight. We still would need to issue security patches and compatibility releases
since they would be official offerings from Solr.
> Migrate Sandbox CrossDC functionality into Solr
> -----------------------------------------------
>
> Key: SOLR-17065
> URL: https://issues.apache.org/jira/browse/SOLR-17065
> Project: Solr
> Issue Type: Task
> Components: CrossDC, module - crossDC
> Reporter: Houston Putman
> Assignee: Houston Putman
> Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> As a part of [SIP-13: Cross Data Center
> Replication|https://cwiki.apache.org/confluence/display/SOLR/SIP-13%3A+Cross+Data+Center+Replication],
> the [solr-sandbox|https://github.com/apache/solr-sandbox] repository was
> created and a new Cross DC implementation has been developed.
> This CrossDC implementation relies on Kafka (for now, maybe other queues in
> the future), and has two parts.
> * A producer, which is a Solr plugin (An updateRequestProcessor), that sends
> documents to kafka after successfully indexing them.
> * A consumer, which is a standalone application, that reads documents for
> kafka and sends them to the mirrored SolrClouds.
> This has been in development for a year now and is used at multiple companies
> (even in production).
> We believe that the project has reached a level of maturity that it can be
> "graduated" from the sandbox module and be included in future Solr releases.
>
> As a part of the move I propose the following:
> * The Producer will be branded as the "cross-dc" Solr module, and be
> available just as any other module is.
> * The Consumer will be branded as the "cross-dc-manager" and be packaged
> just as the prometheus-exporter is, a standalone application that comes with
> Solr and uses many of the same Jars. The reason for the name change is that
> we might expand the role of this application to something beyond just
> "consuming" from the Kafka queue. In that case its much easier to rename it
> now than later.
>
> We will let this bake in the main branch for a while before backporting to
> 9.x.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
