Ah, I see - yes, that's one option, to just point the Connect to use an already exiting and running Kafka cluster.
Would there be issues with potential version mis-match of dependencies between Kafka Connect (that uses, say, newer libs from the Confluent 3.3 distribution) and Apache Kafka of earlier (0.10) version that uses its own dependencies ? Thanks, Stephen! Sent with [ProtonMail](https://protonmail.com) Secure Email. > -------- Original Message -------- > Subject: Re: how to use Confluent connector with Apache Kafka > Local Time: September 29, 2017 11:49 AM > UTC Time: September 29, 2017 3:49 PM > From: sjdur...@gmail.com > To: users@kafka.apache.org, Marina Popova <ppine7...@protonmail.com> > > You can choose to run just kafka connect in the confluent platform (just run > the kafka connect shell script(s)) and configure the connectors to point > towards your kafka installation. The confluent platform uses vanilla kafka > under the covers, but there isn't anything requiring you to run kafka found > in that download. > > Kafka connect will run independently from kafka, as kafka connect is a web > service. Maybe I'm misunderstanding something in your question. > > On Fri, Sep 29, 2017 at 10:42 AM, Marina Popova <ppine7...@protonmail.com> > wrote: > >> Thanks, Stephen, >> Yes, I have no problem getting the Confluent distribution working. >> But my goal right now is to use the vanilla Apache Kafka distribution which >> we already have setup and fully automated in our environments. Switching to >> another distribution would invalidate most of that work, and is not an >> option for now, for at least a few months... >> >> So, I'm trying to understand how I can add the Confluent Elasticsearch >> Connector to the existing Apache Kafka installation... >> >> Thank you!Marina >> >> Sent with [ProtonMail](https://protonmail.com) Secure Email. >> >>> -------- Original Message -------- >>> Subject: Re: how to use Confluent connector with Apache Kafka> Local Time: >>> September 29, 2017 10:50 AM >>> UTC Time: September 29, 2017 2:50 PM >>> From: sjdur...@gmail.com >>> To: users@kafka.apache.org, Marina Popova <ppine7...@protonmail.com> >>> >>> The confluent platform download has everything pre-configured for >>> everything to work after unzipping the download, including any dependencies >>> that it needs to run. The shell script that starts up the kafka connect >>> worker ensures that everything that needs to be on the classpath is on the >>> classpath (take a look at them to see how everything is being wired >>> together, but its essentially a series of scripts that are building up the >>> classpath and setting any appropriate environment variables). The >>> elasticsearch connector is available bundled with the platform, so you >>> shouldn"t need to do anything to get that working beyond configuration of >>> the plugin and submitting that to the worker. >>> >>> Just follow the quickstart guide here: >>> https://docs.confluent.io/current/connect/connect-elasticsearch/docs/elasticsearch_connector.html >>> >>> On Fri, Sep 29, 2017 at 8:24 AM, Marina Popova <ppine7...@protonmail.com> >>> wrote: >>> >>>> Thanks, Matthias, >>>> the "download" part is what I am not clear about.... >>>> I don"t see a separate download for just a connector, only for the whole >>>> Confluent platform. >>>> Is it just a matter of taking one jar, say, the >>>> kafka-connect-elasticsearch-3.3.0.jar from the .../confluent-3.3.0/share/ >>>> java/kafka-connect-elasticsearch/ dir? and adding it into the plugin.path >>>> dir for Kafka? Or do I package the whole >>>> ./confluent-3.3.0/share/java/kafka-connect-elasticsearch/ >>>> dir as an Uber Jar to make sure all dependencies are also included? >>>> >>>> Thanks, >>>> Marina >>>> >>>> Sent with [ProtonMail](https://protonmail.com) Secure Email. >>>> >>>> > -------- Original Message -------- >>>> > Subject: Re: how to use Confluent connector with Apache Kafka >>>> > Local Time: September 27, 2017 2:45 PM >>>> > UTC Time: September 27, 2017 6:45 PM >>>> > From: matth...@confluent.io >>>> > To: users@kafka.apache.org >>>> > >>>> > All connectors are compatible with vanilla AK, as Confluent Open Source >>>> > ships with "plain" Apache Kafka under the hood. >>>> > >>>> > So you can just download the connector, plug it in, and configure it as >>>> > any other connector, too. >>>> > >>>> > https://www.confluent.io/product/connectors/ >>>> > >>>> > -Matthias >>>> > >>>> > On 9/26/17 1:15 PM, Marina Popova wrote: >>>> >> Hi, >>>> >> we have an existing Kafka cluster (0.10) already setup and working in >>>> production. >>>> >> I woudl like to explore using Confluent"s Elasticsearch Connector - >>>> however, I see it comes as part of the Confluent distribution of Kafka >>>> (with separate confluent scripts, libs, etc.). >>>> >> >>>> >> Is there an easy way to just use the Confluent Connector with the plain >>>> (non-Confluent) Apache distribution of Kafka? >>>> >> >>>> >> thanks! >>>> >> Marina >>>> >> >>>>
