nathanhagemann opened a new issue #336:
URL: https://github.com/apache/camel-kafka-connector-examples/issues/336
First, thank you so much for creating these excellent Camel libraries.
Second, this may not be the right place to put this question. I don't have
an issue with the connector. My inexperience is the real issue.
Background: I'm wanting to sink a JSON message from a Kafka (open-source)
topic to Cassandra (open-source) table with Kafka Connect
Kafka
Version: 2.13-3.0.0 (Scala: 2.13, Kafka: 3.0.0)
IP/Ports: 10.188.5.86:9092,10.188.5.86:9093,10.188.5.86:9094
Topic: pie (./bin/kafka-topics.sh --bootstrap-server
10.188.5.86:9092,10.188.5.86:9093,10.188.5.86:9094 --create
--replication-factor 1 --partitions 1 --topic pie)
JSON
Simple Pie Data:
{"Type:":"Apple","Invented":1381}
{"Type:":"Pecan","Invented":1870}
{"Type:":"Cherry","Invented":1500}
Cassandra
Version:4.0.3
IP:10.66.16.10
Port:9042
User:cameldevloader
Pass:newpassword
CREATE TABLE pie(type varchar, invented double, PRIMARY KEY (type));
INSERT INTO pie (type, invented) VALUES ('Chess',1750);
SELECT * FROM pie;
yields: Chess 1750.0
Kafka Connect
Using this Camel connector:
https://camel.apache.org/camel-kafka-connector/1.0.x/reference/connectors/camel-cassandra-sink-kafka-sink-connector.html
Downloaded from:
https://repo.maven.apache.org/maven2/org/apache/camel/kafkaconnector/camel-cassandra-sink-kafka-connector/1.0.0/camel-cassandra-sink-kafka-connector-1.0.0-package.tar.gz
copied the camel-cassandra-sink-kafka-connector-1.0.0-package.tar.gz to
Kafka server at location
/root/sofware/camel-cassandra-sink-kafka-connector-1.0.0-package.tar.gz
on my Kafka server in the /root/connectors/ directory unzipped:
~/connectors/tar -xvzf
../software/camel-cassandra-sink-kafka-connector-1.0.0-package.tar.gz
Updated /root/kafka/config/connect-standalone.properties
`# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# These are defaults. This file just demonstrates how to override some
settings.
bootstrap.servers=10.188.5.86:9092,10.188.5.86:9093,10.188.5.86:9094
# The converters specify the format of data in Kafka and how to translate it
into Connect data. Every Connect user will
# need to configure these based on the format they want their data in when
loaded from or stored into Kafka
key.converter=org.apache.kafka.connect.json.JsonConverter
value.converter=org.apache.kafka.connect.json.JsonConverter
# Converter-specific settings can be passed in by prefixing the Converter's
setting with the converter we want to apply
# it to
key.converter.schemas.enable=true
value.converter.schemas.enable=true
offset.storage.file.filename=/tmp/connect.offsets
# Flush much faster than normal, which is useful for testing/debugging
offset.flush.interval.ms=10000
# Set to a list of filesystem paths separated by commas (,) to enable class
loading isolation for plugins
# (connectors, converters, transformations). The list should consist of top
level directories that include
# any combination of:
# a) directories immediately containing jars with plugins and their
dependencies
# b) uber-jars with plugins and their dependencies
# c) directories immediately containing the package directory structure of
classes of plugins and their dependencies
# Note: symlinks will be followed to discover dependencies or plugins.
# Examples:
plugin.path=/root/connectors/`
Created (from docs/examples)
/root/myconnectorproperties/CamelCassandraPieSink.properties
`---------------------------------------------------------------------------
## Licensed to the Apache Software Foundation (ASF) under one or more
## contributor license agreements. See the NOTICE file distributed with
## this work for additional information regarding copyright ownership.
## The ASF licenses this file to You under the Apache License, Version 2.0
## (the "License"); you may not use this file except in compliance with
## the License. You may obtain a copy of the License at
##
## http://www.apache.org/licenses/LICENSE-2.0
##
## Unless required by applicable law or agreed to in writing, software
## distributed under the License is distributed on an "AS IS" BASIS,
## WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
## See the License for the specific language governing permissions and
## limitations under the License.
##
---------------------------------------------------------------------------
name=CamelCassandra-sinkSinkConnector
connector.class=org.apache.camel.kafkaconnector.cassandrasink.CamelCassandrasinkSinkConnector
tasks.max=1
# use the kafka converters that better suit your needs, these are just
defaults:
key.converter=org.apache.kafka.connect.storage.StringConverter
value.converter=org.apache.kafka.connect.storage.StringConverter
# comma separated topics to get messages from
topics=pie
# mandatory properties (for a complete properties list see the connector
documentation):
# Hostname(s) cassandra server(s). Multiple hosts can be separated by comma.
Example: localhost
camel.kamelet.cassandra-sink.connectionHost=10.66.16.10
# Port number of cassandra server(s) Example: 9042
camel.kamelet.cassandra-sink.connectionPort=9042
# user
camel.kamelet.cassandra-sink.username=cameldevloader
# password
camel.kamelet.cassandra-sink.password=newpassword
# Keyspace to use Example: customers
camel.kamelet.cassandra-sink.keyspace=dev
# The query to execute against the Cassandra cluster table
camel.kamelet.cassandra-sink.query=INSERT INTO pie (type, invented) VALUES
(?,?)`
Connect command:
/kafka # ./bin/connect-standalone.sh
/root/kafka/config/connect-standalone.properties
/root/myconnectorproperties/CamelCassandraPieSink.properties
Error: Kamelet with id cassandra-sink not found in locations:
classpath:/kamelets
Full log attache
[pie_result.txt](https://github.com/apache/camel-kafka-connector-examples/files/8307658/pie_result.txt)
d
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
