stuartm21 opened a new issue #1351:
URL: https://github.com/apache/camel-kafka-connector/issues/1351
Hello @oscerd ,
We are planning to use the following connect and connector for restoring the
topic data from s3 to kafka.
**kafka-connect.yaml:**
```
apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaConnect
metadata:
name: strimzi-kafka
namespace: kafka
annotations:
strimzi.io/use-connector-resources: "true"
spec:
version: 3.1.0
replicas: 1
image: dockerhub/camel-aws-s3-source-kafka-connector:v1.0.0
bootstrapServers: kafka-bootstrap.myhost.com:443
externalConfiguration:
volumes:
- name: aws-credentials
secret:
secretName: aws-credentials
tls:
trustedCertificates:
- secretName: strimzi-cluster-ca-cert
certificate: ca.crt
authentication:
type: tls
certificateAndKey:
certificate: user.crt
key: user.key
secretName: kafka-user
config:
config.providers: file
config.providers.file.class:
org.apache.kafka.common.config.provider.FileConfigProvider
key.converter: org.apache.kafka.connect.json.JsonConverter
value.converter: org.apache.kafka.connect.json.JsonConverter
key.converter.schemas.enable: false
value.converter.schemas.enable: false
config.storage.replication.factor: 1
offset.storage.replication.factor: 1
status.storage.replication.factor: 1
```
**source-connector.yaml**
```
apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaConnector
metadata:
name: s3-source-connector
namespace: kafka
labels:
strimzi.io/cluster: "strimzi-kafka"
spec:
class:
org.apache.camel.kafkaconnector.awss3source.CamelAwss3sourceSourceConnector
tasksMax: 2
config:
key.converter: org.apache.kafka.connect.storage.StringConverter
value.converter: org.apache.kafka.connect.storage.StringConverter
topics: kafka-topic
camel.kamelet.aws-s3-source.bucketNameOrArn: test-kafka-connect
camel.kamlet.aws-s3.configuration.autocloseBody: false
camel.kamelet.aws-s3-source.accessKey:
${file:/opt/kafka/external-configuration/aws-credentials/aws-credentials.properties:aws_access_key_id}
camel.kamelet.aws-s3-source.secretKey:
${file:/opt/kafka/external-configuration/aws-credentials/aws-credentials.properties:aws_secret_access_key}
camel.kamelet.aws-s3-source.region: ca-central-1
```
So our questions are:
1. We need to restore the topics of a single user from s3 to kafka with a
specific year or month or time, so how can we do that.?
2. Once we restored the data to kafka cluster, what will happen with this
connect and connector? Do we need to continue to run this on the k8s cluster?
3. This issue is also connected to my another issue:
https://github.com/apache/camel-kafka-connector/issues/1347 Because the sink
need to take the backup in yearly, monthly, daily and etc, so that we can
restore it to kafka.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
