Thanks all.
I was using an older Kafka version.
I upgraded Kafka in the cluster from kafka_2.12-1.1.0 to the latest stable
version kafka_2.12-2.7.0. I also upgraded zookeeper from
zookeeper-3.4.6 to apache-zookeeper-3.6.2-bin
version.
In addition in the run file I added the following:
#!/bin/k
Ok this try to stream kafka topic to BigQuery has moved on but I am now
getting this error
[2021-03-12 20:17:41,870] ERROR Stopping due to error
(org.apache.kafka.connect.cli.ConnectStandalone:122)
j*ava.lang.NoSuchMethodError:
org.apache.kafka.common.acl.AclBindingFilter.(Lorg/apache/kafka/common
Thanks again Liam.
This is the error
[2021-03-12 14:17:54,670] ERROR Stopping after connector error
(org.apache.kafka.connect.cli.ConnectStandalone:113)
java.lang.NoClassDefFoundError:
org/apache/kafka/common/config/ConfigDef$CaseInsensitiveValidString
at
com.wepay.kafka.connect.bigquery.
I feel your CLASSPATH env var might need to include $KAFKA_HOME/lib (it
should already have the common conf lib in there).
On Fri, 12 Mar. 2021, 10:33 pm Mich Talebzadeh,
wrote:
> Thanks Liam for the suggestion.
>
> This is the redone sink file (plain text)
>
> name=bigquery-sink
> connector.ty
Thanks Liam for the suggestion.
This is the redone sink file (plain text)
name=bigquery-sink
connector.type=bigquery-connector
connector.class=com.wepay.kafka.connect.bigquery.BigQuerySinkConnector
defaultDataset=test
project=axial-glow-224522
topics=md
autoCreateTables=false
gcsBucketName=tmp_st
Hi Mich,
Your bigquery-sink.properties file is in a JSON format - which won't work.
It needs to follow the usual format of a Java properties file.
Kind regards,
Liam Clarke-Hutchinson
On Fri, Mar 12, 2021 at 12:13 AM Mich Talebzadeh
wrote:
> Hi,
>
>
> Trying to stream from Kafka to Google Big
