Re: Kafka-connect

2018-05-03 Thread Burton Williams
whats in docker? kafka? kafka-connnect? Did you try setting up outside of a container first? -B On Thu, May 3, 2018 at 2:22 PM, Jagannath Bilgi wrote: > Hi Team, > Trying to load data from Cassandra to Kafka using kafka-connect. Tried > results from Google search. However unable to complete suc

Re: Fw: Re: Kafka-connect

2018-05-05 Thread Burton Williams
rce > <https://medium.com/walmartlabs/getting-started-with-the-kafka-connect-cassandra-source-e6e06ec72e97> > > Getting below error while starting kafka. > > [image: Inline image] > > > [image: Inline image] > > Thanks and regards > > Jagannath S Bilgi > &

Re: Fw: Re: Kafka-connect

2018-05-08 Thread Burton Williams
path > Thanks and regards > Jagannath S Bilgi > > > On Saturday, May 5, 2018, 4:36:09 PM GMT+5:30, Burton Williams < > burton.b.willi...@gmail.com> wrote: > > Your ZK path is incorrect. It has a `#` vs a '/' in it. Try changing that. > > -B > >

Re: Fw: Re: Kafka-connect

2018-05-09 Thread Burton Williams
Datacqlsh:blog> select * from pack_events; > event_id | event_ts | event_data--+- > +---500 | 2018-01-22 > 20:28:50.869000+ | {"foo":"bar"}501 | 2018-05-08 > 20:28:50.869000+ | {"

Re: Get count of messages

2018-10-29 Thread Burton Williams
you can user kafkacat starting at that offset to head and pipe the output to "wc -l" (word count). -BW On Mon, Oct 29, 2018 at 3:39 AM Sachit Murarka wrote: > Hi All, > > Could you please help me in getting count of all messages stored in kafka > from a particular offset? > I have tried GetOffs

Re: Get count of messages

2018-10-30 Thread Burton Williams
; > > -Matthias > > On 10/29/18 3:02 PM, Burton Williams wrote: > > you can user kafkacat starting at that offset to head and pipe the output > > to "wc -l" (word count). > > > > -BW > > > > On Mon, Oct 29, 2018 at 3:39 AM Sachit Murarka > >

Kafka Connect Limits

2024-07-03 Thread Burton Williams
Hi, I have 100+ sink connectors running with 100+ topics each with roughly 3 partitions per topic. There are running on K8s on 10 pods with 6 cpus and 32Gig mem. The connector in question is Snowflake's sink connector v2.2.0. This worked in the mini batch mode SNOWPIPE, but once i switched over to

Re: Kafka Connect Limits

2024-07-04 Thread Burton Williams
l.java#L908 > [4] > > https://github.com/snowflakedb/snowflake-kafka-connector/blob/b08ee569de6b943f8f624cd4421f7a6c7a10d532/src/main/java/com/snowflake/kafka/connector/internal/streaming/SnowflakeSinkServiceV2.java#L442 > > On Wed, Jul 3, 2024 at 12:25 PM Burton Williams < > burton.b.

Kafka Connect Worker Provisioning/Optimization Considerations

2024-07-04 Thread Burton Williams
I have 100+ sink connectors running with 100+ topics each with roughly 3 partitions per topic. How would you configure resources (mem and cpu) to optimally handle this level of load. What would be your considerations? Also, When considering this load, should i be thinking about it as an aggregated