Re: Apache Kafka and Zookeeper Compatibility matrix

There is a compatibility matrix for Zk versions available until v3.8 here - https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=240882784#KIP902:UpgradeZookeeperto3.8.2-Compatibility,Deprecation,andMigrationPlan -- Divij Vaidya On Tue, Dec 10, 2024 at 3:27 PM Chia-Ping Tsai wrote

Re: Apache Kafka and Zookeeper Compatibility matrix

hi > I have to upgrade Apache Kafka from 3.3 to the latest stable version. The latest version that supports ZooKeeper is *3.9.0*. By the way, there is a pull request #18091 aiming to add a comprehensive compatibility matrix. You can find more informa

Re: Apache Kafka Installation and Env Set up

Hi Venkat, are you planning to use Open source Apache Kafka Or Confluent? what is your use case apart from streaming? regards, Sunil. On Fri, 29 Sep 2023 at 12:27 PM, ANANTHA VENKATA GANJAM wrote: > TCS Confidential > > Hi Team, > > We are planning to set up Lab environment for Kafka in TCS. Pl

Re: Apache Kafka consumer consumes messages with "partition" option, but not with "group" option

Hi, If you have a large number of partitions in your topic, it can take a really long while before you start seeing messages on the console. So, using the partition id is the right approach. But just need to be patient at the command-line. Out of interest, how long did you wait for the output f

Re: Apache Kafka version - End of support

Many thanks Joseph for your response On Mon, Mar 27, 2023 at 4:50 PM Josep Prat wrote: > Hello there, > > You can find the general policy here: > > https://cwiki.apache.org/confluence/display/KAFKA/Time+Based+Release+Plan#TimeBasedReleasePlan-WhatIsOurEOLPolicy > > Basically, the last 3 releases

Re: Apache Kafka version - End of support

Hello there, You can find the general policy here: https://cwiki.apache.org/confluence/display/KAFKA/Time+Based+Release+Plan#TimeBasedReleasePlan-WhatIsOurEOLPolicy Basically, the last 3 releases are covered and one can upgrade to the latest version from the last previous 3. Bug fixes are ported

RE: Apache Kafka EOL policy

Thanks a lot Josep. This cleared all our doubts. Thanks Gautham -Original Message- From: Josep Prat Sent: Tuesday, February 21, 2023 2:24 PM To: Users Subject: Re: Apache Kafka EOL policy Hi Gautham, "The last 3 releases are covered and one can upgrade to the latest version

Re: Apache Kafka EOL policy

Is that supported with bug fixes and security > patches? And if we want to upgrade from 2.8, can we upgrade to 3.4 from 2.8? > > Thanks > Gautham > > -Original Message- > From: Josep Prat > Sent: Tuesday, February 21, 2023 12:18 PM > To: Users > Subject: Re: Apac

RE: Apache Kafka EOL policy

Kafka 2.8? Is that supported with bug fixes and security patches? And if we want to upgrade from 2.8, can we upgrade to 3.4 from 2.8? Thanks Gautham -Original Message- From: Josep Prat Sent: Tuesday, February 21, 2023 12:18 PM To: Users Subject: Re: Apache Kafka EOL policy Hi Gautham

Re: Apache Kafka EOL policy

Hi Gautham, You can find the general policy here: https://cwiki.apache.org/confluence/display/KAFKA/Time+Based+Release+Plan#TimeBasedReleasePlan-WhatIsOurEOLPolicy ? Basically, the last 3 releases are covered and one can upgrade to the latest version from the last previous 3. Bug fixes are ported

Re: Apache Kafka Connect

Hi Namita and Pasi, Logstash as middlemen is good if and only if: 1. You dont need buffer in between and you are Ok with tight coupling between source and destination. 2. There are sufficient number of logstash servers available in case of large data. In case of logstash, source and destinations sh

Re: Apache Kafka Connect

Hi Namita, You can use Logstash as middleman: https://www.elastic.co/guide/en/logstash/current/plugins-inputs-elasticsearch.html https://www.elastic.co/guide/en/logstash/current/plugins-outputs-kafka.html -pasi From: Namita Jaokar Date: Friday, 30. September 2022 at 19.56 To: u

Re: Apache Kafka Connect

Hi Namita, For Moving data from Elasticsearch to Kafka you need Elasticsearch Source connector. I guess this is not supported connector. You may have to rely on some community developed connector where you may not get instant support. https://github.com/DarioBalinzo/kafka-connect-elasticsearch-sou

Re: Apache Kafka :-Producer-Consumer at different network

Happy to help, Namita. Regards, Jim Ziesig On Fri, Sep 30, 2022 at 7:51 AM Namita Jaokar wrote: > Hi James, > > Thanks alot James for the solution provided. The below configurations did > work when producer and consumer applications were checked with the > scenarios mentioned. > > > Thanks & R

Re: Apache Kafka :-Producer-Consumer at different network

Hi James, Thanks alot James for the solution provided. The below configurations did work when producer and consumer applications were checked with the scenarios mentioned. Thanks & Regards, Namita Jaokar On Tue, Sep 27, 2022 at 7:35 PM James Ziesig wrote: > Hi Namita, > > This is likely a lis

Re: Apache Kafka :-Producer-Consumer at different network

Hi Namita, You need to use an address that the producer system can reach on the server system as the listener address. If you are using an IP address in the bootstrap server value on the producer, you likely need to use the IP address in the listener as well. Either that or add the IP address an

Re: Apache Kafka :-Producer-Consumer at different network

Hi James, As suggested, I did try below steps but still my producer is unable to connect with the consumer. Below are the changes checked upon: *Producer changes* In My java producer configuration class which is at client network and different system, Changed the configuration of bootstrap server

Re: Apache Kafka :-Producer-Consumer at different network

Hi Namita, This is likely a listener configuration issue. The bootstrap server setting only gets you so far, your client must be able to contact the server via a listener address. I have seen this occur on a few systems where a hostname could not be identified by Kafka, so the server advertises

Re: Apache Kafka :-Producer-Consumer at different network

++ Updates. On Tue, Sep 27, 2022 at 12:54 PM Namita Jaokar wrote: > Hi All, > > I am currently exploring a scenario where my producer and consumer are at > different networks and systems. > > Below is my scenario: > > My kafka producer is on a client network on a system that does not have > Apac

Re: Apache Kafka acknowledgement

Hi Raj, I think for your proble Rabbitmq RPC usecase is more suitable. By the way its not recommended to make such tight couple between 2 services if it's possible. Regards On Fri, Jul 29, 2022, 12:06 PM Richard Bosch wrote: > Hi Raj, > > The Kafka design is based on fully decoupled producers an

Re: Apache Kafka acknowledgement

Hi Raj, The Kafka design is based on fully decoupled producers and consumers, where the responsibility of the producer ends after a record has been successfully produced. Since a producer is not aware of other applications that might be consuming the record it can continue with its own flow. The s

Re: Apache Kafka Questions

Hello Team Thank you for answering the questions I reached out earlier. We have a new set, and I would really like to have them verified and confirmed by an expert. Request to answer them with specification in a way they can be replicated in local user environment. f. How are Consumer Groups c

Re: Apache Kafka Questions

From: Jatin Chhabriya Date: Wednesday, March 16, 2022 at 9:20 AM To: users@kafka.apache.org Cc: Murali Krishna Subject: [EXTERNAL] Apache Kafka Questions Hello Team Upon careful perusal of documentation and tutorials, our team has a few open questions, and we would appreciate to have these c

Re: Apache Kafka : start up scripts

Start here https://github.com/apache/kafka/blob/trunk/bin/kafka-server-start.sh https://github.com/apache/kafka/tree/trunk/core/src/main/scala/kafka/server Also take a look at the logs when the server starts up. That should give you some insights. On Wed, Oct 27, 2021 at 5:03 PM Kafka Life wro

Re: Apache Kafka mirrormaker2 : sync.group.offsets.enabled and sync.group.offsets.interval.seconds

Hello All, Any inputs on my question below ? Regards, Neeraj Sent from my iPhone > On 26 Oct 2021, at 10:29 am, Neeraj Vaidya > wrote: > > Anyone has inputs on my question below ? > > Regards, > Neeraj > > Sent from my iPhone > >> On 24 Oct 2021, at 10:23 pm, Neeraj Vaidya >> wrote: >>

Re: Apache Kafka mirrormaker2 : sync.group.offsets.enabled and sync.group.offsets.interval.seconds

Anyone has inputs on my question below ? Regards, Neeraj Sent from my iPhone > On 24 Oct 2021, at 10:23 pm, Neeraj Vaidya > wrote: > > When replicating data from one datacentre (DC1) to another (DC2) using MM2, > I can see that the consumer offsets are actually getting synchronized from >

Re: Apache Kafka 2.8.0 and Zookeeper

Thanks for pointing out the relevant documentation. :) That helps a lot ! On Wed, Jul 14, 2021 at 4:24 AM Shilin Wu wrote: > Kafka 2.8 supports ZooKeeper 3.4.10 through 3.5.9 (3.5.9 recommended). > > According to > > https://docs.confluent.io/platform/current/installation/versions-interoperabili

Re: Apache Kafka 2.8.0 and Zookeeper

Kafka 2.8 supports ZooKeeper 3.4.10 through 3.5.9 (3.5.9 recommended). According to https://docs.confluent.io/platform/current/installation/versions-interoperability.html Note that Confluent Platform 6.2.x is compatible of Kafka 2.8, and Confluent Platform 6.2.x supports ZooKeeper 3.4.10 ~ 3.5.9.

Re: Apache Kafka Streams : Out-of-Order messages & uses of TimeStamp extractor

Replied on StackOverflow: https://stackoverflow.com/questions/67158317/apache-kafka-streams-out-of-order-messages -Matthias On 4/20/21 4:21 PM, Neeraj Vaidya wrote: > Hi, > I have asked this on StackOverflow, but will ask it here as well. > > I have an Apache Kafka 2.6 Producer which writes t

Re: Apache Kafka project CLA requirement

advise? In case you are requiring contributors to sign the > ASF CLA, may you please provide me with an URL to the documentation? > > > > > > Thanks, > > > > Radi > > > > *From: *Radoslava Zheleva > *Date: *Monday, January 18, 2021 at 6:03 PM > *To: *"u

Re: Apache Kafka project CLA requirement

requiring contributors to sign the ASF CLA, may you please provide me with an URL to the documentation? Thanks, Radi From: Radoslava Zheleva Date: Monday, January 18, 2021 at 6:03 PM To: "users@kafka.apache.org" Cc: Lauren Britton Subject: Re: Apache Kafka project CLA requirement Hello

Re: Apache Kafka project CLA requirement

Hi Radoslava, I believe you can get your answers from here ASF Contributor Agreements (apache.org) specifically I refer you to this sentence and I quote: "The ASF desires that all contributors of ideas, code, or documentation to any

Re: Apache Kafka project CLA requirement

Hello Team, May you please look at my question below and advise? Thanks, Radi From: Radoslava Zheleva Date: Thursday, January 14, 2021 at 4:33 PM To: "users@kafka.apache.org" Cc: Lauren Britton Subject: Apache Kafka project CLA requirement Hello Team, May you please advise if the Apache Ka

RE: Apache Kafka Powered By : Proposition La Redoute

urray Subject: Re: Apache Kafka Powered By : Proposition La Redoute Hey Antonie, I hope your week is going well. The interactive team has completed the webpage update. Your company's logo is now displayed on the Powered By Kafka webpage - http://kafka.apache.org/powered-by<

RE: Apache Kafka Powered By : Proposition La Redoute

Hello John, Thanks for your feedback and no problem, I just did the PR with the text and the logo file https://github.com/apache/kafka-site/pull/315 Antoine -Original Message- From: John Roesler Sent: 15 December 2020 17:57 To: users@kafka.apache.org Subject: Re: Apache Kafka Powered

Re: Apache Kafka Powered By : Proposition La Redoute

Hi Antoine, Thanks for your message! I couldn't see the logo; I think the mailing list software doesn't transmit attachments. Perhaps you could just send a pull request to add La Redoute to https://github.com/apache/kafka-site/edit/asf-site/powered-by.html ? Feel free to reply here if you have

Re: Apache Kafka website "videos" page clarification

o<http://twitter.com/ppatierno> > Linkedin : paolopatierno<http://it.linkedin.com/in/paolopatierno> > Blog : DevExperience<http://paolopatierno.wordpress.com/> > > From: Ben Stopford > Sent: Thursday, September 10, 2020 12:54 PM > To: Kafka Users > Subject: Re: A

Re: Apache Kafka website "videos" page clarification

tember 10, 2020 12:54 PM To: Kafka Users Subject: Re: Apache Kafka website "videos" page clarification Hi Paulo The reason for using Kafka Summit videos is there is an extensive, community-driven selection process that goes into Kafka Summit driven by the Kafka Summit Program Committee

Re: Apache Kafka Windows

You can probably look at maxwell by zendesk as well for CDC. They too have a very simple producer for Kafka. On Mon, 5 Oct 2020, 04:42 Naveen D, wrote: > Hi Guys, > > We have below scenario to achieve via KAfka windows stand alone > > 1. Download apache kafka for windows > 2.Start Zookeeper > 3.

Re: Apache Kafka Windows

Hi Guys, We have below scenario to achieve via KAfka windows stand alone 1. Download apache kafka for windows 2.Start Zookeeper 3. Basic consumer /producer communication should work. 4. use debezium SQL connector, uses CDC sql push data to azure data bricks We achieved step 1 -3,actually it wa

Re: Apache Kafka website "videos" page clarification

Hi Paulo The reason for using Kafka Summit videos is there is an extensive, community-driven selection process that goes into Kafka Summit driven by the Kafka Summit Program Committee. This is then further filtered by the community itself: those members of the community that attend the summit and

Re: Apache Kafka cluster to cluster

Is there documentation or example for mirror maker 2.0 ? On 4/29/20, 9:04 PM, "Liam Clarke-Hutchinson" wrote: [External] Hi Blake, Replicator is, AFAIK, not FOSS - however, Mirror Maker 2.0, which is built along very similar lines (i.e., on top of Kafka Connect) is, as is

Re: Apache Kafka cluster to cluster

Hi Clark My source cluster is on localhost:9092 My Target cluster was localhost:9091 I am not aware of something like Mirror maker 1 and 2 Because I am using mirror maker which come default from Apache Kafka 2.3.0 On Thu, Apr 30, 2020, 09:43 Liam Clarke-Hutchinson < liam.cla...@adscale.co.nz>

Re: Apache Kafka cluster to cluster

Hi Vishnu, I can see you're using Mirror Maker 1, unlike Mirror Maker 2, this doesn't auto-create topics on the target cluster. Does your target cluster on port 9091 have the same topics created on it that exist on the source cluster on port 9092? Also note that a Kafka consumer defaults to "late

Re: Apache Kafka cluster to cluster

Hi Blake, Replicator is, AFAIK, not FOSS - however, Mirror Maker 2.0, which is built along very similar lines (i.e., on top of Kafka Connect) is, as is Mirror Maker 1.0. On Thu, Apr 30, 2020 at 6:51 AM Blake Miller wrote: > Oh, and it looks like Confluent has released a newer replacement for >

Re: Apache Kafka cluster to cluster

i start the kafka mirror using the below configuration .\bin\windows\kafka-mirror-maker.bat --consumer.config .\config\consumer.properties --producer.config .\config\producer.properties --whitelist=".*" *Consumer.properties:* # format: host1:port1,host2:port2 ... bootstrap.servers=localhost:9092

Re: Apache Kafka cluster to cluster

Thanks Blake..😊😊 More over can we use any connector types for this situation? Like source is a topic and sink is also an another topic Is this possible... On Thu, Apr 30, 2020, 00:19 Blake Miller wrote: > Hi Vishnu, > > Check out MirrorMaker > https://cwiki.apache.org/confluence/pages/viewpag

Re: Apache Kafka cluster to cluster

Oh, and it looks like Confluent has released a newer replacement for MirrorMaker called Replicator https://docs.confluent.io/current/multi-dc-deployments/replicator/migrate-replicator.html On Wed, Apr 29, 2020 at 6:49 PM Blake Miller wrote: > Hi Vishnu, > > Check out MirrorMaker > https://cwi

Re: Apache Kafka cluster to cluster

Hi Vishnu, Check out MirrorMaker https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=27846330 This can do what you want. Note that the offsets are not copied, nor are the message timestamps. HTH On Wed, Apr 29, 2020 at 6:47 PM vishnu murali wrote: > Hi Guys, > > I am having two

Re: Apache Kafka Streams with Sprinng Boot

Hi Gary, Thank you very much for the help. Thanks C Suresh On Wednesday, April 29, 2020, Gary Russell wrote: > There's a simple example in this answer on Stack Overflow [1]. > > I am the project lead for the Spring for Apache Kafka project [2]. > > If you have further questions; ask on Stack O

Re: Apache Kafka Streams with Sprinng Boot

Hi ivan, Thank you for the help. Thanks C Suresh On Wednesday, April 29, 2020, Ivan Ponomarev wrote: > Hi everyone, > > take a look at https://github.com/inponomarev/kstreams-examples > > These are small examples for my training course about Kafka Streams. All > the examples utilize Spring Boo

Re: Apache Kafka Streams with Sprinng Boot

Hi everyone, take a look at https://github.com/inponomarev/kstreams-examples These are small examples for my training course about Kafka Streams. All the examples utilize Spring Boot, and they use quite recent versions of everything. Supporting notes and slides are in Russian, unfortunately.

Re: Apache Kafka Streams with Sprinng Boot

There's a simple example in this answer on Stack Overflow [1]. I am the project lead for the Spring for Apache Kafka project [2]. If you have further questions; ask on Stack Overflow tagged with spring-kafka (I don't monitor this group closely) [1]: https://stackoverflow.com/questions/58193901/u

Re: Apache Kafka Streams with Sprinng Boot

Sure Khaja. Thanks C Suresh On Tuesday, April 28, 2020, KhajaAsmath Mohammed wrote: > Hello Suresh, > > I am also looking for the same. Let me know if you find anything > > Sent from my iPhone > > > On Apr 28, 2020, at 8:25 PM, Suresh Chidambaram > wrote: > > > > Hi Team, > > > > Greetings. >

Re: Apache Kafka Streams with Sprinng Boot

Hello Suresh, I am also looking for the same. Let me know if you find anything Sent from my iPhone > On Apr 28, 2020, at 8:25 PM, Suresh Chidambaram > wrote: > > Hi Team, > > Greetings. > > I have been looking for an example application which uses Kafka Streams > with Spring Boot, but I'm

Re: Apache Kafka training

Disclaimer: I work for Confluent, but have you consider the online or onsite trainings we provide? the team is really good at delivering and the content is really deep in details. -- Pere Missatge de M. Manna del dia dj., 26 de set. 2019 a les 13:54: > If you have access to Pluralsight, Apache

Re: Apache Kafka training

If you have access to Pluralsight, Apache Kafka has a good coverage there. On Thu, 26 Sep 2019 at 12:28, Arvind Sharma wrote: > If you like to study online then there's a series by Stephane Maarek on > udemy , till now it's one of the best content I ever saw . > > Btw I am not promoting either o

Re: Apache Kafka training

If you like to study online then there's a series by Stephane Maarek on udemy , till now it's one of the best content I ever saw . Btw I am not promoting either of the 2 (udemy or Stephane ) in any way Regards On Thu, 26 Sep 2019, 3:59 pm Sachin Bal, wrote: > Hi All, > > Can someone suggest a

Re: apache kafka community container image

There’s no official image signed by ASF Kafka community like we have for some other projects. Containerising Kafka has been more of a casual project than open source initiative (e.g. tomcat containers). On Thu, 23 May 2019 at 16:48, Zain Malik wrote: > I was hoping to just get a hold on some o

Re: apache kafka community container image

I was hoping to just get a hold on some official image of kafka where the community can rely on rather than images living in 3rd party repos. On 2019/05/23 15:35:33, Pavel Sapozhnikov wrote: > If you're running on Kubernetes or Openshift check out strimzi project. > > On Thu, May 23, 2019, 11

Re: apache kafka community container image

If you're running on Kubernetes or Openshift check out strimzi project. On Thu, May 23, 2019, 11:23 AM Zain Malik wrote: > Hi, > > Is there any plans to support a community container image for kafka? > > Right now there are bitnami and confluent based images but nothing > official without bitnam

Re: Apache Kafka Process showing high CPU (100 to 200+) usage in Linux when Idle

Controller is not running in this node. We found that one of the new producer added was publishing only to a specific partition which was in this node, that explains the high utilization only in this node. The publisher was running in an Async thread invoking the send() asynchronously but the messa

Re: Apache Kafka Process showing high CPU (100 to 200+) usage in Linux when Idle

After you deleted a topic was it a clean delete. Did you verify in zookeeper and kafka logs directory? if not you may need to do some clean up if there are inconsistency in kafka logs dir and zookeeper. did you try to move the replicas assignment to different machines for this topic and see if i

Re: Apache Kafka Process showing high CPU (100 to 200+) usage in Linux when Idle

bq. only one specific node is showing this issue Is controller running on this node ? updating the metrics is expensive. Cheers On Sat, Aug 4, 2018 at 3:00 PM Abhijith Sreenivasan < abhijithonl...@gmail.com> wrote: > Hello > > We are seeing high CPU usage for the Kafka process. I am using 0.11

Re: Apache Kafka QuickStart

> After executing the first command to start zookeeper, do i have to open a > Terminal to run the Kafka Server? Yes. What is the problem with this? "but run into problem in Step 2" If you follow the quickstart, you download the binaries and start multiple processes (Zookeeper, Kafka Broker, Ka

Re: Apache Kafka QuickStart

Hi Nicholas, The quickstart is meant to run in terminals. The two commands in Step 2 should be run in different terminals unless you're sending the ZooKeeper process to the backgroud. If you are facing particular errors please share so we can better assist you. Thanks. --Vahid From: Nich

Re: Apache Kafka / Spark Integration - Exception - The server disconnected before a response was received.

Hi Daniel: Yes I am working with Spark Structured Streaming. The exception is emanating from spark kafka connector but I was wondering if someone has encountered this issue and resolved it by some configuration parameter in kafka client/broker or OS settings. Thanks Mans On Tuesday, April

Re: Apache Kafka / Spark Integration - Exception - The server disconnected before a response was received.

This looks more like a spark issue than it does a Kafka judging by the stack trace, are you using Spark structured streaming with Kafka integration by chance? On Mon, Apr 9, 2018 at 8:47 AM, M Singh wrote: > Hi Folks: > Just wanted to see if anyone has any suggestions on this issue. > Thanks > >

Re: Apache Kafka / Spark Integration - Exception - The server disconnected before a response was received.

Hi Folks: Just wanted to see if anyone has any suggestions on this issue. Thanks On Monday, March 26, 2018, 11:04:02 AM PDT, M Singh wrote: Hi Ted: Here is the exception trace (Note - The exception is occuring in the kafka spark writer class). I will try to check broker logs.  Is th

Re: Apache Kafka / Spark Integration - Exception - The server disconnected before a response was received.

Hi Ted: Here is the exception trace (Note - The exception is occuring in the kafka spark writer class). I will try to check broker logs.  Is there anything specific I should look for ? Driver stacktrace: at org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$failJob

Re: Apache Kafka / Spark Integration - Exception - The server disconnected before a response was received.

Can you post the stack trace for NetworkException (pastebin) ? Please also check the broker logs to see if there was some clue around the time this happened. Thanks On Mon, Mar 26, 2018 at 9:30 AM, M Singh wrote: > Hi: > I am working with spark 2.2.1 and spark kafka 0.10 client integration wit

Re: Apache Kafka TLV Meetup group

Hello Ori, I have added the meetup on the events list. Cheers, Guozhang On Thu, Nov 2, 2017 at 2:24 AM, ApacheKafka TLV wrote: > Hi, > > I am the organizer of Apache Kafka TLV Meetup in Tel-Aviv Israel, please > add our meetup group to your global events list. > > https://www.meetup.com/Apac

Re: Apache Kafka integration using Apache Camel

t; > > > > > org.apache.camel > > camel-core > > 2.17.0 > > > > > > Thanks & Regards > > Swati > > > > -Original Message- > > From: Gupta, Swati [mailto:swati.gu...@an

Re: Apache Kafka integration using Apache Camel

org.apache.camel > camel-core > 2.17.0 > > > Thanks & Regards > Swati > > -Original Message- > From: Gupta, Swati [mailto:swati.gu...@anz.com] > Sent: Friday, 6 January 2017 4:01 PM > To: users@kafka.apache.org > Subjec

RE: Apache Kafka integration using Apache Camel

org.apache.camel camel-core 2.17.0 Thanks & Regards Swati -Original Message- From: Gupta, Swati [mailto:swati.gu...@anz.com] Sent: Friday, 6 January 2017 4:01 PM To: users@kafka.apache.org Subject: RE: Apache Kafka integra

RE: Apache Kafka integration using Apache Camel

pic=test] PID_IS_UNDEFINED: INFO DefaultCamelContext - Total 1 routes, of which 1 are started. PID_IS_UNDEFINED: INFO DefaultCamelContext -Original Message- From: Ewen Cheslack-Postava [mailto:e...@confluent.io] Sent: Friday, 6 January 2017 3:58 PM To: users@kafka.apache.org Subject: Re

Re: Apache Kafka integration using Apache Camel

More generally, do you have any log errors/messages or additional info? It's tough to debug issues like this from 3rd party libraries if they don't provide logs/exception info that indicates why processing a specific message failed. -Ewen On Thu, Jan 5, 2017 at 8:29 PM, UMESH CHAUDHARY wrote: >

Re: Apache Kafka integration using Apache Camel

Did you test that kafka console consumer is displaying the produced message? On Fri, Jan 6, 2017 at 9:18 AM, Gupta, Swati wrote: > Hello All, > > > > I am trying to create a Consumer using Apache Camel for a topic in Apache > Kafka. > I am using Camel 2.17.0 and Kafka 0.10 and JDK 1.8. > I have

Re: Apache Kafka Meetup today, at San Jose Convention Center

Unfortunately it will not be recorded this time, but we will try to upload the slides after the session. And for those who can make it, you are NOT required a Strata pass to attend! Guozhang On Tue, Mar 29, 2016 at 9:04 AM, Marina wrote: > Sounds like a great Meetup! Unfortunately, not all of

Re: Apache Kafka Meetup today, at San Jose Convention Center

Sounds like a great Meetup! Unfortunately, not all of us are lucky enough to be in CA :) - any chance this Meetup will be recorded? thanks! From: Guozhang Wang To: "d...@kafka.apache.org" ; "users@kafka.apache.org" Sent: Tuesday, March 29, 2016 12:00 PM Subject: Apache Kafka Meetup t

Re: Apache Kafka Case Studies

Talk I gave about Kafka at the Wikimedia Foundation at Kafka NYC Meetup in 2014. https://www.hakkalabs.co/articles/apache-kafka-wikimedia On Wed, Feb 3, 2016 at 1:56 PM, Joe San wrote: > The OReilly online training seems to be interesting! Is there anything else > that is similiar to this that

Re: Apache Kafka Case Studies

The OReilly online training seems to be interesting! Is there anything else that is similiar to this that talks more of a real time experience from any organisation just like the one at LinkedIn? On Wed, Feb 3, 2016 at 6:10 PM, Todd Palino wrote: > To follow up on the blog post of mine that Jens

Re: Apache Kafka Case Studies

To follow up on the blog post of mine that Jens pasted, I also have a number of talk presentations up at http://www.slideshare.net/toddpalino If you download the actual presentation files, you’ll be able to access the slide notes in most of those that have a lot more information. On Wed, Feb 3, 2

Re: Apache Kafka Case Studies

Hi Joe, This might be interesting: https://engineering.linkedin.com/kafka/running-kafka-scale Cheers, Jens On Wed, Feb 3, 2016 at 4:15 PM, Joe San wrote: > Dear Kafka users, > > I'm looking for some case studies around using Kafka on big projects. > Specifically, I'm looking for some architect

Re: Apache Kafka Case Studies

Hello Joe, Consider registering for http://www.oreilly.com/online-training/building-distributed-pipelines-for-data-science.html and/or for http://www.kafka-summit.org/ Just as an idea where you could get ideas :) Btw, there's no such thing as best practice - only practice/approach that worked in p

Re: Apache Kafka Topic Design

Hi again Joe, I would suggest keeping things simple and using a single topic and a single partition with replication in the beginning if possible. You can split your topic up into multiple topics later if needed. You can split up your topic into more partitions later if needed. If you need strict

Re: Apache Kafka Topic Design

Hi Jens, Thanks for your suggestion. So here is a bit more text: I have a List of signals for a specific device: Map[DeviceId, List[Signal]], where Signal is case class Signal(name: String, value: String) Now I will write a producer to connect to the device and read these signals at constant in

Re: Apache Kafka Topic Design

Hi Joe, I think you are leaving out all your requirements to be able to get decent answer from anyone here. On Tue, Jan 19, 2016 at 8:47 AM, Joe San wrote: > I soon realized that this is a doomed approach. > Could you elaborate? Why would it be doomed? > Is there a better way to model my top

Re: Apache Kafka 0.8.2 Consumer Example

I'm guessing the upgrade changed your broker configuration file (server.properties). Perhaps take a look and see if things like max.message.bytes are still where you want them? Gwen On Sun, Feb 8, 2015 at 11:24 AM, Ricardo Ferreira < jricardoferre...@gmail.com> wrote: > Hi Gwen, > > Sorry, both

Re: Apache Kafka 0.8.2 Consumer Example

High level consumer of 0.8.1 works fine with 0.8.2. In extra, you can change the config to use kafka for offsets storage instead of zookeeper. There are some extra config parameters added as well as explained in the wiki. http://kafka.apache.org/documentation.html#consumerconfigs For low-level con

Re: Apache Kafka 0.8.2 Consumer Example

Ricardo, What does your consumer code look like? If it's too big to have that code inline in this mail, you can perhaps put it in a github repo or even a gist. -Jaikiran On Monday 09 February 2015 12:54 AM, Ricardo Ferreira wrote: Hi Gwen, Sorry, both the consumer and the broker are 0.8.2?

Re: Apache Kafka 0.8.2 Consumer Example

Hi Gwen, Sorry, both the consumer and the broker are 0.8.2? A = Yes So what's on 0.8.1? A = It works fine using 0.8.1 for server AND client. You probably know the consumer group of your application. Can you use the offset checker tool on that? A = Yes, I know from the consumer, and the offset ch

Re: Apache Kafka 0.8.2 Consumer Example

Sorry, both the consumer and the broker are 0.8.2? So what's on 0.8.1? I seriously doubt downgrading is the solution. You probably know the consumer group of your application. Can you use the offset checker tool on that? Gwen On Sun, Feb 8, 2015 at 9:01 AM, Ricardo Ferreira wrote: > Hi Gwen,

Re: Apache Kafka 0.8.2 Consumer Example

Hi Gwen, Thanks for the response. In my case, I have both consumer application and the server versions in 0.8.2, Scala 2.10. No errors are thrown, and my *zookeeper.session.timeout.ms * property is set to 500, although I tried 5000 and also didn't worked. I

Re: Apache Kafka 0.8.2 Consumer Example

I have a 0.8.1 high level consumer working fine with 0.8.2 server. Few of them actually :) AFAIK the API did not change. Do you see any error messages? Do you have timeout configured on the consumer? What does the offset checker tool say? On Fri, Feb 6, 2015 at 4:49 PM, Ricardo Ferreira wrote:

RE: Apache Kafka Consumers in java 8

How many topics do you have and can you share message throughput ?. What's the functionality you are using for ?. -Original Message- From: Michael Nussbaum [mailto:michaelnussbau...@gmail.com] Sent: Wednesday, October 29, 2014 1:57 PM To: users Subject: Re: Apache Kafka Consume

Re: Apache Kafka Consumers in java 8

; > Did you experience any performance or bottleneck ?. > > -Original Message- > From: Michael Nussbaum [mailto:michaelnussbau...@gmail.com] > Sent: Wednesday, October 29, 2014 1:39 PM > To: users@kafka.apache.org > Subject: Re: Apache Kafka Consumers in java 8 > > I'

RE: Apache Kafka Consumers in java 8

Are you using it in production ?. Did you experience any performance or bottleneck ?. -Original Message- From: Michael Nussbaum [mailto:michaelnussbau...@gmail.com] Sent: Wednesday, October 29, 2014 1:39 PM To: users@kafka.apache.org Subject: Re: Apache Kafka Consumers in java 8 I&#x

Re: Apache Kafka Consumers in java 8

I've been using clojure on open jdk 8 for my producers and consumers for about a month now without any issue. Anything specific you're interested in? Cheers, Michael Nussbaum On Wed, Oct 29, 2014 at 2:32 PM, Seshadri, Balaji wrote: > Have anybody used Kafka with Java 8 ?. > > -Original Mess

RE: Apache Kafka Consumers in java 8

Have anybody used Kafka with Java 8 ?. -Original Message- From: Seshadri, Balaji [mailto:balaji.sesha...@dish.com] Sent: Wednesday, October 29, 2014 11:11 AM To: 'd...@kafka.apache.org'; 'users@kafka.apache.org' Subject: Apache Kafka Consumers in java 8 Hi All, Can you please share your

  1   2   >