RE: Spark and Kafka integration

Cool! Thanks for your inputs Jacek and Mark! From: Mark Hamstra [mailto:m...@clearstorydata.com] Sent: 13 January 2017 12:59 To: Phadnis, Varun Cc: user@spark.apache.org Subject: Re: Spark and Kafka integration See "API compatibility" in http://spark.apache.org/versioning-policy.h

Re: Spark and Kafka integration

See "API compatibility" in http://spark.apache.org/versioning-policy.html While code that is annotated as Experimental is still a good faith effort to provide a stable and useful API, the fact is that we're not yet confident enough that we've got the public API in exactly the form that we want to

Re: Spark and Kafka integration

Hi Phadnis, I found this in http://spark.apache.org/docs/latest/streaming-kafka-0-10-integration.html: > This version of the integration is marked as experimental, so the API is > potentially subject to change. Pozdrawiam, Jacek Laskowski https://medium.com/@jaceklaskowski/ Mastering Apach

Re: Spark and Kafka direct approach problem

This works spark 1.61, using Scala version 2.10.5 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_77) Kafka version 0.9.0.1 using scala-library-2.11.7.jar Dr Mich Talebzadeh LinkedIn * https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw

Re: Spark and Kafka direct approach problem

It's because the Scala version of Spark and the Scala version of Kafka don't match. Please check them. On Wed, May 4, 2016 at 6:17 AM, أنس الليثي wrote: > NoSuchMethodError usually appears because of a difference in the library > versions. > > Check the version of the libraries you downloaded, t

Re: Spark and Kafka direct approach problem

NoSuchMethodError usually appears because of a difference in the library versions. Check the version of the libraries you downloaded, the version of spark, the version of Kafka. On 4 May 2016 at 16:18, Luca Ferrari wrote: > Hi, > > I’m new on Apache Spark and I’m trying to run the Spark Streami

RE: Spark and Kafka Integration

For Q2. The order of the logs in each partition is guaranteed but there cannot be any such thing as global order. From: Prashant Bhardwaj [mailto:prashant2006s...@gmail.com] Sent: Monday, December 07, 2015 5:46 PM To: user@spark.apache.org Subject: Spark and Kafka Integration Hi Some Background

Re: Spark and Kafka

This is my window: reduceByKeyAndWindow( new Function2() { @Override public Integer call(Integer i1, Integer i2) { return i1 + i2; } }, new Function2() { public Integer call(Integer i1, Integer i2) { return i1 - i2; } }, new Duration(60