will try scala.
Only Reason of not using scala is - never worked on that.
On Wed, Aug 19, 2015 at 7:34 PM, Cody Koeninger wrote:
> Is there a reason not to just use scala? It's not a lot of code... and
> it'll be even less code in scala ;)
>
> On Wed, Aug 19, 2015 at 4:14 AM, Shushant Arora >
Is there a reason not to just use scala? It's not a lot of code... and
it'll be even less code in scala ;)
On Wed, Aug 19, 2015 at 4:14 AM, Shushant Arora
wrote:
> To correct myself - KafkaRDD[K, V, U, T, R] is subclass of RDD[R] but
> Option is not subclass of Option;
>
> In scala C[T’] is a
To correct myself - KafkaRDD[K, V, U, T, R] is subclass of RDD[R] but
Option is not subclass of Option;
In scala C[T’] is a subclass of C[T] as per
https://twitter.github.io/scala_school/type-basics.html
but this is not allowed in java.
So is there any workaround to achieve this in java for over
But KafkaRDD[K, V, U, T, R] is not subclass of RDD[R] as per java generic
inheritance is not supported so derived class cannot return different
genric typed subclass from overriden method.
On Wed, Aug 19, 2015 at 12:02 AM, Cody Koeninger wrote:
> Option is covariant and KafkaRDD is a subclass o
Option is covariant and KafkaRDD is a subclass of RDD
On Tue, Aug 18, 2015 at 1:12 PM, Shushant Arora
wrote:
> Is it that in scala its allowed for derived class to have any return type ?
>
> And streaming jar is originally created in scala so its allowed for
> DirectKafkaInputDStream to return
Is it that in scala its allowed for derived class to have any return type ?
And streaming jar is originally created in scala so its allowed for
DirectKafkaInputDStream to return Option[KafkaRDD[K, V, U, T, R]]
compute method ?
On Tue, Aug 18, 2015 at 8:36 PM, Shushant Arora
wrote:
> looking a
looking at source code of
org.apache.spark.streaming.kafka.DirectKafkaInputDStream
override def compute(validTime: Time): Option[KafkaRDD[K, V, U, T, R]] = {
val untilOffsets = clamp(latestLeaderOffsets(maxRetries))
val rdd = KafkaRDD[K, V, U, T, R](
context.sparkContext, kafkaParams
The superclass method in DStream is defined as returning an Option[RDD[T]]
On Tue, Aug 18, 2015 at 8:07 AM, Shushant Arora
wrote:
> Getting compilation error while overriding compute method of
> DirectKafkaInputDStream.
>
>
> [ERROR] CustomDirectKafkaInputDstream.java:[51,83]
> compute(org.apach
Getting compilation error while overriding compute method of
DirectKafkaInputDStream.
[ERROR] CustomDirectKafkaInputDstream.java:[51,83]
compute(org.apache.spark.streaming.Time) in CustomDirectKafkaInputDstream
cannot override compute(org.apache.spark.streaming.Time) in
org.apache.spark.streaming
Look at the definitions of the java-specific KafkaUtils.createDirectStream
methods (the ones that take a JavaStreamingContext)
On Mon, Aug 17, 2015 at 5:13 AM, Shushant Arora
wrote:
> How to create classtag in java ?Also Constructor
> of DirectKafkaInputDStream takes Function1 not Function but
>
How to create classtag in java ?Also Constructor of DirectKafkaInputDStream
takes Function1 not Function but kafkautils.createDirectStream allows
function.
I have below as overriden DirectKafkaInputDStream.
public class CustomDirectKafkaInputDstream extends
DirectKafkaInputDStream{
public Custo
11 matches
Mail list logo
