Back-pressure to Spark Kafka Streaming?

Hey all, Does backressure actually work on spark kafka streaming? According to the latest spark streaming document: *http://spark.apache.org/docs/latest/streaming-programming-guide.html * "*In Spark 1.5, we have introduced a fea

Package org.apache.spark.annotation no longer exist in Spark 2.0?

I just upgrade from Spark 1.6.1 to 2.0, and got an java compile error: *error: cannot access DeveloperApi* * class file for org.apache.spark.annotation.DeveloperApi not found* >From the Spark 2.0 document ( https://spark.apache.org/docs/2.0.0/api/java/overview-summary.html), the package org.apac

Re: Dynamic Allocation & Spark Streaming

It's has been a while since last update on the thread. Now Spark 2.0 is available, do you guys know if there's any progress on Dynamic Allocation & Spark Streaming? On Mon, Oct 19, 2015 at 1:13 PM, robert towne wrote: > I have watched a few videos from Databricks/Andrew Or around the Spark 1.2 >