We are definitely investigating a Python API for Streaming, but no announced 
deadline at this point.

Matei

On Jun 4, 2014, at 5:02 PM, John Omernik <j...@omernik.com> wrote:

> So Python is used in many of the Spark Ecosystem products, but not Streaming 
> at this point. Is there a roadmap to include Python APIs in Spark Streaming? 
> Anytime frame on this? 
> 
> Thanks!
> 
> John
> 
> 
> On Thu, May 29, 2014 at 4:19 PM, Matei Zaharia <matei.zaha...@gmail.com> 
> wrote:
> Quite a few people ask this question and the answer is pretty simple. When we 
> started Spark, we had two goals — we wanted to work with the Hadoop 
> ecosystem, which is JVM-based, and we wanted a concise programming interface 
> similar to Microsoft’s DryadLINQ (the first language-integrated big data 
> framework I know of, that begat things like FlumeJava and Crunch). On the 
> JVM, the only language that would offer that kind of API was Scala, due to 
> its ability to capture functions and ship them across the network. Scala’s 
> static typing also made it much easier to control performance compared to, 
> say, Jython or Groovy.
> 
> In terms of usage, however, we see substantial usage of our other languages 
> (Java and Python), and we’re continuing to invest in both. In a user survey 
> we did last fall, about 25% of users used Java and 30% used Python, and I 
> imagine these numbers are growing. With lambda expressions now added to Java 
> 8 (http://databricks.com/blog/2014/04/14/Spark-with-Java-8.html), I think 
> we’ll see a lot more Java. And at Databricks I’ve seen a lot of interest in 
> Python, which is very exciting to us in terms of ease of use.
> 
> Matei
> 
> On May 29, 2014, at 1:57 PM, Benjamin Black <b...@b3k.us> wrote:
> 
>> HN is a cesspool safely ignored.
>> 
>> 
>> On Thu, May 29, 2014 at 1:55 PM, Nick Chammas <nicholas.cham...@gmail.com> 
>> wrote:
>> I recently discovered Hacker News and started reading through older posts 
>> about Scala. It looks like the language is fairly controversial on there, 
>> and it got me thinking.
>> 
>> Scala appears to be the preferred language to work with in Spark, and Spark 
>> itself is written in Scala, right?
>> 
>> I know that often times a successful project evolves gradually out of 
>> something small, and that the choice of programming language may not always 
>> have been made consciously at the outset.
>> 
>> But pretending that it was, why is Scala the preferred language of Spark?
>> 
>> Nick
>> 
>> 
>> View this message in context: Why Scala?
>> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>> 
> 
> 

Reply via email to