We are definitely investigating a Python API for Streaming, but no announced deadline at this point.
Matei On Jun 4, 2014, at 5:02 PM, John Omernik <j...@omernik.com> wrote: > So Python is used in many of the Spark Ecosystem products, but not Streaming > at this point. Is there a roadmap to include Python APIs in Spark Streaming? > Anytime frame on this? > > Thanks! > > John > > > On Thu, May 29, 2014 at 4:19 PM, Matei Zaharia <matei.zaha...@gmail.com> > wrote: > Quite a few people ask this question and the answer is pretty simple. When we > started Spark, we had two goals — we wanted to work with the Hadoop > ecosystem, which is JVM-based, and we wanted a concise programming interface > similar to Microsoft’s DryadLINQ (the first language-integrated big data > framework I know of, that begat things like FlumeJava and Crunch). On the > JVM, the only language that would offer that kind of API was Scala, due to > its ability to capture functions and ship them across the network. Scala’s > static typing also made it much easier to control performance compared to, > say, Jython or Groovy. > > In terms of usage, however, we see substantial usage of our other languages > (Java and Python), and we’re continuing to invest in both. In a user survey > we did last fall, about 25% of users used Java and 30% used Python, and I > imagine these numbers are growing. With lambda expressions now added to Java > 8 (http://databricks.com/blog/2014/04/14/Spark-with-Java-8.html), I think > we’ll see a lot more Java. And at Databricks I’ve seen a lot of interest in > Python, which is very exciting to us in terms of ease of use. > > Matei > > On May 29, 2014, at 1:57 PM, Benjamin Black <b...@b3k.us> wrote: > >> HN is a cesspool safely ignored. >> >> >> On Thu, May 29, 2014 at 1:55 PM, Nick Chammas <nicholas.cham...@gmail.com> >> wrote: >> I recently discovered Hacker News and started reading through older posts >> about Scala. It looks like the language is fairly controversial on there, >> and it got me thinking. >> >> Scala appears to be the preferred language to work with in Spark, and Spark >> itself is written in Scala, right? >> >> I know that often times a successful project evolves gradually out of >> something small, and that the choice of programming language may not always >> have been made consciously at the outset. >> >> But pretending that it was, why is Scala the preferred language of Spark? >> >> Nick >> >> >> View this message in context: Why Scala? >> Sent from the Apache Spark User List mailing list archive at Nabble.com. >> > >
