So Python is used in many of the Spark Ecosystem products, but not
Streaming at this point. Is there a roadmap to include Python APIs in Spark
Streaming? Anytime frame on this?

Thanks!

John


On Thu, May 29, 2014 at 4:19 PM, Matei Zaharia <matei.zaha...@gmail.com>
wrote:

> Quite a few people ask this question and the answer is pretty simple. When
> we started Spark, we had two goals — we wanted to work with the Hadoop
> ecosystem, which is JVM-based, and we wanted a concise programming
> interface similar to Microsoft’s DryadLINQ (the first language-integrated
> big data framework I know of, that begat things like FlumeJava and Crunch).
> On the JVM, the only language that would offer that kind of API was Scala,
> due to its ability to capture functions and ship them across the network.
> Scala’s static typing also made it much easier to control performance
> compared to, say, Jython or Groovy.
>
> In terms of usage, however, we see substantial usage of our other
> languages (Java and Python), and we’re continuing to invest in both. In a
> user survey we did last fall, about 25% of users used Java and 30% used
> Python, and I imagine these numbers are growing. With lambda expressions
> now added to Java 8 (
> http://databricks.com/blog/2014/04/14/Spark-with-Java-8.html), I think
> we’ll see a lot more Java. And at Databricks I’ve seen a lot of interest in
> Python, which is very exciting to us in terms of ease of use.
>
> Matei
>
> On May 29, 2014, at 1:57 PM, Benjamin Black <b...@b3k.us> wrote:
>
> HN is a cesspool safely ignored.
>
>
> On Thu, May 29, 2014 at 1:55 PM, Nick Chammas <nicholas.cham...@gmail.com>
> wrote:
>
>> I recently discovered Hacker News and started reading through older
>> posts about Scala
>> <https://hn.algolia.com/?q=scala#!/story/forever/0/scala>. It looks like
>> the language is fairly controversial on there, and it got me thinking.
>>
>> Scala appears to be the preferred language to work with in Spark, and
>> Spark itself is written in Scala, right?
>>
>> I know that often times a successful project evolves gradually out of
>> something small, and that the choice of programming language may not always
>> have been made consciously at the outset.
>>
>> But pretending that it was, why is Scala the preferred language of Spark?
>>
>> Nick
>>
>>
>> ------------------------------
>> View this message in context: Why Scala?
>> <http://apache-spark-user-list.1001560.n3.nabble.com/Why-Scala-tp6536.html>
>> Sent from the Apache Spark User List mailing list archive
>> <http://apache-spark-user-list.1001560.n3.nabble.com/> at Nabble.com.
>>
>
>
>

Reply via email to