spark kafka batch integration

hello all, we at tresata wrote a library to provide for batch integration between spark and kafka (distributed write of rdd to kafa, distributed read of rdd from kafka). our main use cases are (in lambda architecture jargon): * period appends to the immutable master dataset on hdfs from kafka using

Re: Spark JIRA Report

Taking after Andrew’s suggestion, perhaps the report can just focus on Stale issues (no updates in > 90 days), since those are probably the easiest to act on. For example: Stale Issues

Re: jenkins downtime: 730-930am, 12/12/14

josh rosen has this PR open to address the streaming test failures: https://github.com/apache/spark/pull/3687 On Sun, Dec 14, 2014 at 8:21 AM, WangTaoTheTonic wrote: > Jenkins is still not available now as some unit tests(about streaming) > failed > all the time. Does it have something to do wi

Re: Is there any document to explain how to build the hive jars for spark?

The modified version of hive can be found here: https://github.com/pwendell/hive On Thu, Dec 11, 2014 at 5:47 PM, Yi Tian wrote: > > Hi, all > > We found some bugs in hive-0.12, but we could not wait for hive community > fixing them. > > We want to fix these bugs in our lab and build a new releas

Re: jenkins downtime: 730-930am, 12/12/14

Jenkins is still not available now as some unit tests(about streaming) failed all the time. Does it have something to do with this update? -- View this message in context: http://apache-spark-developers-list.1001551.n3.nabble.com/jenkins-downtime-730-930am-12-12-14-tp9583p9778.html Sent from th

Re: Spark JIRA Report

I formatted this report using Markdown; I'm open to changing the structure or formatting or reducing the amount of information to make the report more easily consumable. Regarding just sending links or whether this would just be mailing list noise, those are a good questions. I've sent out links