Hi All, I am using pyspark streaming to ETL raw data files as they land on HDFS. While researching this topic I found this great presentation about Spark and Spark Streaming at Uber (http://www.slideshare.net/databricks/spark-meetup-at-uber), where they mention this INotifyDStream that sounds very interesting and like it may suit my use case well.
Does anyone know if this code has been submitted to apache, or how I might otherwise come upon it? Reference: https://issues.apache.org/jira/browse/SPARK-10555 - Add INotifyDStream to Spark Streaming Thanks! --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
