We are looking to incorporate Spark into a timeseries data investigation application, but we are having a hard time transforming our workflow into the required transformations-on-data model. The crux of the problem is that we don’t know a priori which data will be required for our transformations. For example, a common request might be `average($series2.within($ranges))`, where in order to fetch the right sections of data from $series2, $ranges will need to be computed first and then used to define data boundaries. Is there a way to get around the need to define data first in Spark?
-- View this message in context: http://apache-spark-developers-list.1001551.n3.nabble.com/Using-Spark-when-data-definitions-are-unknowable-at-compile-time-tp17371.html Sent from the Apache Spark Developers List mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
