Hi, In an attempt to keep processing logic as simple as possible, I'm trying to use spark streaming for processing historic as well as real-time data. This works quite well, using big intervals that match the window size for historic data, and small intervals for real-time.
I found this discussion on a (very) similar situation: https://groups.google.com/forum/#!topic/spark-users/ES8X1l_xn5s To be able to do this historic data processing, I need access to the ManualClock that's used by the job generator. That's unfortunately only accessible with reflection. Since I couldn't find JIRA or GitHub issues, I wonder if there's perhaps a better alternative solution, or if it would be worthwhile improving support for manual time management? cheers, Frank -- Amsterdam - Oosteinde 11, 1017 WT Amsterdam Boston - 1 Broadway, Cambridge, MA 02142 US +1 877 414 4776 (toll free) Europe +31(0)20 522 4466 www.onehippo.com
