Hi, I am running streaming application reading data from kafka and performing window operations on it. I have a usecase where all incoming events have a fixed latency of 10s, which means data belonging to minute 10:00:00 will arrive 10s late at 10:00:10.
I want to set the spark clock to "Manualclock" and set the time behind by 10s so that the batch calculation triggers at 10:00:10, during which time all the events for the previous minute has arrived. But, I see that "spark.streaming.clock" is hardcoded to "org.apache.spark.util.SystemClock" in the code. Is there a way to easily hack this property to use Manual clock. -- Regards Hemalatha
