Hi Bill,
Hope the following is what you need.
val zerotime = System.currentTimeMillis()
Then in foreach do the following
//difference = RDDtimeparameter - zerotime //only to find the constant value to
be used later
starttime = (RDDtimeparameter - (zerotime + difference)) - intervalsize
endt
Hi Tathagata,
On Thu, Jul 17, 2014 at 6:12 PM, Tathagata Das
wrote:
> The RDD parameter in foreachRDD contains raw/transformed data from the
> last batch. So when forearchRDD is called with the time parameter as 5:02:01
> and batch size is 1 minute, then the rdd will contain data based on the
The RDD parameter in foreachRDD contains raw/transformed data from the last
batch. So when forearchRDD is called with the time parameter as 5:02:01 and
batch size is 1 minute, then the rdd will contain data based on the data
received by between 5:02:00 and 5:02:01.
If you want to do custom interva
Hi Tathagata，
Thanks for your answer. Please see my further question below:
On Wed, Jul 16, 2014 at 6:57 PM, Tathagata Das
wrote:
> Answers inline.
>
>
> On Wed, Jul 16, 2014 at 5:39 PM, Bill Jay
> wrote:
>
>> Hi all,
>>
>> I am currently using Spark Streaming to conduct a real-time data
>> a
Answers inline.
On Wed, Jul 16, 2014 at 5:39 PM, Bill Jay
wrote:
> Hi all,
>
> I am currently using Spark Streaming to conduct a real-time data
> analytics. We receive data from Kafka. We want to generate output files
> that contain results that are based on the data we receive from a specific
Hi all,
I am currently using Spark Streaming to conduct a real-time data analytics.
We receive data from Kafka. We want to generate output files that contain
results that are based on the data we receive from a specific time
interval.
I have several questions on Spark Streaming's timestamp:
1) I
