Move your count operation outside the foreach and use a broadcast to access
it inside the foreach.
On Aug 17, 2015 10:34 AM, "Priya Ch" wrote:
> Looks like because of Spark-5063
> RDD transformations and actions can only be invoked by the driver, not
> inside of other transformations; for example
Looks like because of Spark-5063
RDD transformations and actions can only be invoked by the driver, not
inside of other transformations; for example, rdd1.map(x =>
rdd2.values.count() * x) is invalid because the values transformation and
count action cannot be performed inside of the rdd1.map trans
The error could be because of the missing brackets after the word cache -
.ticketRdd.cache()
> On Aug 17, 2015, at 7:26 AM, Priya Ch wrote:
>
> Hi All,
>
> Thank you very much for the detailed explanation.
>
> I have scenario like this-
> I have rdd of ticket records and another rdd of book
Hi All,
Thank you very much for the detailed explanation.
I have scenario like this-
I have rdd of ticket records and another rdd of booking records. for each
ticket record, i need to check whether any link exists in booking table.
val ticketCachedRdd = ticketRdd.cache
ticketRdd.foreach{
ticke
