arn, logs are aggregated from each containers to hdfs. You can use
> yarn CLI or ui to view. For spark, you would have a history server which
> consolidate s the logs
> On 21 Sep 2016 19:03, "Nisha Menon" wrote:
>
>> I looked at the driver logs, that reminded me that I ne
e driver?
>
>
> On Thursday, September 8, 2016, Nisha Menon
> wrote:
>
>> I have an RDD created as follows:
>>
>> *JavaPairRDD inputDataFiles =
>> sparkContext.wholeTextFiles("hdfs://ip:8020/user/cdhuser/inputFolder/");*
>>
>> On this RDD I
I have an RDD created as follows:
*JavaPairRDD inputDataFiles =
sparkContext.wholeTextFiles("hdfs://ip:8020/user/cdhuser/inputFolder/");*
On this RDD I perform a map to process individual files and invoke a
foreach to trigger the same map.
* JavaRDD output = inputDataFiles.map(new
Functio
