Just a correction - what the strange symbol which was auto generated is
for “. . . . “
Thanks again,
Ranjan
On 3/9/14, 11:33 PM, "Sen, Ranjan [USA]" <sen_ran...@bah.com> wrote:
>Hi Patrick
>
>How do I know which part of the code is in the driver and which in task?
>The structure of my code is as below-
>
>Š
>
>Static boolean done=false;
>Š
>
>Public static void main(..
>
>..
>
>JavaRDD<String> lines = ..
>
>..
>
>While (!done) {
>
>..
>While (..) {
>
>JavaPairRDD<Integer, List<Integer>> labs1 = labs.map (new PairFunction<Š
>);
>
>!! Here I have System.out.println (A)
>
>} // inner while
>
>!! Here I have System.out.println (B)
>
>
>If (Š) {
> Done = true;
>
> !! Also here some System.out.println (C)
>
> Break;
>}
>
>Else {
>
> If (Š) {
>
> !! More System.out.println (D)
>
>
> labs = labs.map(Š) ;
>
> }
> }
>
>} // outer while
>
>!! Even more System.out.println (E)
>
>} // main
>
>} //class
>
>I get the console outputs on the master for (B) and (E). I do not see any
>stdout in the worker node. I find the stdout and stderr in the
><spark>/work/<appid>/0/. I see output
>in stderr but not in stdout.
>
>I do get all the outputs on the console when I run it in local mode.
>
>Sorry I am new and may be asking some naïve question but it is really
>confusing to me. Thanks for your help.
>
>Ranjan
>
>On 3/9/14, 10:50 PM, "Patrick Wendell" <pwend...@gmail.com> wrote:
>
>>Hey Sen,
>>
>>Is your code in the driver code or inside one of the tasks?
>>
>>If it's in the tasks, the place you would expect these to be is in
>>stdout file under <spark>/<appid>/work/[stdout/stderr]. Are you seeing
>>at least stderr logs in that folder? If not then the tasks might not
>>be running on the workers machines. If you see stderr but not stdout
>>that's a bit of a puzzler since they both go through the same
>>mechanism.
>>
>>- Patrick
>>
>>On Sun, Mar 9, 2014 at 2:32 PM, Sen, Ranjan [USA] <sen_ran...@bah.com>
>>wrote:
>>> Hi
>>> I have some System.out.println in my Java code that is working ok in a
>>>local
>>> environment. But when I run the same code on a standalone mode in a
>>>EC2
>>> cluster I do not see them at the worker stdout (in the worker node
>>>under
>>> <spark location>/work ) or at the driver console. Could you help me
>>> understand how do I troubleshoot?
>>>
>>> Thanks
>>> Ranjan
>
