Zepplin 0.7.3 DEBUG console output, how do I switch it off!!! Spark 2.2.0-bin-hadoop2.7

2017-10-12 Thread amersh...@gmail.com
Hello there

I have installed Spark 2.20 and from the spark-shell, my scala code works 
perfectly and performs well.

The same scala executed from Zeppelin causes lots of DEBUG statements to be 
printed in the console (almost non-stop) and the performance is very slow.

I have just installed binaries and haven't changed any configurations, but it 
seems Zeppelin is running in debug mode...

Help..

I use bin\zeppelin.cmd to start zeppelin. I'm on Windows Server 2012

Please help...




Re: Zepplin 0.7.3 DEBUG console output, how do I switch it off!!! Spark 2.2.0-bin-hadoop2.7

2017-10-12 Thread amersh...@gmail.com


On 2017-10-12 10:41, "amersh...@gmail.com" wrote: 
> Hello there
> 
> I have installed Spark 2.20 and from the spark-shell, my scala code works 
> perfectly and performs well.
> 
> The same scala executed from Zeppelin causes lots of DEBUG statements to be 
> printed in the console (almost non-stop) and the performance is very slow.
> 
> I have just installed binaries and haven't changed any configurations, but it 
> seems Zeppelin is running in debug mode...
> 
> Help..
> 
> I use bin\zeppelin.cmd to start zeppelin. I'm on Windows Server 2012
> 
> Please help...
> 
> 
> PS

The scala works perfectly within Zeppelin... Just very slow performance and 
lots of Debug output...


Re: Zepplin 0.7.3 DEBUG console output, how do I switch it off!!! Spark 2.2.0-bin-hadoop2.7

2017-10-16 Thread amersh...@gmail.com


On 2017-10-14 06:48, Jeff Zhang  wrote: 
> I just suspect it is due to different spark configuration in zeppelin spark
> interpreter and spark-shell. Please make sure they are using the same
> configuration.
> 
> 
> Amer Sheikh 于2017年10月14日周六 
> 下午1:44写道:
> 
> > Hi Jeff
> >
> > Thanks for your reply. The zeppelin console output is continuous when my
> > scala code (spark sql) is running.
> > The same code runs very from a separate spark-shell, so im suspecting that
> > Zeppelin is executing the code in DEBUG mode.
> >
> > When I installed Zeppelin, I did not change any settings.  My main concern
> > is the very slow performance.
> >
> > Kind Regards
> >
> > Amer
> >
> > On Sat, Oct 14, 2017 at 2:16 AM, Jianfeng (Jeff) Zhang <
> > jzh...@hortonworks.com> wrote:
> >
> >>
> >> These are zeppelin server log which is controlled by log4.properties
> >> under ZEPPELIN_CONF_DIR. And I don’t understand why would you say these
> >> log would affect the performance.
> >>
> >> Could you explain more ?
> >>
> >>
> >>
> >>
> >> Best Regard,
> >> Jeff Zhang
> >>
> >>
> >> From: Amer Sheikh 
> >> Reply-To: "users@zeppelin.apache.org" 
> >> Date: Saturday, October 14, 2017 at 12:15 AM
> >> To: "users@zeppelin.apache.org" 
> >> Subject: Re: Zepplin 0.7.3 DEBUG console output, how do I switch it
> >> off!!! Spark 2.2.0-bin-hadoop2.7
> >>
> >> Hi there
> >>
> >> Please see the console output
> >>
> >>
> >>
> >> On Thu, Oct 12, 2017 at 1:25 PM, Jeff Zhang  wrote:
> >>
> >>>
> >>> What kind of DEBUG log you see ? Can you paste it ?
> >>>
> >>>
> >>> amersh...@gmail.com 于2017年10月12日周四 
> >>> 下午5:44写道:
> >>>
> >>>>
> >>>>
> >>>> On 2017-10-12 10:41, "amersh...@gmail.com" wrote:
> >>>> > Hello there
> >>>> >
> >>>> > I have installed Spark 2.20 and from the spark-shell, my scala code
> >>>> works perfectly and performs well.
> >>>> >
> >>>> > The same scala executed from Zeppelin causes lots of DEBUG statements
> >>>> to be printed in the console (almost non-stop) and the performance is 
> >>>> very
> >>>> slow.
> >>>> >
> >>>> > I have just installed binaries and haven't changed any
> >>>> configurations, but it seems Zeppelin is running in debug mode...
> >>>> >
> >>>> > Help..
> >>>> >
> >>>> > I use bin\zeppelin.cmd to start zeppelin. I'm on Windows Server 2012
> >>>> >
> >>>> > Please help...
> >>>> >
> >>>> >
> >>>> > PS
> >>>>
> >>>> The scala works perfectly within Zeppelin... Just very slow performance
> >>>> and lots of Debug output...
> >>>>
> >>>
> >>
> >
>  Hi there. Since the same spark sql is 3 times SLOWER on ZEPPELIN, how do i 
> ensure that a query through ZEPPELIN uses the same as that in SPARK-SHELL. 
> Please help


PLEASE HELP: Spark query through Zeppelin 0.7.3 runs so much slower than the same query through spark-shell

2017-10-16 Thread amersh...@gmail.com
Hello there

This questions relates to a previous query. Im filtering a single row from a 5 
million row dataframe.

Through Zeppelin the query takes 7minutes.

Through spark-shell, it takes 1.5minutes.

I need help with this to see why the Zeppelin query is taking a much slower 
path.

Please help

Amer