you can increase driver memory by setting spark.driver.memory
Jongyoul Lee <jongy...@gmail.com>于2018年8月18日周六 下午2:35写道： > Hi, > > 1. AKAIF, it’s a problem of spark-shell. Z’s spark interpreter uses > spark-shell internally. Thus it cannot be solved easily. > > 2. You could try ‘per user’ setting in your interpreter. > > 3. Currently, there’s no way to figure it out. > > On Tue, 14 Aug 2018 at 5:59 PM Chintan Patel <chintan.pa...@qdata.io> > wrote: > >> Hello, >> >> I'm running Zeppelin in yarn-client mode. I'm using SQL interpreter and >> pyspark interpreter to run some query and python jobs in shared mode per >> note. Sometimes when I run multiple jobs at same time. It's using lot's of >> CPU. I try to check the problem and I found that It's because of It creates >> spark driver for each notebook. >> >> >> My Question are >> 1. How I can tune Zeppelin to Handle large amount of concurrent jobs to >> fix "GC overhead limit exceeded" ? >> 2. How can I scale the zeppelin with number of users ? >> 3. If memory or CPU is not available, Is there any way to backlog the >> jobs ? >> >> Thanks & Regards >> Chintan >> > -- > 이종열, Jongyoul Lee, 李宗烈 > http://madeng.net >
