Where and how do i pass this or other JVM argument ? -XX:MaxPermSize=512m On Wed, Mar 25, 2015 at 11:36 PM, Zhan Zhang <zzh...@hortonworks.com> wrote:
> I solve this by increase the PermGen memory size in driver. > > -XX:MaxPermSize=512m > > Thanks. > > Zhan Zhang > > On Mar 25, 2015, at 10:54 AM, ÐΞ€ρ@Ҝ (๏̯͡๏) <deepuj...@gmail.com> wrote: > > I am facing same issue, posted a new thread. Please respond. > > On Wed, Jan 14, 2015 at 4:38 AM, Zhan Zhang <zzh...@hortonworks.com> > wrote: > >> Hi Folks, >> >> I am trying to run hive context in yarn-cluster mode, but met some error. >> Does anybody know what cause the issue. >> >> I use following cmd to build the distribution: >> >> ./make-distribution.sh -Phive -Phive-thriftserver -Pyarn -Phadoop-2.4 >> >> 15/01/13 17:59:42 INFO cluster.YarnClusterScheduler: >> YarnClusterScheduler.postStartHook done >> 15/01/13 17:59:42 INFO storage.BlockManagerMasterActor: Registering block >> manager cn122-10.l42scl.hortonworks.com:56157 with 1589.8 MB RAM, >> BlockManagerId(2, cn122-10.l42scl.hortonworks.com, 56157) >> 15/01/13 17:59:43 INFO parse.ParseDriver: Parsing command: CREATE TABLE >> IF NOT EXISTS src (key INT, value STRING) >> 15/01/13 17:59:43 INFO parse.ParseDriver: Parse Completed >> 15/01/13 17:59:44 INFO metastore.HiveMetaStore: 0: Opening raw store with >> implemenation class:org.apache.hadoop.hive.metastore.ObjectStore >> 15/01/13 17:59:44 INFO metastore.ObjectStore: ObjectStore, initialize >> called >> 15/01/13 17:59:44 INFO DataNucleus.Persistence: Property >> datanucleus.cache.level2 unknown - will be ignored >> 15/01/13 17:59:44 INFO DataNucleus.Persistence: Property >> hive.metastore.integral.jdo.pushdown unknown - will be ignored >> 15/01/13 17:59:44 WARN DataNucleus.Connection: BoneCP specified but not >> present in CLASSPATH (or one of dependencies) >> 15/01/13 17:59:44 WARN DataNucleus.Connection: BoneCP specified but not >> present in CLASSPATH (or one of dependencies) >> 15/01/13 17:59:52 INFO metastore.ObjectStore: Setting MetaStore object >> pin classes with >> hive.metastore.cache.pinobjtypes="Table,StorageDescriptor,SerDeInfo,Partition,Database,Type,FieldSchema,Order" >> 15/01/13 17:59:52 INFO metastore.MetaStoreDirectSql: MySQL check failed, >> assuming we are not on mysql: Lexical error at line 1, column 5. >> Encountered: "@" (64), after : "". >> 15/01/13 17:59:53 INFO DataNucleus.Datastore: The class >> "org.apache.hadoop.hive.metastore.model.MFieldSchema" is tagged as >> "embedded-only" so does not have its own datastore table. >> 15/01/13 17:59:53 INFO DataNucleus.Datastore: The class >> "org.apache.hadoop.hive.metastore.model.MOrder" is tagged as >> "embedded-only" so does not have its own datastore table. >> 15/01/13 17:59:59 INFO DataNucleus.Datastore: The class >> "org.apache.hadoop.hive.metastore.model.MFieldSchema" is tagged as >> "embedded-only" so does not have its own datastore table. >> 15/01/13 17:59:59 INFO DataNucleus.Datastore: The class >> "org.apache.hadoop.hive.metastore.model.MOrder" is tagged as >> "embedded-only" so does not have its own datastore table. >> 15/01/13 18:00:00 INFO metastore.ObjectStore: Initialized ObjectStore >> 15/01/13 18:00:00 WARN metastore.ObjectStore: Version information not >> found in metastore. hive.metastore.schema.verification is not enabled so >> recording the schema version 0.13.1aa >> 15/01/13 18:00:01 INFO metastore.HiveMetaStore: Added admin role in >> metastore >> 15/01/13 18:00:01 INFO metastore.HiveMetaStore: Added public role in >> metastore >> 15/01/13 18:00:01 INFO metastore.HiveMetaStore: No user is added in admin >> role, since config is empty >> 15/01/13 18:00:01 INFO session.SessionState: No Tez session required at >> this point. hive.execution.engine=mr. >> 15/01/13 18:00:02 INFO log.PerfLogger: <PERFLOG method=Driver.run >> from=org.apache.hadoop.hive.ql.Driver> >> 15/01/13 18:00:02 INFO log.PerfLogger: <PERFLOG method=TimeToSubmit >> from=org.apache.hadoop.hive.ql.Driver> >> 15/01/13 18:00:02 INFO ql.Driver: Concurrency mode is disabled, not >> creating a lock manager >> 15/01/13 18:00:02 INFO log.PerfLogger: <PERFLOG method=compile >> from=org.apache.hadoop.hive.ql.Driver> >> 15/01/13 18:00:03 INFO log.PerfLogger: <PERFLOG method=parse >> from=org.apache.hadoop.hive.ql.Driver> >> 15/01/13 18:00:03 INFO parse.ParseDriver: Parsing command: CREATE TABLE >> IF NOT EXISTS src (key INT, value STRING) >> 15/01/13 18:00:03 INFO parse.ParseDriver: Parse Completed >> 15/01/13 18:00:03 INFO log.PerfLogger: </PERFLOG method=parse >> start=1421190003030 end=1421190003031 duration=1 >> from=org.apache.hadoop.hive.ql.Driver> >> 15/01/13 18:00:03 INFO log.PerfLogger: <PERFLOG method=semanticAnalyze >> from=org.apache.hadoop.hive.ql.Driver> >> 15/01/13 18:00:03 INFO parse.SemanticAnalyzer: Starting Semantic Analysis >> 15/01/13 18:00:03 INFO parse.SemanticAnalyzer: Creating table src >> position=27 >> 15/01/13 18:00:03 INFO metastore.HiveMetaStore: 0: get_table : db=default >> tbl=src >> 15/01/13 18:00:03 INFO HiveMetaStore.audit: ugi=zzhang >> ip=unknown-ip-addr cmd=get_table : db=default tbl=src >> 15/01/13 18:00:03 INFO metastore.HiveMetaStore: 0: get_database: default >> 15/01/13 18:00:03 INFO HiveMetaStore.audit: ugi=zzhang >> ip=unknown-ip-addr cmd=get_database: default >> 15/01/13 18:00:03 INFO ql.Driver: Semantic Analysis Completed >> 15/01/13 18:00:03 INFO log.PerfLogger: </PERFLOG method=semanticAnalyze >> start=1421190003031 end=1421190003406 duration=375 >> from=org.apache.hadoop.hive.ql.Driver> >> 15/01/13 18:00:03 INFO ql.Driver: Returning Hive schema: >> Schema(fieldSchemas:null, properties:null) >> 15/01/13 18:00:03 INFO log.PerfLogger: </PERFLOG method=compile >> start=1421190002998 end=1421190003416 duration=418 >> from=org.apache.hadoop.hive.ql.Driver> >> 15/01/13 18:00:03 INFO log.PerfLogger: <PERFLOG method=Driver.execute >> from=org.apache.hadoop.hive.ql.Driver> >> 15/01/13 18:00:03 INFO ql.Driver: Starting command: CREATE TABLE IF NOT >> EXISTS src (key INT, value STRING) >> 15/01/13 18:00:03 INFO log.PerfLogger: </PERFLOG method=TimeToSubmit >> start=1421190002995 end=1421190003421 duration=426 >> from=org.apache.hadoop.hive.ql.Driver> >> 15/01/13 18:00:03 INFO log.PerfLogger: <PERFLOG method=runTasks >> from=org.apache.hadoop.hive.ql.Driver> >> 15/01/13 18:00:03 INFO log.PerfLogger: <PERFLOG method=task.DDL.Stage-0 >> from=org.apache.hadoop.hive.ql.Driver> >> 15/01/13 18:00:03 INFO exec.DDLTask: Default to LazySimpleSerDe for table >> src >> 15/01/13 18:00:05 INFO log.PerfLogger: </PERFLOG method=Driver.execute >> start=1421190003416 end=1421190005498 duration=2082 >> from=org.apache.hadoop.hive.ql.Driver> >> Exception in thread "Driver" >> Exception: java.lang.OutOfMemoryError thrown from the >> UncaughtExceptionHandler in thread "Driver" >> -- >> CONFIDENTIALITY NOTICE >> NOTICE: This message is intended for the use of the individual or entity >> to >> which it is addressed and may contain information that is confidential, >> privileged and exempt from disclosure under applicable law. If the reader >> of this message is not the intended recipient, you are hereby notified >> that >> any printing, copying, dissemination, distribution, disclosure or >> forwarding of this communication is strictly prohibited. If you have >> received this communication in error, please contact the sender >> immediately >> and delete it from your system. Thank You. >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org >> For additional commands, e-mail: user-h...@spark.apache.org >> >> > > > -- > Deepak > > > -- Deepak
