Hive metatool JVM Heap error

2013-11-11 Thread Mark Schramm
Hive Metatool experts, Trying to use the hive "metatool" to update the locations for all of our tables after a configuration change did not do this automatically. The tool gets a heap error even though we upped the JVM to 4 GB. Does anyone know of a workaround? Thanks, Mark

Map-Joining the same small table multiple times

2012-06-18 Thread Mark Schramm
All, We have a small table that we use the map-join technique to join to several large tables in separate hive query scripts. As I understand it, the map-join will do some preparatory work to get the small table into the distributed cache for the map-join. These steps are (from my understandi

RE: derby related files show up when using MySQL

2012-01-24 Thread Mark Schramm (tetrascend)
automatically collect table stats ONLY during 'insert overwrite' queries. This feature is still being worked on and is only useful for indexing. You should set this in your hive-site.xml hive.stats.autogather false Edward On Tue, Jan 24, 2012 at 1:58 PM, Mark Schramm (

derby related files show up when using MySQL

2012-01-24 Thread Mark Schramm (tetrascend)
I have configured hive to use MySQL. This all seems to work fine, but occasionally derby.log files and TempStatsStore directories get created as if I was using the default derby database. Is this an indication that I have something misconfigured? Can anyone explain why this happens? Thanks,

Is there a way to track Hive jobs

2012-01-09 Thread Mark Schramm (tetrascend)
Anyone, I would like to be able to submit Hive queries/jobs and track their progress (from a Java App). For example, I would like to submit a hive sql command string as a job, returning a job-ID and to be able query the hive server for status of jobs (e.g. queued, completed, executing hadoop j