Hi All,
I am using hive 0.7 on Amazon EMR. I need to merge a large number of small
files into a few larger files( basically merging a number of partitions for
a table into one). On doing the obvious query, i.e.( insert into a new
partition select * from all partitions), a large number of small file
Hey Alex,
This indeed is a bug. I have done a patch for it on
https://issues.apache.org/jira/browse/HIVE-2616 Would you like to try it
out to see if that works for you?
Ashutosh
On Tue, Nov 29, 2011 at 02:45, Alex Holmes wrote:
> Running mysql as the metastore doesn't change the behavior of the
Hi William,
I have hands-on experience with Pentaho for Hadoop, that is the PDI
(Pentaho Data Integration) module. There are components out there (called
"steps") that can check whether a file is there (in HDFS or somewhere
else). If the file is not there yet, you could check again every X minutes
William,
Oozie workflow jobs support Hive actions and Oozie coordinator jobs support
time/data activation of workflow jobs.
Cheers.
Alejandro
On Tue, Nov 29, 2011 at 4:27 PM, William Kornfeld wrote:
> We are building an application that involves chains of M/R jobs, most
> likely all will be w
Azkaban is worth to look at
On Tue, Nov 29, 2011 at 4:27 PM, William Kornfeld wrote:
> We are building an application that involves chains of M/R jobs, most
> likely all will be written in Hive. We need to start a Hive job when one
> or more prerequisite data sets appear (defined in the Hive se
We are building an application that involves chains of M/R jobs, most likely
all will be written in Hive. We need to start a Hive job when one or more
prerequisite data sets appear (defined in the Hive sense as a new partition
having been populated with data) - OR- a particular time has been re
On Nov 29, 2011, at 3:24 PM, Jakob Homan wrote:
> I'm trying to find documentation as to what changes in the metastore
> structure are necessary going from 0.7 to the 0.8RCs, and am failing.
> Does that mean there is none, or I'm just not very good at finding it?
README.txt, section "Upgrading f
I'm trying to find documentation as to what changes in the metastore
structure are necessary going from 0.7 to the 0.8RCs, and am failing.
Does that mean there is none, or I'm just not very good at finding it?
Thanks,
Jakob
Thanks for all the help and replies. I tracked this error down to the fact
that I was using the --warehouse-dir option in swoop to be the directory of
the Hive warehouse. That meant the Hive import step in Sqoop was trying to
overwrite the source of the import, namely the data that was produced by
Hi Jurgen,
May be you can find some hints in
http://www.slideshare.net/kate_ting/habits-of-effective-sqoop-users
Regards,
Miguel
On 29 Nov 2011, at 00:44, arv...@cloudera.com wrote:
> Hi Jurgen,
>
> What version of Hive and Sqoop are you using? Also, please look under
> /tmp/${USER}/hive.lo
I'm using
* Sqoop 1.3.0-cdh3u2
* Hive 0.7.1-cdh3u2
My /tmp/${USER}/hive.log file is not very informative:
2011-11-29 08:04:22,636 ERROR DataNucleus.Plugin
(Log4JLogger.java:error(115)) - Bundle "org.eclipse.jdt.core" requires
"org.eclipse.core.resources" but it cannot be resolved.
2011-11-29 08:0
Add it to the distributed cache
add jar derby.jar;
or
use environment variable HIVE_AUX_JARS_PATH
On Mon, Nov 28, 2011 at 1:29 PM, Ghousia wrote:
> Hi Chinna,
>
> I have the derby.jar in the lib folder, hive runs fine and we are able to
> execute the queries. Even then this error shows up in t
Hi,
In the below stack trace i found that shims not able to get the version
number or it is giving wrong numbers so it is not able get the class name and
class name coming as null. "Could not load shims in class null"
Can u post what version of distributions u r using (Hadoop and hive) a
Running mysql as the metastore doesn't change the behavior of the HDFS
operations, and more importantly who (the ugi) they are executed as.
Does anyone have any thoughts as to why Hive HDFS operations are run
as different users?
Many thoughts,
Alex
On Tue, Nov 29, 2011 at 2:47 AM, Alexander C.H
Please go through this blog ..
http://ankitasblogger.blogspot.com/2011/01/installing-hbase-in-cluster-complete.html
On Tue, Nov 29, 2011 at 2:11 PM, shashwat shriparv <
dwivedishash...@gmail.com> wrote:
> Hey Ankit, Thanks. my host file i tried using 127.0.0.1 still same
> problem, can you please
Hey Ankit, Thanks. my host file i tried using 127.0.0.1 still same problem,
can you please tell me what should be there in the host file now my host
file contains following entries:
192.168.2.125 shashwat.pointcrossblr ubuntu
127.0.0.1 localhost
#127.0.0.1localhost
#192.168.2.125ubuntu.
Hi,
Please use 127.0.0.1 instead of ubuntu.ubuntu-domain .
or Open the Hbase shell and run command 'status'.
On Tue, Nov 29, 2011 at 1:34 PM, shashwat shriparv <
dwivedishash...@gmail.com> wrote:
> I have followed
> https://cwiki.apache.org/confluence/display/Hive/HBaseIntegration my
> hbase is
17 matches
Mail list logo
