FYI:
hive@orca:~/hive-0.10.0/src$ g++ --version
g++ (Debian 4.4.5-8) 4.4.5
Seb
On 04/30/2013 04:13 PM, Sebastien FLAESCH wrote:
Making some progress...
After disabling some options to build thrift - because it installs some
files in the system directory (/usr/lib/php), and I - do not - want that,
I did following configure for thrift:
./configure --prefix=/home/hive/thrift-0.9.0 \
--with-qt4=no \
--with-csharp=no \
--with-erlang=no \
--with-python=no \
--with-perl=no \
--with-php=no \
--with-php_extension=no \
--with-ruby=no \
--with-haskell=no \
--with-go=no \
--with-d=no
Then I could build thrift and fb303, but not when I try to build Hive ODBC:
hive@orca:~/hive-0.10.0/src$ ant compile-cpp -Dthrift.home=$THRIFT_HOME
...
...
...
[exec] g++ -Wall -g -fPIC -m32 -DARCH32
-I/home/hive/thrift-0.9.0/include/thrift
-I/home/hive/thrift-0.9.0/include/thrift/fb303 -I/include
-I/home/hive/hive-0.10.0/src/service/src/gen/thrift/gen-cpp
-I/home/hive/hive-0.10.0/src/ql/src/gen/thrift/gen-cpp
-I/home/hive/hive-0.10.0/src/metastore/src/gen/thrift/gen-cpp
-I/home/hive/hive-0.10.0/src/odbc/src/cpp -c
/home/hive/hive-0.10.0/src/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp
-o /home/hive/hive-0.10.0/src/build/metastore/objs/ThriftHiveMetastore.o
[exec] In file included from
/usr/local/include/thrift/protocol/TProtocol.h:23,
[exec] from /usr/local/include/thrift/TProcessor.h:24,
[exec] from /usr/local/include/thrift/TDispatchProcessor.h:22,
[exec] from
/home/hive/hive-0.10.0/src/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.h:10,
[exec] from
/home/hive/hive-0.10.0/src/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp:7:
[exec] /usr/local/include/thrift/transport/TTransport.h:34: error:
expected constructor, destructor, or type conversion before ‘readAll’
[exec] /usr/local/include/thrift/transport/TTransport.h:107: error:
‘uint32_t’ does not name a type
[exec] /usr/local/include/thrift/transport/TTransport.h:111: error:
‘uint32_t’ does not name a type
[exec] /usr/local/include/thrift/transport/TTransport.h:124: error:
‘uint32_t’ does not name a type
[exec] /usr/local/include/thrift/transport/TTransport.h:128: error:
‘uint32_t’ does not name a type
[exec] /usr/local/include/thrift/transport/TTransport.h:139: error:
‘uint32_t’ does not name a type
Any idea?
Thanks
Seb
On 04/30/2013 11:00 AM, Sebastien FLAESCH wrote:
Thank you Carl, but I still have problems to build Hive ODBC:
I followed the instruction from this page (I believe the doc is wrong):
https://cwiki.apache.org/Hive/hiveodbc.html
Where it says:
"Build the Hive client by running the following command from
HIVE_HOME..."
Here is my env:
----------------------------------------------------------
hive@orca:~$ cat env-apache.sh
JAVA_HOME="/usr/lib/jvm/java-6-sun"
export JAVA_HOME
JRE_HOME="$JAVA_HOME/jre"
export JRE_HOME
PATH=$JAVA_HOME/bin:$PATH
HADOOP_HOME=/home/hive/hadoop-0.23.7
export HADOOP_HOME
PATH=$HADOOP_HOME/bin:$PATH
export PATH
HIVE_HOME=/home/hive/hive-0.10.0
export HIVE_HOME
PATH=$HIVE_HOME/bin:$PATH
export PATH
THRIFT_HOME=/home/hive/thrift-0.9.0
export THRIFT_HOME
----------------------------------------------------------
But:
----------------------------------------------------------
hive@orca:~$ cd $HIVE_HOME
hive@orca:~/hive-0.10.0$ ant compile-cpp -Dthrift.home=$THRIFT_HOME
Buildfile: build.xml does not exist!
Build failed
----------------------------------------------------------
And when I go to the src directory, it goes further, but then
I have missing Thrift header files:
----------------------------------------------------------
hive@orca:~/hive-0.10.0$ cd src
hive@orca:~/hive-0.10.0/src$ ant compile-cpp -Dthrift.home=$THRIFT_HOME
...
compile-cpp:
[echo] Project: odbc
[exec] mkdir -p /home/hive/hive-0.10.0/src/build/metastore/objs
[exec] g++ -Wall -g -fPIC -m32 -DARCH32
-I/home/hive/thrift-0.9.0/include/thrift
-I/home/hive/thrift-0.9.0/include/thrift/fb303 -I/include
-I/home/hive/hive-0.10.0/src/service/src/gen/thrift/gen-cpp
-I/home/hive/hive-0.10.0/src/ql/src/gen/thrift/gen-cpp
-I/home/hive/hive-0.10.0/src/metastore/src/gen/thrift/gen-cpp
-I/home/hive/hive-0.10.0/src/odbc/src/cpp -c
/home/hive/hive-0.10.0/src/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp
-o /home/hive/hive-0.10.0/src/build/metastore/objs/ThriftHiveMetastore.o
[exec] In file included from
/home/hive/hive-0.10.0/src/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp:7:
[exec]
/home/hive/hive-0.10.0/src/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.h:10:39:
error: thrift/TDispatchProcessor.h: No such file or directory
[exec] In file included from
/home/hive/hive-0.10.0/src/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.h:11,
[exec] from
/home/hive/hive-0.10.0/src/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp:7:
[exec]
/home/hive/hive-0.10.0/src/metastore/src/gen/thrift/gen-cpp/hive_metastore_types.h:10:27:
error: thrift/Thrift.h: No such file or directory
[exec]
/home/hive/hive-0.10.0/src/metastore/src/gen/thrift/gen-cpp/hive_metastore_types.h:11:42:
error: thrift/TApplicationException.h: No such file or directory
...
----------------------------------------------------------
In fact there is no such directory:
/home/hive/thrift-0.9.0/include/thrift/fb303
I see only:
----------------------------------------------------------
hive@orca:~/hive-0.10.0/src$ ls /home/hive/thrift-0.9.0/include/thrift
c_glib
----------------------------------------------------------
According to:
http://mail-archives.apache.org/mod_mbox/hive-user/201110.mbox/%3c556325346ca26341b6f0530e07f90d960169f0fd4...@gbgh-exch-cms.sig.ads%3E
I need to build fb303 component of thrift...
So I tried to build "fb303" in the thrift sources.
Note that I ran thrift's configure with a non-system installation target
directory:
----------------------------------------------------------
hive@orca:~$ cd tmp/thrift-0.9.0/contrib/fb303
hive@orca:~/tmp/thrift-0.9.0/contrib/fb303$ ./configure
--prefix=/home/hive/thrift-0.9.0
----------------------------------------------------------
But it could not find /usr/local/bin/thrift and
/usr/local/include/thrift...
So I created links:
----------------------------------------------------------
/usr/local/bin/thrift -> /home/hive/thrift-0.9.0/bin/thrift
/usr/local/include/thrift -> /home/hive/thrift-0.9.0/include/thrift
----------------------------------------------------------
But:
----------------------------------------------------------
hive@orca:~/tmp/thrift-0.9.0/contrib/fb303$ make
make all-recursive
make[1]: Entering directory `/home/hive/tmp/thrift-0.9.0/contrib/fb303'
Making all in .
make[2]: Entering directory `/home/hive/tmp/thrift-0.9.0/contrib/fb303'
make[2]: Nothing to be done for `all-am'.
make[2]: Leaving directory `/home/hive/tmp/thrift-0.9.0/contrib/fb303'
Making all in cpp
make[2]: Entering directory
`/home/hive/tmp/thrift-0.9.0/contrib/fb303/cpp'
make all-am
make[3]: Entering directory
`/home/hive/tmp/thrift-0.9.0/contrib/fb303/cpp'
g++ -DPACKAGE_NAME=\"libfb303\" -DPACKAGE_TARNAME=\"libfb303\"
-DPACKAGE_VERSION=\"20080209\" -DPACKAGE_STRING=\"libfb303\ 20080209\"
-DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DHAVE_BOOST=/\*\*/
-DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1
-DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1
-DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1
-DHAVE_INTTYPES_H=1 -DHAVE_NETINET_IN_H=1 -I. -I.. -Igen-cpp
-I/usr/local/include/thrift -I/usr/include -Wall -O3 -MT
FacebookService.o -MD -MP -MF .deps/FacebookService.Tpo -c -o
FacebookService.o `test -f 'gen-cpp/FacebookService.cpp' || echo
'./'`gen-cpp/FacebookService.cpp
In file included from gen-cpp/FacebookService.cpp:7:
gen-cpp/FacebookService.h:10:39: error: thrift/TDispatchProcessor.h: No
such file or directory
In file included from gen-cpp/FacebookService.h:11,
from gen-cpp/FacebookService.cpp:7:
----------------------------------------------------------
...
Seb
On 04/29/2013 08:48 PM, Carl Steinbach wrote:
Hi Sebastien,
Most (if not all) of the ant targets will fail if run from the submodule
directories. You must run the targets from the root source directory
instead.
Thanks.
Carl
On Fri, Apr 26, 2013 at 6:09 AM, Sebastien FLAESCH <s...@4js.com
<mailto:s...@4js.com>> wrote:
I have now installed Thrift, but I still get the error with the
"osfamily" task:
Please help!
hive@orca:~/hive-0.10.0/src/__odbc$ ant compile-cpp
-Dthrift.home=/home/hive/__thrift-0.9.0
Buildfile: /home/hive/hive-0.10.0/src/__odbc/build.xml
BUILD FAILED
/home/hive/hive-0.10.0/src/__odbc/build.xml:30: The following error
occurred while executing this line:
/home/hive/hive-0.10.0/src/__build-common.xml:117: Problem: failed
to create task or type osfamily
Cause: The name is undefined.
Action: Check the spelling.
Action: Check that any custom tasks/types have been declared.
Action: Check that any <presetdef>/<macrodef> declarations have
taken place.
Total time: 0 seconds
Seb
On 04/26/2013 01:40 PM, Sebastien FLAESCH wrote:
Go it, need to install Apache Thrift first ... sorry for this mail.
Seb
On 04/26/2013 01:33 PM, Sebastien FLAESCH wrote:
Hi all,
Just started with an Hadoop / Hive POC to write a
So far, I have installed:
So far I have downloaded the Hadoop and Hive packages from:
http://hadoop.apache.org/__releases.html
<http://hadoop.apache.org/releases.html>
http://hive.apache.org/__releases.html
<http://hive.apache.org/releases.html>
I took:
http://hadoop.apache.org/__releases.html#18+April%2C+__2013%3A+Release+0.23.7+__available
<http://hadoop.apache.org/releases.html#18+April%2C+2013%3A+Release+0.23.7+available>
and
http://hive.apache.org/__releases.html#11+January%2C+__2013%3A+release+0.10.0+__available
<http://hive.apache.org/releases.html#11+January%2C+2013%3A+release+0.10.0+available>
After setting the env, it seems that hive is working:
hive@orca:~$ $HIVE_HOME/bin/hive
...
hive> create table t1 ( k int, s string );
OK
hive> select count(*) from t1;
Total MapReduce jobs = 1
...
OK
0
Time taken: 7.634 seconds
No I want to connect through ODBC...
Following the instructions of this page:
https://cwiki.apache.org/__confluence/display/Hive/__HiveODBC
<https://cwiki.apache.org/confluence/display/Hive/HiveODBC>
Section "Hive Client Build/Setup", I tried:
hive@orca:~/hive-0.10.0$ cd $HIVE_HOME
hive@orca:~/hive-0.10.0$ ant compile-cpp
-Dthrift.home=/home/hive/hive-__0.10.0/odbc
Buildfile: build.xml does not exist!
Build failed
Then, from the sources directory src/odbc:
hive@orca:~/hive-0.10.0$ cd src/odbc/
hive@orca:~/hive-0.10.0/src/__odbc$ ant compile-cpp
-Dthrift.home=/home/hive/hive-__0.10.0/odbc
Buildfile: /home/hive/hive-0.10.0/src/__odbc/build.xml
BUILD FAILED
/home/hive/hive-0.10.0/src/__odbc/build.xml:30: The
following error
occurred while executing this line:
/home/hive/hive-0.10.0/src/__build-common.xml:117: Problem:
failed to
create task or type osfamily
Cause: The name is undefined.
Action: Check the spelling.
Action: Check that any custom tasks/types have been declared.
Action: Check that any <presetdef>/<macrodef> declarations
have taken
place.
Total time: 0 seconds
Can someone help or point me to an up to date documentation?
Thanks!
Seb
