BTW, I am using hive 0.7
From: Lu, Wei
Sent: Wednesday, January 11, 2012 3:13 PM
To: 'user@hive.apache.org'
Subject:
Hi,
I am using ThriftHive.Client to access a pretty large table.
SQL Statement:
select a11.asin asin,
max(a11.title) title,
a11.salesrank salesrank,
a11.category category,
avg(a11.avg_rating) WJXBFS1,
sum(a11.total_num_reviews) WJXBFS2,
sum(a11.num_subcategories) WJXBFS3
from table_details a11
group by a11.asin,
a11.salesrank,
a11.category.
The statement will select a pretty large result set (1,000,000+ rows). When I
use ThriftHive.Client fetchAll() to get all the row strings, an exception
returns like below:
Exception in thread "main" org.apache.thrift.TApplicationException: Internal
error processing fetchAll
at
org.apache.thrift.TApplicationException.read(TApplicationException.java:108)
at
org.apache.hadoop.hive.service.ThriftHive$Client.recv_fetchAll(ThriftHive.java:224)
at
org.apache.hadoop.hive.service.ThriftHive$Client.fetchAll(ThriftHive.java:208)
... ...
Why does that happen? How could I deal with it? It seem that incremental fetch
is not supported by hive.
Regards,
Wei
