If you do not want compression
set hive.exec.compress.output=false
If you want to compress-
set hive.exec.compress.output=false
and specify
mapred.map.output.compression.codec and mapred.output.compression.codec
depending upon query is map-only or map-reduce.
If your question is about changing
Hi All,
I have enabled my hive to use LZO codec from Hadoop, due to which all the
output from hive is by default LZO compressed unless and until Storage
format as textfile etc has not been specified.
While using the query
INSERT OVERWRITE DIRECTORY '/path' SELECT * FROM tableA. It is storing
out
I recommend trying a daily partitioning scheme over an hourly one. We had a
similar setup and ran into the same problem and ultimately found that daily
works fine for us, even with larger file sizes.
At the very least it is worth evaluating.
Sent from my iPhone
On Jan 5, 2012, at 2:23 PM, Mat
Shoot, I meant to reply to the group, not respond to Mark directly. (Mark
replied offline to me; not sure the etiquette in pasting that response in here
as well!)
Hi Mark, thanks for the response! I tried using the
memory-intensive boostrap action and got a different error; however,
I'm not
Hi Matt,
You might want to try using
s3://elasticmapreduce/bootstrap-actions/configurations/latest/memory-intensive
in your bootstrap action and see if that helps.
I would also suggest that you reconsider if having 15000 partitions is the
right thing to do and make sure you are not suffering fro
