Thanks Once hive.compactor.initiator.on property is set to true whether the merge operation takes place and reuse the blocks after each update or do we need to do Alter statement mensioned in
https://cwiki.apache.org/confluence/display/Hive/Hive+Transactions#HiveTransactions-ConfigurationValuestoSetforCompaction On Tue, Dec 2, 2014 at 2:46 PM, vic0777 <vic0...@163.com> wrote: > > > The compact operation will merge the data and then the blocks may be > resued. > > > > > At 2014-12-02 17:10:43, "unmesha sreeveni" <unmeshab...@gmail.com> wrote: > > So that block will not be reused right? If we are updating the entire > block..and at some point we dont need that record...The block will be > wasted right? > They need to relize the blocks for further writes, right? > Am I correct? > > On Tue, Dec 2, 2014 at 2:36 PM, vic0777 <vic0...@163.com> wrote: > >> >> The document describes how transaction works and what the data layout is: >> https://cwiki.apache.org/confluence/display/Hive/Hive+Transactions. See >> the "Basic design" section. HDFS is immutable. Hive creates a delta >> directory for every transaction and merges it when read, so it does not >> written to the same block. >> >> Wantao >> >> >> >> >> At 2014-12-02 16:58:26, "unmesha sreeveni" <unmeshab...@gmail.com> wrote: >> >> Why hive "UPADTE" is not reusing the blocks. >> the update is not written to same block, why is it so ? >> >> >> On Tue, Dec 2, 2014 at 10:50 AM, unmesha sreeveni <unmeshab...@gmail.com> >> wrote: >> >>> I tried to update my record in hive previous version and also tried out >>> update in hive 0.14.0. The newer version which support hive. >>> >>> I created a table with 3 buckets with 180 MB. In my warehouse the data >>> get stored into 3 different blocks >>> >>> delta_0000012_0000012 >>> --- Block ID: 1073751752 >>> --- Block ID: 1073751750 >>> --- Block ID: 1073751753 >>> >>> After doing an update >>> >>> I am getting 2 directories >>> >>> delta_0000012_0000012 >>> --- Block ID: 1073751752 >>> --- Block ID: 1073751750 >>> --- Block ID: 1073751753 >>> AND >>> delta_0000014_0000014 >>> ---Block ID: 1073752044 >>> >>> Ie the blocks are not reused >>> Whether my understanding is correct? >>> Any pointers? >>> >>> >>> -- >>> *Thanks & Regards * >>> >>> >>> *Unmesha Sreeveni U.B* >>> *Hadoop, Bigdata Developer* >>> *Centre for Cyber Security | Amrita Vishwa Vidyapeetham* >>> http://www.unmeshasreeveni.blogspot.in/ >>> >>> >>> >> >> >> -- >> *Thanks & Regards * >> >> >> *Unmesha Sreeveni U.B* >> *Hadoop, Bigdata Developer* >> *Centre for Cyber Security | Amrita Vishwa Vidyapeetham* >> http://www.unmeshasreeveni.blogspot.in/ >> >> >> >> >> > > > -- > *Thanks & Regards * > > > *Unmesha Sreeveni U.B* > *Hadoop, Bigdata Developer* > *Centre for Cyber Security | Amrita Vishwa Vidyapeetham* > http://www.unmeshasreeveni.blogspot.in/ > > > > > -- *Thanks & Regards * *Unmesha Sreeveni U.B* *Hadoop, Bigdata Developer* *Centre for Cyber Security | Amrita Vishwa Vidyapeetham* http://www.unmeshasreeveni.blogspot.in/
