Hi, Curious to see if this issue been resolved (performance) after compaction?
Thanks Dr Mich Talebzadeh LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>* http://talebzadehmich.wordpress.com *Disclaimer:* Use it at your own risk. Any and all responsibility for any loss, damage or destruction of data or any other property which may arise from relying on this email's technical content is explicitly disclaimed. The author will in no case be liable for any monetary damages arising from such loss, damage or destruction. On 26 June 2016 at 21:11, @Sanjiv Singh <sanjiv.is...@gmail.com> wrote: > Thanks Gopal for your inputs ....For now I have create NON ACID table and > loaded data ....see below from logs proper group splits happening . > > 2016-06-25 12:52:00,160 [INFO] [InputInitializer {Map 1} #0] > |tez.HiveSplitGenerator|: Number of grouped splits: 512 > > > On compaction issue , Compaction enabled with two workers. why compaction > not happened ? will check metastore logs. > > I have too many ACID tables on hive and how many worker should be > configured ? currently it is 2. > > Thanks a lot once again. > > > Regards > Sanjiv Singh > Mob : +091 9990-447-339 > > On Fri, Jun 24, 2016 at 9:14 PM, @Sanjiv Singh <sanjiv.is...@gmail.com> > wrote: > >> Thanks Gopal for your inputs. Let me run compaction explicitly on table >> then see how query works. >> >> >> >> Let >> >> Regards >> Sanjiv Singh >> Mob : +091 9990-447-339 >> >> On Fri, Jun 24, 2016 at 7:53 PM, Gopal Vijayaraghavan <gop...@apache.org> >> wrote: >> >>> >>> > Yes for this tables, ACID enabled. it has only 256 files for each >>> >buckets. these are create only when data initially loaded in this table. >>> >>> Yes, the initial load goes in as an insert DELTA too - that requires >>> another compaction to move into base files. >>> >>> The fact that they haven't been automatically compacted yet, suggests >>> that >>> the compactor isn't working for some reason (check hive metastore logs). >>> >>> > One thing that I am not able to understand that its is running with 1 >>> >MAPPER. >>> >>> The size of deltas shows up as 0, till the compaction goes through - in >>> Hive2, it will be -1 which will be correctly interpreted as "unknown >>> size". >>> >>> >>> > | -rw-r--r-- 3 H56473 hdfs 215973009 2016-06-23 17:38 >>> >>> >/apps/hive/warehouse/PRDDB.db/tuning_dd_key/delta_0001570_0001570/bucket_0 >>> >0000 | >>> >>> Clearly an issue due to the lack of compaction - I see a single delta >>> with >>> 255 buckets and no base_* files at all. >>> >>> Cheers, >>> Gopal >>> >>> >>> >>> >>> >>> >>> >>> >>> >> >
