Hi Ketan,
If we chabge these parameters for the already existing cubes and refresh the already existing segments, that will create the new segment for the required date range, with preferred region size or ? >>> Yes, newly created HBase table will use the new parameters; Maybe after that we can run the kylin cleanup to delete these tables from Hbase.Other option is to manually merge , the regions of this existing segment table into our preferred region numbers. >>> Correct; Merge is also workable, because Kylin will re-estimate the size; Hope Kylin will update its metadata accordingly in this case, without any issues or ? >>> I think so; After upgrade to Kylin 2.5.1, the customization should be cleared from kylin.properties; The new version will estimate that more accurate. kdcool6932 <[email protected]> 于2018年11月6日周二 下午4:31写道： > Thanks ShaoFeng,I will check the code base to understand how these > parameters are effecting the cube estimations, that will help us gauge most > appropriate values for the same. (Till we plan to upgrade 2.5.1)I have a > question though, If we chabge these parameters for the already existing > cubes and refresh the already existing segments, that will create the new > segment for the required date range, with preferred region size or ?Maybe > after that we can run the kylin cleanup to delete these tables from > Hbase.Other option is to manually merge , the regions of this existing > segment table into our preferred region numbers. Hope Kylin will update its > metadata accordingly in this case, without any issues or > ?Thanks,[email protected] Sent from my Samsung Galaxy smartphone. > -------- Original message --------From: ShaoFeng Shi < > [email protected]> Date: 06/11/2018 8:36 am (GMT+05:30) To: dev < > [email protected]> Subject: Re: Understanding about region cut size > for base Hi Ketan,Kylin estimates the HBase table size; The estimation > might be inaccuratewhen there are some advanced measures like TopN, Count > distinct. Theaccuracy was improved in v2.5.0 by KYLIN-3453. For previous > versions, youmay need to manually give smaller value to these > parameters:kylin.cube.size-estimate-ratio=0.25kylin.cube.size-estimate-memhungry-ratio=0.05ketan > dikshit <[email protected]> 于2018年11月5日周一 下午10:13写道：> Hi Team> > I would like to understand how does the> > 'kylin.storage.hbase.region-cut-gb’ property works.> We are currently using > kylin 2.3.1, We are going with the default property> value ie; > kylin.storage.hbase.region-cut-gb=5>> But still we see some segments not > adhering to this property; example:>> Segment: > 20180723000000_20180730000000>> Start Time: 2018-07-23 00:00:00> End Time: > 2018-07-30 00:00:00> Source Count: 447860691> HBase Table: > KYLIN_ENX1MBQAMX> Region Count: 500> Size: 49.57422 GB> Segment: > 20181005000000_20181006000000>> Start Time: 2018-10-05 00:00:00> End Time: > 2018-10-06 00:00:00> Source Count: 52522716> HBase Table: KYLIN_PG5PQBJ910> > Region Count: 47> Size: 6.16309 GB> Segment: > 20181010000000_20181011000000>> Start Time: 2018-10-10 00:00:00> End Time: > 2018-10-11 00:00:00> Source Count: 62012099> HBase Table: KYLIN_I4QS9A4AHL> > Region Count: 52> Size: 6.98145 GB>> Along with the same, we are also using > compression,> 'kylin.storage.hbase.compression-codec=lz4’> The number of > regions need to be kept in control, for our Hbase cluster to> be > performant.>> Please share the understanding, how this property works, and > what can be> the possible reasons why it is not working as intended.>> > Thanks,> Ketan@Exponential>>-- Best regards,Shaofeng Shi 史少锋 -- Best regards, Shaofeng Shi 史少锋
