Hi, Im using normal query like this: select <some dims>, <sum(metrics)> from <table> inner join <table> where <condition> group by <some dims>
Result set is about over 10.000 rows Query time: 3.564 And i have 5 HBase nodes 2017-01-11 9:42 GMT+07:00 Billy Liu <[email protected]>: > What kind of query you are using? How much the result set would be? How > many HBase nodes you are using? > > 2017-01-11 10:11 GMT+08:00 Phong Pham <[email protected]>: > >> Hi, >> i try to edit config on ehcache_test.xml like this: >> <ehcache maxBytesLocalHeap="2048M"> >> >> And i still not use LDAP authentication. And its still not working. >> Result is: >> *Duration: 3.735* >> *Project: metrixa_global_database* >> *Realization Names: [xxx]* >> *Cuboid Ids: [45971]* >> *Total scan count: 556952* >> *Result row count: 50000* >> *Accept Partial: true* >> *Is Partial Result: false* >> *Hit Exception Cache: false* >> *Storage cache used: false* >> *Message: null* >> >> Maybe i will try LDAP and report result for you. >> >> And one more problem that i need your help is: * "Query performance is >> not stable"* >> My query offent take 2-3s. However after a long time i execute that >> query, it takes about 10s. On log file, i saw that process will load >> snapshot data again. I try to configure >> *kylin.table.snapshot.max_mb=1000.* >> So is this ok? >> >> Thanks for your help. >> >> 2017-01-10 16:53 GMT+07:00 ShaoFeng Shi <[email protected]>: >> >>> Hi Phong, >>> >>> Did you enable "LDAP" authentication in Kylin? By default Kylin >>> configures to use 10MB memory cache only; if LDAP is enabled (it thought be >>> used in production), will use 2GB memory cache. >>> >>> You can try to allocate more memory by: >>> >>> cd $KYLIN_HOME >>> vi ./tomcat/webapps/kylin/WEB-INF/classes/ehcache_test.xml >>> In there modify "maxBytesLocalHeap="10M"" to a bigger value like 1G >>> >>> Then restart Kylin. >>> >>> I'm not sure whether it can solve the error, but it worth a try. Please >>> share the result after apply this. Thanks! >>> >>> >>> 2017-01-10 14:39 GMT+08:00 Phong Pham <[email protected]>: >>> >>>> Hi, >>>> I'm still facing with cache problems. After setting: >>>> >>>> >>>> >>>> >>>> *kylin.query.cache-threshold-duration=2kylin.query.cache-threshold-scan-count=5000kylin.query.cache-enabled=true* >>>> >>>> Result is still: >>>> *Is Partial Result: false* >>>> *Hit Exception Cache: false* >>>> *Storage cache used: false* >>>> >>>> i saw some warnings that: >>>> The configured limit of 1,000 object references was reached while >>>> attempting to calculate the size of the object graph. Severe performance >>>> degradation could occur if the sizing operation continues. This can be >>>> avoided by setting the CacheManger or Cache <sizeOfPolicy> elements >>>> maxDepthExceededBehavior to "abort" or adding stop points with >>>> @IgnoreSizeOf annotations. If performance degradation is NOT an issue at >>>> the configured limit, raise the limit value using the CacheManager or Cache >>>> <sizeOfPolicy> elements maxDepth attribute. For more information, see the >>>> Ehcache configuration documentation. >>>> 2017-01-10 00:46:48,423 INFO [Query >>>> 25496a78-149a-4752-8c3b-d988a786b0ec-312] >>>> service.QueryService:284 : >>>> >>>> So how can i fix this? >>>> thanks >>>> >>>> 2017-01-10 9:27 GMT+07:00 ShaoFeng Shi <[email protected]>: >>>> >>>>> Hi Phong, >>>>> >>>>> Kylin cache query results in two scenarios: >>>>> >>>>> 1) the query time exceeds 2 seconds (defined by >>>>> kylin.query.cache-threshold-duration=2000) >>>>> or 2) the query scan lager than 10 thousands rows in hbase (defined by >>>>> kylin.query.cache-threshold-scan-count=10240) >>>>> >>>>> This means, the very fast and small query will not be hit by cache. >>>>> You can give smaller values to the above parameters to use cache more >>>>> frequently. >>>>> >>>>> 2017-01-10 9:03 GMT+08:00 Phong Pham <[email protected]>: >>>>> >>>>>> How can i turn on cache for queries? >>>>>> >>>>>> I turn on *kylin.query.cache.enabled=true and enabled on System >>>>>> tab. * >>>>>> However when i query and submit query again. Result still is: >>>>>> >>>>>> Accept Partial: true >>>>>> Is Partial Result: false >>>>>> Hit Exception Cache: false >>>>>> Storage cache used: false >>>>>> >>>>>> Kylin Version: 1.6.2 >>>>>> Thanks for your help. >>>>>> >>>>>> 2017-01-09 20:36 GMT+07:00 Billy Liu <[email protected]>: >>>>>> >>>>>>> *Hit Exception Cache: false *means your query does not hit not >>>>>>> cache, if you query again, the result would be "Hit Exception Cache: >>>>>>> true" >>>>>>> >>>>>>> 2017-01-09 17:20 GMT+08:00 Phong Pham <[email protected]>: >>>>>>> >>>>>>>> Hi, >>>>>>>> i saw this on log file after a query finished: >>>>>>>> >>>>>>>> *Accept Partial: true* >>>>>>>> *Is Partial Result: false* >>>>>>>> *Hit Exception Cache: false* >>>>>>>> *Storage cache used: false* >>>>>>>> *Message: null* >>>>>>>> >>>>>>>> On setting, i set: >>>>>>>> *kylin.query.cache.enabled=true* >>>>>>>> >>>>>>>> and enable cache on System tab. >>>>>>>> >>>>>>>> So Is my query not cached? >>>>>>>> >>>>>>>> And sometime, my kylin server go down with some bad query or unknow >>>>>>>> reasons. So how can i kill slow queries automatically? >>>>>>>> Thanks >>>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> Best regards, >>>>> >>>>> Shaofeng Shi 史少锋 >>>>> >>>>> >>>> >>> >>> >>> -- >>> Best regards, >>> >>> Shaofeng Shi 史少锋 >>> >>> >> >
