Hi, Thank you so much for your help. I really appreciate it. Im really impressed with your project and trying to apply it to our product. Our live product is still working on Mysql and MongoDb, but data is growing fast. That's why we need your product for the database engine replacement. About our problem with many queries on same time on Apache Kylin, I'm trying to monitor some elements on our system and review cubes. So are there some tutorials about concurrency of Kylin or HBase? I will give you more details abour our system: Hardware: 2 physical machines -> 7 vitural machines Each vitural machine: CPU: 8cores RAM: 24GB We are setup hadoop env with hortonwork 2.5 and setup HBase with 5 RegionServer, 2 Hbase masters Apahce Kylin we setup on 2 machines: + Node 1: using for build cubes + Node 2: using for only queries (this node also contain RegionServer) Cube and Queries: + Size of Cubes: - Cube 1: 20GB/14M rows - 24 segments (maybe we need to meger them into 2-3 segments) - Cube 2: 460MB/3M rows - 24 segments - Cube 3: 1.3GB/1.4M rows - 24 segments + We use one query to read data from 3 cubes and union all into 1 result Test case: + On single request: 3s + On 5 requests on same times: (submit multi-requests from client): 20s/request And that is not acceptable when we go live. So hope you all review our struture and give us some best pratices with Kylin And Hbase. Thanks
2017-01-12 8:24 GMT+07:00 ShaoFeng Shi <[email protected]>: > In this case you need do some profiling to see what's the bottleneck: > Kylin or HBase or other factors like CPU, memory or network; maybe it is > related with the cube design, try to optimize the cube design with the > executed query is also a way; It is hard to give you good answer with a > couple words. > > 2017-01-11 19:50 GMT+08:00 Phong Pham <[email protected]>: > >> Heres about detail on our system: >> >> Hbase: 5 nodes >> Data size: 24M rows >> >> Query result: >> *Success: true* >> *Duration: 20s* >> *Project: metrixa_global_database* >> *Realization Names: [xxx, xxx, xxx]* >> *Cuboid Ids: [45971, 24]* >> >> >> 2017-01-11 18:34 GMT+07:00 Phong Pham <[email protected]>: >> >>> Hi all, >>> I have a problem with concurrency on Apache Kylin. Execute single >>> query, it takes about 3s. Howerver,when i run multiple queries on the same >>> time, each query take about 13-15s. So how can i solve problems? >>> My Kylin Version is 1.6.1 >>> Thanks >>> >> >> > > > -- > Best regards, > > Shaofeng Shi 史少锋 > >
