EmmyMiao87 edited a comment on issue #1095: Unifies load flow URL: https://github.com/apache/incubator-doris/issues/1095#issuecomment-504322449 I propose to test the performance of streaming broker load and mini load. This test will ignore the effect of rollup and query. The result of test will only depends on the performance of new load framework. The environment: Node 1: * Memory: about 3G * Disk: SATA about 2T * Network: about 10000Mb/s Node 2,3,4: * Memory: about 27G * Disk: SSD about 369G * Network: about 10000Mb/s The prepare of test: 1. Install doris: * One frontend and three backends. * Three backends are located in different node(2,3,4). * The frontend is located Node1. 2. Create table: * Situation 1: The table has only one tablet and 3 replicas. SQL: create table xxx (about 34 columns: 16 sum aggregation columns) ENGINE=OLAP AGGREGATE KEY(2 columns) DISTRIBUTED BY HASH(2 columns) BUCKETS 1 PROPERTIES ("storage_type" = "COLUMN"); * Situation 2: The table has 100 tablet and every tablet has 3 replicas. SQL: create table xxx (about 34 columns: sum 16) ENGINE=OLAP AGGREGATE KEY(2 columns) DISTRIBUTED BY HASH(2 columns) BUCKETS 100 PROPERTIES ("storage_type" = "COLUMN"); The data of test: Load different size (2K, 40M, 20G, 280G) of file: catalog_sales.dat The step of test: 1. Load data by non-streaming broker load 2. Load data by streaming broker load 3. Load data by non-streaming mini load 4. Load data by streaming mini load
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org For additional commands, e-mail: dev-h...@doris.apache.org
