Re: Re:Re: The performance issue about Doris mapping table

2021-03-31 Thread Lee Happen
Hi Thomas Cai, I did the work to push down the limit clause for odbc table and mysql table. You can refer to this pr:https://github.com/apache/incubator-doris/pull/4707/files. If you want better performance of mapping table with limit clause, please update doris version after 0.13.15. _

Re:Re: The performance issue about Doris mapping table

2021-03-31 Thread Thomas Cai
Dear Ling Miao, Thanks for your advice. Here is the result by looking for the profile. Any more suggestions? Query: Summary: Query ID: 3ef6a5a504e4466e-aeffd78390f003e6 Start Time: 2021-03-31 20:24:25 End Time: 2021-03-31 20:24:34 Total: 9s452ms Query Type: Query Que

Re: Apache Hop integration

2021-03-31 Thread Matt Casters
Hi Ling Miao, That's right, we'll list Apache Doris in that list and as well in the user interface so that people learn about Doris and if they have it they can immediately connect. Thanks and best of luck, Matt On Wed, Mar 31, 2021 at 1:37 PM ling miao wrote: > Hi Matt Casters, > > It is a

Re: The performance issue about Doris mapping table

2021-03-31 Thread ling miao
Hi Thomas Cai, Doris's mapping table read performance is indeed slightly worse than the direct read mysql table performance. *But your case should not be affected.* It may be that some optimizations of our later versions are not in 0.12 version. You can use this document to print out the profile

Re: Apache Hop integration

2021-03-31 Thread ling miao
Hi Matt Casters, It is a great honor that Doris can be used as one of the data sources supported by Hop and advertised in your document. I read the Hop document, do you mean to put Doris in this directory? https://hop.apache.org/manual/latest/database/databases.html If so, the Doris information i

The performance issue about Doris mapping table

2021-03-31 Thread Thomas Cai
Dear All, I got a performance issue when I selected records(limit 1000) from a Doris mapping table(MySQL,records were around 150K). It took less than 2s to look for the same records from MySQL table directly, but it would take about 20s from the Doris mapping table. My Doris version is 0.12 an

Re: Apache Hop integration

2021-03-31 Thread Matt Casters
Hi Ling Miao, Thank you for your reply. We already support MySQL in all RDBMS transforms and with bulk loading so folks can use that. To "advertise" support I'll create a derivative database plugin for Apache Doris so that folks can find it in the list of databases we support. The MySQL JDBC dri

Re: Apache Hop integration

2021-03-31 Thread ling miao
Hi Matt, Nice to meet Apache Hop (incubating) devs. About question2: Since Doris supports the mysql protocol, general users will use JDBC or other tools that support the mysql protocol to perform bulk load tasks. If you want to use the http api, you can use the stream load method. Stream load can