nice
------------------ Original ------------------ From: 陈明雨 <morning...@163.com> Date: Mon,May 30,2022 1:00 PM To: dev <dev@doris.apache.org> Subject: Re: Re:Re: Re: [Discuss][DSIP] Support Multi Catalog Actually, in my design, after user creating an external datasource, he can access the external databases and tables directly, and no need to create the metadata mapping manuelly. -- 此致！Best Regards 陈明雨 Mingyu Chen Email: chenmin...@apache.org 在 2022-05-30 12:52:49，"张家峰" <zhang...@gmail.com> 写道： >how to create an exterior： > > When using external tables in this management method, it is best not >to manually create the mapping of each table, so that users can use it >without perception, especially the field mapping of the external table, >which can be simplified to the point where the user only needs to create an >external data source. resource , and then create a foreign table. You don't >need to specify the order of fields. You only need to specify the resource >and attributes corresponding to this table (which table corresponds to the >data source). If you can create an external data source in Doris, you can >create database by specifying the external data source. attribute, and then >the user can view the best appearance through the show tables method under >this database; > > > >jiafeng.Zhang > > >陈明雨 <morning...@163.com> 于2022年5月30日周一 12:39写道： > >> 1. Permission >> >> >> In my design, the permission system of the external data source is >> decoupled from Doris' own permission system. >> First, when creating an external data source, the user will specify an >> account of the external data source to connect to (named ”proxy account”), >> and the permissions of this account to the external data source are managed >> by the external data source itself. >> Inside Doris, we still use the current permission management mechanism to >> manage the read and write permissions of the database or table in the >> external data source. >> >> >> For example, if I grant read permission to table A on the Doris side, but >> the proxy account does not have read permission to table A, an error will >> be reported when the actual access is made. These are two logics that do >> not affect each other. >> >> >> 2. Direction >> >> >> Many users' data is stored in external data sources, but they hopes to >> have a unified data access portal for offline and online analysis, >> federated query and other operations. The purpose of this function is to >> give Doris the ability to "unify SQL entry". >> At this stage, we need to solve the problem of metadata mapping and data >> access unification for external data sources such as hive, iceberg, and >> hudi. >> >> >> >> -- >> >> 此致！Best Regards >> 陈明雨 Mingyu Chen >> >> Email: >> chenmin...@apache.org >> >> >> >> >> >> At 2022-05-30 10:54:08, "ling miao" <lingm...@apache.org> wrote: >> >Regarding permissions, there is currently a difference between Doris's >> >permission system and the external permission system. For external data >> >sources, still use Doris's permissions, or use the permissions of external >> >data sources? >> > >> >As far as the current architecture is concerned, Doris is still a system >> >based on querying its own table and has made many optimizations on this >> >basis. >> >Supporting such functions at this stage, what *direction* do you hope >> Doris >> >will develop in the future? What *other features* are planned besides >> this? >> > >> > >> > >> >陈明雨 <morning...@163.com> 于2022年5月27日周五 18:10写道： >> > >> >> Hi all, >> >> I plan to support multi catalog in Doris to manager all external >> >> datasource such as hive, iceberg, hudi, es, odbc, etc. >> >> I have created a DSIP[1] for this. >> >> And here is the first PR to add some new interface and class[2] >> >> >> >> >> >> Please feel free to discuss. >> >> >> >> >> >> >> >> >> >> [1] >> >> >> https://cwiki.apache.org/confluence/display/DORIS/DSIP-014%3A+Multi+Catalog+Support >> >> >> >> [2] https://github.com/apache/incubator-doris/pull/9812 >> >> >> >> >> >> >> >> >> >> -- >> >> >> >> 此致！Best Regards >> >> 陈明雨 Mingyu Chen >> >> >> >> Email: >> >> chenmin...@apache.org >> > >> > >> > >> >-- >> >Ling Miao | Apache Doris >> > > >-- >张家峰
