Hi Alexander Alten, Thanks for your interest and +1 for Kyuubi.
Not being very familiar with Dremel, I just took a quick look at the profile of a very similar and exciting project, Apache Drill. If I understand correctly, Kyuubi is built on top of the compute layer (Apache Spark), while Apache Drill is built on top of the storage layer. Apache Spark has a large user base and usage scenarios all over the world. When users have some basic Spark programming and tuning skills, things are easy for them. But many times, most users do not have all of these abilities when they use Spark directly or indirectly. We want to build a more easy-to-use platform on top of it to help these users. On the one hand, we will work on the interface layer to make it easy to access (of course, this effort will make our project look very similar to other projects, such as HiveServer2). On the other hand, we will work under Kyuubi's multi-tenant architecture to extend our on-prem Spark SQL engine(a.k.a Kyuubi engine) to make it more stable and efficient in our scenario. What's more, we are going to add more on-prem Kyuubi engine types, e.g. Flink. Alexander Alten <a...@scalytics.io> 于2021年5月31日周一 下午2:35写道： > > Hey, > > Looks like an interesting project, competition to Starburst. How does it > compare to Apache Dremel? > > +1 (non-binding) for incubation. > > Thanks and stay safe, > --alex > > On Mon, May 31, 2021, 08:21 Kent Yao <y...@apache.org> wrote: > > > Dear all, > > > > We want to propose Kyuubi as a new Apache Incubator project. > > > > Kyuubi is a distributed multi-tenant Thrift JDBC/ODBC server for > > large-scale data management, processing, and analytics, built on top > > of Apache Spark and designed to support more engines (i.e., Apache > > Flink). We are aiming to make Kyuubi an "out-of-the-box" tool for data > > warehouses and data lakes. Kyuubi concentrates on improving the > > experience for non-Spark users, making it easy for them to use Spark > > as a service and focus on business data of their own, not where it > > stores, how it computes. NetEase has open-sourced Kyuubi since 2018, > > https://github.com/NetEase/kyuubi. > > > > Many thanks to Willem Jiang (Champion), Jeff Zhang, Duo Zhang, Akira > > Ajisaka for being the mentors of the Kyuubi project and helping us set > > up this proposal. The proposal can be found at > > https://cwiki.apache.org/confluence/display/INCUBATOR/KyuubiProposal. > > > > Looking forward to your feedback and thank you very much. > > > > Best regards, > > Kent Yao > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org > > For additional commands, e-mail: general-h...@incubator.apache.org > > > > -- Kent Yao @ Data Science Center, Hangzhou Research Institute, NetEase Corp. a spark enthusiast kyuubiis a unified multi-tenant JDBC interface for large-scale data processing and analytics, built on top of Apache Spark. spark-authorizerA Spark SQL extension which provides SQL Standard Authorization for Apache Spark. spark-postgres A library for reading data from and transferring data to Postgres / Greenplum with Spark SQL and DataFrames, 10~100x faster. itatchiA library that brings useful functions from various modern database management systems to Apache Spark. --------------------------------------------------------------------- To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
