On Wed, Apr 10, 2024 at 9:54 PM Binwei Yang <felix...@apache.org> wrote:
> > Gluten currently already support Velox backend and Clickhouse backend. > data fusion support is also proposed but no one worked on it. > > Gluten isn't a POC. It's under actively developing but some companies > already used it. > > > On 2024/04/11 03:32:01 Dongjoon Hyun wrote: > > I'm interested in your claim. > > > > Could you elaborate or provide some evidence for your claim, *a door for > > all native libraries*, Binwei? > > > > For example, is there any POC for that claim? Maybe, did I miss something > > in that SPIP? > I think the concern here is there are multiple different layers to get from Spark -> Native code and ideally any changes we introduce in Spark would be for common functionality that is useful across them (e.g. data fusion comet & gluten & photon*, etc.) * Photon being harder to guess at since it's closed source. > > > > Dongjoon. > > > > On Wed, Apr 10, 2024 at 8:19 PM Binwei Yang <felix...@apache.org> wrote: > > > > > > > > The SPIP is not for current Gluten, but open a door for all native > > > libraries and accelerators support. > > > > > > On 2024/04/11 00:27:43 Weiting Chen wrote: > > > > Yes, the 1st Apache release(v1.2.0) for Gluten will be in September. > > > > For Spark version support, currently Gluten v1.1.1 support Spark3.2 > and > > > 3.3. > > > > We are planning to support Spark3.4 and 3.5 in Gluten v1.2.0. > > > > Spark4.0 support for Gluten is depending on the release schedule in > > > Spark community. > > > > > > > > On 2024/04/09 07:14:13 Dongjoon Hyun wrote: > > > > > Thank you for sharing, Weiting. > > > > > > > > > > Do you think you can share the future milestone of Apache Gluten? > > > > > I'm wondering when the first stable release will come and how we > can > > > > > coordinate across the ASF communities. > > > > > > > > > > > This project is still under active development now, and doesn't > have > > > a > > > > > stable release. > > > > > > https://github.com/apache/incubator-gluten/releases/tag/v1.1.1 > > > > > > > > > > In the Apache Spark community, Apache Spark 3.2 and 3.3 is the end > of > > > > > support. > > > > > And, 3.4 will have 3.4.3 next week and 3.4.4 (another EOL release) > is > > > > > scheduled in October. > > > > > > > > > > For the SPIP, I guess it's applicable for Apache Spark 4.0.0 only > if > > > there > > > > > is something we need to do from Spark side. > > > > > > > > > > Thanks, > > > > > Dongjoon. > > > > > > > > > > > > > > > On Mon, Apr 8, 2024 at 11:19 PM WeitingChen < > weitingc...@apache.org> > > > wrote: > > > > > > > > > > > Hi all, > > > > > > > > > > > > We are excited to introduce a new Apache incubating project > called > > > Gluten. > > > > > > Gluten serves as a middleware layer designed to offload Spark to > > > native > > > > > > engines like Velox or ClickHouse. > > > > > > For more detailed information, please visit the project > repository at > > > > > > https://github.com/apache/incubator-gluten > > > > > > > > > > > > Additionally, a new Spark SPIP related to Spark + Gluten > > > collaboration has > > > > > > been proposed at > https://issues.apache.org/jira/browse/SPARK-47773. > > > > > > We eagerly await feedback from the Spark community. > > > > > > > > > > > > Thanks, > > > > > > Weiting. > > > > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > To unsubscribe e-mail: dev-unsubscr...@spark.apache.org > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > To unsubscribe e-mail: dev-unsubscr...@spark.apache.org > > > > > > > > > > --------------------------------------------------------------------- > To unsubscribe e-mail: dev-unsubscr...@spark.apache.org > > -- Twitter: https://twitter.com/holdenkarau Books (Learning Spark, High Performance Spark, etc.): https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9> YouTube Live Streams: https://www.youtube.com/user/holdenkarau
