[
https://issues.apache.org/jira/browse/SPARK-51735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Descotte updated SPARK-51735:
-----------------------------
Description:
Frameless is a Scala library for working with Spark using more expressive types.
It leverages Scala meta programming capabilities to add type safety on
dataframes without compromising on performance as all checks are done at
compile time. Including this as part of the Spark project would allow better
type safety than Datasets and Dataframes, with the performances of Dataframes
and at the same time ensuring compatibility with each new Spark release.
Currently it is a risk to use this library as an external dependency as we
don't know for sure how long it will take to be compatible with each new Spark
release if we need to upgrade our cluster.
References :
* [https://typelevel.org/frameless/TypedDatasetVsSparkDataset.html]
* [https://github.com/typelevel/frameless]
was:
Frameless is a Scala library for working with Spark using more expressive types.
It leverages Scala meta programming capabilities to add type safety on
dataframes without compromising on performance as all checks are done at
compile time. Including this as part of the Spark project would allow better
type safety than Datasets and Dataframes, with the performances of Dataframes
and at the same time ensuring compatibility with each new Spark release.
Currently it is a risk to use this library as an external dependency as we
don't know for sure how long it will take to be compatible with each new Spark
release if we need to upgrade our cluster.
Reference :
[https://github.com/typelevel/frameless|https://typelevel.org/frameless/FeatureOverview.html]
> Integrate frameless as part of Spark
> ------------------------------------
>
> Key: SPARK-51735
> URL: https://issues.apache.org/jira/browse/SPARK-51735
> Project: Spark
> Issue Type: New Feature
> Components: Spark Core
> Affects Versions: 3.5.5
> Reporter: Descotte
> Priority: Major
>
> Frameless is a Scala library for working with Spark using more expressive
> types.
> It leverages Scala meta programming capabilities to add type safety on
> dataframes without compromising on performance as all checks are done at
> compile time. Including this as part of the Spark project would allow better
> type safety than Datasets and Dataframes, with the performances of Dataframes
> and at the same time ensuring compatibility with each new Spark release.
> Currently it is a risk to use this library as an external dependency as we
> don't know for sure how long it will take to be compatible with each new
> Spark release if we need to upgrade our cluster.
> References :
> * [https://typelevel.org/frameless/TypedDatasetVsSparkDataset.html]
> * [https://github.com/typelevel/frameless]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
