I think you can use Spark Connect's way and represent DataFrame as a
LogicalPlan that can be serialized to the protobuf message and back.
On Tue, 2024-11-05 at 16:46 +0100, vincent gromakowski wrote:
> Hi spark users,
> I am looking for a way to access the binary representation of a
> Dataframe to
Hello!
I'm trying to create my own Spark Connect Plugin by implementing
`org.apache.sql.connect.plugin.RelationPlugin`. My target is spark 4.0;
What I did:
1. I added compile time dependency `spark-connect-common_2.13`
2. I added `import public "spark/connect/base.proto";` to my message
3. I add
