This is from the vote for spark connector. Is this you are looking for? The goal of the SPIP is to introduce a Dataframe based client/server API for Spark
Please also refer to: - Previous discussion in dev mailing list: [DISCUSS] SPIP: Spark Connect - A client and server interface for Apache Spark. <https://lists.apache.org/thread/3fd2n34hlyg872nr55rylbv5cg8m1556> - Design doc: Spark Connect - A client and server interface for Apache Spark. <https://docs.google.com/document/d/1Mnl6jmGszixLW4KcJU5j9IgpG9-UabS0dcM6PM2XGDc/edit?usp=sharing> - JIRA: SPARK-39375 <https://issues.apache.org/jira/browse/SPARK-39375> ons. 23. nov. 2022 kl. 17:36 skrev Mitch Shepherd < mitch.sheph...@marklogic.com>: > Hello, > > > > I’m wondering if anyone can point me in the right direction for a Spark > connector developer guide. > > > > I’m looking for information on writing a new connector for Spark to move > data between Apache Spark and other systems. > > > > Any information would be helpful. I found a similar thing for Kafka > <https://docs.confluent.io/platform/current/connect/devguide.html> but > haven’t been able to track down documentation for Spark. > > > > Best, > > Mitch > > This message and any attached documents contain information of MarkLogic > and/or its customers that may be confidential and/or privileged. If you are > not the intended recipient, you may not read, copy, distribute, or use this > information. If you have received this transmission in error, please notify > the sender immediately by reply e-mail and then delete this message. This > email may contain pricing or other suggested contract terms related to > MarkLogic software or services. Any such terms are not binding on MarkLogic > unless and until they are included in a definitive agreement executed by > MarkLogic. > -- Bjørn Jørgensen Vestre Aspehaug 4, 6010 Ålesund Norge +47 480 94 297
