Hi all, I've been prototyping an implementation of the DataSource V2 writer for the MongoDB Spark Connector and I have a couple of questions about how its intended to be used with database systems. According to the Javadoc for DataWriter.commit():
*"this method should still "hide" the written data and ask the DataSourceWriter at driver side to do the final commit via WriterCommitMessage"* Although, MongoDB now has transactions, it doesn't have a way to "hide" the data once it has been written. So as soon as the DataWriter has committed the data, it has been inserted/updated in the collection and is discoverable - thereby breaking the documented contract. I was wondering how other databases systems plan to implement this API and meet the contract as per the Javadoc? Many thanks Ross
