+1 On Sat, Mar 22, 2025 at 7:01 PM Peter Toth <peter.t...@gmail.com> wrote:
> +1 > > On Fri, Mar 21, 2025 at 10:24 PM Szehon Ho <szehon.apa...@gmail.com> > wrote: > >> +1 (non binding) >> >> Agree with Anton, data sources like the open table formats define the >> requirement, and definitely need engines to write to it accordingly. >> >> Thanks, >> Szehon >> >> On Fri, Mar 21, 2025 at 1:31 PM Anton Okolnychyi <aokolnyc...@gmail.com> >> wrote: >> >>> -1 (non-binding): Breaks the Chain of Responsibility. Constraints should >>>> be defined and enforced by the data sources themselves, not Spark. Spark is >>>> a processing engine, and enforcing constraints at this level blurs >>>> architectural boundaries, making Spark responsible for something it does >>>> not control. >>>> >>> >>> I disagree that this breaks the chain of responsibility. It may be quite >>> the opposite, in fact. Spark is already responsible for enforcing NOT NULL >>> constraints by adding AssertNotNull for required columns today. Connectors >>> like Iceberg and Delta store constraint definitions but rely on engines >>> like Spark to enforce them during INSERT, DELETE, UPDATE, and MERGE >>> operations. Without this API, each connector would need to reimplement the >>> same logic, creating duplication. >>> >>> The proposal is aligned with the SQL standard and other relational >>> databases. In my view, it simply makes Spark a better engine, facilitates >>> data accuracy and consistency, and enables performance optimizations. >>> >>> - Anton >>> >>> пт, 21 бер. 2025 р. о 12:59 Ángel Álvarez Pascua < >>> angel.alvarez.pas...@gmail.com> пише: >>> >>>> -1 (non-binding): Breaks the Chain of Responsibility. Constraints >>>> should be defined and enforced by the data sources themselves, not Spark. >>>> Spark is a processing engine, and enforcing constraints at this level blurs >>>> architectural boundaries, making Spark responsible for something it does >>>> not control. >>>> >>>> El vie, 21 mar 2025 a las 20:18, L. C. Hsieh (<vii...@gmail.com>) >>>> escribió: >>>> >>>>> +1 >>>>> >>>>> On Fri, Mar 21, 2025 at 12:13 PM huaxin gao <huaxin.ga...@gmail.com> >>>>> wrote: >>>>> > >>>>> > +1 >>>>> > >>>>> > On Fri, Mar 21, 2025 at 12:08 PM Denny Lee <denny.g....@gmail.com> >>>>> wrote: >>>>> >> >>>>> >> +1 (non-binding) >>>>> >> >>>>> >> On Fri, Mar 21, 2025 at 11:52 Gengliang Wang <ltn...@gmail.com> >>>>> wrote: >>>>> >>> >>>>> >>> +1 >>>>> >>> >>>>> >>> On Fri, Mar 21, 2025 at 11:46 AM Anton Okolnychyi < >>>>> aokolnyc...@gmail.com> wrote: >>>>> >>>> >>>>> >>>> Hi all, >>>>> >>>> >>>>> >>>> I would like to start a vote on adding support for constraints to >>>>> DSv2. >>>>> >>>> >>>>> >>>> Discussion thread: >>>>> https://lists.apache.org/thread/njqjcryq0lot9rkbf10mtvf7d1t602bj >>>>> >>>> SPIP: >>>>> https://docs.google.com/document/d/1EHjB4W1LjiXxsK_G7067j9pPX0y15LUF1Z5DlUPoPIo >>>>> >>>> PR with the API changes: >>>>> https://github.com/apache/spark/pull/50253 >>>>> >>>> JIRA: https://issues.apache.org/jira/browse/SPARK-51207 >>>>> >>>> >>>>> >>>> Please vote on the SPIP for the next 72 hours: >>>>> >>>> >>>>> >>>> [ ] +1: Accept the proposal as an official SPIP >>>>> >>>> [ ] +0 >>>>> >>>> [ ] -1: I don’t think this is a good idea because … >>>>> >>>> >>>>> >>>> - Anton >>>>> >>>>> --------------------------------------------------------------------- >>>>> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org >>>>> >>>>>
