Hi Mark, The proposal sounds good to me, +1 as it's a real gap in Spark. GROUPS is SQL:2011.
Also, this should be feasible, given that GROUPS is a relatively close cousin of RANGE. It will be a well scoped, standard defined, additive change. No broad arch changes. Your design doc looks good, please attach it to the Jira, for any further discussions. Regarding scope, Python can be deferred initially, and done in an immediate followup. With this in mind, I think a normal Jira and PR will be fine here. No need for SPIP. Thank you for the well scoped proposal, looking forward to the seeing the PR! Best, Uroš On 2026/08/10 20:36:22 Mark Andreev wrote: > Hi all, > > I’d like to propose adding SQL-standard `GROUPS` window frames to Spark SQL. > > Spark currently supports `ROWS` and `RANGE`, but not `GROUPS`, which > defines frame boundaries in terms of peer groups. This is useful where > group sizes and ordering-key gaps vary, and cannot in general be expressed > correctly with a fixed `ROWS` or `RANGE` frame. > > I’ve prepared a design proposal covering semantics, compatibility, > implementation feasibility, testing, and behaviour across other SQL engines: > > https://docs.google.com/document/d/1BOVWBHz6vQi39uQ_8SJfMWg55oJrUqNMQAgtHXmTZhA/edit?usp=sharing > > I intend to implement this and open a draft PR. > > Before starting, I’d particularly appreciate feedback on: > > * whether this should go through SPIP or a normal JIRA + PR; > * whether `GROUPS` support for Python/Pandas window UDAFs should be > included initially or deferred. > > Any feedback on the proposed semantics or implementation direction would be > very welcome. > > -- > Best regards, > Mark Andreev > --------------------------------------------------------------------- To unsubscribe e-mail: [email protected]
