Hi Andrew. My personal goal here is exclusively to support aggregate functions, which in SQLite terms actually have `Step(V)` and `Final() R` callbacks (rather than `Step(V)` and `Value() R`), because those semantics more closely match iterators and `iter.Seq`. I have a different interface for window aggregates (that also has `Inverse(V)`). Reusing `Value(R)` for aggregates and moving cleanup to `io.Closer` is my attempt to make the interface more Go-like.
Other databases take an even less "iterative" approach to window functions, instead requiring a `Merge(A, A) A` operation (rather than `Inverse(V)`) which merges two intermediate results, and then they find a tree of the hopefully minimal amount of aggregations and merges to cover a window, reusing aggregations as they go. This is more suitable to be implemented by certain data structures, and also more amenable to computing subtrees in parallel, which is why it's preferred, I guess. But this is all even more removed from iteration, so I didn't consider it, even though I'm searching for something that's more generally useful than my own narrow needs. Regards, Nuno On Tuesday 25 February 2025 at 21:48:00 UTC Andrew Harris wrote: > I just want to highlight a theoretical cliff around `iter.Push` or `func > processor(seq iter.Seq[V]) R` contrasted with solutions involving step and > value callbacks (built-in <https://www.sqlite.org/lang_aggfunc.html> or > custom <https://www.sqlite.org/c3ref/create_function.html> sqlite > aggregate functions, also of interest: windowed variants). Aggregate > functions expressed like the latter, or some of the pretty amazing > aggregate algorithms, really tend towards stepwise, inductive proof over > potentially infinite inputs. The stepwise operations have nice algebraic > properties, (e.g. they are left- and right-associative, or at least the > difference isn't considered essential, or sequence order is assumed a > priori, etc. - sometimes the technique is expressing the problem with > clever operations). Invariants about the result hold at each step (and for > windowed aggregate functions, at inverse steps). > > With this kind of setup, `func processor(R, V) R` or the step / value pair > `func Step(V)`, `func Value() R` make a ton of sense. However, without this > kind of setup, the same callbacks can still be used ... I think what really > changes is that `func Value() R` shouldn't be called until the sequence has > terminated, and if it's really necessary there's correspondingly less > clarity about the time and space complexity needed to support `func Value() > R`. > > The concurrency of `iter.Push` is definitely interesting, and `func > processor(seq iter.Seq[V]) R` is nice, but the combination of the two seems > less nice (still interesting) relative to other approaches. > > On Tuesday, February 25, 2025 at 1:12:06 PM UTC-8 Nuno Cruces wrote: > >> Ian, just to note that your message is not at all lost. I just need to >> look at it carefully in front of a computer, which I won't be able to do >> today. >> >> But if it works, I agree that looks like a nicer, more general API than >> mine proposal, which is exactly what I was hoping for, so many thanks! >> >> Regards, >> Nuno >> >> On Tue 25 Feb 2025, 20:24 Ian Lance Taylor, <ia...@golang.org> wrote: >> >>> On Tue, Feb 25, 2025 at 11:52 AM 'Brian Candler' via golang-nuts >>> <golan...@googlegroups.com> wrote: >>> > >>> > You can curry a function? >>> > >>> > On Tuesday, 25 February 2025 at 18:52:09 UTC Jason E. Aten wrote: >>> >> >>> >> Hi Ian, I'm not quite understanding -- is Push meant to take an input >>> seq too? like >>> >> >>> >> func Push[E any](inputSeq iter.Seq[E]) (seq iter.Seq[E], yield >>> func(E), stop func()) >>> >> ^^^^^^^^ >>> >> ? >>> >>> I'm suggesting that Push returns, conceptually, two things: an >>> iter.Seq, and a pair of functions. You hand the iter.Seq off to >>> something that expects an iter.Seq. You use the two functions to push >>> values into the iter.Seq. Just as iter.Pull gives you flexibility to >>> fetch values from the sequence however you like, iter.Push gives you >>> flexibility to push values into the sequence however you like, without >>> being tied to the lifespan of a single function. In the original >>> example, the values to push into the sequence would come from a method >>> call. >>> >>> This can all be done already with channels, of course, as shown by the >>> earlier examples. >>> >>> Ian >>> >>> >>> >> On Tuesday, February 25, 2025 at 6:15:40 PM UTC Ian Lance Taylor >>> wrote: >>> >>> >>> >>> On Tue, Feb 25, 2025 at 6:17 AM Nuno Cruces <ncr...@gmail.com> >>> wrote: >>> >>> > >>> >>> > I wanted a solution that doesn't necessarily involve goroutines >>> and channels, for the same reason that iter.Pull was created: because >>> goroutines and channels add unnecessary parallelism that has the potential >>> to introduce data races, when what is needed is concurrency without >>> parallelism. >>> >>> >>> >>> I think what you're presenting is an argument for >>> >>> >>> >>> package iter >>> >>> >>> >>> // Push returns an iterator, a yield function, and a stop function. >>> >>> // The iterator will return all the values passed to the yield >>> function. >>> >>> // The iterator will stop when the stop function is called. >>> >>> // This provides a way to flexibly convert a sequence of values into >>> a Seq. >>> >>> func Push[E any]() (seq iter.Seq[E], yield func(E), stop func()) >>> >>> >>> >>> Ian >>> > >>> > -- >>> > You received this message because you are subscribed to the Google >>> Groups "golang-nuts" group. >>> > To unsubscribe from this group and stop receiving emails from it, send >>> an email to golang-nuts...@googlegroups.com. >>> > To view this discussion visit >>> https://groups.google.com/d/msgid/golang-nuts/b46f0864-9dfd-4c1c-b34a-dff83decae7fn%40googlegroups.com >>> . >>> >>> -- >>> >> You received this message because you are subscribed to a topic in the >>> Google Groups "golang-nuts" group. >>> To unsubscribe from this topic, visit >>> https://groups.google.com/d/topic/golang-nuts/-Kmtx-sr3G8/unsubscribe. >>> To unsubscribe from this group and all its topics, send an email to >>> golang-nuts...@googlegroups.com. >>> To view this discussion visit >>> https://groups.google.com/d/msgid/golang-nuts/CAOyqgcWmTubCfieowziq9BgyqiJddp6sv2g%3Dm4v44FF5P-5aHg%40mail.gmail.com >>> . >>> >> -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion visit https://groups.google.com/d/msgid/golang-nuts/65d276a3-36ba-4ad2-98be-ea294d22deb9n%40googlegroups.com.
