Hi Steffen
> The short answer is that the compact notation turned out to work much better > for me in my code, especially, if multiple transducers are involved. But > that's my personal taste. You can choose which suits you better. In fact, > > 1000 take. > > just sits on top and simply calls > > Take number: 1000. To me this is much much better. > If the need arises, we could of course factor the compact notation out into > a separate package. Good idea Btw, would you prefer (Take n: 1000) over (Take number: > 1000)? I tend to prefer explicit selector :) > Damien, you're right, I experimented with additional styles. Right now, we > already have in the basic Transducer package: > > (collection transduce: #squared map * 1000 take. "which is equal to" > (collection transduce: #squared map) transduce: 1000 take. > > Basically, one can split #transduce:reduce:init: into single calls of > #transduce:, #reduce:, and #init:, depending on the needs. > I also have an (unfinished) extension, that allows to write: > > (collection transduce map: #squared) take: 1000. To me this is much mre readable. I cannot and do not want to use the other forms. > This feels familiar, but becomes a bit hard to read if more than two steps > are needed. > > collection transduce > map: #squared; > take: 1000. Why this is would hard to read. We do that all the time everywhere. > I think, this alternative would reads nicely. But as the message chain has > to modify the underlying object (an eduction), very snaky side effects may > occur. E.g., consider > > eduction := collection transduce. > squared := eduction map: #squared. > take := squared take: 1000. > > Now, all three variables hold onto the same object, which first squares all > elements and than takes the first 1000. This is because the programmer did not understand what he did. No? Stef PS: I played with infinite stream and iteration back in 1993 in CLOS. Now I do not like to mix things because it breaks my flow of thinking. > > Best, > Steffen > > > > > > Am .06.2017, 21:28 Uhr, schrieb Damien Pollet > <damien.pollet+ph...@gmail.com>: > >> If I recall correctly, there is an alternate protocol that looks more like >> xtreams or the traditional select/collect iterations. >> >> On 2 June 2017 at 21:12, Stephane Ducasse <stepharo.s...@gmail.com> wrote: >> >>> I have a design question >>> >>> why the library is implemented in functional style vs messages? >>> I do not see why this is needed. To my eyes the compact notation >>> goes against readibility of code and it feels ad-hoc in Smalltalk. >>> >>> >>> I really prefer >>> >>> square := Map function: #squared. >>> take := Take number: 1000. >>> >>> Because I know that I can read it and understand it. >>> From that perspective I prefer Xtreams. >>> >>> Stef >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> On Wed, May 31, 2017 at 2:23 PM, Steffen Märcker <merk...@web.de> wrote: >>> >>>> Hi, >>>> >>>> I am the developer of the library 'Transducers' for VisualWorks. It was >>>> formerly known as 'Reducers', but this name was a poor choice. I'd like >>>> to >>>> port it to Pharo, if there is any interest on your side. I hope to learn >>>> more about Pharo in this process, since I am mainly a VW guy. And most >>>> likely, I will come up with a bunch of questions. :-) >>>> >>>> Meanwhile, I'll cross-post the introduction from VWnc below. I'd be very >>>> happy to hear your optinions, questions and I hope we can start a >>>> fruitful >>>> discussion - even if there is not Pharo port yet. >>>> >>>> Best, Steffen >>>> >>>> >>>> >>>> Transducers are building blocks that encapsulate how to process elements >>>> of a data sequence independently of the underlying input and output >>>> source. >>>> >>>> >>>> >>>> # Overview >>>> >>>> ## Encapsulate >>>> Implementations of enumeration methods, such as #collect:, have the >>>> logic >>>> how to process a single element in common. >>>> However, that logic is reimplemented each and every time. Transducers >>>> make >>>> it explicit and facilitate re-use and coherent behavior. >>>> For example: >>>> - #collect: requires mapping: (aBlock1 map) >>>> - #select: requires filtering: (aBlock2 filter) >>>> >>>> >>>> ## Compose >>>> In practice, algorithms often require multiple processing steps, e.g., >>>> mapping only a filtered set of elements. >>>> Transducers are inherently composable, and hereby, allow to make the >>>> combination of steps explicit. >>>> Since transducers do not build intermediate collections, their >>>> composition >>>> is memory-efficient. >>>> For example: >>>> - (aBlock1 filter) * (aBlock2 map) "(1.) filter and (2.) map elements" >>>> >>>> >>>> ## Re-Use >>>> Transducers are decoupled from the input and output sources, and hence, >>>> they can be reused in different contexts. >>>> For example: >>>> - enumeration of collections >>>> - processing of streams >>>> - communicating via channels >>>> >>>> >>>> >>>> # Usage by Example >>>> >>>> We build a coin flipping experiment and count the occurrence of heads >>>> and >>>> tails. >>>> >>>> First, we associate random numbers with the sides of a coin. >>>> >>>> scale := [:x | (x * 2 + 1) floor] map. >>>> sides := #(heads tails) replace. >>>> >>>> Scale is a transducer that maps numbers x between 0 and 1 to 1 and 2. >>>> Sides is a transducer that replaces the numbers with heads an tails by >>>> lookup in an array. >>>> Next, we choose a number of samples. >>>> >>>> count := 1000 take. >>>> >>>> Count is a transducer that takes 1000 elements from a source. >>>> We keep track of the occurrences of heads an tails using a bag. >>>> >>>> collect := [:bag :c | bag add: c; yourself]. >>>> >>>> Collect is binary block (reducing function) that collects events in a >>>> bag. >>>> We assemble the experiment by transforming the block using the >>>> transducers. >>>> >>>> experiment := (scale * sides * count) transform: collect. >>>> >>>> From left to right we see the steps involved: scale, sides, count and >>>> collect. >>>> Transforming assembles these steps into a binary block (reducing >>>> function) >>>> we can use to run the experiment. >>>> >>>> samples := Random new >>>> reduce: experiment >>>> init: Bag new. >>>> >>>> Here, we use #reduce:init:, which is mostly similar to #inject:into:. >>>> To execute a transformation and a reduction together, we can use >>>> #transduce:reduce:init:. >>>> >>>> samples := Random new >>>> transduce: scale * sides * count >>>> reduce: collect >>>> init: Bag new. >>>> >>>> We can also express the experiment as data-flow using #<~. >>>> This enables us to build objects that can be re-used in other >>>> experiments. >>>> >>>> coin := sides <~ scale <~ Random new. >>>> flip := Bag <~ count. >>>> >>>> Coin is an eduction, i.e., it binds transducers to a source and >>>> understands #reduce:init: among others. >>>> Flip is a transformed reduction, i.e., it binds transducers to a >>>> reducing >>>> function and an initial value. >>>> By sending #<~, we draw further samples from flipping the coin. >>>> >>>> samples := flip <~ coin. >>>> >>>> This yields a new Bag with another 1000 samples. >>>> >>>> >>>> >>>> # Basic Concepts >>>> >>>> ## Reducing Functions >>>> >>>> A reducing function represents a single step in processing a data >>>> sequence. >>>> It takes an accumulated result and a value, and returns a new >>>> accumulated >>>> result. >>>> For example: >>>> >>>> collect := [:col :e | col add: e; yourself]. >>>> sum := #+. >>>> >>>> A reducing function can also be ternary, i.e., it takes an accumulated >>>> result, a key and a value. >>>> For example: >>>> >>>> collect := [:dic :k :v | dict at: k put: v; yourself]. >>>> >>>> Reducing functions may be equipped with an optional completing action. >>>> After finishing processing, it is invoked exactly once, e.g., to free >>>> resources. >>>> >>>> stream := [:str :e | str nextPut: each; yourself] completing: >>>> #close. >>>> absSum := #+ completing: #abs >>>> >>>> A reducing function can end processing early by signaling Reduced with a >>>> result. >>>> This mechanism also enables the treatment of infinite sources. >>>> >>>> nonNil := [:res :e | e ifNil: [Reduced signalWith: res] ifFalse: >>>> [res]]. >>>> >>>> The primary approach to process a data sequence is the reducing protocol >>>> with the messages #reduce:init: and #transduce:reduce:init: if >>>> transducers >>>> are involved. >>>> The behavior is similar to #inject:into: but in addition it takes care >>>> of: >>>> - handling binary and ternary reducing functions, >>>> - invoking the completing action after finishing, and >>>> - stopping the reduction if Reduced is signaled. >>>> The message #transduce:reduce:init: just combines the transformation and >>>> the reducing step. >>>> >>>> However, as reducing functions are step-wise in nature, an application >>>> may >>>> choose other means to process its data. >>>> >>>> >>>> ## Reducibles >>>> >>>> A data source is called reducible if it implements the reducing >>>> protocol. >>>> Default implementations are provided for collections and streams. >>>> Additionally, blocks without an argument are reducible, too. >>>> This allows to adapt to custom data sources without additional effort. >>>> For example: >>>> >>>> "XStreams adaptor" >>>> xstream := filename reading. >>>> reducible := [[xstream get] on: Incomplete do: [Reduced signal]]. >>>> >>>> "natural numbers" >>>> n := 0. >>>> reducible := [n := n+1]. >>>> >>>> >>>> ## Transducers >>>> >>>> A transducer is an object that transforms a reducing function into >>>> another. >>>> Transducers encapsulate common steps in processing data sequences, such >>>> as >>>> map, filter, concatenate, and flatten. >>>> A transducer transforms a reducing function into another via #transform: >>>> in order to add those steps. >>>> They can be composed using #* which yields a new transducer that does >>>> both >>>> transformations. >>>> Most transducers require an argument, typically blocks, symbols or >>>> numbers: >>>> >>>> square := Map function: #squared. >>>> take := Take number: 1000. >>>> >>>> To facilitate compact notation, the argument types implement >>>> corresponding >>>> methods: >>>> >>>> squareAndTake := #squared map * 1000 take. >>>> >>>> Transducers requiring no argument are singletons and can be accessed by >>>> their class name. >>>> >>>> flattenAndDedupe := Flatten * Dedupe. >>>> >>>> >>>> >>>> # Advanced Concepts >>>> >>>> ## Data flows >>>> >>>> Processing a sequence of data can often be regarded as a data flow. >>>> The operator #<~ allows define a flow from a data source through >>>> processing steps to a drain. >>>> For example: >>>> >>>> squares := Set <~ 1000 take <~ #squared map <~ (1 to: 1000). >>>> fileOut writeStream <~ #isSeparator filter <~ fileIn readStream. >>>> >>>> In both examples #<~ is only used to set up the data flow using reducing >>>> functions and transducers. >>>> In contrast to streams, transducers are completely independent from >>>> input >>>> and output sources. >>>> Hence, we have a clear separation of reading data, writing data and >>>> processing elements. >>>> - Sources know how to iterate over data with a reducing function, e.g., >>>> via #reduce:init:. >>>> - Drains know how to collect data using a reducing function. >>>> - Transducers know how to process single elements. >>>> >>>> >>>> ## Reductions >>>> >>>> A reduction binds an initial value or a block yielding an initial value >>>> to >>>> a reducing function. >>>> The idea is to define a ready-to-use process that can be applied in >>>> different contexts. >>>> Reducibles handle reductions via #reduce: and #transduce:reduce: >>>> For example: >>>> >>>> sum := #+ init: 0. >>>> sum1 := #(1 1 1) reduce: sum. >>>> sum2 := (1 to: 1000) transduce: #odd filter reduce: sum. >>>> >>>> asSet := [:set :e | set add: e; yourself] initializer: [Set new]. >>>> set1 := #(1 1 1) reduce: asSet. >>>> set2 := #(1 to: 1000) transduce: #odd filter reduce: asSet. >>>> >>>> By combining a transducer with a reduction, a process can be further >>>> modified. >>>> >>>> sumOdds := sum <~ #odd filter >>>> setOdds := asSet <~ #odd filter >>>> >>>> >>>> ## Eductions >>>> >>>> An eduction combines a reducible data sources with a transducer. >>>> The idea is to define a transformed (virtual) data source that needs not >>>> to be stored in memory. >>>> >>>> odds1 := #odd filter <~ #(1 2 3) readStream. >>>> odds2 := #odd filter <~ (1 to 1000). >>>> >>>> Depending on the underlying source, eductions can be processed once >>>> (streams, e.g., odds1) or multiple times (collections, e.g., odds2). >>>> Since no intermediate data is stored, transducers actions are lazy, >>>> i.e., >>>> they are invoked each time the eduction is processed. >>>> >>>> >>>> >>>> # Origins >>>> >>>> Transducers is based on the same-named Clojure library and its ideas. >>>> Please see: >>>> http://clojure.org/transducers >>>> >>>> >
