Hi Kenneth, thanks for the clarification. I was not aware of bundles. Now it makes sense.
Vaclav ________________________________ From: Kenneth Knowles <[email protected]> Sent: Thursday, May 10, 2018 4:57:34 PM To: dev Subject: Re: triggers in direct runner Hi Vaclav, Slightly stale but still maybe a good reference is https://s.apache.org/beam-triggers. Triggers are per key and window, and they give the runner permission to fire but do not require it. The runner can thus amortize the cost of output. The bundle is the unit of commit in Beam so firing sub-bundle is wasted. Also relevant is https://s.apache.org/beam-sink-triggers. It emphasizes the other point: triggering is about completeness, latency, and cost. It is (mostly) a bug that triggering affects the observable elements. Kenn On Thu, May 10, 2018, 06:38 Plajt, Vaclav <[email protected]<mailto:[email protected]>> wrote: Hello Beam Devs, I'm working on DSL-Euphoria. And I found that when GroupByKey transform is executed on direct runner, window triggers are evaluated element-wise (ReduceFnRunner#processElement) but not actually fired element-wise. They are fired (pane is emitted) when whole batch of elements is processed (ReduceFnRunner#processElements). I also suspects that elements are already grouped by key when they reach ReduceFnRunner#processElements so triggers are fired only once per whole group. I'm confused by this behavior since Beam Programming guide has examples with element-wise triggerning (see paragraph 8.4.1.1.). Is this a wanted behavior? And does every Beam Runner do it the same (portability)? Or do I miss something? Any kind of help or guidance is appreciated. Thank you. Vaclav Plajt Je dobré vědět, že tento e-mail a přílohy jsou důvěrné. Pokud spolu jednáme o uzavření obchodu, vyhrazujeme si právo naše jednání kdykoli ukončit. Pro fanoušky právní mluvy - vylučujeme tím ustanovení občanského zákoníku o předsmluvní odpovědnosti. Pravidla o tom, kdo u nás a jak vystupuje za společnost a kdo může co a jak podepsat naleznete zde<https://onas.seznam.cz/cz/podpisovy-rad-cz.html> You should know that this e-mail and its attachments are confidential. If we are negotiating on the conclusion of a transaction, we reserve the right to terminate the negotiations at any time. For fans of legalese—we hereby exclude the provisions of the Civil Code on pre-contractual liability. The rules about who and how may act for the company and what are the signing procedures can be found here<https://onas.seznam.cz/cz/signature-rules.html>.
