Re: Renaming SideOutput

2017-04-13 Thread Thomas Groh
All outputs are logically the same "type" of output. Within the SDKs, the "main" output is just the one that is output if no output tag is specified, and the one that matches the output type parameter of the DoFn. However, because there are multiple methods, including something that is a reasonable

Join to external table

2017-04-13 Thread JingsongLee
Hi all, I've seen repeatedly the following pattern: Consider a sql (Joining stream to table, from Calcite): SELECT STREAM o.rowtime, o.productId, o.orderId, o.units,   p.name, p.unitPrice FROM Orders AS o JOIN Products AS p   ON o.productId = p.productId; A stream-to-table join is straightforward

Re: splitIntoBundles vs. generateInitialSplits

2017-04-13 Thread Etienne Chauchot
Hi all, Just to give feedback on the ML, we got a consensus in the PR about using "split()" name. So I'll update the PR Best Etienne Le 13/04/2017 à 13:35, Jean-Baptiste Onofré a écrit : Hi, Thanks for that. I gonna review and it's not a big deal with the IOs PR (easy to rebase and updat

Re: splitIntoBundles vs. generateInitialSplits

2017-04-13 Thread Jean-Baptiste Onofré
Hi, Thanks for that. I gonna review and it's not a big deal with the IOs PR (easy to rebase and update). I would have preferred simply split() but ok with splitIntoSubSources(). Regards JB On 04/13/2017 01:31 PM, Etienne Chauchot wrote: Hi all, It seems that we have an agreement on the nam

Re: splitIntoBundles vs. generateInitialSplits

2017-04-13 Thread Etienne Chauchot
Hi all, It seems that we have an agreement on the name, and as the stable release date is coming soon, I did two simple PRs to rename both methods to splitIntoSubSources as suggested bellow. One is for the beam code base and the other is for the website. I did not change the python-sdk https

Jenkins build became unstable: beam_Release_NightlySnapshot #387

2017-04-13 Thread Apache Jenkins Server
See

Re: Question regarding loops in BEAM programs

2017-04-13 Thread Robert Bradshaw
There is no way (short of inspecting stack traces and bytecodes) for Beam to distinguish between for (int i=0; i<3; i++) { pc = pc.apply(MyTransform()); } from pc.apply(MyTransform()).apply(MyTransform()).apply(MyTransform()); However, PTransforms are hierarchal, so you coul