One more note: When you specify the stages in the Pipeline, they need to be in topological order according to the DAG.
On Sun, Jun 12, 2016 at 10:47 AM, Joseph Bradley <jos...@databricks.com> wrote: > Hi Pranay, > > Yes, you can do this. The DAG structure should be specified via the > various Transformers' input and output columns, where a Transformer can > have multiple input and/or output columns. Most of the classification and > regression Models are good examples of Transformers with multiple input and > output columns. > > Hope this helps! > Joseph > > On Wed, Jun 8, 2016 at 9:59 PM, Pranay Tonpay <pton...@gmail.com> wrote: > >> Hi, >> Pipeline as of now seems to be having a series of transformers and >> estimators in a serial fashion. >> Is it possible to create a DAG sort of thing - >> Eg - >> Two transformers running in parallel to cleanse data (a custom built >> Transformer) in some way and then their outputs ( two outputs ) used for >> some sort of correlation ( another custom built Transformer ) >> >> Let me know - >> >> thx >> pranay >> > >
