Paul said: > I'm guessing the atypical case is a stage such as FANOUT which necessarily copies the data.
Not sure what you mean by atypical. FANOUT is typical in the respect that it doesn't create an actual copy of the input record. It just looks like it. FANOUT, and non-record-changing stages, pass the same input record pointer to their downstream stage(s). This is what makes Pipes so efficient. No working set expansion and less reloading of just purged cache data. OREXXMan Would you rather pass data in move mode (*nix piping) or locate mode (Pipes) or via disk (JCL)? Why do you think you rarely see *nix commands with more than a dozen filters, while Pipelines specifications are commonly over 100s of stages, and 1000s of stages are not uncommon. IBM has been looking for an HLL for program products; REXX is that language. On Wed, Sep 22, 2021 at 3:13 AM Martin Packer <[email protected]> wrote: > Conversely a pipe as input is not necessarily a good input medium for a > sort. 10 years ago I contributed to a Batch Modernization Redbook on this, > emphasising the need for BatchPipes input to DFSORT to be accompanied by a > FILSZ / AVGRLEN estimate pair. > > Bringing it back to pipes, I wonder if it's feasible to tell a sorting > stage (whether DFSORT (yes please Sri Hari) or otherwise) the input size. > Otherwise we could have blow ups or bad performance at scale. > > BTW I'm all in favour of pipes as a first class citizen but note I have > little influence in this regard. > > Cheers, Martin > > Martin Packer > > WW z/OS Performance, Capacity and Architecture, IBM Technology Sales > > +44-7802-245-584 > > email: [email protected] > > Twitter / Facebook IDs: MartinPacker > > Blog: https://mainframeperformancetopics.com > > Mainframe, Performance, Topics Podcast Series (With Marna Walle): > https://anchor.fm/marna-walle > > Youtube channel: https://www.youtube.com/channel/UCu_65HaYgksbF6Q8SQ4oOvA > > > > From: "Paul Gilmartin" <[email protected]> > To: [email protected] > Date: 22/09/2021 03:50 > Subject: [EXTERNAL] Re: The Business Case for Pipes in the z/OS > Base (was: Re: REXX - Interpret or Value - Which is better?) > Sent by: "IBM Mainframe Discussion List" <[email protected]> > > > > On Tue, 21 Sep 2021 21:03:14 -0500, Mike Schwab wrote: > > >If a SORT (or other similar temporary data store) program is one of > >the pipe programs, when the EXEC PGM= program closes the output file > >then the program holding the data needs to output the stored data to > >output ddnames (pipe or output files). > > > Are you thinking of MS-DOS pseudo-"pipes" where the upstream program > wrote a temporary file under-the-covers and the downstream program > processed it? A pipe in syntax only. Even Windows is better nowadays. > > SORT is a bad conceptual example for Pipethink because SORT can't > write its first output record until it has read its last input record. > Better > to envision a filter which re-formats log records from a long-running (or > never-terminating) program, writing a file to be browsed with SDSF or > tail -f in real time. > > -- gil > > ---------------------------------------------------------------------- > For IBM-MAIN subscribe / signoff / archive access instructions, > send email to [email protected] with the message: INFO IBM-MAIN > > > > Unless stated otherwise above: > IBM United Kingdom Limited - Registered in England and Wales with number > 741598. > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU > > > ---------------------------------------------------------------------- > For IBM-MAIN subscribe / signoff / archive access instructions, > send email to [email protected] with the message: INFO IBM-MAIN > ---------------------------------------------------------------------- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [email protected] with the message: INFO IBM-MAIN
