450mill * 110 * 1.5 = 74250000000 = 69GB so maybe SORTWK01 - SORTWK10
Billy,
Based on your above description, you would need about 1.345 million tracks of
sortwk space if the sorting is to be done entirely using Disk space.
//SORTWK01 DD UNIT=WORK,DSNTYPE=LARGE,SPACE=(CYL,(4000,1000))
It is highly unlikely that you have 10 SINGLE volumes that have 4000 cylinders
of contiguous space. So I would reduce the primary to reasonable amount
depending on your shop disk space availability.
//SORTWK01 DD UNIT=(WORK,5)...
and this will not work, is that right, Sri Kolusu? I would only get the first
volume anyway?
Well, you will NOT a JCL error or any DFSORT message as it would ONLY consider
the FIRST volume. SORTWK space cannot span multiple volumes. Only the First
volume is considered.
Ideally, I would code up Dynamic Allocation and let DFSORT figure out how much
disk space it needs.
//SYSIN DD *
OPTION DYNALLOC=(,16),MOSIZE=MAX
SORT FIELDS=(P,M,F,X)
/*
This would allocate 16 + 2(reserve based on DYNAPCT=10) = 18 sortwk datasets
along with setting the MOSIZE to max so that you get the max of 64 bit memory
objects.
If you know the esoteric name for disk pool space then you can specify on
dynalloc parm. For example if you diskspace has an esoteric SYSWRK
//SYSIN DD *
OPTION DYNALLOC=(SYSWRK,16),MOSIZE=MAX
SORT FIELDS=(P,M,F,X)
/*
If I have a COBOL program doing a SORT in the middle of the program (I am not
sure if the big file is doing this, too or not), is using this DFSORT the best
way to force the use of 15 Dynalloc SORTWK files, so I would not have to code
SORTWK DD statements?
Well if a COBOL program is passing the records then DFSORT has no ability of
Calculating the filesize and hence you would receive ICE118I message.
Generally, DFSORT can automatically determine the input file size. However, in
a few cases, such as when an E15 supplies all of the input records,
or when information about a tape data set is not available from a tape
management system, DFSORT cannot determine an accurate file size.
I believe you are using Cobol SORT key word to sort the data. In such cases you
have to code DFSPARM along with dynamic allocation and the estimated file size,
so that DFSORT can determine the size of the file that is being sorted. So
something like this. I assumed that the size of the file to be sorted has 500
million (450 + 10% for future growth).
//DFSPARM DD *
OPTION DYNALLOC=(,16),MOSIZE=MAX, FILSZ=E500000000
/*
Also make sure that your JCL has //SORTDIAG DD DUMMY Which will produce more
diagnostic messages
Thanks,
Kolusu
DFSORT Development
IBM Corporation
-----Original Message-----
From: IBM Mainframe Discussion List <IBM-MAIN@LISTSERV.UA.EDU> On Behalf Of
Billy Ashton
Sent: Monday, July 3, 2023 11:14 AM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: [EXTERNAL] Re: SORTWK space usage
Thanks everyone - these are good recommendations.
However, the guy who came to me first has a job that sorts 450 million records
of 110 bytes, and I can't see how I could run SORT without specifying SORTWK DD
statements. Are there special configuration options I can verify that we have
in place to help my comfort level that I could eliminate SORTWK DD?
If the programming team are not convinced, does this look right, based on the
discussion - and based on a 100% Disk sort, which is not likely?
450mill * 110 * 1.5 = 74250000000 = 69GB so maybe SORTWK01 - SORTWK10
//SORTWK01 DD UNIT=WORK,DSNTYPE=LARGE,SPACE=(CYL,(4000,1000))
to
//SORTWK10 DD UNIT=WORK,DSNTYPE=LARGE,SPACE=(CYL,(4000,1000))
and this will give me 4000 + (15*1000) on each, or 40000c primary + (up to 150
* 1000) secondary
While we have been talking here, I have looked, and found some jobs with
//SORTWK01 DD UNIT=(WORK,5)...
and this will not work, is that right, Sri Kolusu? I would only get the first
volume anyway?
Thank you and best regards,
Billy Ashton
------ Original Message ------
From "Farley, Peter"
<0000031df298a9da-dmarc-requ...@listserv.ua.edu<mailto:0000031df298a9da-dmarc-requ...@listserv.ua.edu>>
To IBM-MAIN@listserv.ua.edu<mailto:IBM-MAIN@listserv.ua.edu>
Date 7/3/2023 12:44:43 PM
Subject Re: SORTWK space usage
I will add on to Sri's excellent answer with my very STRONG recommendation NOT
to use hard-coded SORTWK's in your JCL. Both of the major SORT vendors (IBM
and Syncsort) do a far, far better job of estimating necessary SORTWK space and
memory utilization than any human could hope to do.
I also STRONGLY recommend that you give your JCL and program-controlled SORT
steps as much memory in the REGION parameter as your installation allows for
production and test jobs. Both of the major SORT programs will figure out how
much of that memory to use, whether and how much of it to use in the current
WLM environment, and will make VERY effective use of as much of it as they can
to cut down on SORT execution and elapsed time. Memory is (relatively) cheap,
don't be afraid to use all you've got available to shorten your SORT times.
My basic advice is don't try to second-guess these very intelligent programs -
they each have more than half a century of practice and experience that none of
us can match, even those of us who have been around that long.
Peter
-----Original Message-----
From: IBM Mainframe Discussion List
<IBM-MAIN@LISTSERV.UA.EDU<mailto:IBM-MAIN@LISTSERV.UA.EDU>> On
Behalf Of Sri h Kolusu
Sent: Monday, July 3, 2023 12:28 PM
To: IBM-MAIN@LISTSERV.UA.EDU<mailto:IBM-MAIN@LISTSERV.UA.EDU>
Subject: Re: SORTWK space usage
I will only get the primary space of 5000 cylinders, and the other 14x2000
cylinders is never used. Is that right?
Billy,
No. Incorrect. DFSORT will make use of BOTH primary and secondary space
allocations ( 1 primary + 15 Secondary) for a total of 16 extents. So if you
allocated 1 sortwk dataset with (CYL,(5000,2000)), then DFSORT would use 5000 +
15* 2000 = 5000 + 30000 = 35,000 cylinders. Also note that the secondary
extents will only come into picture ONLY when needed.
Is there a written explanation I can forward to the programmers so they
understand this?
Also (since I know it will come), is there any good way to calculate how much
DASD sortwork would be used? I know this depends on what is in memory at the
time, but want to get a better handle on how Sort determines what it needs.
I would suggest that you use DFSORT's Dynamic Allocation as it will allocate
the required workspace optimally rather than programmers calculating it. The
reason is you don't want to change the allocation every time there is an
increase/decrease in the number of records to be sorted.
Having said that, here is a general formula that you want to use.
The amount of sortwk space required depends on the size of the file. It usually
ranges from 1.3X to 1.8X of the size of the file to be sorted depending on the
sorting path that DFSORT chose.
Filesize = Number of records * Avg length of the record ( for Fixed
length RECFM=F or FB , it is the LRECL value, or RECFM=V or RECFM=VB is
the average length of the record)
However, that range is applicable ONLY when the entire file is being sorted
using Disk workspace. DFSORT has the capability of using memory (real and
auxiliary storage) and if it runs out of it, it will then use disk workspace.
Thanks,
Kolusu
--
This message and any attachments are intended only for the use of the addressee
and may contain information that is privileged and confidential. If the reader
of the message is not the intended recipient or an authorized representative of
the intended recipient, you are hereby notified that any dissemination of this
communication is strictly prohibited. If you have received this communication
in error, please notify us immediately by e-mail and delete the message and any
attachments from your system.
----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions, send
email to lists...@listserv.ua.edu<mailto:lists...@listserv.ua.edu> with the
message: INFO IBM-MAIN
----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions, send email to
lists...@listserv.ua.edu<mailto:lists...@listserv.ua.edu> with the message:
INFO IBM-MAIN
----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
