David, I will have to complain that your python benchmark is not a fair
comparison. Your python script uses a module named pyzfile to access z/OS
files (which I see from PYPI is authored by you but for which you have
published no source yet). A fairer comparison would be a python script that
only used standard python modules and shell commands. My python-vs-Rexx
testing was done using python's subprocess.run to execute the "cat" command to
copy data from z/OS files (PDS, QSAM, VSAM) to STDOUT captured by the
subprocess.run routine and then using the captured STDOUT data for the
processing. Writing to z/OS files (PDS member and QSAM only) was accomplished
by first writing the output file data to the Unix file system (with encoding
1047 to write in EBCDIC) and then again using subprocess.run to execute "cp" to
copy the written Unix file to the z/OS file. In both read and write cases I
used the "//'DSN'" file name format for the z/OS files, supported by both "cat"
and "cp".
Rexx using EXECIO or RXVSAM from CBT beats that type of python script by a
small margin but not by a lot -- the process I was measuring averaged 23-24
"real" seconds per test for the python version and 19-20 "real" seconds per
test for the Rexx version.
This was all done on the IBM Zxplore z/OS platform, which is x86 under the
covers rather than "real iron", so probably zPDT. In any case, "students" on
Zxplore aren't permitted to install any python packages and venv/virtualenv are
not available for the same reason (DASD-and-CPU-constrained system). The
platform only permits you to use standard python packages or one of the few
non-standard ones pre-installed by the admins there.
Peter
-----Original Message-----
From: IBM Mainframe Discussion List <IBM-MAIN@LISTSERV.UA.EDU> On Behalf Of
David Crayford
Sent: Tuesday, February 28, 2023 1:53 AM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: zOSMF and zOWE for non-mainframers
On 28/2/23 13:47, David Crayford wrote:
>
> On 25/2/23 01:23, Farley, Peter wrote:
>> Python on the mainframe is pretty good, but still can't beat out Rexx
>> in performance even when the Rex script needs to use BPXWUNIX and
>> friends to access z/OS Unix file systems,
>>
>
> I have conducted a series of benchtests, and the results suggest that
> REXX is not as fast as Python. In my testing, I compare the
> performance of C, Lua, Python, and REXX, and the results are clear: C
> is the fastest, followed by Lua, which is within an order of magnitude
> of C. Python comes next, within an order of magnitude of Lua, and REXX
> consistently performs the poorest. In addition to the performance
> factor, the vast Python ecosystem compared to the limited options
> available for REXX also make it an easy decision. Python is also
> simpler to extend with packages, while REXX requires more effort and
> potentially complex steps, such as using modern libraries that require
> Language Environment (LE).
My benchtests
Lua
local file = assert(io.open(arg[1], "rb, type=record, noseek")) while true do
local rec = file:read()
if not rec then break end
end
Python
import sys
from pyzfile import *
try:
with ZFile(sys.argv[1], "rb,type=record,noseek") as file:
for rec in file:
pass
except ZFileError as e:
print(e)
REXX
/* REXX */
arg dsname
address MVS
call bpxwdyn "ALLOC FI(INPUT) DA("dsname") SHR"
do until eof
"EXECIO 10000 DISKR INPUT ( STEM rec."
eof = (rc > 0)
end
"EXECIO 0 DISKR INPUT ( FINIS"
The results: Add user+system to get total CPU time
> time lua benchio.lua "//'CPA000.QADATA.PMR99999.SSA.HR1315PM'" && time
> python3 benchio.py "//'CPA000.QADATA.PMR99999.SSA.HR1315PM'" && time
> ./benchio.rex "CPA000.QADATA.PMR99999.SSA.HR1315PM"
real 0m47.019s
user 0m3.255s
sys 0m1.097s
real 1m0.710s
user 0m8.001s
sys 0m2.678s
real 1m17.772s
user 0m13.575s
sys 0m4.536s
--
This message and any attachments are intended only for the use of the addressee
and may contain information that is privileged and confidential. If the reader
of the message is not the intended recipient or an authorized representative of
the intended recipient, you are hereby notified that any dissemination of this
communication is strictly prohibited. If you have received this communication
in error, please notify us immediately by e-mail and delete the message and any
attachments from your system.
----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
