Sadly true - perhaps it's time to go over to the https://zosopentools.github.io/meta/#/ and encourage them to enhance the performance of cp - assuming you have permission to use these tools.
Lionel B. Dyck <>< Website: https://www.lbdsoftware.com Github: https://github.com/lbdyck “Worry more about your character than your reputation. Character is what you are, reputation merely what others think you are.” - - - John Wooden -----Original Message----- From: IBM Mainframe Discussion List <IBM-MAIN@LISTSERV.UA.EDU> On Behalf Of Farley, Peter Sent: Monday, June 5, 2023 11:02 AM To: IBM-MAIN@LISTSERV.UA.EDU Subject: Re: z/OS 3.1: Now UNIXR Certified Which is fine for those who have access to those tools; many of us do not have that luxury, nor the permission nor the disk space to "try them out" for ourselves. -----Original Message----- From: IBM Mainframe Discussion List <IBM-MAIN@LISTSERV.UA.EDU> On Behalf Of Lionel B. Dyck Sent: Monday, June 5, 2023 11:44 AM To: IBM-MAIN@LISTSERV.UA.EDU Subject: Re: z/OS 3.1: Now UNIXR Certified Peter - I believe you'll find that the Co:Z folks, at coztoolkit.com, have tools that may provide the performance improvements that you are looking for. The ZIGI tool will take advantage of their getpds and putpds commands if they are available and achieve significantly improved performance compared to native cp. Lionel B. Dyck <>< Website: https://urldefense.com/v3/__https://www.lbdsoftware.com__;!!Ebr-cpPeAnfNniQ8HSAI-g_K5b7VKg!LAIovDokCs-oEgMZ6WCSnTOL8NyW8GN8r0RbiJPkLNybyfjLR0U7xWJijkAwrCmc7-VI_n5W5CsQYcDqizc$ Github: https://urldefense.com/v3/__https://github.com/lbdyck__;!!Ebr-cpPeAnfNniQ8HSAI-g_K5b7VKg!LAIovDokCs-oEgMZ6WCSnTOL8NyW8GN8r0RbiJPkLNybyfjLR0U7xWJijkAwrCmc7-VI_n5W5CsQyKQdnzg$ “Worry more about your character than your reputation. Character is what you are, reputation merely what others think you are.” - - - John Wooden -----Original Message----- From: IBM Mainframe Discussion List <IBM-MAIN@LISTSERV.UA.EDU> On Behalf Of Farley, Peter Sent: Monday, June 5, 2023 10:37 AM To: IBM-MAIN@LISTSERV.UA.EDU Subject: Re: z/OS 3.1: Now UNIXR Certified Where the performance of the USS file system falls down is in the transfer between z/OS files/data stores and USS files/data stores. Practical experimentation has shown me that the "cp" and "cat" commands are the fastest way to sequentially transfer non-DB2 data using only a pre-written utility (and both do support VSAM files), but that is not to say those are fast processes compared to directly reading and writing files in the "other system". When the business-important permanent data store is not originally in the USS file system you must deal with that transfer time in the application design or write your own data transfer code to achieve an efficiently running system. Neither zowe nor zoau provide good performance in the data transfer process. The zowe transfer time is reasonable for one-offs (but still not as fast as "cp" or "cat") but zoau is horribly slow no matter how it is used. The DB2CLI is a significant time consumer; the python-ibmdb module does it far faster, though that product is still officially in "beta" mode. As usual, the best way to optimize data access time is good design and a thorough knowledge of the technical benefits and flaws of your system. -----Original Message----- From: IBM Mainframe Discussion List <IBM-MAIN@LISTSERV.UA.EDU> On Behalf Of David Crayford Sent: Monday, June 5, 2023 7:06 AM To: IBM-MAIN@LISTSERV.UA.EDU Subject: Re: z/OS 3.1: Now UNIXR Certified One compelling reason to embrace zFS is its potential for modernization and facilitating the development of contemporary tools. While acknowledging the significance of QSAM, VSAM KSDS, and other older technologies, it is crucial to recognize the advancements made in data structure formats for disk files since the days of VSAM. In the present era, LSM-trees have gained popularity for their application in NoSQL key-value stores, blazing-fast TSDBs, and highly optimized logging systems. Attempting to implement an LSM-tree using VSAM would be an arduous endeavor, bordering on a nightmare. Even with the assistance of Media Manager, it remains a Herculean task to reconcile these two disparate technologies. I dedicated a couple of hours to porting RocksDB, and the results have been nothing short of exceptional. It operates seamlessly on z/OS, demonstrating its prowess and resilience. Another noteworthy aspect of LSM-trees is their inherent ability to merge and compact while in operation, eliminating the need for reorgs. https://urldefense.com/v3/__https://github.com/facebook/rocksdb__;!!Ebr-cpPeAnfNniQ8HSAI-g_K5b7VKg!LzT4O_YlJ0b4g_9FS6BO4zVHzZ55N3E8k4QPWtQx_hnCZfUjLh3mDAsW-PYPhMIqZnaz3WNQfEhFE3UKfk3uzfY$ https://urldefense.com/v3/__https://www.youtube.com/watch?v=I6jB0nM9SKU__;!!Ebr-cpPeAnfNniQ8HSAI-g_K5b7VKg!LzT4O_YlJ0b4g_9FS6BO4zVHzZ55N3E8k4QPWtQx_hnCZfUjLh3mDAsW-PYPhMIqZnaz3WNQfEhFE3UKiaUQef4$ On 5/6/2023 5:55 pm, David Crayford wrote: > > > On 2/6/2023 11:31 pm, René Jansen wrote: >> What I remember of it is that he was convinced it was a lot slower. > > He was mistaken! I've tested it out, and QSAM is no match for zFS. You > can find the details in this gist: > https://urldefense.com/v3/__https://gist.github.com/daveyc/14b45d6d70d8dd9af1848e539d78881f__;!!Ebr-cpPeAnfNniQ8HSAI-g_K5b7VKg!LzT4O_YlJ0b4g_9FS6BO4zVHzZ55N3E8k4QPWtQx_hnCZfUjLh3mDAsW-PYPhMIqZnaz3WNQfEhFE3UKrdRm7Qs$ > . > Adding an fsync() call after writing each record barely incurs any > overhead. zFS, operating with highly optimized Media Manager APIs, > handles it efficiently. Additionally, zFS functions as a caching file > system. > > I have observed a certain degree of snobbery among many > traditionalists when it comes to USS. I can recall an incident from > approximately 15 years ago when I advocated for the use of sqlite in > one of our products. My boss dismissed the idea, expressing concerns > that customers might be deterred by using the UNIX file system. > Consequently, we opted for a VSAM KSDS, despite its inherent > limitations. Interestingly, it is worth noting that there are now > numerous IBM z/OS products that embrace sqlite, with some even > integrating it with HLASM. > >> So I told him that nobody forced him not to use QSAM for datasets just >> because it ran in USS. And it think that is a great asset of it. Just >> because Unix forces you to have a hierarchical directory system does not >> mean, in USS, that you need to use it for all I/O. >> >> René. >> >>> On 2 Jun 2023, at 17:03, Seymour J Metz<sme...@gmu.edu> wrote: >>> >>> Dubbing is part of the setup overhead for a task, and only occurs once, so >>> except for very short tasks it is just noise in measuring performance. >>> >>> As for the general overhead of Unix System Services, the Devil is in the >>> details. For a comparison to be reasonable, the two programs have to be >>> using the services in a comparable fashion. Was your COBOL programmer >>> really comparing the overhead of conventional access methods to Unix file >>> I/O, or were the numbers drowned out by, e.g., differences in application >>> logic? >>> >>> >>> -- >>> Shmuel (Seymour J.) Metz >>> https://urldefense.com/v3/__http://mason.gmu.edu/*smetz3__;fg!!Ebr-c >>> pPeAnfNniQ8HSAI-g_K5b7VKg!LzT4O_YlJ0b4g_9FS6BO4zVHzZ55N3E8k4QPWtQx_h >>> nCZfUjLh3mDAsW-PYPhMIqZnaz3WNQfEhFE3UKsIUJTkY$ -- This message and any attachments are intended only for the use of the addressee and may contain information that is privileged and confidential. If the reader of the message is not the intended recipient or an authorized representative of the intended recipient, you are hereby notified that any dissemination of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by e-mail and delete the message and any attachments from your system. ---------------------------------------------------------------------- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN ---------------------------------------------------------------------- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN This message and any attachments are intended only for the use of the addressee and may contain information that is privileged and confidential. If the reader of the message is not the intended recipient or an authorized representative of the intended recipient, you are hereby notified that any dissemination of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by e-mail and delete the message and any attachments from your system. ---------------------------------------------------------------------- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN ---------------------------------------------------------------------- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
