Re: [Libcdio-devel] Can we merge branch "joliet_multi_extent" ?
Hi, to revive the processing of branch "joliet_multi_extent" i made some experiments: The branch merges without problems with a branch that was freshly created from "master". So i assume that it is good for "master", too. I hacked GNU xorriso to create extents of only 16 KiB each. Macros MAX_ISO_FILE_SECTION_SIZE and ISO_EXTENT_SIZE in libisofs/ecma119.h were set to 16384. Then i created an ISO image with Joliet and without Rock Ridge of a file with ~360 KiB (actually /bin/tar): xorriso -compliance no_emul_toc -padding 0 -joliet on -rockridge off \ -outdev joliet_multi_extent.iso \ -map /bin/tar /FileWithManyExtents Inspection by xorriso -indev joliet_multi_extent.iso -toc -find / -exec report_lba -- shows that Joliet was loaded and the file is represented by 22 extents: Media blocks : 208 readable , 188491924 writable , 188492148 overall ISO offers : Joliet ISO loaded : Joliet ... Report layout: xt , Startlba , Blocks , Filesize , ISO image path File data lba: 0 , 33 ,8 , 358072 , '/FileWithManyExtents' ... File data lba: 21 , 201 ,7 , 358072 , '/FileWithManyExtents' I get from the programs of branch "joliet_multi_extent": $ iso-info -i joliet_multi_extent.iso -l ... Joliet Level: 3 ... - [LSN 33]358072 Nov 08 2014 19:48:38 FileWithManyExtents $ iso-read --image=joliet_multi_extent.iso \ --extract=FileWithManyExtents --output-file=../tar_from_iso $ diff tar_from_iso /bin/tar $ So for me it looks ready for merging. Other, more realistic experiments would be welcome. Have a nice day :) Thomas
Re: [Libcdio-devel] Can we merge branch "joliet_multi_extent" ?
Here is a suggestion - in that branch the new ISO image that you created. And then add a new test for this feature. After that passes, then merge the whole thing back into master. My suggestion is to merge to master and let's move forward given that you've tested this. If there are further problems we can address them down the line. Thanks - and have a good (and NOT stressful) day. On Mon, Mar 27, 2023 at 3:23 AM Thomas Schmitt wrote: > Hi, > > to revive the processing of branch "joliet_multi_extent" i made some > experiments: > > The branch merges without problems with a branch that was freshly > created from "master". So i assume that it is good for "master", too. > > I hacked GNU xorriso to create extents of only 16 KiB each. > Macros MAX_ISO_FILE_SECTION_SIZE and ISO_EXTENT_SIZE in libisofs/ecma119.h > were set to 16384. Then i created an ISO image with Joliet and without > Rock Ridge of a file with ~360 KiB (actually /bin/tar): > > xorriso -compliance no_emul_toc -padding 0 -joliet on -rockridge off \ > -outdev joliet_multi_extent.iso \ > -map /bin/tar /FileWithManyExtents > > Inspection by > xorriso -indev joliet_multi_extent.iso -toc -find / -exec report_lba -- > shows that Joliet was loaded and the file is represented by 22 extents: > > Media blocks : 208 readable , 188491924 writable , 188492148 overall > ISO offers : Joliet > ISO loaded : Joliet > ... > Report layout: xt , Startlba , Blocks , Filesize , ISO image path > File data lba: 0 , 33 ,8 , 358072 , > '/FileWithManyExtents' > ... > File data lba: 21 , 201 ,7 , 358072 , > '/FileWithManyExtents' > > I get from the programs of branch "joliet_multi_extent": > > $ iso-info -i joliet_multi_extent.iso -l > ... > Joliet Level: 3 > ... > - [LSN 33]358072 Nov 08 2014 19:48:38 FileWithManyExtents > > $ iso-read --image=joliet_multi_extent.iso \ > --extract=FileWithManyExtents --output-file=../tar_from_iso > $ diff tar_from_iso /bin/tar > $ > > So for me it looks ready for merging. > Other, more realistic experiments would be welcome. > > > Have a nice day :) > > Thomas > > >
Re: [Libcdio-devel] Can we merge branch "joliet_multi_extent" ?
in that branch -> in that branch *add* On Mon, Mar 27, 2023 at 3:56 AM Rocky Bernstein wrote: > Here is a suggestion - in that branch the new ISO image that you created. > And then add a new test for this feature. > > After that passes, then merge the whole thing back into master. > > My suggestion is to merge to master and let's move forward given that > you've tested this. > > If there are further problems we can address them down the line. > > Thanks - and have a good (and NOT stressful) day. > > On Mon, Mar 27, 2023 at 3:23 AM Thomas Schmitt wrote: > >> Hi, >> >> to revive the processing of branch "joliet_multi_extent" i made some >> experiments: >> >> The branch merges without problems with a branch that was freshly >> created from "master". So i assume that it is good for "master", too. >> >> I hacked GNU xorriso to create extents of only 16 KiB each. >> Macros MAX_ISO_FILE_SECTION_SIZE and ISO_EXTENT_SIZE in libisofs/ecma119.h >> were set to 16384. Then i created an ISO image with Joliet and without >> Rock Ridge of a file with ~360 KiB (actually /bin/tar): >> >> xorriso -compliance no_emul_toc -padding 0 -joliet on -rockridge off \ >> -outdev joliet_multi_extent.iso \ >> -map /bin/tar /FileWithManyExtents >> >> Inspection by >> xorriso -indev joliet_multi_extent.iso -toc -find / -exec report_lba -- >> shows that Joliet was loaded and the file is represented by 22 extents: >> >> Media blocks : 208 readable , 188491924 writable , 188492148 overall >> ISO offers : Joliet >> ISO loaded : Joliet >> ... >> Report layout: xt , Startlba , Blocks , Filesize , ISO image path >> File data lba: 0 , 33 ,8 , 358072 , >> '/FileWithManyExtents' >> ... >> File data lba: 21 , 201 ,7 , 358072 , >> '/FileWithManyExtents' >> >> I get from the programs of branch "joliet_multi_extent": >> >> $ iso-info -i joliet_multi_extent.iso -l >> ... >> Joliet Level: 3 >> ... >> - [LSN 33]358072 Nov 08 2014 19:48:38 FileWithManyExtents >> >> $ iso-read --image=joliet_multi_extent.iso \ >> --extract=FileWithManyExtents --output-file=../tar_from_iso >> $ diff tar_from_iso /bin/tar >> $ >> >> So for me it looks ready for merging. >> Other, more realistic experiments would be welcome. >> >> >> Have a nice day :) >> >> Thomas >> >> >>
Re: [Libcdio-devel] Can we merge branch "joliet_multi_extent" ?
Actually, let me change this slightly. After adding the test, then push the *branch *back and let everyone know. I (and possibly others) can check out the branch first. The most common kind of mistake is not adding the files into git and not ensuring this gets into the tarball. There is a "make distcheck" to help with the latter. On Mon, Mar 27, 2023 at 3:57 AM Rocky Bernstein wrote: > in that branch -> in that branch *add* > > On Mon, Mar 27, 2023 at 3:56 AM Rocky Bernstein wrote: > >> Here is a suggestion - in that branch the new ISO image that you created. >> And then add a new test for this feature. >> >> After that passes, then merge the whole thing back into master. >> >> My suggestion is to merge to master and let's move forward given that >> you've tested this. >> >> If there are further problems we can address them down the line. >> >> Thanks - and have a good (and NOT stressful) day. >> >> On Mon, Mar 27, 2023 at 3:23 AM Thomas Schmitt wrote: >> >>> Hi, >>> >>> to revive the processing of branch "joliet_multi_extent" i made some >>> experiments: >>> >>> The branch merges without problems with a branch that was freshly >>> created from "master". So i assume that it is good for "master", too. >>> >>> I hacked GNU xorriso to create extents of only 16 KiB each. >>> Macros MAX_ISO_FILE_SECTION_SIZE and ISO_EXTENT_SIZE in >>> libisofs/ecma119.h >>> were set to 16384. Then i created an ISO image with Joliet and without >>> Rock Ridge of a file with ~360 KiB (actually /bin/tar): >>> >>> xorriso -compliance no_emul_toc -padding 0 -joliet on -rockridge off \ >>> -outdev joliet_multi_extent.iso \ >>> -map /bin/tar /FileWithManyExtents >>> >>> Inspection by >>> xorriso -indev joliet_multi_extent.iso -toc -find / -exec report_lba -- >>> shows that Joliet was loaded and the file is represented by 22 extents: >>> >>> Media blocks : 208 readable , 188491924 writable , 188492148 overall >>> ISO offers : Joliet >>> ISO loaded : Joliet >>> ... >>> Report layout: xt , Startlba , Blocks , Filesize , ISO image path >>> File data lba: 0 , 33 ,8 , 358072 , >>> '/FileWithManyExtents' >>> ... >>> File data lba: 21 , 201 ,7 , 358072 , >>> '/FileWithManyExtents' >>> >>> I get from the programs of branch "joliet_multi_extent": >>> >>> $ iso-info -i joliet_multi_extent.iso -l >>> ... >>> Joliet Level: 3 >>> ... >>> - [LSN 33]358072 Nov 08 2014 19:48:38 FileWithManyExtents >>> >>> $ iso-read --image=joliet_multi_extent.iso \ >>> --extract=FileWithManyExtents --output-file=../tar_from_iso >>> $ diff tar_from_iso /bin/tar >>> $ >>> >>> So for me it looks ready for merging. >>> Other, more realistic experiments would be welcome. >>> >>> >>> Have a nice day :) >>> >>> Thomas >>> >>> >>>
Re: [Libcdio-devel] Can we merge branch "joliet_multi_extent" ?
Hi, Rocky Bernstein wrote: > Here is a suggestion - in that branch add the new ISO image that you created. > And then add a new test for this feature. It seems easiest to extend test "check_multiextent.sh". (The main obstacle which gave me riddles was missing TZ=CUT when i created multiextent_joliet.right.) The Joliet test ISO is significantly smaller than the old one for Rock Ridge: -rw-r--r-- 1 . . 524288 Jun 17 2020 test/data/multi_extent_8k.iso -rw-r--r-- 1 . . 122880 Mar 27 10:56 test/data/multi_extent_8k_joliet.iso But strangely du says that it uses more disk space on ext4: 100 test/data/multi_extent_8k.iso 120 test/data/multi_extent_8k_joliet.iso I guess this is the effect of sparse file storing. But i am not aware that i ever gave multi_extent_8k.iso a special treatment back in 2018. Is git known to work with sparse file generating seeks when checking out files ? The old multi_extent_8k.iso is nominally so much larger, because i obviously did not apply space saving options when creating it: 64+ KiB are used for multi-session TOC emulation. 300 KiB are trailing padding against the Linux TAO CD bug. I could create it freshly with 122880 bytes. Shall i do ? Have a nice day :) Thomas
Re: [Libcdio-devel] Can we merge branch "joliet_multi_extent" ?
On Mon, Mar 27, 2023 at 7:23 AM Thomas Schmitt wrote: > Hi, > > Rocky Bernstein wrote: > > Here is a suggestion - in that branch add the new ISO image that you > created. > > And then add a new test for this feature. > > It seems easiest to extend test "check_multiextent.sh". > (The main obstacle which gave me riddles was missing TZ=CUT when i > created multiextent_joliet.right.) > > The Joliet test ISO is significantly smaller than the old one for > Rock Ridge: > -rw-r--r-- 1 . . 524288 Jun 17 2020 test/data/multi_extent_8k.iso > -rw-r--r-- 1 . . 122880 Mar 27 10:56 test/data/multi_extent_8k_joliet.iso > But strangely du says that it uses more disk space on ext4: > 100 test/data/multi_extent_8k.iso > 120 test/data/multi_extent_8k_joliet.iso > I guess this is the effect of sparse file storing. But i am not aware that > i ever gave multi_extent_8k.iso a special treatment back in 2018. > Is git known to work with sparse file generating seeks when checking out > files ? > > > The old multi_extent_8k.iso is nominally so much larger, because i > obviously did not apply space saving options when creating it: > 64+ KiB are used for multi-session TOC emulation. 300 KiB are trailing > padding against the Linux TAO CD bug. > > I could create it freshly with 122880 bytes. Shall i do ? > Sure, try that or try creating a different image. BTW, when creating ISO images please try to use the smallest amount of data that will do the job. > > > Have a nice day :) > > Thomas > > >
Re: [Libcdio-devel] Can we merge branch "joliet_multi_extent" ?
On Mon, Mar 27, 2023 at 8:49 AM Rocky Bernstein wrote: > > > On Mon, Mar 27, 2023 at 7:23 AM Thomas Schmitt wrote: > >> Hi, >> >> Rocky Bernstein wrote: >> > Here is a suggestion - in that branch add the new ISO image that you >> created. >> > And then add a new test for this feature. >> >> It seems easiest to extend test "check_multiextent.sh". >> (The main obstacle which gave me riddles was missing TZ=CUT when i >> created multiextent_joliet.right.) >> >> The Joliet test ISO is significantly smaller than the old one for >> Rock Ridge: >> -rw-r--r-- 1 . . 524288 Jun 17 2020 test/data/multi_extent_8k.iso >> -rw-r--r-- 1 . . 122880 Mar 27 10:56 >> test/data/multi_extent_8k_joliet.iso >> But strangely du says that it uses more disk space on ext4: >> 100 test/data/multi_extent_8k.iso >> 120 test/data/multi_extent_8k_joliet.iso >> I guess this is the effect of sparse file storing. But i am not aware that >> i ever gave multi_extent_8k.iso a special treatment back in 2018. >> Is git known to work with sparse file generating seeks when checking out >> files ? >> >> >> The old multi_extent_8k.iso is nominally so much larger, because i >> obviously did not apply space saving options when creating it: >> 64+ KiB are used for multi-session TOC emulation. 300 KiB are trailing >> padding against the Linux TAO CD bug. >> >> I could create it freshly with 122880 bytes. Shall i do ? >> > > Sure, try that or try creating a different image. BTW, when creating ISO > images please try to use the smallest amount of data that will do the job. > Actually, a new image is better. We want to be able to handle historical images that were not created optimally. > > >> >> >> Have a nice day :) >> >> Thomas >> >> >>
Re: [Libcdio-devel] CD-Text support for macOS and Windows
Am 26.03.2023 um 22:58 schrieb Rocky Bernstein: On Fri, Mar 24, 2023 at 3:58 AM Rocky Bernstein wrote: For what it is worth, I was able to build the current master on Windows 10 pro and Msys2. I suspect though that master doesn't have Robert's latest patches. No, the Windows CD-Text patches are in the cdtext-win32 branch. I didn't want to push this directly to master without someone else testing it before. It looks like CDDB reading has bit rot. But that is a different story. The freedb database that libcdio defaults to shut down in 2020. I just pushed a commit to update the references to GnuDB (gnudb.org) which is the only viable alternative I know of. CDDB queries work again with this change.
Re: [Libcdio-devel] Can we merge branch "joliet_multi_extent" ?
Hi, i pushed branch "joliet_multi_extent" with a new commit cb03533aaba6113465b19157577e631e26406c7e "Add test for proper handling of multi-extent files in the Joliet tree" Please review and/or test. Especially (cd test && ./check_multiextent.sh ) should report: -- ./check_multiextent.sh: ISO 9660 + Rock Ridge multiextent listing test ok. -- ./check_multiextent.sh: ISO 9660 + Rock Ridge multiextent read test ok. -- ./check_multiextent.sh: Joliet multiextent listing test ok. -- ./check_multiextent.sh: Joliet multiextent read test ok. I tested that this branch can be merged into a branch made from "master" (15801fb), compiles, and yields above test result. Rocky Bernstein wrote: > BTW, when creating ISO > images please try to use the smallest amount of data that will do the job. That's why i hack xorriso before making the test ISOs. Normally a multi-extent ISO would be at least 4 GiB large. The overhead of the old multi_extent_8k.iso consists entirely of zeros. So within git and tar.gz there should be nearly no cost. Nevertheless it is worthwhile to avoid those zeros. So i made a new one by: xorriso -compliance no_emul_toc -padding 0 -joliet off -rockridge on \ -outdev test/data/multi_extent_8k.iso \ -map test/data/multi_extent_file /multi_extent_file Have a nice day :) Thomas
