Hi Bret, maybe virtual 512 byte sectors are actually not that evil: Imagine a NORMAL 4096 byte sector based FAT32 filesystem.
Each cluster and each FAT will be a multiple of 4096 byte in size, as will be the boot and fsinfo sectors. In FAT32 the root directory is just any directory, so like all the directories, it will consist of 1 or more normal clusters. So what are the differences between that FAT32 system and a system with 512 byte sectors where all structures only HAPPEN to be multiples of 4096 bytes in size and offset? BPB bytes per sector differs (512 vs 4096) BPB sectors per cluster differs (? vs ?/8) BPB total sectors differs (? vs ?/8) BPB sectors per fat differs (? vs ?/8) BPB CHS geometry differs - but does a disk with 4096 byte sectors allow CHS based access at all? I hope it does not. BPB "hidden" sectors differs, as in "offset of partition" *but* a DOS kernel which uses a disk via an external block device driver will not have access to the raw disk at all because the accessible area starts at the partition then. BPB FAT32 root dir position is the same (a cluster number) BPB FAT32 fsinfo sector and backup boot sector positions differ and are ? versus ?/8 again. Common values are NOT multiples of 8, but DOS of course does allow them... :-) BPB "reserved" sectors differs, as in how many sectors are reserved for boot, fsinfo, backup boot and really reserved areas. Common values are multiples of 16 for a 512 byte per sector filesystem, not sure about 4096 byte filesystems. As long as you can express the size of the reserved area of your 4096 byte per sector disk in units of 512 bytes, things work - which is always true. In short: To "mount" a 4096 byte per sector disk with a driver that allows you to access numbered 512 byte areas AS IF the sector size would be 512 bytes per sector, you have to multiply some BPB values by 8 because N sectors of 4096 bytes each equal 8 times N areas of 512 bytes... You also have to change the "sector" size from 4096 to 512 to tell DOS that access is in units of 512 bytes. No other steps should be needed. Also, because 4096 is just a multiple of 512, this will always (*) work. Only going the OTHER way round is hard: You cannot install a simple diskimage of a 512 byte per sector FAT filesystem on a disk with 4096 bytes per sector, because you cannot know that all structures align to multiples of 4096 bytes and because you cannot have fractional locations of sizes. * Note that copying a diskimage of a 4096 byte per sector disk to a 512 byte per sector disk, or using a driver to access a native 4096 byte per sector disk in VIRTUAL 512 byte per chunk "sectors" can SOMETIMES overflow values: 1. Large sector disks can have cluster sizes above 64 kB Such filesystems cannot be transformed to 512 byte/sec 2. Large sector disks can hold FAT32 filesystems > 2 TB. Such filesystems cannot be transformed to 512 byte/sec 3. Large sector disks can have reserved area sizes above 32 MB, which could not be expressed as uint16 count of sectors after the transform to 512 bytes per "sector" but in practice, reserved areas are always below 1 MB. In short: "Expressing" a 4096 byte per sector disk with a FAT32 filesystem on it in terms of a virtual 512 byte per sector FAT32 system ALWAYS works, simply by converting a few values in the BPB of that disk, UNLESS the filesystem or cluster size are larger than would be possible on 512 byte per sector classic harddisks in the first place :-) > Internally, there are lots of technical issues with things like DMA > and buffering that will probably cause at least as many, if not more, > problems than trying to use hard-coded-512-byte programs with disks > that aren't 512-bytes. There is also a significant speed and memory > usage problem -- all disk transfers will need to be "double-buffered" While it is true that performance is lost if writing a virtual 512 byte sector means reading-modifying-writing a real 4096 byte sector, this loss in performance is a general DOS problem: Other operating systems would gain performance by pooling writes to MULTIPLE sectors which makes a very noticeable difference on flash based disks such as SSD or USB flash sticks. As for the DMA and memory usage, you have to suffer the real 4096 byte sectors at SOME place anyway... If it is not in the "allow access to virtual 512 byte per sector disk representation" block device driver, you will have to suffer larger sector buffers in the DOS kernel's SDA and BUFFERS areas instead for native 4096 byte sectors. So you arguably even suffer more with the 4k/sec kernel. > (the driver will need to internally store the entire large sector and > then transfer only the part that was requested). Again, possible, > but NOT a good idea, IMHO. Yet that is exactly what many caches including LBACACHE and SMARTDRV do: To reduce the size of bookkeeping data they store data in multi sector bites of memory. Still transfers need not be rounded up to full bites. For the native 4096 byte sector disk case, you of course do have to do transfers in multiples of native sectors anyway. Only for WRITING data which is not an aligned multiple of native sectors, you do get a speed loss with virtual 512 byte sector drivers compared to DOS KNOWING that a sector is actually 4096 bytes and not just 512 bytes as you end up doing slow read-modify-write access... Fun fact: If you stream data to a file in FreeDOS, it will read-modify-write your FAT and data sectors all the time anyway, as it does not pool the writes afair. This is why I suggest to first grow such a file to the final size, putting all FAT writes into ONE access and then writing the file's contents in large N kB chunks. >> From what I understand, in MS-DOS and IBM-DOS, the maximum sector >> size is stored in only one place (part of the List of Lists). This >> value is "adjusted" while booting to match the largest sector size >> of any disk that the kernel finds and assigns drive letters to. Then I expect the side effect to be that the kernel "inflates" all sector-sized areas to 4096 bytes, to be prepared to handle 4096 byte sized sectors in SOME disks. For all old disks with classic 512 byte sectors, this means a lot of wasted space in RAM for each such buffer in memory. Unless I misunderstood the details, MS DOS would not adjust sizes on a per-buffer basis. >> the kernel code, it always uses the value stored there in its >> various calculations, and never uses any hard-coded values. That's >> the way FreeDOS _should_ be as well, though I don't know if it >> actually is. It is not - the situation described in my replayed 2005 mail is quite similar in the current FreeDOS kernel. Sector sizes are hardcoded in several places instead of being processed on the fly. You are right, editable size would be more elegant. >> Interesting. Any known side effects? > > Not that I know of. I've been using a 2048-byte sector size in > MS-DOS for a long time now (since USBDRIVE came out) You mean you know any sort of USB storage device which has native 2048 byte sectors? I thought that for flash and large disk formats, only 4096 was popular and 512? > any problems. I should point out that I don't actually have any > disks with other than 512-byte sectors, so my testing is not very So you actually do the "virtual trick" the other way around? As mentioned above, that only works when all relevant structures on the actually-512-byte-per-sec disk are aligned to and sized in 2 kB boundaries, not counting the global offset of the partition, though. >> Of course it will not help tools like FORMAT or CHKDSK I guess that has slightly lower priority - if we can "mount" a 4096 byte per sector disk in DOS, no matter what tool and operating system formatted / fdisked it then we can already enjoy using those disks :-) Later we can start to support format, fdisk, dosfsck etc. > In my opinion, there's no legitimate reason for programs > like FORMAT and CHKDSK to disallow non-512-byte sectors. In FORMAT, it is probably mostly laziness: You would have to do more calculations in more flexible ways and the sector buffer memory management would also change. By the way, is the 55aa boot sector thing and similar flags for fsinfo and 2nd stage boot sectors on FAT32 always at the end of the sector or rather just at the end of the first 512 bytes of the sector? ------------------------------------------------------------------------------ RSA(R) Conference 2012 Mar 27 - Feb 2 Save $400 by Jan. 27 Register now! http://p.sf.net/sfu/rsa-sfdev2dev2 _______________________________________________ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
