On Wed, Feb 11, 2026 at 12:23 PM Gordan Bobic <[email protected]> wrote: > > On Wed, 11 Feb 2026 at 07:12, Marko Mäkelä <[email protected]> wrote: > > > > Hi Gordan, > > > > On Wed, Feb 11, 2026 at 1:56 AM Gordan Bobic via discuss > > <[email protected]> wrote: > > > > > > Ah - apologies - I just noticed that it was reintroduced again in a > > > later version. I needed a bigger version jump. Was the auto-detection > > > also removed, or is just outright not working with ZFS? > > > > When I designed the variable-size-block format that was introduced by > > MDEV-14425 in 10.8, I chose the log file header size to be 12288 > > bytes, or three 4096-byte blocks. The motivation for this was to allow > > the two checkpoint headers to be safely written in an alternating > > fashion, also when using O_DIRECT. According to my research back then, > > the largest physical block size that is available is 4096 bytes. > > I can assure you SSDs, especially large enterprise grade ones, were > coming with erase block sizes greater than that even back then. > Not to mention that virtualisation brings a whole new layer of > complexity that can result in large underlying block sizes.
I did think about the erase block size, which indeed can be much larger than the write block size, already in the flash ROM chips that I am familiar with, from some 20+ years ago. However, if we are talking about an update-in-place file system such as ext4 or XFS stored on a single device, I think that it is best to use a write unit of the physical block size. If the erase block size were (say) 1 MiB and the write block size 4 KiB, we would have 256-fold write amplification when (over)writing the last log block, as new data is being appended to the log. It would be better to write just 4 KiB at a time and let the flash translation layer of the SSD to take care of managing the erase queue (and adding some more write amplification when relocating stray blocks away from an almost-empty bunch of blocks). > > If yes, what would be the compression block size, and is it > > aligned in some way? Say, each 1 MiB block from the start of a file > > would be compressed separately? If yes, then maybe the maximum write > > block size should be 1 MiB, or whatever the input block size of the > > built-in compression is. > > I don't think compression should factor into this from trying to guess > it from the WAL side. That should be left to the user, because the > compression could be happening at the exposed block device level, or > it could be happening underneath the exposed block device level. I do not have any plans to implement any further guessing beyond the current default value. I asked the question for my own understanding. > For example, if I am exposing 16KB logical sectors and using > compression underneath that, then I would not use compression at the > top level. There could also be a thinly provisioned computational storage device (CSD) or LVM storage underneath. We once got impressive results when testing a ScaleFlux drive, which presented itself as an SSD with 4096-byte physical block size. > Or pad the header out with a reservation up to innodb_page_size. Let's > face it, nobody is going to care about wasting potentially up to 52KB > of header space on a multi MB/GB file. No, thanks, I do not want to change the file format. The log records will wrap from the end of the file to byte offset 12288. Even in the upcoming innodb_log_archive=ON format (https://jira.mariadb.org/browse/MDEV-37949) I am retaining the 12288-byte log file header. Marko -- Marko Mäkelä, Lead Developer InnoDB MariaDB plc _______________________________________________ discuss mailing list -- [email protected] To unsubscribe send an email to [email protected]
