On Thu, Jan 2, 2020 at 11:23 AM Mick <michaelkintz...@gmail.com> wrote: > > On Thursday, 2 January 2020 14:43:58 GMT Rich Freeman wrote: > > > > Out of curiosity, what model drive is it? Is it by chance an SMR / > > archive drive? > > Good catch! I hadn't thought of this - the Linux kernel will need to have > DM_ZONED enabled I think, for the OS to manager the shingled writes > sequentially, but I don't have this enabled here because AFAIK I have no such > drives in my possession.
I haven't looked into the details of how it works. Certainly the kernel should be able to optimize writes to such disks and utilize TRIM if supported by the drive. But it isn't strictly needed as you go on to say. > > Due to the limitations on how those write data out I > > could see them implementing an internal filesystem that journals > > incoming data and then writes it back out after the fact. > > SMR drives which implement a 'device managed' write mechanism, will use their > own firmware to control data storage. The OS would not be aware of anything > being different to a conventional drive. Correct. > > If so then > > that might happen even after the kernel thinks it is unmounted. > > However, such a drive firmware would probably use a journal that > > ensures data is safe even if power is cut mid-operation. The drive > > isn't supposed to report that a write is completed until it is > > durable. > > Which I take it to mean the drive would not be unmounted by the OS until it is > safe to do so and for all intends and purposes it will also be safe to be > powered down thereafter. Yes - even if the drive is doing its own data shuffling after being unmounted, it should still be safe to power off. In any case, I was just speculating as to why it might be doing writes when not mounted. I don't know the internal details of how these drives all work. > I would think this would be within seconds of > successfully unmounting it. Spinning for 30 minutes or more after it is > unmounted sounds excessive to me, if it is only being spun by the firmware for > flushing its journal buffers. Indeed. That really makes me wonder whether it is actually writing anything. It could just be that the drive has some vibration or otherwise is giving the sensation that it is doing something when it isn't. Though, if the thing really does have a large journal inside to improve performance it could actually buffer a pretty large number of writes. SMR drives can have a very large amount of write amplification. If each erase region on the disk contains 10k blocks then every 1MB of data in the journal could potentially lead to 10GB of disk rewrites, assuming that writes are randomly distributed. It also makes sense that when replaying the journal the drive is going to prioritize erase regions with the most updates, which means that by the time you're unmounting the drive you're going to expect the stuff left in the journal to lead to the most write amplification. I would imagine that a drive that works in this manner is going to use a logfs for the journal (if the journal is even in SMR format), and then it would just keep an erase region free at all times. Anytime a partial write happens in a region the data would get written to the free region, then that region would be remapped in place of the old region, and now the old region is unallocated for the next write. This could be interrupted due to power loss at any time without any real loss to data, since data is not overwritten in place. Before the journal record is closed out the old region is still valid, and after the new region is valid. An interrupted write is just repeated on power up since the new region can just be overwritten again safely. In any case, the bottom line is that a drive should be safe to unplug if all filesystems on it are umounted and the umount commands all return. If the disk loses data after that point it is a manufacturing design flaw. -- Rich
