Folks,
I'd like to have a discussion about FAT robustness.
Please give your thought, comments and related issues.
About few years ago, we added some features to FAT, called xvfat,
so that System and FAT have robustness against unexpected media hot
unplug and ability to let applications correctly be aware the event.
Just for your reference, I put a patch to 2.4.20 kernel at
http://www.celinuxforum.org/CelfPubWiki/XvFatDiscussion?action=AttachFile&do=get&target=20050715-xvfat-2.4.20.patch
This includes following features;
Handle media removed during “mount”
Notification of media removal to application
Cancellation of I/O Elevator for Block device
Block system calls until a completion of writing
Control order of meta-data updates, using transaction
control implemented in fs/xvfat/fwrq.c
File syscall return “error”, except umount
Japanese file name support
possible 1-N mapping issues SJIS <-> UNICODE
Dirty Flag support
TIME ZONE support
On moving to 2.6, we consider and categorize issues, again.
And we are planing to have open source project for these features
to add 2.6 kernel. I'd like to open discussion about these features
and how to implement on 2.6 kernel.
1. Issues to be addressed
- Issues around FAT with CE devices
- Hot unplug issues
- File System corruption on unplug media/storage device
Almost same as power down without umount
- Notification of the event
Application need to know the event precisely
Need to more investigation
- System stability after unplug
Almost same as I/O error recovery issues discussed
at LKLM
http://developer.osdl.jp/projects/doubt/fs-consistency-and-coherency/index.html
http://groups.google.co.jp/group/linux.kernel/browse_thread/thread/b9c11bccd59e0513/4a4dd84b411c6d32?q=[RFD]+FS+behavior+(I%2FO+failure)+in+kernel+summit++lkml&rnum=1&hl=ja#4a4dd84b411c6d32
- Other issues
- Time stamp issues
using always local time
time resolution is 2sec unit
- Issues around mapping with UNICODE and local char code
1-N mapping SJIS<-> UNICODE
Potential directory cache problem due to 1 –N mapping
Possible inconsistency problems with application side
- Support file size over 2GB
- Support dirty flag
Q1 : First issue for discussion is "Do you have any other issues
about this?" and "Do you have any other idea to categorize
the issues?"
2. FAT corruption on unplug media/storage device
On starting the open source project, we focus to the following issue,
first.
- File System corruption on unplug media/storage device
Almost same as power down without umount
And, we are planing to focus on HDD device and treat system power down
instead of unplug media, because
A. Damages and it's counter methods may depend on property of lower
layer
E.g.
- Memory Card
Some controller can guaranty atomicity of certain
operations
- Flush Memory (NAND, NOR)
I/O operations may be constrained by Block Size
(e,g, 128KB) or Page Size (e.g. 2KB)
- HDD
- Cache memory my resident inside in
- Sector which is under writing
on power down may be corrupted(can't read anymore)
B. It may make the problem easier
- Sector size is 512 Byte
- Many developers may check with PC
Q2 : Do you know any other storage devices and it's property, to
be address later?
3. Features to be developed for FAT corruption.
We currently plan to add following features to address FAT corruption.
- Utilize standard 2.6 features as much as possible
- Implement as options of fat, vfat and uvfat
- Utilize existent journal block device (JBD) for transaction control
- Utilize noop elevator to cancel unexpected operation
reordering
- Coordinate order of operations so that update data first, meta
data later with transaction control
- With O_SYNC, close() make flush all related data and
meta-data, then wait completion of I/O
Q3 : I'm not sure JBD can be used for FAT improvements.
Do you have any comments ?
Thanks,
Hiroyuki Machida
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
