On Thu, 17 Oct 2013, David Vasek wrote:
On Fri, 11 Oct 2013, Joel Sing wrote:
On Thu, 10 Oct 2013, Manuel Giraud wrote:
Hi,
I have a ntfs partition with rather large (about 3GB) files on it. When
I copy these files on a ffs partition they are corrupted. When I try to
checksum them directly from the ntfs partition the checksum is not
correct (compared to the same file on a fat32 partition copied with
Windows).
I tried this (with same behaviour) on i386 5.3 release and on i386 last
week current. I'm willing to do some testing to fix this issue but don't
really know where to start.
See if you can isolate the smallest possible reproducable test case. If you
create a 3GB file with known content (e.g. the same byte repeated), does
the
same issue occur? If so, how small do you need to go before the problem
goes
away? Also, what operating system (and version) was used to write the files
to the NTFS volume?
Hello, I encountered the same issue. Anything over the 2 GB limit is wrong. I
mean, first exactly 2 GB of the file are read correctly, following that I get
wrong data till the end of the file. It is reproducible with any file over 2
GB in size so far. Smells like int somewhere... I get the same wrong data
with any release since at least 5.0, didn't test anything older, but I bet it
is the same.
The filesystem is a Windows XP NTFS system disk, 32-bit, the files were
copied there with explorer.exe.
Some additional notes and findings:
(1)
The data I receive after first 2 GB are not part of the file, the data is
from another file (from the same directory, if that fact could be
important). The data is taken in uninterrupted sequence and the starting
offset of that sequence is way less than 2 GB in the other file where the
data belong.
(2)
While reading past 2 GB in larger blocks gives me just wrong data, reading
in smaller blocks (2kB and less) gives me kernel panic in KASSERT
immediately when I read past the 2 GB limit. It is 100% reproducible with
any file larger than 2 GB so far.
# mount -r /dev/wd0i /mnt
# ls -lo /mnt/DATA/ntfs_2gb_test.bin
-rwxr-xr-x 1 root wheel - 3054813184 Oct 17 22:11 /mnt/DATA/ntfs_2gb_test.bin
# cat /mnt/DATA//ntfs_2gb_test.bin > /dev/null
# dd if=/mnt/DATA/ntfs_2gb_test.bin bs=4k of=/dev/null
745804+0 records in
745804+0 records out
3054813184 bytes transferred in 108.518 secs (28150083 bytes/sec)
# dd if=/mnt/DATA/ntfs_2gb_test.bin bs=2k count=1m of=/dev/null
1048576+0 records in
1048576+0 records out
2147483648 bytes transferred in 78.783 secs (27258052 bytes/sec)
# dd if=/mnt/DATA/ntfs_2gb_test.bin bs=1k count=2m of=/dev/null
2097152+0 records in
2097152+0 records out
2147483648 bytes transferred in 81.210 secs (26443280 bytes/sec)
# dd if=/mnt/DATA/ntfs_2gb_test.bin bs=4k skip=512k of=/dev/null
221516+0 records in
221516+0 records out
907329536 bytes transferred in 32.314 secs (28077667 bytes/sec)
# dd if=/mnt/DATA/ntfs_2gb_test.bin bs=2k skip=1m of=/dev/null
panic: kernel diagnostic assertion "cl == 1 && tocopy <= ntfs_cntob(1)" failed: file
"../../../../ntfs/ntfs_subr.c", line 1556
Stopped at Debugger+0x4: popl %ebp
RUN AT LEAST 'trace' AND 'ps' AND INCLUDE OUTPUT WHEN REPORTING THIS PANIC!
DO NOT EVEN BOTHER REPORTING THIS WITHOUT INCLUDING THAT INFORMATION!
ddb> trace
Debugger(d08fdcbc,f544fb88,d08dc500,f544fb88,200) at Debugger+0x4
panic(d08dc500,d085fc0e,d08dfe60,d08e00b0,614) at panic+0x5d
__assert(d085fc0e,d08e00b0,614,d08dfe60,80000) at __assert+0x2e
ntfs_readntvattr_plain(d1a2d200,d1a36200,d1a5bc00,80000800,0) at ntfs_readntvat
tr_plain+0x2e6
ntfs_readattr_plain(d1a2d200,d1a36200,80,0,80000800) at ntfs_readattr_plain+0x1
41
ntfs_readattr(d1a2d200,d1a36200,80,0,80000800) at ntfs_readattr+0x156
ntfs_read(f544fddc,d64e5140,d6522a60,f544fea0,0) at ntfs_read+0xa8
VOP_READ(d6522a60,f544fea0,0,d6599000,d64e5140) at VOP_READ+0x35
vn_read(d65290a8,d65290c4,f544fea0,d6599000,0) at vn_read+0xb5
dofilereadv(d65365d4,3,d65290a8,f544ff08,1) at dofilereadv+0x13a
sys_read(d65365d4,f544ff64,f544ff84,106,d653f100) at sys_read+0x89
syscall() at syscall+0x227
--- syscall (number 0) ---
0x2:
ddb> ps
PID PPID PGRP UID S FLAGS WAIT COMMAND
*19967 9961 19967 0 7 0 dd
9961 1 9961 0 3 0x88 pause sh
14 0 0 0 3 0x100200 aiodoned aiodoned
13 0 0 0 3 0x100200 syncer update
12 0 0 0 3 0x100200 cleaner cleaner
11 0 0 0 3 0x100200 reaper reaper
10 0 0 0 3 0x100200 pgdaemon pagedaemon
9 0 0 0 3 0x100200 bored crypto
8 0 0 0 3 0x100200 pftm pfpurge
7 0 0 0 3 0x100200 usbtsk usbtask
6 0 0 0 3 0x100200 usbatsk usbatsk
5 0 0 0 3 0x100200 acpi0 acpi0
4 0 0 0 3 0x100200 bored syswq
3 0 0 0 3 0x40100200 idle0
2 0 0 0 3 0x100200 kmalloc kmthread
1 0 1 0 3 0x80 wait init
0 -1 0 0 3 0x200 scheduler swapper
ddb> show registers
ds 0x10
es 0x10
fs 0x20
gs 0
edi 0xd08dc500 addrmask+0x2920
esi 0x100
ebp 0xf544fb3c
ebx 0xf544fb88
edx 0x1
ecx 0xd09e6f94 kprintf_mutex
eax 0x1
eip 0xd057b3f4 Debugger+0x4
cs 0x8
eflags 0x202
esp 0xf544fb3c
ss 0x10
Debugger+0x4: popl %ebp
ddb> show panic
kernel diagnostic assertion "cl == 1 && tocopy <= ntfs_cntob(1)" failed: file "
../../../../ntfs/ntfs_subr.c", line 1556
ddb>
A few other important bits:
OpenBSD 5.3 (GENERIC) #50: Tue Mar 12 18:35:23 MDT 2013
dera...@i386.openbsd.org:/usr/src/sys/arch/i386/compile/GENERIC
# fdisk wd0
Disk: wd0 geometry: 20673/240/63 [312581808 Sectors]
Offset: 0 Signature: 0xAA55
Starting Ending LBA Info:
#: id C H S - C H S [ start: size ]
-------------------------------------------------------------------------------
*0: 07 0 1 1 - 20671 239 63 [ 63: 312560577 ] NTFS
1: 00 0 0 0 - 0 0 0 [ 0: 0 ] unused
2: 00 0 0 0 - 0 0 0 [ 0: 0 ] unused
3: 00 0 0 0 - 0 0 0 [ 0: 0 ] unused
# disklabel wd0
# /dev/rwd0c:
type: ESDI
disk: ESDI/IDE disk
label: Hitachi HTS54161
duid: 0000000000000000
flags:
bytes/sector: 512
sectors/track: 63
tracks/cylinder: 240
sectors/cylinder: 15120
cylinders: 20673
total sectors: 312581808
boundstart: 0
boundend: 312581808
drivedata: 0
16 partitions:
# size offset fstype [fsize bsize cpg]
c: 312581808 0 unused
i: 312560577 63 NTFS
Regards,
David
