Re: [zfs-discuss] Digging in the bowels of ZFS

2012-12-11 Thread Jim Klimov
On 2012-12-11 16:44, Jim Klimov wrote: For single-break-per-row tests based on hypotheses from P parities, D data disks and R broken rows, we need to checksum P*(D^R) userdata recombinations in order to determine that we can't recover the block. A small maths correction: the formula above refle

Re: [zfs-discuss] Digging in the bowels of ZFS

2012-12-11 Thread Jim Klimov
On 2012-12-02 05:42, Jim Klimov wrote: My plan is to dig out the needed sectors of the broken block from each of the 6 disks and try any and all reasonable recombinations of redundancy and data sectors to try and match the checksum - this should be my definite answer on whether ZFS (of that oi151

Re: [zfs-discuss] Digging in the bowels of ZFS

2012-12-10 Thread Jim Klimov
On 2012-12-10 07:35, Timothy Coalson wrote: The corrupted area looks like a series of "0xFC 0x42" bytes about half a kilobyte long, followed by zero bytes to the end of sector. Start of this area is not aligned to a multiple of 512 bytes. Just a guess, but that might be how the sect

Re: [zfs-discuss] Digging in the bowels of ZFS

2012-12-09 Thread Timothy Coalson
On Sun, Dec 9, 2012 at 1:27 PM, Jim Klimov wrote: > In two of three cases, some of the sectors (in the range which > mismatches the parity data) are not only clearly invalid, like > being filled with long stretches of zeroes with other sectors > being uniformly-looking binary data (results of com

Re: [zfs-discuss] Digging in the bowels of ZFS

2012-12-09 Thread Jim Klimov
more below... On 2012-12-06 03:06, Jim Klimov wrote: It also happens that on disks 1,2,3 the first row's sectors (d0, d2, d3) are botched - ranges from 0x9C0 to 0xFFF (end of 4KB sector) are zeroes. The neighboring blocks, located a few sectors away from this one, also have compressed data and

Re: [zfs-discuss] Digging in the bowels of ZFS

2012-12-05 Thread Jim Klimov
more below... On 2012-12-05 23:16, Timothy Coalson wrote: On Tue, Dec 4, 2012 at 10:52 PM, Jim Klimov mailto:jimkli...@cos.ru>> wrote: On 2012-12-03 18:23, Jim Klimov wrote: On 2012-12-02 05:42, Jim Klimov wrote: >> 4) Where are the redundancy algorithms specified? Is there

Re: [zfs-discuss] Digging in the bowels of ZFS

2012-12-05 Thread Jim Klimov
On 2012-12-05 05:52, Jim Klimov wrote: For undersized allocations, i.e. of compressed data, it is possible to see P-sizes not divisible by 4 (disks) in 4KB sectors, however, some sectors do apparently get wasted because the A-size in the DVA is divisible by 6*4KB. With columnar allocation of disk

Re: [zfs-discuss] Digging in the bowels of ZFS

2012-12-05 Thread Timothy Coalson
On Tue, Dec 4, 2012 at 10:52 PM, Jim Klimov wrote: > On 2012-12-03 18:23, Jim Klimov wrote: > >> On 2012-12-02 05:42, Jim Klimov wrote: >> >>> >>> >> 4) Where are the redundancy algorithms specified? Is there any simple > >> tool that would recombine a given algo-N redundancy sector with > >>

Re: [zfs-discuss] Digging in the bowels of ZFS

2012-12-04 Thread Jim Klimov
On 2012-12-03 18:23, Jim Klimov wrote: On 2012-12-02 05:42, Jim Klimov wrote: So... here are some applied questions: Well, I am ready to reply a few of my own questions now :) Continuing the desecration of my deceased files' resting grounds... 2) Do I understand correctly that for the offs

Re: [zfs-discuss] Digging in the bowels of ZFS

2012-12-03 Thread Jim Klimov
On 2012-12-02 05:42, Jim Klimov wrote: So... here are some applied questions: Well, I am ready to reply a few of my own questions now :) I've staged an experiment by taking a 128Kb block from that file and appending it to a new file in a test dataset, where I changed the compression settings b