Re: ZFS PANIC: HELP.

2022-02-27 Thread Larry Rosenman
On 02/27/2022 3:58 pm, Mark Johnston wrote: On Sun, Feb 27, 2022 at 01:16:44PM -0600, Larry Rosenman wrote: On 02/26/2022 11:08 am, Larry Rosenman wrote: > On 02/26/2022 10:57 am, Larry Rosenman wrote: >> On 02/26/2022 10:37 am, Juraj Lutter wrote: On 26 Feb 2022, at 03:03, Larry Rosenman

Re: ZFS PANIC: HELP.

2022-02-27 Thread Mark Johnston
On Sun, Feb 27, 2022 at 01:16:44PM -0600, Larry Rosenman wrote: > On 02/26/2022 11:08 am, Larry Rosenman wrote: > > On 02/26/2022 10:57 am, Larry Rosenman wrote: > >> On 02/26/2022 10:37 am, Juraj Lutter wrote: > On 26 Feb 2022, at 03:03, Larry Rosenman wrote: > I'm running this script:

Re: ZFS PANIC: HELP.

2022-02-27 Thread Michael Butler
On 2/27/22 16:09, Larry Rosenman wrote: On 02/27/2022 3:03 pm, Michael Butler wrote: [ cc list trimmed ] On 2/27/22 14:16, Larry Rosenman wrote: I was able to export the rest of the datasets, and re-install 14-CURRENT from a recent snapshot, and restore the datasets I care about. I'm now

Re: ZFS PANIC: HELP.

2022-02-27 Thread Larry Rosenman
On 02/27/2022 3:03 pm, Michael Butler wrote: [ cc list trimmed ] On 2/27/22 14:16, Larry Rosenman wrote: I was able to export the rest of the datasets, and re-install 14-CURRENT from a recent snapshot, and restore the datasets I care about. I'm now seeing: mfi0: IOCTL 0x40086481 not handle

Re: ZFS PANIC: HELP.

2022-02-27 Thread Michael Butler
[ cc list trimmed ] On 2/27/22 14:16, Larry Rosenman wrote: I was able to export the rest of the datasets, and re-install 14-CURRENT from a recent snapshot, and restore the datasets I care about. I'm now seeing: mfi0: IOCTL 0x40086481 not handled mfi0: IOCTL 0x40086481 not handled mfi0: IOC

Re: ZFS PANIC: HELP.

2022-02-27 Thread Larry Rosenman
On 02/26/2022 11:08 am, Larry Rosenman wrote: On 02/26/2022 10:57 am, Larry Rosenman wrote: On 02/26/2022 10:37 am, Juraj Lutter wrote: On 26 Feb 2022, at 03:03, Larry Rosenman wrote: I'm running this script: #!/bin/sh for i in $(zfs list -H | awk '{print $1}') do FS=$1 FN=$(echo ${FS} | s

Re: ZFS PANIC: HELP.

2022-02-26 Thread Larry Rosenman
On 02/26/2022 10:57 am, Larry Rosenman wrote: On 02/26/2022 10:37 am, Juraj Lutter wrote: On 26 Feb 2022, at 03:03, Larry Rosenman wrote: I'm running this script: #!/bin/sh for i in $(zfs list -H | awk '{print $1}') do FS=$1 FN=$(echo ${FS} | sed -e s@/@_@g) sudo zfs send -vecLep ${FS}@RE

Re: ZFS PANIC: HELP.

2022-02-26 Thread Larry Rosenman
On 02/26/2022 10:37 am, Juraj Lutter wrote: On 26 Feb 2022, at 03:03, Larry Rosenman wrote: I'm running this script: #!/bin/sh for i in $(zfs list -H | awk '{print $1}') do FS=$1 FN=$(echo ${FS} | sed -e s@/@_@g) sudo zfs send -vecLep ${FS}@REPAIR_SNAP | ssh l...@freenas.lerctr.org cat -

Re: ZFS PANIC: HELP.

2022-02-26 Thread Alexander Leidinger
Quoting Larry Rosenman (from Fri, 25 Feb 2022 20:03:51 -0600): On 02/25/2022 2:11 am, Alexander Leidinger wrote: Quoting Larry Rosenman (from Thu, 24 Feb 2022 20:19:45 -0600): I tried a scrub -- it panic'd on a fatal double fault.  Suggestions? The safest / cleanest (but

Re: ZFS PANIC: HELP.

2022-02-25 Thread Larry Rosenman
On 02/25/2022 2:11 am, Alexander Leidinger wrote: Quoting Larry Rosenman (from Thu, 24 Feb 2022 20:19:45 -0600): I tried a scrub -- it panic'd on a fatal double fault. Suggestions? The safest / cleanest (but not fastest) is data export and pool re-creation. If you export dataset by dat

Re: ZFS PANIC: HELP.

2022-02-25 Thread Alexander Leidinger
Quoting Larry Rosenman (from Thu, 24 Feb 2022 20:19:45 -0600): I tried a scrub -- it panic'd on a fatal double fault.  Suggestions? The safest / cleanest (but not fastest) is data export and pool re-creation. If you export dataset by dataset (instead of recursively all), you can eve

Re: ZFS PANIC: HELP.

2022-02-24 Thread Larry Rosenman
On 02/24/2022 8:07 pm, Larry Rosenman wrote: On 02/24/2022 1:27 pm, Larry Rosenman wrote: On 02/24/2022 10:48 am, Rob Wing wrote: even with those set, I still get the panid. :( Let me see if I can compile a 14 non-INVARIANTS kernel on the 13-REL system. UGH. I chroot'd to the pool, and

Re: ZFS PANIC: HELP.

2022-02-24 Thread Larry Rosenman
On 02/24/2022 1:27 pm, Larry Rosenman wrote: On 02/24/2022 10:48 am, Rob Wing wrote: even with those set, I still get the panid. :( Let me see if I can compile a 14 non-INVARIANTS kernel on the 13-REL system. UGH. I chroot'd to the pool, and built a no invariants kernel. It booted an

Re: ZFS PANIC: HELP.

2022-02-24 Thread Larry Rosenman
On 02/24/2022 10:48 am, Rob Wing wrote: Yes, I believe so. On Thu, Feb 24, 2022 at 7:42 AM Larry Rosenman wrote: On 02/24/2022 10:36 am, Rob Wing wrote: You might try setting `sysctl vfs.zfs.recover=1` and `sysctl vfs.zfs.spa.load_verify_metadata=0`. I had a similar error the other day

Re: ZFS PANIC: HELP.

2022-02-24 Thread Rob Wing
Yes, I believe so. On Thu, Feb 24, 2022 at 7:42 AM Larry Rosenman wrote: > On 02/24/2022 10:36 am, Rob Wing wrote: > > You might try setting `sysctl vfs.zfs.recover=1` and `sysctl > vfs.zfs.spa.load_verify_metadata=0`. > > I had a similar error the other day (couple months ago). The best I did >

Re: ZFS PANIC: HELP.

2022-02-24 Thread Larry Rosenman
On 02/24/2022 10:36 am, Rob Wing wrote: You might try setting `sysctl vfs.zfs.recover=1` and `sysctl vfs.zfs.spa.load_verify_metadata=0`. I had a similar error the other day (couple months ago). The best I did was being able to import the pool read only. I ended up restoring from backup.

Re: ZFS PANIC: HELP.

2022-02-24 Thread Rob Wing
You might try setting `sysctl vfs.zfs.recover=1` and `sysctl vfs.zfs.spa.load_verify_metadata=0`. I had a similar error the other day (couple months ago). The best I did was being able to import the pool read only. I ended up restoring from backup. On Thu, Feb 24, 2022 at 7:30 AM Alexander Motin

Re: ZFS PANIC: HELP.

2022-02-24 Thread Larry Rosenman
On 02/24/2022 10:29 am, Alexander Motin wrote: On 24.02.2022 10:57, Larry Rosenman wrote: On 02/23/2022 9:27 pm, Larry Rosenman wrote: It crashes just after root mount (this is the boot pool and only pool on the system), seeL https://www.lerctr.org/~ler/14-BOOT-Crash.png Where do I go from he

Re: ZFS PANIC: HELP.

2022-02-24 Thread Alexander Motin
On 24.02.2022 10:57, Larry Rosenman wrote: On 02/23/2022 9:27 pm, Larry Rosenman wrote: It crashes just after root mount (this is the boot pool and only pool on the system), seeL https://www.lerctr.org/~ler/14-BOOT-Crash.png Where do I go from here? I see 2 ways: 1) Since it is only an asser

Re: ZFS PANIC: HELP.

2022-02-24 Thread Larry Rosenman
On 02/23/2022 9:27 pm, Larry Rosenman wrote: On 02/23/2022 9:15 pm, Alexander Motin wrote: On 23.02.2022 22:01, Larry Rosenman wrote: On 02/23/2022 8:58 pm, Alexander Motin wrote: On 23.02.2022 21:52, Larry Rosenman wrote: On 02/23/2022 8:41 pm, Alexander Motin wrote: Hi Larry, The panic yo

Re: ZFS PANIC: HELP.

2022-02-23 Thread Larry Rosenman
On 02/23/2022 9:15 pm, Alexander Motin wrote: On 23.02.2022 22:01, Larry Rosenman wrote: On 02/23/2022 8:58 pm, Alexander Motin wrote: On 23.02.2022 21:52, Larry Rosenman wrote: On 02/23/2022 8:41 pm, Alexander Motin wrote: Hi Larry, The panic you are getting is an assertion, enabled by kern

Re: ZFS PANIC: HELP.

2022-02-23 Thread Alexander Motin
On 23.02.2022 22:01, Larry Rosenman wrote: On 02/23/2022 8:58 pm, Alexander Motin wrote: On 23.02.2022 21:52, Larry Rosenman wrote: On 02/23/2022 8:41 pm, Alexander Motin wrote: Hi Larry, The panic you are getting is an assertion, enabled by kernel built with INVARIANTS option.  On 13 you may

Re: ZFS PANIC: HELP.

2022-02-23 Thread Larry Rosenman
On 02/23/2022 8:58 pm, Alexander Motin wrote: On 23.02.2022 21:52, Larry Rosenman wrote: On 02/23/2022 8:41 pm, Alexander Motin wrote: Hi Larry, The panic you are getting is an assertion, enabled by kernel built with INVARIANTS option.  On 13 you may just not have that debugging enabled to hit

Re: ZFS PANIC: HELP.

2022-02-23 Thread Alexander Motin
On 23.02.2022 21:52, Larry Rosenman wrote: On 02/23/2022 8:41 pm, Alexander Motin wrote: Hi Larry, The panic you are getting is an assertion, enabled by kernel built with INVARIANTS option.  On 13 you may just not have that debugging enabled to hit the issue.  But that may be only a consequence

Re: ZFS PANIC: HELP.

2022-02-23 Thread Larry Rosenman
On 02/23/2022 8:41 pm, Alexander Motin wrote: Hi Larry, The panic you are getting is an assertion, enabled by kernel built with INVARIANTS option. On 13 you may just not have that debugging enabled to hit the issue. But that may be only a consequence. Original problem I guess in possibly corru

Re: ZFS PANIC: HELP.

2022-02-23 Thread Alexander Motin
Hi Larry, The panic you are getting is an assertion, enabled by kernel built with INVARIANTS option. On 13 you may just not have that debugging enabled to hit the issue. But that may be only a consequence. Original problem I guess in possibly corrupted ZFS intent log records (or false posi