Re: [zfs-discuss] ZFS snapshot send/recv "hangs" X4540 servers

2009-06-30 Thread Ian Collins
Tim Haley wrote: Ian Collins wrote: Ian Collins wrote: Tim Haley wrote: Brent Jones wrote: On the sending side, I CAN kill the ZFS send process, but the remote side leaves its processes going, and I CANNOT kill -9 them. I also cannot reboot the receiving system, at init 6, the system will ju

Re: [zfs-discuss] ZFS snapshot send/recv "hangs" X4540 servers

2009-06-11 Thread Brent Jones
>> >> > After examining the dump we got from you (thanks again), we're relatively > sure you are hitting > > 6826836 Deadlock possible in dmu_object_reclaim() > > This was introduced in nv_111 and fixed in nv_113. > > Sorry for the trouble. > > -tim > > Do you know when new builds will show up on

Re: [zfs-discuss] ZFS snapshot send/recv "hangs" X4540 servers

2009-06-10 Thread Tim Haley
Brent Jones wrote: On Mon, Jun 8, 2009 at 9:38 PM, Richard Lowe wrote: Brent Jones writes: I've had similar issues with similar traces. I think you're waiting on a transaction that's never going to come. I thought at the time that I was hitting: CR 6367701 "hang because tx_state_t is in

Re: [zfs-discuss] ZFS snapshot send/recv "hangs" X4540 servers

2009-06-08 Thread Brent Jones
On Mon, Jun 8, 2009 at 9:38 PM, Richard Lowe wrote: > Brent Jones writes: > > > I've had similar issues with similar traces.  I think you're waiting on > a transaction that's never going to come. > > I thought at the time that I was hitting: >   CR 6367701 "hang because tx_state_t is inconsistent

Re: [zfs-discuss] ZFS snapshot send/recv "hangs" X4540 servers

2009-06-08 Thread Richard Lowe
Brent Jones writes: >> >> I haven't figured out a way to identify the problem, still trying to >> find a 100% way to reproduce this problem. >> Seemingly the more snapshots I send at a given time, the likelihood of >> this happening goes up, but, correlation is not causation  :) >> >> I might try

Re: [zfs-discuss] ZFS snapshot send/recv "hangs" X4540 servers

2009-06-08 Thread Brent Jones
> > I haven't figured out a way to identify the problem, still trying to > find a 100% way to reproduce this problem. > Seemingly the more snapshots I send at a given time, the likelihood of > this happening goes up, but, correlation is not causation  :) > > I might try to open a support case with

Re: [zfs-discuss] ZFS snapshot send/recv "hangs" X4540 servers

2009-06-08 Thread Brent Jones
On Sun, Jun 7, 2009 at 3:50 AM, Ian Collins wrote: > Ian Collins wrote: >> >> Tim Haley wrote: >>> >>> Brent Jones wrote: On the sending side, I CAN kill the ZFS send process, but the remote side leaves its processes going, and I CANNOT kill -9 them. I also cannot reboot the rec

Re: [zfs-discuss] ZFS snapshot send/recv "hangs" X4540 servers

2009-06-07 Thread Ian Collins
Ian Collins wrote: Tim Haley wrote: Brent Jones wrote: On the sending side, I CAN kill the ZFS send process, but the remote side leaves its processes going, and I CANNOT kill -9 them. I also cannot reboot the receiving system, at init 6, the system will just hang trying to unmount the file sys

Re: [zfs-discuss] ZFS snapshot send/recv "hangs" X4540 servers

2009-06-06 Thread Brent Jones
> > Well, I think I found a specific file system that is causing this. > I kicked off a zpool scrub to see if there might be corruption on > either end, but that takes well over 40 hours on these servers. > > > -- > Brent Jones > br...@servuhome.net > It turns out that the file system I believed w

Re: [zfs-discuss] ZFS snapshot send/recv "hangs" X4540 servers

2009-06-05 Thread Brent Jones
On Fri, Jun 5, 2009 at 4:20 PM, Tim Haley wrote: > Brent Jones wrote: >> >> Hello all, >> I had been running snv_106 for about 3 or 4 months on a pair of X4540's. >> I would ship snapshots from the primary server to the secondary server >> nightly, which was working really well. >> >> However, I h

Re: [zfs-discuss] ZFS snapshot send/recv "hangs" X4540 servers

2009-06-05 Thread Brent Jones
On Fri, Jun 5, 2009 at 4:20 PM, Tim Haley wrote: > Brent Jones wrote: >> >> Hello all, >> I had been running snv_106 for about 3 or 4 months on a pair of X4540's. >> I would ship snapshots from the primary server to the secondary server >> nightly, which was working really well. >> >> However, I h

Re: [zfs-discuss] ZFS snapshot send/recv "hangs" X4540 servers

2009-06-05 Thread Ian Collins
Tim Haley wrote: Brent Jones wrote: On the sending side, I CAN kill the ZFS send process, but the remote side leaves its processes going, and I CANNOT kill -9 them. I also cannot reboot the receiving system, at init 6, the system will just hang trying to unmount the file systems. I have to phys

Re: [zfs-discuss] ZFS snapshot send/recv "hangs" X4540 servers

2009-06-05 Thread Tim Haley
Brent Jones wrote: Hello all, I had been running snv_106 for about 3 or 4 months on a pair of X4540's. I would ship snapshots from the primary server to the secondary server nightly, which was working really well. However, I have upgraded to 2009.06, and my replication scripts appear to "hang" w

Re: [zfs-discuss] ZFS snapshot send/recv "hangs" X4540 servers

2009-06-05 Thread Ian Collins
Brent Jones wrote: On Fri, Jun 5, 2009 at 3:25 PM, Ian Collins wrote: Brent Jones wrote: On the sending side, I CAN kill the ZFS send process, but the remote side leaves its processes going, and I CANNOT kill -9 them. I also cannot reboot the receiving system, at init 6, the system wil

Re: [zfs-discuss] ZFS snapshot send/recv "hangs" X4540 servers

2009-06-05 Thread Brent Jones
On Fri, Jun 5, 2009 at 3:25 PM, Ian Collins wrote: > Brent Jones wrote: >> >> On the sending side, I CAN kill the ZFS send process, but the remote >> side leaves its processes going, and I CANNOT kill -9 them. I also >> cannot reboot the receiving system, at init 6, the system will just >> hang tr

Re: [zfs-discuss] ZFS snapshot send/recv "hangs" X4540 servers

2009-06-05 Thread Ian Collins
Brent Jones wrote: On the sending side, I CAN kill the ZFS send process, but the remote side leaves its processes going, and I CANNOT kill -9 them. I also cannot reboot the receiving system, at init 6, the system will just hang trying to unmount the file systems. I have to physically cut power t

[zfs-discuss] ZFS snapshot send/recv "hangs" X4540 servers

2009-06-05 Thread Brent Jones
Hello all, I had been running snv_106 for about 3 or 4 months on a pair of X4540's. I would ship snapshots from the primary server to the secondary server nightly, which was working really well. However, I have upgraded to 2009.06, and my replication scripts appear to "hang" when performing zfs se