Hi, reminds me about this dedup bug, don't use the "-d" switch in zfs send, it
produces broken stream that you won't be able to receive.
--
This message posted from opensolaris.org
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.ope
On Wed, Jan 5, 2011 at 1:16 PM, Cindy Swearingen
wrote:
> You might review your zpool history -il rpool output for clues.
This isn't a critical problem, it's just a point of annoyance since it
seems like something that shouldn't happen. It's also just a test host
that's led a hard life full of ab
We installed b151a and couldn't reproduce a failed receive of a
recursive root pool snapshot and also tested on b152 and b155.
The original error message isn't very helpful, but your test below
points to a problem in your root pool environment.
You might review your zpool history -il rpool outpu
On Wed, Jan 5, 2011 at 11:57 AM, Cindy Swearingen
wrote:
> In the meantime, you could rule out a problem with zfs send/recv on your
> system if you could create another non-BE dataset with descendent
> datasets, create a recursive snapshot, and retry the recursive send/recv
> operation.
That appe
Okay. We are trying again to reproduce this on b151a.
In the meantime, you could rule out a problem with zfs send/recv on your
system if you could create another non-BE dataset with descendent
datasets, create a recursive snapshot, and retry the recursive send/recv
operation.
Thanks,
Cindy
On Wed, Jan 5, 2011 at 9:44 AM, Cindy Swearingen
wrote:
> In your follow-up, I think you are saying that rp...@copy is a recursive
> snapshot and you are able to receive the individual rpool snapshots. You
> just can't receive the recursive snapshot. Is this correct?
Sorry, I didn't really expla
Hi Brandon,
I'm not the right person to evaluate your zstreamdump output, but I
can't reproduce this error on my b152 system, which as close as I
could get to b151a. See below.
Are the rpool and radar pool versions reasonably equivalent?
In your follow-up, I think you are saying that rp...@copy
On an snv_151a system I'm trying to do a send of rpool, and works when
using -n, but when I actually try to receive it's failing.
scrubs pass without issue, it's just the recv that fails.
# zfs send -R rp...@copy | zfs recv -n -vduF radar/foo
would receive full stream of rp...@copy into radar/f..
Ross wrote:
[context, please!]
Why are you sending from s1? If you've already sent that, the logical thing to
do is send from s3 the next time.
If you really do need to send from the start every time, you can do that with
the -f option on zfs receive, to force it to overwrite newer changes,
Why are you sending from s1? If you've already sent that, the logical thing to
do is send from s3 the next time.
If you really do need to send from the start every time, you can do that with
the -f option on zfs receive, to force it to overwrite newer changes, but you
are going to be sending f
Raghav Ilavarasu wrote:
Hi there,
While receiving incremental streams, "zfs recv" ignores the existing
snapshots and stops without processing rest of the streams.
Here is the scenario.
# zfs snapshot sp...@s1
# zfs send sp...@s1 | zfs recv dpath
# zfs snapshot sp...@s2
# zfs snapshot sp..
Hi there,
While receiving incremental streams, "zfs recv" ignores the existing
snapshots and stops without processing rest of the streams.
Here is the scenario.
# zfs snapshot sp...@s1
# zfs send sp...@s1 | zfs recv dpath
# zfs snapshot sp...@s2
# zfs snapshot sp...@s3
# zfs send -I sp...@s1
I created http://defect.opensolaris.org/bz/show_bug.cgi?id=12249
--
Robert Milkowski
http://milek.blogspot.com
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Hi,
snv_123, x64
zfs recv -F complains it can't open a snapshot it just destroyed itself as it
was destroyed on a sending side. Other than complaining about it it finishes
successfully.
Below is an example where I created a filesystem fs1 with three snapshots of it
called snap1, snap2, snap3.
Tristan Ball wrote:
OK, Thanks for that.
From reading the RFE, it sound's like having a faster machine on the
receive side will be enough to alleviate the problem in the short term?
That's correct.
--matt
___
zfs-discuss mailing list
zfs-discuss@o
OK, Thanks for that.
From reading the RFE, it sound's like having a faster machine on the
receive side will be enough to alleviate the problem in the short term?
The hardware I'm using at the moment is quite old, and not particularly
fast - although this is the first out & out performance lim
Tristan Ball wrote:
Hi Everyone,
I have a couple of systems running opensolaris b118, one of which sends
hourly snapshots to the other. This has been working well, however as of
today, the receiving zfs process has started running extremely slowly,
and is running at 100% CPU on one core, comp
Hi Everyone,
I have a couple of systems running opensolaris b118, one of which sends
hourly snapshots to the other. This has been working well, however as
of today, the receiving zfs process has started running extremely
slowly, and is running at 100% CPU on one core, completely in kernel
mode
I'm running OpenSolaris 2009.06, and when I attempt to restore a ZFS snapshot,
the machine hangs in an odd fashion.
I create a backup of fs1 (roughly 15GB):
zfs send -R tank/f...@1 | gzip > /backups/test_1.gz
I create a new zpool to accept the backup:
zpool create testdev1 testpool
Then I att
Hello Brent,
Friday, February 13, 2009, 8:15:55 AM, you wrote:
BJ> Sad to report that I am seeing the slow zfs recv issue cropping up
BJ> again while running b105 :(
BJ> Not sure what has triggered the change, but I am seeing the same
BJ> behavior again: massive amounts of reads on the receivin
On Mon, Feb 2, 2009 at 6:55 AM, Robert Milkowski wrote:
> It definitely does. I made some tests today comparing b101 with b105 while
> doing 'zfs send -R -I A B >/dev/null' with several dozen snapshots between A
> and B. Well, b105 is almost 5x faster in my case - that's pretty good.
>
> --
> Ro
It definitely does. I made some tests today comparing b101 with b105 while
doing 'zfs send -R -I A B >/dev/null' with several dozen snapshots between A
and B. Well, b105 is almost 5x faster in my case - that's pretty good.
--
Robert Milkowski
http://milek.blogspot.com
--
This message posted fr
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Brent Jones:
> My results are much improved, on the order of 5-100 times faster
> (either over Mbuffer or SSH).
this is good news - although not quite soon enough for my current 5TB zfs send
;-)
have you tested if this also improves the performance
Brent Jones wrote:
> On Fri, Jan 9, 2009 at 11:41 PM, Brent Jones wrote:
>
>> On Fri, Jan 9, 2009 at 7:53 PM, Ian Collins wrote:
>>
>>> Ian Collins wrote:
>>>
Send/receive speeds appear to be very data dependent. I have several
different filesystems containing differing
On Fri, Jan 9, 2009 at 11:41 PM, Brent Jones wrote:
> On Fri, Jan 9, 2009 at 7:53 PM, Ian Collins wrote:
>> Ian Collins wrote:
>>> Send/receive speeds appear to be very data dependent. I have several
>>> different filesystems containing differing data types. The slowest to
>>> replicate is ma
On Fri, Jan 9, 2009 at 7:53 PM, Ian Collins wrote:
> Ian Collins wrote:
>> Send/receive speeds appear to be very data dependent. I have several
>> different filesystems containing differing data types. The slowest to
>> replicate is mail and my guess it's the changes to the index files that
>
Ian Collins wrote:
> Send/receive speeds appear to be very data dependent. I have several
> different filesystems containing differing data types. The slowest to
> replicate is mail and my guess it's the changes to the index files that takes
> the time. Similar sized filesystems with similar
Hello
> Yah, the incrementals are from a 30TB volume, with about 1TB used.
> Watching iostat on each side during the incremental sends, the sender
> side is hardly doing anything, maybe 50iops read, and that could be
> from other machines accessing it, really light load.
> The receiving side howev
On Thu 08/01/09 08:08 , "Brent Jones" br...@servuhome.net sent:
>
> I have yet to devise a script that starts Mbuffer zfs recv on the
> receiving side with proper parameters, then start an Mbuffer ZFS send
> on the sending side, but I may work on one later this week.
> I'd like the snapshots to be
On Wed, Jan 7, 2009 at 12:36 AM, Andrew Gabriel wrote:
> Brent Jones wrote:
>
>> Reviving an old discussion, but has the core issue been addressed in
>> regards to zfs send/recv performance issues? I'm not able to find any
>> new bug reports on bugs.opensolaris.org related to this, but my search
>
Brent Jones wrote:
> Reviving an old discussion, but has the core issue been addressed in
> regards to zfs send/recv performance issues? I'm not able to find any
> new bug reports on bugs.opensolaris.org related to this, but my search
> kung-fu may be weak.
I raised:
CR 6729347 Poor zfs receive p
On Wed 07/01/09 20:31 , Carsten Aulbert carsten.aulb...@aei.mpg.de sent:
> Brent Jones wrote:
> >
> > Using mbuffer can speed it up dramatically, but
> > this seems like a hack> without addressing a real problem with zfs
> > send/recv.> Trying to send any meaningful sized snapshots
> > from say an
Hi,
Brent Jones wrote:
>
> Using mbuffer can speed it up dramatically, but this seems like a hack
> without addressing a real problem with zfs send/recv.
> Trying to send any meaningful sized snapshots from say an X4540 takes
> up to 24 hours, for as little as 300GB changerate.
I have not found
On Sat, Dec 6, 2008 at 11:40 AM, Ian Collins wrote:
> Richard Elling wrote:
>> Ian Collins wrote:
>>> Ian Collins wrote:
Andrew Gabriel wrote:
> Ian Collins wrote:
>> I've just finished a small application to couple zfs_send and
>> zfs_receive through a socket to remove ssh from t
Richard Elling wrote:
> Ian Collins wrote:
>> Ian Collins wrote:
>>> Andrew Gabriel wrote:
Ian Collins wrote:
> I've just finished a small application to couple zfs_send and
> zfs_receive through a socket to remove ssh from the equation and the
> speed up is better than 2x.
Ian Collins wrote:
> Andrew Gabriel wrote:
>
>> Ian Collins wrote:
>>
>>> I've just finished a small application to couple zfs_send and
>>> zfs_receive through a socket to remove ssh from the equation and the
>>> speed up is better than 2x. I have a small (140K) buffer on the sending
>>> s
Andrew Gabriel wrote:
> Ian Collins wrote:
>> I've just finished a small application to couple zfs_send and
>> zfs_receive through a socket to remove ssh from the equation and the
>> speed up is better than 2x. I have a small (140K) buffer on the sending
>> side to ensure the minimum number of sen
Ian Collins wrote:
> I've just finished a small application to couple zfs_send and
> zfs_receive through a socket to remove ssh from the equation and the
> speed up is better than 2x. I have a small (140K) buffer on the sending
> side to ensure the minimum number of sent packets
>
> The times I g
Andrew Gabriel wrote:
> Ian Collins wrote:
>>
>>
>> I don't see the 5 second bursty behaviour described in the bug
>> report. It's more like 5 second interval gaps in the network traffic
>> while the
>> data is written to disk.
>
> That is exactly the issue. When the zfs recv data has been written,
[EMAIL PROTECTED] wrote:
>
>
> >BTW: a lot of numbers in Solaris did not grow since a long time and
> >thus create problems now. Just think about the maxphys values
> >63 kB on x86 does not even allow to write a single BluRay disk sector
> >with a single transfer.
>
>
> Any "fixed value" will
>
> Seems like there's a strong case to have such a program bundled in Solaris.
>
I think, the idea of having a separate configurable buffer program with a high
feature set fits into UNIX philosophy of having small programs that can be used
as building blocks to solve larger problems.
mbuffe
Bob Friesenhahn wrote:
> On Fri, 14 Nov 2008, Joerg Schilling wrote:
>>> -----
>>> Disk RPM 3,600 10,000x3
>>
>> The best rate I did see in 1985 was 800 kB/s (w. linear reads)
>> now I see 120 MB/s this is more than x100 ;
>On Fri, 14 Nov 2008, Joerg Schilling wrote:
>>> -----
>>> Disk RPM 3,600 10,000x3
>>
>> The best rate I did see in 1985 was 800 kB/s (w. linear reads)
>> now I see 120 MB/s this is more than x100 ;-)
>
>Yes. And how tha
Bob Friesenhahn <[EMAIL PROTECTED]> wrote:
> On Fri, 14 Nov 2008, Joerg Schilling wrote:
> >> -----
> >> Disk RPM 3,600 10,000x3
> >
> > The best rate I did see in 1985 was 800 kB/s (w. linear reads)
> > now I see 120 MB/
On Fri, 14 Nov 2008, Joerg Schilling wrote:
>> -----
>> Disk RPM 3,600 10,000x3
>
> The best rate I did see in 1985 was 800 kB/s (w. linear reads)
> now I see 120 MB/s this is more than x100 ;-)
Yes. And how that SSDs ar
[EMAIL PROTECTED] wrote:
> But zfs could certainly use bigger buffers; just like mbuffer, I also
> wrote my own "pipebuffer" which does pretty much the same.
You too? (My "buffer" program which I used to diagnose the problem is
attached to the bugid ;-)
I know Chris Gerhard wrote one too.
Seems
Andrew Gabriel <[EMAIL PROTECTED]> wrote:
> I have put together a simple set of figures I use to compare how disks
> and systems have changed over the 25 year life of ufs/ffs, which I
> sometimes use when I give ZFS presentations...
>
>25 years ago Nowfactor
>
>BTW: a lot of numbers in Solaris did not grow since a long time and
>thus create problems now. Just think about the maxphys values
>63 kB on x86 does not even allow to write a single BluRay disk sector
>with a single transfer.
Any "fixed value" will soon be too small (think about ufs_throt
- original Nachricht
Betreff: Re: [zfs-discuss] 'zfs recv' is very slow
Gesendet: Fr, 14. Nov 2008
Von: Bob Friesenhahn<[EMAIL PROTECTED]>
> On Fri, 14 Nov 2008, Joerg Schilling wrote:
> >
> > On my first Sun at home (a Sun 2/50 with 1 MB of RAM) in 1
Joerg Schilling wrote:
> Andrew Gabriel <[EMAIL PROTECTED]> wrote:
>> Andrew Gabriel wrote:
>>> Interesting idea, but for 7200 RPM disks (and a 1Gb ethernet link), I
>>> need a 250GB buffer (enough to buffer 4-5 seconds worth of data). That's
>>> many orders of magnitude bigger than SO_RCVBUF can
On Fri, 14 Nov 2008, Joerg Schilling wrote:
>
> On my first Sun at home (a Sun 2/50 with 1 MB of RAM) in 1986, I could
> set the socket buffer size to 63 kB. 63kB : 1 MB is the same ratio
> as 256 MB : 4 GB.
>
> BTW: a lot of numbers in Solaris did not grow since a long time and
> thus create probl
On Fri, Nov 14, 2008 at 10:04 AM, Joerg Schilling
<[EMAIL PROTECTED]> wrote:
> Andrew Gabriel <[EMAIL PROTECTED]> wrote:
>
>> Andrew Gabriel wrote:
>> > Interesting idea, but for 7200 RPM disks (and a 1Gb ethernet link), I
>> > need a 250GB buffer (enough to buffer 4-5 seconds worth of data). That'
Andrew Gabriel <[EMAIL PROTECTED]> wrote:
> Andrew Gabriel wrote:
> > Interesting idea, but for 7200 RPM disks (and a 1Gb ethernet link), I
> > need a 250GB buffer (enough to buffer 4-5 seconds worth of data). That's
> > many orders of magnitude bigger than SO_RCVBUF can go.
>
> No -- that's wro
Andrew Gabriel wrote:
> Interesting idea, but for 7200 RPM disks (and a 1Gb ethernet link), I
> need a 250GB buffer (enough to buffer 4-5 seconds worth of data). That's
> many orders of magnitude bigger than SO_RCVBUF can go.
No -- that's wrong -- should read 250MB buffer!
Still some orders of m
Joerg Schilling wrote:
> Andrew Gabriel <[EMAIL PROTECTED]> wrote:
>
>> That is exactly the issue. When the zfs recv data has been written, zfs
>> recv starts reading the network again, but there's only a tiny amount of
>> data buffered in the TCP/IP stack, so it has to wait for the network to
Joerg Schilling schrieb:
> Andrew Gabriel <[EMAIL PROTECTED]> wrote:
>
>> That is exactly the issue. When the zfs recv data has been written, zfs
>> recv starts reading the network again, but there's only a tiny amount of
>> data buffered in the TCP/IP stack, so it has to wait for the network to
Andrew Gabriel <[EMAIL PROTECTED]> wrote:
> That is exactly the issue. When the zfs recv data has been written, zfs
> recv starts reading the network again, but there's only a tiny amount of
> data buffered in the TCP/IP stack, so it has to wait for the network to
> heave more data across. In e
If anyone out there has a support contract with sun that covers Solaris 10
support. Feel free to email me and/or sun and have them add you to my
support case.
The Sun Case is 66104157 and I am seeking to have 6333409 and 6418042
putback into Solaris 10.
CR 6712788 was closed as a duplicate of CR
River Tarnell wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> Andrew Gabriel:
>
>> This is quite easily worked around by putting a buffering program
>> between the network and the zfs receive.
>>
>
> i tested inserting mbuffer with a 250MB buffer between the zfs send and zfs
>
I have an open ticket to have these putback into Solaris 10.
On Fri, Nov 7, 2008 at 3:24 PM, Ian Collins <[EMAIL PROTECTED]> wrote:
> Brent Jones wrote:
> > Theres been a couple threads about this now, tracked some bug
> ID's/ticket:
> >
> > 6333409
> > 6418042
> I see these are fixed in build 10
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Andrew Gabriel:
> This is quite easily worked around by putting a buffering program
> between the network and the zfs receive.
i tested inserting mbuffer with a 250MB buffer between the zfs send and zfs
recv. unfortunately, it seems to make very lit
Ian Collins wrote:
> Andrew Gabriel wrote:
>> Ian Collins wrote:
>>> Andrew Gabriel wrote:
Given the issue described is slow zfs recv over network, I suspect
this is:
6729347 Poor zfs receive performance across networks
This is quite easily worked around by putting a b
Andrew Gabriel wrote:
> Ian Collins wrote:
>> Andrew Gabriel wrote:
>>> Ian Collins wrote:
>>>
Brent Jones wrote:
> Theres been a couple threads about this now, tracked some bug
> ID's/ticket:
>
> 6333409
> 6418042
>
I see these are fixed in build
Ian Collins wrote:
> Andrew Gabriel wrote:
>> Ian Collins wrote:
>>
>>> Brent Jones wrote:
>>>
Theres been a couple threads about this now, tracked some bug ID's/ticket:
6333409
6418042
>>> I see these are fixed in build 102.
>>>
>>> Are they targeted to get
Andrew Gabriel wrote:
> Ian Collins wrote:
>
>> Brent Jones wrote:
>>
>>> Theres been a couple threads about this now, tracked some bug ID's/ticket:
>>>
>>> 6333409
>>> 6418042
>>>
>> I see these are fixed in build 102.
>>
>> Are they targeted to get back to Solaris 10 via a patch?
Ian Collins wrote:
> Brent Jones wrote:
>> Theres been a couple threads about this now, tracked some bug ID's/ticket:
>>
>> 6333409
>> 6418042
> I see these are fixed in build 102.
>
> Are they targeted to get back to Solaris 10 via a patch?
>
> If not, is it worth escalating the issue with supp
Brent Jones wrote:
> Theres been a couple threads about this now, tracked some bug ID's/ticket:
>
> 6333409
> 6418042
I see these are fixed in build 102.
Are they targeted to get back to Solaris 10 via a patch?
If not, is it worth escalating the issue with support to get a patch?
--
Ian.
River Tarnell wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> hi,
>
> i have two systems, A (Solaris 10 update 5) and B (Solaris 10 update 6). i'm
> using 'zfs send -i' to replicate changes on A to B. however, the 'zfs recv'
> on
> B is running extremely slowly.
I'm sorry, I didn'
River Tarnell wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> Ian Collins:
>
>> That's very slow. What's the nature of your data?
>>
>
> mainly two sets of mid-sized files; one of 200KB-2MB in size and other under
> 50KB. they are organised into subdirectories, A/B/C/. eac
On Thu, Nov 6, 2008 at 4:19 PM, River Tarnell
<[EMAIL PROTECTED]> wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> Ian Collins:
>> That's very slow. What's the nature of your data?
>
> mainly two sets of mid-sized files; one of 200KB-2MB in size and other under
> 50KB. they are organi
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Ian Collins:
> That's very slow. What's the nature of your data?
mainly two sets of mid-sized files; one of 200KB-2MB in size and other under
50KB. they are organised into subdirectories, A/B/C/. each directory
has 18,000-25,000 files. total data
On Fri 07/11/08 12:09 , River Tarnell [EMAIL PROTECTED] sent:
>
> hi,
>
> i have two systems, A (Solaris 10 update 5) and B (Solaris 10 update 6).
> i'musing 'zfs send -i' to replicate changes on A to B. however, the 'zfs
> recv' onB is running extremely slowly. if i run the zfs send on A and
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
hi,
i have two systems, A (Solaris 10 update 5) and B (Solaris 10 update 6). i'm
using 'zfs send -i' to replicate changes on A to B. however, the 'zfs recv' on
B is running extremely slowly. if i run the zfs send on A and redirect output
to a file,
Bill Shannon wrote:
> datsun# zfs recv -d test < d.0
> cannot open 'test/tckuser': dataset does not exist
Despite the error message, the recv does seem to work.
Is it a bug that it prints the error message, or is it a bug that it
restores the data?
___
I'm trying to figure out how to restore a filesystem using zfs recv.
Obviously there's some important concept I don't understand. I'm
using my zfsdump script to create the dumps that I'm going to restore.
Here's what I tried:
Save a "level 0" dump in d.0:
datsun# zfsdump 0 home/tckuser > d.0
zfs
Is there a zfs recv-like command that will list a table of contents
for what's in a stream?
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Matthew Ahrens wrote:
Robert Milkowski wrote:
Hello zfs-discuss,
zfs recv -v at the end reported:
received 928Mb stream in 6346 seconds (150Kb/sec)
I'm not sure but shouldn't it be 928MB and 150KB ?
Or perhaps we're counting bits?
That's correct, it is in bytes and
Robert Milkowski wrote:
Hello zfs-discuss,
zfs recv -v at the end reported:
received 928Mb stream in 6346 seconds (150Kb/sec)
I'm not sure but shouldn't it be 928MB and 150KB ?
Or perhaps we're counting bits?
That's correct, it is in bytes and should use capital B
Hello zfs-discuss,
zfs recv -v at the end reported:
received 928Mb stream in 6346 seconds (150Kb/sec)
I'm not sure but shouldn't it be 928MB and 150KB ?
Or perhaps we're counting bits?
--
Best regards,
Robert mailto:[
79 matches
Mail list logo
