zfs receive -> space quota exceeded

2015-11-25 Thread Stefan Wendler
Hi,

I am trying to setup a zfs snapshot backup. And I am using quotas on most of 
the datasets that I am trying to backup with send/receive.

The snapshot data on the backup pool is around 100% larger than the original 
data (compression is lz4 on both pools, dedup=off, no refquota). So I am 
exceeding the quota for every dataset. 

The send/receive always fails with: "space quota exceeded"

What can I do other than disabling the quotas? Which I won't do.

The command I am using for the first send/receive is:

fs send -R storage@autobackup-2015-11-25 | mbuffer | zfs receive -Fduv 
backup/storage

I know that -R is also sending attributes. Which is fine and wanted. But I 
don't think that a quota should be a problem here?

FreeBSD Version is still 10.1 unfortunately. There is no way to have a 
downtime right now and upgrade to 10.2.

zpool/zfs versions are 28/5

Cheers,
Stefan
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"


Re: sshpass

2015-11-25 Thread Andrey Fesenko
I'm make patch for port
https://bitbucket.org/f_andrey/redports_andrey/src/482b0b1f9ea7cd5e212d6c81961ad5fa545fb6d6/security/sshpass/?at=sshpass
contain last two upstream patch
Although the program hardkoded /dev/tty is not in the FreeBSD
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"


FreeBSD_STABLE_9-i386 - Build #239 - Still Failing

2015-11-25 Thread jenkins-admin
FreeBSD_STABLE_9-i386 - Build #239 - Still Failing:

Build information: https://jenkins.FreeBSD.org/job/FreeBSD_STABLE_9-i386/239/
Full change log: 
https://jenkins.FreeBSD.org/job/FreeBSD_STABLE_9-i386/239/changes
Full build log: 
https://jenkins.FreeBSD.org/job/FreeBSD_STABLE_9-i386/239/console

Change summaries:

291297 by ngie:
MFstable/10 r291294:

MFC r258245:
r258245 (by eadler):

Add missing include files for the printf_l and scanf_l man pages.

Reported by:swild...@dragonflybsd.org



The end of the build log:

[...truncated 52195 lines...]
gzip -cn /usr/src/secure/usr.bin/openssl/man/genrsa.1 > genrsa.1.gz
gzip -cn /usr/src/secure/usr.bin/openssl/man/nseq.1 > nseq.1.gz
gzip -cn /usr/src/secure/usr.bin/openssl/man/ocsp.1 > ocsp.1.gz
gzip -cn /usr/src/secure/usr.bin/openssl/man/openssl.1 > openssl.1.gz
gzip -cn /usr/src/secure/usr.bin/openssl/man/passwd.1 > passwd.1.gz
gzip -cn /usr/src/secure/usr.bin/openssl/man/pkcs12.1 > pkcs12.1.gz
gzip -cn /usr/src/secure/usr.bin/openssl/man/pkcs7.1 > pkcs7.1.gz
gzip -cn /usr/src/secure/usr.bin/openssl/man/pkcs8.1 > pkcs8.1.gz
gzip -cn /usr/src/secure/usr.bin/openssl/man/rand.1 > rand.1.gz
gzip -cn /usr/src/secure/usr.bin/openssl/man/req.1 > req.1.gz
gzip -cn /usr/src/secure/usr.bin/openssl/man/rsa.1 > rsa.1.gz
gzip -cn /usr/src/secure/usr.bin/openssl/man/rsautl.1 > rsautl.1.gz
gzip -cn /usr/src/secure/usr.bin/openssl/man/s_client.1 > s_client.1.gz
gzip -cn /usr/src/secure/usr.bin/openssl/man/s_server.1 > s_server.1.gz
gzip -cn /usr/src/secure/usr.bin/openssl/man/s_time.1 > s_time.1.gz
gzip -cn /usr/src/secure/usr.bin/openssl/man/sess_id.1 > sess_id.1.gz
gzip -cn /usr/src/secure/usr.bin/openssl/man/smime.1 > smime.1.gz
gzip -cn /usr/src/secure/usr.bin/openssl/man/speed.1 > speed.1.gz
gzip -cn /usr/src/secure/usr.bin/openssl/man/spkac.1 > spkac.1.gz
gzip -cn /usr/src/secure/usr.bin/openssl/man/verify.1 > verify.1.gz
gzip -cn /usr/src/secure/usr.bin/openssl/man/version.1 > version.1.gz
gzip -cn /usr/src/secure/usr.bin/openssl/man/x509.1 > x509.1.gz
gzip -cn /usr/src/secure/usr.bin/openssl/man/x509v3_config.1 > 
x509v3_config.1.gz
cc -O2 -pipe  -DTERMIOS -DANSI_SOURCE 
-I/usr/src/secure/usr.bin/openssl/../../../crypto/openssl 
-I/usr/src/secure/usr.bin/openssl/../../../crypto/openssl/crypto 
-I/usr/obj/usr/src/secure/usr.bin/openssl -DOPENSSL_THREADS -DDSO_DLFCN 
-DHAVE_DLFCN_H -DOPENSSL_NO_IDEA -DL_ENDIAN -DMONOLITH 
-I/usr/src/secure/usr.bin/openssl -DNO_IDEA -std=gnu99  -fstack-protector 
-Wno-pointer-sign  -o openssl app_rand.o apps.o asn1pars.o ca.o ciphers.o cms.o 
crl.o crl2p7.o dgst.o dh.o dhparam.o dsa.o dsaparam.o ec.o ecparam.o enc.o 
engine.o errstr.o gendh.o gendsa.o genrsa.o nseq.o ocsp.o openssl.o passwd.o 
pkcs12.o pkcs7.o pkcs8.o prime.o rand.o req.o rsa.o rsautl.o s_cb.o s_client.o 
s_server.o s_socket.o s_time.o sess_id.o smime.o speed.o spkac.o verify.o 
version.o x509.o -lssl -lcrypto
===> secure/usr.bin/scp (all)
cc -O2 -pipe  -I/usr/src/secure/usr.bin/scp/../../../crypto/openssh -include 
ssh_namespace.h -DNO_IDEA -std=gnu99  -fstack-protector -Wno-pointer-sign -c 
/usr/src/secure/usr.bin/scp/../../../crypto/openssh/scp.c -o scp.o
cc -O2 -pipe  -I/usr/src/secure/usr.bin/scp/../../../crypto/openssh -include 
ssh_namespace.h -DNO_IDEA -std=gnu99  -fstack-protector -Wno-pointer-sign -c 
/usr/src/secure/usr.bin/scp/../../../crypto/openssh/roaming_dummy.c -o 
roaming_dummy.o
gzip -cn /usr/src/secure/usr.bin/scp/../../../crypto/openssh/scp.1 > scp.1.gz
cc -O2 -pipe  -I/usr/src/secure/usr.bin/scp/../../../crypto/openssh -include 
ssh_namespace.h -DNO_IDEA -std=gnu99  -fstack-protector -Wno-pointer-sign  
-L/usr/obj/usr/src/tmp/usr/lib/private -rpath /usr/lib/private -o scp scp.o 
roaming_dummy.o -lssh -lcrypt -lcrypto -lz
===> secure/usr.bin/sftp (all)
cc -O2 -pipe  -I/usr/src/secure/usr.bin/sftp/../../../crypto/openssh -include 
ssh_namespace.h -DNO_IDEA -std=gnu99  -fstack-protector -Wno-pointer-sign -c 
/usr/src/secure/usr.bin/sftp/../../../crypto/openssh/sftp.c -o sftp.o
cc -O2 -pipe  -I/usr/src/secure/usr.bin/sftp/../../../crypto/openssh -include 
ssh_namespace.h -DNO_IDEA -std=gnu99  -fstack-protector -Wno-pointer-sign -c 
/usr/src/secure/usr.bin/sftp/../../../crypto/openssh/sftp-client.c -o 
sftp-client.o
cc -O2 -pipe  -I/usr/src/secure/usr.bin/sftp/../../../crypto/openssh -include 
ssh_namespace.h -DNO_IDEA -std=gnu99  -fstack-protector -Wno-pointer-sign -c 
/usr/src/secure/usr.bin/sftp/../../../crypto/openssh/sftp-common.c -o 
sftp-common.o
cc -O2 -pipe  -I/usr/src/secure/usr.bin/sftp/../../../crypto/openssh -include 
ssh_namespace.h -DNO_IDEA -std=gnu99  -fstack-protector -Wno-pointer-sign -c 
/usr/src/secure/usr.bin/sftp/../../../crypto/openssh/sftp-glob.c -o sftp-glob.o
cc -O2 -pipe  -DGCCVER=\"4.2\" -DIN_GCC -DHAVE_CONFIG_H -DPREFIX=\"/usr\" 
-I/usr/obj/usr/src/gnu/usr.bin/cc/cpp/../cc_tools 
-I/usr/src/gnu/usr.bin/cc/cpp/../cc_tools 
-I/usr/src/gnu/usr.bin/cc/cpp/../../../../contrib/gcc 
-I/usr/src/gnu/usr.bin/cc/cpp/../../../../

FreeBSD_STABLE_10-i386 - Build #668 - Still Failing

2015-11-25 Thread jenkins-admin
FreeBSD_STABLE_10-i386 - Build #668 - Still Failing:

Build information: https://jenkins.FreeBSD.org/job/FreeBSD_STABLE_10-i386/668/
Full change log: 
https://jenkins.FreeBSD.org/job/FreeBSD_STABLE_10-i386/668/changes
Full build log: 
https://jenkins.FreeBSD.org/job/FreeBSD_STABLE_10-i386/668/console

Change summaries:

291295 by smh:
MFC r291012:

Document loader(8) dumpdev option

Sponsored by:   Multiplay

291294 by ngie:
MFC r258245:
r258245 (by eadler):

Add missing include files for the printf_l and scanf_l man pages.

Reported by:swild...@dragonflybsd.org

291293 by ngie:
MFC r264737:

Discussed with: jilles

r264737 (by jilles):

libc/stdio: Fail fdopen() on an execute-only fd.

An execute-only fd (opened with O_EXEC) allows neither read() nor write()
and is therefore incompatible with all stdio modes. Therefore, the [EINVAL]
error applies.

Also adjust the similar check in freopen() with a NULL path, even though
this checks an fd which is already from a FILE.



The end of the build log:

[...truncated 101650 lines...]
--- secure.all__D ---
--- t_req.po ---
cc  -pg  -O2 -pipe   -DTERMIOS -DANSI_SOURCE 
-I/usr/src/secure/lib/libcrypto/../../../crypto/openssl 
-I/usr/src/secure/lib/libcrypto/../../../crypto/openssl/crypto 
-I/usr/obj/usr/src/secure/lib/libcrypto -DOPENSSL_THREADS -DDSO_DLFCN 
-DHAVE_DLFCN_H -DL_ENDIAN -DOPENSSL_IA32_SSE2 -DAES_ASM -DVPAES_ASM 
-DOPENSSL_BN_ASM_PART_WORDS -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_GF2m 
-DMD5_ASM -DGHASH_ASM -DRMD160_ASM -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM 
-DWHIRLPOOL_ASM 
-I/usr/src/secure/lib/libcrypto/../../../crypto/openssl/crypto/asn1 
-I/usr/src/secure/lib/libcrypto/../../../crypto/openssl/crypto/evp 
-I/usr/src/secure/lib/libcrypto/../../../crypto/openssl/crypto/modes -std=gnu89 
-Qunused-arguments  -fstack-protector -Wno-pointer-sign -Wno-empty-body 
-Wno-string-plus-int -Wno-unused-const-variable -Wno-tautological-compare 
-Wno-unused-value -Wno-parentheses-equality -Wno-unused-function 
-Wno-enum-conversion -Wno-switch -Wno-switch-enum -Wno-knr-promoted-parameter 
-Wno-parentheses -c /usr/src/sec
 ure/lib/libcrypto/../../../crypto/openssl/crypto/asn1/t_req.c -o t_req.po
--- lib.all__D ---
--- uucplock.po ---
cc  -pg  -O2 -pipe   -DLIBC_SCCS -DINET6 -I/usr/src/lib/libutil 
-I/usr/src/lib/libutil/../libc/gen/ -std=gnu99 -Qunused-arguments  
-fstack-protector -Wsystem-headers -Werror -Wall -Wno-format-y2k -W 
-Wno-unused-parameter -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith 
-Wreturn-type -Wcast-qual -Wwrite-strings -Wswitch -Wshadow -Wunused-parameter 
-Wcast-align -Wchar-subscripts -Winline -Wnested-externs -Wredundant-decls 
-Wold-style-definition -Wmissing-variable-declarations -Wno-pointer-sign 
-Wno-empty-body -Wno-string-plus-int -Wno-unused-const-variable -c 
/usr/src/lib/libutil/uucplock.c -o uucplock.po
--- all_subdir_libvgl ---
--- bitmap.po ---
cc  -pg  -O2 -pipe   -Wall -I/usr/src/lib/libvgl -std=gnu99 -Qunused-arguments  
-fstack-protector -Wsystem-headers -Werror -Wall -Wno-format-y2k 
-Wno-uninitialized -Wno-pointer-sign -Wno-empty-body -Wno-string-plus-int 
-Wno-unused-const-variable -Wno-tautological-compare -Wno-unused-value 
-Wno-parentheses-equality -Wno-unused-function -Wno-enum-conversion -Wno-switch 
-Wno-switch-enum -Wno-knr-promoted-parameter -c /usr/src/lib/libvgl/bitmap.c -o 
bitmap.po
--- all_subdir_libutil ---
--- libutil_p.a ---
building profiled util library
ranlib -D libutil_p.a
--- expand_number.3.gz ---
gzip -cn /usr/src/lib/libutil/expand_number.3 > expand_number.3.gz
--- flopen.3.gz ---
gzip -cn /usr/src/lib/libutil/flopen.3 > flopen.3.gz
--- fparseln.3.gz ---
gzip -cn /usr/src/lib/libutil/fparseln.3 > fparseln.3.gz
--- hexdump.3.gz ---
gzip -cn /usr/src/lib/libutil/hexdump.3 > hexdump.3.gz
--- secure.all__D ---
--- t_spki.po ---
cc  -pg  -O2 -pipe   -DTERMIOS -DANSI_SOURCE 
-I/usr/src/secure/lib/libcrypto/../../../crypto/openssl 
-I/usr/src/secure/lib/libcrypto/../../../crypto/openssl/crypto 
-I/usr/obj/usr/src/secure/lib/libcrypto -DOPENSSL_THREADS -DDSO_DLFCN 
-DHAVE_DLFCN_H -DL_ENDIAN -DOPENSSL_IA32_SSE2 -DAES_ASM -DVPAES_ASM 
-DOPENSSL_BN_ASM_PART_WORDS -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_GF2m 
-DMD5_ASM -DGHASH_ASM -DRMD160_ASM -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM 
-DWHIRLPOOL_ASM 
-I/usr/src/secure/lib/libcrypto/../../../crypto/openssl/crypto/asn1 
-I/usr/src/secure/lib/libcrypto/../../../crypto/openssl/crypto/evp 
-I/usr/src/secure/lib/libcrypto/../../../crypto/openssl/crypto/modes -std=gnu89 
-Qunused-arguments  -fstack-protector -Wno-pointer-sign -Wno-empty-body 
-Wno-string-plus-int -Wno-unused-const-variable -Wno-tautological-compare 
-Wno-unused-value -Wno-parentheses-equality -Wno-unused-function 
-Wno-enum-conversion -Wno-switch -Wno-switch-enum -Wno-knr-promoted-parameter 
-Wno-parentheses -c /usr/src/sec
 ure/lib/libcrypto/../../../crypto/openssl/crypto/asn1/t_spki.c -o t_spki.po
--- lib.all__D ---
--- humanize_number.3.gz ---
gzip -cn /usr/src/lib/libutil/humanize_number.

Re: FreeBSD_STABLE_10-i386 - Build #661 - Still Failing

2015-11-25 Thread Glen Barber
On Mon, Nov 23, 2015 at 07:15:40PM +0100, Baptiste Daroussin wrote:
> > The build happens in a jail built every time the job starts, by fetching
> > the latest snapshot from:
> > 
> > http://ftp.freebsd.org/pub/FreeBSD/snapshots/i386/i386/11.0-CURRENT/base.txz
> > 
> > It looks there is no /usr/bin/colldef.  I haven't looked into what's
> > going on here.
> > 
> 
> Ok so it will be fixed by the next snapshot
> 

The Jenkins builders absolutely *cannot* rely on files within the
snapshots/ directory on the mirrors to exist.  Relying on files that
possibly could not exist (failed builds for extended periods of time,
for example), is an implementation bug.

The Jenkins configurations need to be updated to use the base.txz under
the releases/ directory, and for building -CURRENT, must use the latest
major release base.txz (not *snapshot* build).

In other words:

 9.3-STABLE: built from 9.3-RELEASE
 10.2-STABLE: built from 10.2-RELEASE
 11.0-CURRENT: built from 10.2-RELEASE

This ensures:

1) edge cases where a snapshot (which is *not* a release) is built
   between a problematic commit does not prevent Jenkins spam;
2) source-based upgrades from the last major version to -CURRENT (which
   in turn, ensures 10.2-RELEASE can be source-upgraded to 11.0-RELEASE)
   is tested.

This must be fixed by Jenkins Admins.

Glen



signature.asc
Description: PGP signature


Re: FreeBSD_STABLE_10-i386 - Build #661 - Still Failing

2015-11-25 Thread Glen Barber
On Wed, Nov 25, 2015 at 10:31:28AM +, Glen Barber wrote:
> On Mon, Nov 23, 2015 at 07:15:40PM +0100, Baptiste Daroussin wrote:
> > > The build happens in a jail built every time the job starts, by fetching
> > > the latest snapshot from:
> > > 
> > > http://ftp.freebsd.org/pub/FreeBSD/snapshots/i386/i386/11.0-CURRENT/base.txz
> > > 
> > > It looks there is no /usr/bin/colldef.  I haven't looked into what's
> > > going on here.
> > > 
> > 
> > Ok so it will be fixed by the next snapshot
> > 
> 
> The Jenkins builders absolutely *cannot* rely on files within the
> snapshots/ directory on the mirrors to exist.  Relying on files that
> possibly could not exist (failed builds for extended periods of time,
> for example), is an implementation bug.
> 
> The Jenkins configurations need to be updated to use the base.txz under
> the releases/ directory, and for building -CURRENT, must use the latest
> major release base.txz (not *snapshot* build).
> 
> In other words:
> 
>  9.3-STABLE: built from 9.3-RELEASE
>  10.2-STABLE: built from 10.2-RELEASE
>  11.0-CURRENT: built from 10.2-RELEASE
> 
> This ensures:
> 
> 1) edge cases where a snapshot (which is *not* a release) is built
>between a problematic commit does not prevent Jenkins spam;

s/does not prevent/does not cause/

> 2) source-based upgrades from the last major version to -CURRENT (which
>in turn, ensures 10.2-RELEASE can be source-upgraded to 11.0-RELEASE)
>is tested.
> 
> This must be fixed by Jenkins Admins.
> 

Glen



signature.asc
Description: PGP signature


Re: ZFS - poor performance with "large" directories

2015-11-25 Thread krad
consumer SSDs are cheap enough now not to bother with usb drives I would
imagine.

On 25 November 2015 at 07:12, Gerrit Kühn  wrote:

> On Tue, 24 Nov 2015 17:11:54 +0100 Albert Cervin 
> wrote about Re: ZFS - poor performance with "large" directories:
>
> AC> Will try a bit with the meta limit.
>
> You can also put metadata on a flash device to speed things up. To
> check if this is really the bottleneck in your case, something simple like
> a USB stick might suffice to try out:
> <
> https://pthree.org/2013/05/09/zfs-administration-appendix-b-using-usb-drives/
> >
>
>
> cu
>   Gerrit
> ___
> freebsd-stable@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-stable
> To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
>
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: ZFS - poor performance with "large" directories

2015-11-25 Thread Gerrit Kühn
On Wed, 25 Nov 2015 12:06:38 + krad  wrote about Re:
ZFS - poor performance with "large" directories:

K> consumer SSDs are cheap enough now not to bother with usb drives I would
K> imagine.

Sure. I was just suggesting a USB drive as a quick way to check if this
might help at all. Most people have USB drives lying around, and they can
simply be plugged into any computer. A SSD (cheap or not) more likely
needs to be bought first, and the server box might need to be opened to
have it installed.


cu
  Gerrit
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"


Re: ZFS - poor performance with "large" directories

2015-11-25 Thread Albert Cervin
Hi again,

I have conducted a few other experiments and concluded that ZFS is NOT
the culprit here and my suspicions now go to Samba instead which seems
to behave really badly under these conditions...

Thanks for all the ZFS help though, much appreciated!

Cheers,
Albert

On Wed, Nov 25, 2015 at 1:16 PM, Gerrit Kühn  wrote:
> On Wed, 25 Nov 2015 12:06:38 + krad  wrote about Re:
> ZFS - poor performance with "large" directories:
>
> K> consumer SSDs are cheap enough now not to bother with usb drives I would
> K> imagine.
>
> Sure. I was just suggesting a USB drive as a quick way to check if this
> might help at all. Most people have USB drives lying around, and they can
> simply be plugged into any computer. A SSD (cheap or not) more likely
> needs to be bought first, and the server box might need to be opened to
> have it installed.
>
>
> cu
>   Gerrit
> ___
> freebsd-stable@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-stable
> To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: sshpass

2015-11-25 Thread Mark Linimon
Please submit this via bugzilla so that it will not get lost in all
the mailing list traffic.  Thank you.

mcl
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"


FreeBSD_STABLE_10-i386 - Build #669 - Fixed

2015-11-25 Thread jenkins-admin
FreeBSD_STABLE_10-i386 - Build #669 - Fixed:

Build information: https://jenkins.FreeBSD.org/job/FreeBSD_STABLE_10-i386/669/
Full change log: 
https://jenkins.FreeBSD.org/job/FreeBSD_STABLE_10-i386/669/changes
Full build log: 
https://jenkins.FreeBSD.org/job/FreeBSD_STABLE_10-i386/669/console

Change summaries:

No changes
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"


FreeBSD_STABLE_9-i386 - Build #240 - Fixed

2015-11-25 Thread jenkins-admin
FreeBSD_STABLE_9-i386 - Build #240 - Fixed:

Build information: https://jenkins.FreeBSD.org/job/FreeBSD_STABLE_9-i386/240/
Full change log: 
https://jenkins.FreeBSD.org/job/FreeBSD_STABLE_9-i386/240/changes
Full build log: 
https://jenkins.FreeBSD.org/job/FreeBSD_STABLE_9-i386/240/console

Change summaries:

No changes
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"


Re: ZFS - poor performance with "large" directories

2015-11-25 Thread Albert Cervin
Just to close this off, when using Samba with ZFS it seems to be very
important (if you have many files in a directory) to make it case
sensitive as per:
https://wiki.samba.org/index.php/Performance_tuning#Handling_Large_Directories.

Everything is now roses and works as expected.

Sorry ZFS that I accused you! ;)

Thanks again!

// Albert

On Wed, Nov 25, 2015 at 2:23 PM, Albert Cervin  wrote:
> Hi again,
>
> I have conducted a few other experiments and concluded that ZFS is NOT
> the culprit here and my suspicions now go to Samba instead which seems
> to behave really badly under these conditions...
>
> Thanks for all the ZFS help though, much appreciated!
>
> Cheers,
> Albert
>
> On Wed, Nov 25, 2015 at 1:16 PM, Gerrit Kühn  wrote:
>> On Wed, 25 Nov 2015 12:06:38 + krad  wrote about Re:
>> ZFS - poor performance with "large" directories:
>>
>> K> consumer SSDs are cheap enough now not to bother with usb drives I would
>> K> imagine.
>>
>> Sure. I was just suggesting a USB drive as a quick way to check if this
>> might help at all. Most people have USB drives lying around, and they can
>> simply be plugged into any computer. A SSD (cheap or not) more likely
>> needs to be bought first, and the server box might need to be opened to
>> have it installed.
>>
>>
>> cu
>>   Gerrit
>> ___
>> freebsd-stable@freebsd.org mailing list
>> https://lists.freebsd.org/mailman/listinfo/freebsd-stable
>> To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: FreeBSD_STABLE_10-i386 - Build #661 - Still Failing

2015-11-25 Thread Li-Wen Hsu
On Wed, Nov 25, 2015 at 10:34:47 +, Glen Barber wrote:
> On Wed, Nov 25, 2015 at 10:31:28AM +, Glen Barber wrote:
> > On Mon, Nov 23, 2015 at 07:15:40PM +0100, Baptiste Daroussin wrote:
> > > > The build happens in a jail built every time the job starts, by fetching
> > > > the latest snapshot from:
> > > > 
> > > > http://ftp.freebsd.org/pub/FreeBSD/snapshots/i386/i386/11.0-CURRENT/base.txz
> > > > 
> > > > It looks there is no /usr/bin/colldef.  I haven't looked into what's
> > > > going on here.
> > > > 
> > > 
> > > Ok so it will be fixed by the next snapshot
> > > 
> > 
> > The Jenkins builders absolutely *cannot* rely on files within the
> > snapshots/ directory on the mirrors to exist.  Relying on files that
> > possibly could not exist (failed builds for extended periods of time,
> > for example), is an implementation bug.
> > 
> > The Jenkins configurations need to be updated to use the base.txz under
> > the releases/ directory, and for building -CURRENT, must use the latest
> > major release base.txz (not *snapshot* build).
> > 
> > In other words:
> > 
> >  9.3-STABLE: built from 9.3-RELEASE
> >  10.2-STABLE: built from 10.2-RELEASE
> >  11.0-CURRENT: built from 10.2-RELEASE
> > 
> > This ensures:
> > 
> > 1) edge cases where a snapshot (which is *not* a release) is built
> >between a problematic commit does not prevent Jenkins spam;
> 
> s/does not prevent/does not cause/
> 
> > 2) source-based upgrades from the last major version to -CURRENT (which
> >in turn, ensures 10.2-RELEASE can be source-upgraded to 11.0-RELEASE)
> >is tested.
> > 
> > This must be fixed by Jenkins Admins.
> > 

This is changed in most of the jobs which execute the build process in
jail.  As you can see. i386 10-STABLE and i386 9-STABLE build is back to
normal. For arm64 and i386 11-CURRENT, I have switched the configuration
in test jobs and will modify the main jobs when I see them successfully
ends.

Li-Wen



-- 
Li-Wen Hsu 
http://lwhsu.org


pgpFpPKSuyuCJ.pgp
Description: PGP signature


Re: FreeBSD_STABLE_10-i386 - Build #661 - Still Failing

2015-11-25 Thread Glen Barber
On Thu, Nov 26, 2015 at 01:39:37AM +0800, Li-Wen Hsu wrote:
> On Wed, Nov 25, 2015 at 10:34:47 +, Glen Barber wrote:
> > On Wed, Nov 25, 2015 at 10:31:28AM +, Glen Barber wrote:
> > > On Mon, Nov 23, 2015 at 07:15:40PM +0100, Baptiste Daroussin wrote:
> > > > > The build happens in a jail built every time the job starts, by 
> > > > > fetching
> > > > > the latest snapshot from:
> > > > > 
> > > > > http://ftp.freebsd.org/pub/FreeBSD/snapshots/i386/i386/11.0-CURRENT/base.txz
> > > > > 
> > > > > It looks there is no /usr/bin/colldef.  I haven't looked into what's
> > > > > going on here.
> > > > > 
> > > > 
> > > > Ok so it will be fixed by the next snapshot
> > > > 
> > > 
> > > The Jenkins builders absolutely *cannot* rely on files within the
> > > snapshots/ directory on the mirrors to exist.  Relying on files that
> > > possibly could not exist (failed builds for extended periods of time,
> > > for example), is an implementation bug.
> > > 
> > > The Jenkins configurations need to be updated to use the base.txz under
> > > the releases/ directory, and for building -CURRENT, must use the latest
> > > major release base.txz (not *snapshot* build).
> > > 
> > > In other words:
> > > 
> > >  9.3-STABLE: built from 9.3-RELEASE
> > >  10.2-STABLE: built from 10.2-RELEASE
> > >  11.0-CURRENT: built from 10.2-RELEASE
> > > 
> > > This ensures:
> > > 
> > > 1) edge cases where a snapshot (which is *not* a release) is built
> > >between a problematic commit does not prevent Jenkins spam;
> > 
> > s/does not prevent/does not cause/
> > 
> > > 2) source-based upgrades from the last major version to -CURRENT (which
> > >in turn, ensures 10.2-RELEASE can be source-upgraded to 11.0-RELEASE)
> > >is tested.
> > > 
> > > This must be fixed by Jenkins Admins.
> > > 
> 
> This is changed in most of the jobs which execute the build process in
> jail.  As you can see. i386 10-STABLE and i386 9-STABLE build is back to
> normal. For arm64 and i386 11-CURRENT, I have switched the configuration
> in test jobs and will modify the main jobs when I see them successfully
> ends.
> 

Thank you very much for resolving this.

Glen



signature.asc
Description: PGP signature


Re: sshpass

2015-11-25 Thread Andrey Fesenko
On Wed, Nov 25, 2015 at 4:52 PM, Mark Linimon  wrote:
> Please submit this via bugzilla so that it will not get lost in all
> the mailing list traffic.  Thank you.
>
> mcl

Done https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=204819
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"


Re: zfs receive -> space quota exceeded

2015-11-25 Thread Stefan Wendler
oh well, I think this is solved. I should have used refquota instead of quota ^^

Cheers 

Am 25.11.2015 10:51 schrieb Stefan Wendler :
>
> Hi, 
>
> I am trying to setup a zfs snapshot backup. And I am using quotas on most of 
> the datasets that I am trying to backup with send/receive. 
>
> The snapshot data on the backup pool is around 100% larger than the original 
> data (compression is lz4 on both pools, dedup=off, no refquota). So I am 
> exceeding the quota for every dataset. 
>
> The send/receive always fails with: "space quota exceeded" 
>
> What can I do other than disabling the quotas? Which I won't do. 
>
> The command I am using for the first send/receive is: 
>
> fs send -R storage@autobackup-2015-11-25 | mbuffer | zfs receive -Fduv 
> backup/storage 
>
> I know that -R is also sending attributes. Which is fine and wanted. But I 
> don't think that a quota should be a problem here? 
>
> FreeBSD Version is still 10.1 unfortunately. There is no way to have a 
> downtime right now and upgrade to 10.2. 
>
> zpool/zfs versions are 28/5 
>
> Cheers, 
> Stefan 
> ___ 
> freebsd-stable@freebsd.org mailing list 
> https://lists.freebsd.org/mailman/listinfo/freebsd-stable 
> To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org" 
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"


FreeBSD_stable_10 - Build #1842 - Failure

2015-11-25 Thread jenkins-admin
FreeBSD_stable_10 - Build #1842 - Failure:

Build information: https://jenkins.FreeBSD.org/job/FreeBSD_stable_10/1842/
Full change log: https://jenkins.FreeBSD.org/job/FreeBSD_stable_10/1842/changes
Full build log: https://jenkins.FreeBSD.org/job/FreeBSD_stable_10/1842/console

Change summaries:

291336 by ngie:
MFC r288006,r288031,r288032,r288033:

r288006 (by rodrigc):

Add declarations to eliminate -Wmissing-prototypes warnings

r288031 (by rodrigc):

Remove names from some prototypes

r288032 (by rodrigc):

Remove names from some prototypes

r288033 (by rodrigc):

Use ANSI C prototypes.  Eliminates -Wold-style-definition warnings.



The end of the build log:

[...truncated 107793 lines...]
(cd /builds/FreeBSD_stable_10/lib/libc/tests/net && make -f 
/builds/FreeBSD_stable_10/lib/libc/tests/net/Makefile _RECURSING_PROGS=  
SUBDIR= PROG=h_dns_server  DEPENDFILE=.depend.h_dns_server 
.MAKE.DEPENDFILE=.depend.h_dns_server   depend)
--- gnu.depend__D ---
--- _sub.depend ---
===> gnu/lib/libreadline/history (depend)
--- kerberos5.depend__D ---
--- lib.depend__D ---
--- .depend.h_dns_server ---
--- kerberos5.depend__D ---
===> kerberos5 (depend)
--- lib.depend__D ---
rm -f .depend.h_dns_server
CC='cc ' mkdep -f .depend.h_dns_server -a
-I/builds/FreeBSD_stable_10/lib/libc/tests/net -std=gnu99   
/builds/FreeBSD_stable_10/contrib/netbsd-tests/lib/libc/net/h_dns_server.c
--- gnu.depend__D ---
--- _sub.depend ---
===> gnu/lib/libreadline/history/doc (depend)
--- bin.depend__D ---
echo ps: 
/builds/FreeBSD_stable_10/obj/builds/FreeBSD_stable_10/tmp/usr/lib/libc.a 
/builds/FreeBSD_stable_10/obj/builds/FreeBSD_stable_10/tmp/usr/lib/libm.a 
/builds/FreeBSD_stable_10/obj/builds/FreeBSD_stable_10/tmp/usr/lib/libkvm.a 
/builds/FreeBSD_stable_10/obj/builds/FreeBSD_stable_10/tmp/usr/lib/libjail.a >> 
.depend
--- depend_subdir_pwait ---
===> bin/pwait (depend)
--- kerberos5.depend__D ---
--- depend_subdir_lib ---
===> kerberos5/lib (depend)
--- gnu.depend__D ---
===> gnu/lib/libreadline/readline (depend)
--- bin.depend__D ---
--- .depend ---
rm -f .depend
CC='cc ' mkdep -f .depend -a -std=gnu99   
/builds/FreeBSD_stable_10/bin/pwait/pwait.c
--- kerberos5.depend__D ---
--- _sub.depend ---
===> kerberos5/lib/libasn1 (depend)
--- lib.depend__D ---
echo h_dns_server: 
/builds/FreeBSD_stable_10/obj/builds/FreeBSD_stable_10/tmp/usr/lib/libc.a  >> 
.depend.h_dns_server
(cd /builds/FreeBSD_stable_10/lib/libc/tests/net && make -f 
/builds/FreeBSD_stable_10/lib/libc/tests/net/Makefile _RECURSING_PROGS=  
SUBDIR= PROG=ether_test  DEPENDFILE=.depend.ether_test 
.MAKE.DEPENDFILE=.depend.ether_test   depend)
--- gnu.depend__D ---
--- _sub.depend ---
===> gnu/lib/libreadline/readline/doc (depend)
--- kerberos5.depend__D ---
===> kerberos5/lib/libgssapi_krb5 (depend)
--- lib.depend__D ---
--- .depend.ether_test ---
rm -f .depend.ether_test
--- gnu.depend__D ---
===> gnu/lib/libssp (depend)
--- lib.depend__D ---
CC='cc ' mkdep -f .depend.ether_test -a
-I/builds/FreeBSD_stable_10/lib/libc/tests/net -std=gnu99   
/builds/FreeBSD_stable_10/lib/libc/tests/net/ether_test.c
--- bin.depend__D ---
echo pwait: 
/builds/FreeBSD_stable_10/obj/builds/FreeBSD_stable_10/tmp/usr/lib/libc.a  >> 
.depend
--- depend_subdir_pwd ---
===> bin/pwd (depend)
--- gnu.depend__D ---
--- _sub.depend ---
===> gnu/lib/libssp/libssp_nonshared (depend)
--- lib.depend__D ---
echo ether_test: 
/builds/FreeBSD_stable_10/obj/builds/FreeBSD_stable_10/tmp/usr/lib/libc.a 
/builds/FreeBSD_stable_10/obj/builds/FreeBSD_stable_10/tmp/usr/lib/private/libatf-c.a
 >> .depend.ether_test
--- bin.depend__D ---
--- .depend ---
--- lib.depend__D ---
(cd /builds/FreeBSD_stable_10/lib/libc/tests/net && make -f 
/builds/FreeBSD_stable_10/lib/libc/tests/net/Makefile _RECURSING_PROGS=  
SUBDIR= PROG=eui64_aton_test  DEPENDFILE=.depend.eui64_aton_test 
.MAKE.DEPENDFILE=.depend.eui64_aton_test   depend)
--- bin.depend__D ---
rm -f .depend
CC='cc ' mkdep -f .depend -a -std=gnu99   
/builds/FreeBSD_stable_10/bin/pwd/pwd.c
--- kerberos5.depend__D ---
===> kerberos5/lib/libgssapi_ntlm (depend)
--- gnu.depend__D ---
===> gnu/lib/tests (depend)
--- lib.depend__D ---
--- .depend.eui64_aton_test ---
rm -f .depend.eui64_aton_test
CC='cc ' mkdep -f .depend.eui64_aton_test -a
-I/builds/FreeBSD_stable_10/lib/libc/tests/net -std=gnu99   
/builds/FreeBSD_stable_10/lib/libc/tests/net/eui64_aton_test.c
--- bin.depend__D ---
echo pwd: 
/builds/FreeBSD_stable_10/obj/builds/FreeBSD_stable_10/tmp/usr/lib/libc.a  >> 
.depend
--- gnu.depend__D ---
===> gnu/tests (depend)
--- bin.depend__D ---
--- depend_subdir_rcp ---
===> bin/rcp (depend)
--- lib.depend__D ---
echo eui64_aton_test: 
/builds/FreeBSD_stable_10/obj/builds/FreeBSD_stable_10/tmp/usr/lib/libc.a 
/builds/FreeBSD_stable_10/obj/builds/FreeBSD_stable_10/tmp/usr/lib/private/libatf-c.a
 >> .depend.eui64_aton_test
(cd /builds/FreeBSD_stable_10/lib/libc/tests/net && make -f 
/builds/FreeBSD_stable_10/lib/libc/tests/net/Makefile _RECURSING_PROG

FreeBSD_stable_10 - Build #1843 - Fixed

2015-11-25 Thread jenkins-admin
FreeBSD_stable_10 - Build #1843 - Fixed:

Build information: https://jenkins.FreeBSD.org/job/FreeBSD_stable_10/1843/
Full change log: https://jenkins.FreeBSD.org/job/FreeBSD_stable_10/1843/changes
Full build log: https://jenkins.FreeBSD.org/job/FreeBSD_stable_10/1843/console

Change summaries:

291355 by gnn:
MFC 290028:
Turning on IPSEC used to introduce a slight amount of performance
degradation (7%) for host host TCP connections over 10Gbps links,
even when there were no secuirty policies in place. There is no
change in performance on 1Gbps network links. Testing GENERIC vs.
GENERIC-NOIPSEC vs. GENERIC with this change shows that the new
code removes any overhead introduced by having IPSEC always in the
kernel.

Differential Revision:  D3993
Sponsored by:   Rubicon Communications (Netgate)

291345 by ngie:
MFC r291172:

Use __MAKE_SHELL instead of HOST_SHELL when generating aton_ether_subr.c
(HOST_SHELL is used in NetBSD)

This fixes permission denied issues when gen_ether_subr is not executable

Reported by: José Pérez 
Suggested by: bdrewery, sjg

291343 by ngie:
Fix bad MFC (r291175)

Replace SRCTOP with the relevant path via .CURDIR

Pointyhat to: ngie
Sponsored by: EMC / Isilon Storage Division

___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"