Re: svn commit: r289405 - head/sys/ufs/ffs

2015-10-16 Thread Hans Petter Selasky

On 10/16/15 08:21, Bruce Evans wrote:

In addition, making the file contiguous in LBA space doesn't
  improve the access times from flash devices because they have no seek
time.


Hi,

This is not exactly true, like Bruce pointed out too. Maybe there should 
be a check, that if the block is too small reallocate it, else leave it 
for the sake of the flash. Doing 1K accesses versus 64K accesses will 
typically show up in the performance benchmark regardless of how fast 
the underlying medium is.


--HPS
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"


Re: svn commit: r289405 - head/sys/ufs/ffs

2015-10-16 Thread Bruce Evans

On Fri, 16 Oct 2015, Hans Petter Selasky wrote:


On 10/16/15 08:21, Bruce Evans wrote:

[Bruce Evans didn't write:]
In addition, making the file contiguous in LBA space doesn't
  improve the access times from flash devices because they have no seek
time.


This is not exactly true, like Bruce pointed out too. Maybe there should be a 
check, that if the block is too small reallocate it, else leave it for the 
sake of the flash. Doing 1K accesses versus 64K accesses will typically show 
up in the performance benchmark regardless of how fast the underlying medium 
is.


Now I don't unerstand the whole point of the change.  Anything that reduces
i/o's is good, but AFAIK ffs_doreallocblks() is all in software.  Writes
should be delayed so that it doesn't have to do extra i/o's to back out of
committed writes.  Often it reduces the number of writes and increases
their size by making blocks contiguous so that the write can be clustered.
Increasing the write size is especially good for flash devices, but maybe
ffs's default block size is already large enough.

Bruce
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"


svn commit: r289415 - in head: . sys/sys

2015-10-16 Thread Alexander Motin
Author: mav
Date: Fri Oct 16 08:57:11 2015
New Revision: 289415
URL: https://svnweb.freebsd.org/changeset/base/289415

Log:
  Bump version and add notice about incompatibility introduced by resumable
  send/receive support in ZFS.

Modified:
  head/UPDATING
  head/sys/sys/param.h

Modified: head/UPDATING
==
--- head/UPDATING   Fri Oct 16 08:22:21 2015(r289414)
+++ head/UPDATING   Fri Oct 16 08:57:11 2015(r289415)
@@ -31,6 +31,12 @@ NOTE TO PEOPLE WHO THINK THAT FreeBSD 11
disable the most expensive debugging functionality run
"ln -s 'abort:false,junk:false' /etc/malloc.conf".)
 
+20151015:
+   Added ZFS support for resumable send/receive changed respective
+   IOCTL API/ABI.  Compatibility ABI shims were provided for other
+   functionality, while receive require version match between world
+   and kernel.
+
 20151012:
If you specify SENDMAIL_MC or SENDMAIL_CF in make.conf, mergemaster
and etcupdate will now use this file. A custom sendmail.cf is now

Modified: head/sys/sys/param.h
==
--- head/sys/sys/param.hFri Oct 16 08:22:21 2015(r289414)
+++ head/sys/sys/param.hFri Oct 16 08:57:11 2015(r289415)
@@ -58,7 +58,7 @@
  * in the range 5 to 9.
  */
 #undef __FreeBSD_version
-#define __FreeBSD_version 1100080  /* Master, propagated to newvers */
+#define __FreeBSD_version 1100081  /* Master, propagated to newvers */
 
 /*
  * __FreeBSD_kernel__ indicates that this system uses the kernel of FreeBSD,
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"


svn commit: r289417 - head/sys/modules/cloudabi64

2015-10-16 Thread Ed Schouten
Author: ed
Date: Fri Oct 16 10:26:15 2015
New Revision: 289417
URL: https://svnweb.freebsd.org/changeset/base/289417

Log:
  Use the right variable name.
  
  MACHINE_CPUARCH expands to aarch64 for arm64, whereas MACHINE always
  corresponds to the directory name under sys/ that contains the sources
  for that architecture.

Modified:
  head/sys/modules/cloudabi64/Makefile

Modified: head/sys/modules/cloudabi64/Makefile
==
--- head/sys/modules/cloudabi64/MakefileFri Oct 16 09:37:15 2015
(r289416)
+++ head/sys/modules/cloudabi64/MakefileFri Oct 16 10:26:15 2015
(r289417)
@@ -1,7 +1,7 @@
 # $FreeBSD$
 
 .PATH: ${.CURDIR}/../../compat/cloudabi64
-.PATH: ${.CURDIR}/../../${MACHINE_CPUARCH}/cloudabi64
+.PATH: ${.CURDIR}/../../${MACHINE}/cloudabi64
 
 KMOD=  cloudabi64
 SRCS=  cloudabi64_fd.c cloudabi64_poll.c cloudabi64_sock.c \
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"


svn commit: r289419 - head/lib/libfetch

2015-10-16 Thread Dag-Erling Smørgrav
Author: des
Date: Fri Oct 16 12:21:44 2015
New Revision: 289419
URL: https://svnweb.freebsd.org/changeset/base/289419

Log:
  Fix two bugs in HTTPS tunnelling:
   - If the proxy returns a non-200 result, set the error code accordingly
 so the caller / user gets a somewhat meaningful error message.
   - Consume and discard any HTTP response header following the result line.
  
  PR:   194483
  Tested by:Fabian Keil 
  MFC after:1 week

Modified:
  head/lib/libfetch/http.c

Modified: head/lib/libfetch/http.c
==
--- head/lib/libfetch/http.cFri Oct 16 12:16:35 2015(r289418)
+++ head/lib/libfetch/http.cFri Oct 16 12:21:44 2015(r289419)
@@ -1376,8 +1376,12 @@ http_connect(struct url *URL, struct url
 {
struct url *curl;
conn_t *conn;
+   hdr_t h;
+   http_headerbuf_t headerbuf;
+   const char *p;
int verbose;
int af, val;
+   int serrno;
 
 #ifdef INET6
af = AF_UNSPEC;
@@ -1398,6 +1402,7 @@ http_connect(struct url *URL, struct url
if ((conn = fetch_connect(curl->host, curl->port, af, verbose)) == NULL)
/* fetch_connect() has already set an error code */
return (NULL);
+   init_http_headerbuf(&headerbuf);
if (strcasecmp(URL->scheme, SCHEME_HTTPS) == 0 && purl) {
http_cmd(conn, "CONNECT %s:%d HTTP/1.1",
URL->host, URL->port);
@@ -1405,10 +1410,26 @@ http_connect(struct url *URL, struct url
URL->host, URL->port);
http_cmd(conn, "");
if (http_get_reply(conn) != HTTP_OK) {
-   fetch_close(conn);
-   return (NULL);
+   http_seterr(conn->err);
+   goto ouch;
}
-   http_get_reply(conn);
+   /* Read and discard the rest of the proxy response */
+   if (fetch_getln(conn) < 0) {
+   fetch_syserr();
+   goto ouch;
+   }
+   do {
+   switch ((h = http_next_header(conn, &headerbuf, &p))) {
+   case hdr_syserror:
+   fetch_syserr();
+   goto ouch;
+   case hdr_error:
+   http_seterr(HTTP_PROTOCOL_ERROR);
+   goto ouch;
+   default:
+   /* ignore */ ;
+   }
+   } while (h < hdr_end);
}
if (strcasecmp(URL->scheme, SCHEME_HTTPS) == 0 &&
fetch_ssl(conn, URL, verbose) == -1) {
@@ -1416,13 +1437,20 @@ http_connect(struct url *URL, struct url
/* grrr */
errno = EAUTH;
fetch_syserr();
-   return (NULL);
+   goto ouch;
}
 
val = 1;
setsockopt(conn->sd, IPPROTO_TCP, TCP_NOPUSH, &val, sizeof(val));
 
+   clean_http_headerbuf(&headerbuf);
return (conn);
+ouch:
+   serrno = errno;
+   clean_http_headerbuf(&headerbuf);
+   fetch_close(conn);
+   errno = serrno;
+   return (NULL);
 }
 
 static struct url *
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"


Re: svn commit: r289350 - head/sys/netinet

2015-10-16 Thread Gleb Smirnoff
On Thu, Oct 15, 2015 at 01:44:32AM +, Bjoern A. Zeeb wrote:
B> Author: bz
B> Date: Thu Oct 15 01:44:32 2015
B> New Revision: 289350
B> URL: https://svnweb.freebsd.org/changeset/base/289350
B> 
B> Log:
B>   Hopefully also unbreak VIMAGE kernels replacing the &V_... with
B>   &VNET_NAME(...).
B>   Everything else is just a whitespace wrapping change.
B> 
B> Modified:
B>   head/sys/netinet/tcp_pcap.c
B> 
B> Modified: head/sys/netinet/tcp_pcap.c
B> 
==
B> --- head/sys/netinet/tcp_pcap.c  Thu Oct 15 01:09:14 2015
(r289349)
B> +++ head/sys/netinet/tcp_pcap.c  Thu Oct 15 01:44:32 2015
(r289350)
B> @@ -70,9 +70,9 @@ SYSCTL_INT(_net_inet_tcp, OID_AUTO, tcp_
B>  
B>  VNET_DEFINE(int, tcp_pcap_packets) = 0;
B>  #define V_tcp_pcap_packets  VNET(tcp_pcap_packets)
B> -SYSCTL_INT(_net_inet_tcp, OID_AUTO, tcp_pcap_packets, CTLFLAG_RW,
B> -&V_tcp_pcap_packets, 0, "Default number of packets saved per direction "
B> -"per TCPCB");
B> +SYSCTL_INT(_net_inet_tcp, OID_AUTO, tcp_pcap_packets,
B> +CTLFLAG_RW, &VNET_NAME(tcp_pcap_packets), 0,
B> +"Default number of packets saved per direction per TCPCB");

It also lacks CTLFLAG_VNET flag. Will panic on write now.

-- 
Totus tuus, Glebius.
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"


svn commit: r289420 - head/lib/libfetch

2015-10-16 Thread Dag-Erling Smørgrav
Author: des
Date: Fri Oct 16 12:53:22 2015
New Revision: 289420
URL: https://svnweb.freebsd.org/changeset/base/289420

Log:
  Use fopen()'s newfangled "e" flag instead of explicit fcntl() calls.
  
  PR:   199801
  Submitted by: Jukka Ukkonen 
  MFC after:1 week

Modified:
  head/lib/libfetch/file.c

Modified: head/lib/libfetch/file.c
==
--- head/lib/libfetch/file.cFri Oct 16 12:21:44 2015(r289419)
+++ head/lib/libfetch/file.cFri Oct 16 12:53:22 2015(r289420)
@@ -48,7 +48,7 @@ fetchXGetFile(struct url *u, struct url_
if (us && fetchStatFile(u, us, flags) == -1)
return (NULL);
 
-   f = fopen(u->doc, "r");
+   f = fopen(u->doc, "re");
 
if (f == NULL) {
fetch_syserr();
@@ -61,7 +61,6 @@ fetchXGetFile(struct url *u, struct url_
return (NULL);
}
 
-   fcntl(fileno(f), F_SETFD, FD_CLOEXEC);
return (f);
 }
 
@@ -77,9 +76,9 @@ fetchPutFile(struct url *u, const char *
FILE *f;
 
if (CHECK_FLAG('a'))
-   f = fopen(u->doc, "a");
+   f = fopen(u->doc, "ae");
else
-   f = fopen(u->doc, "w+");
+   f = fopen(u->doc, "w+e");
 
if (f == NULL) {
fetch_syserr();
@@ -92,7 +91,6 @@ fetchPutFile(struct url *u, const char *
return (NULL);
}
 
-   fcntl(fileno(f), F_SETFD, FD_CLOEXEC);
return (f);
 }
 
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"


Re: svn commit: r289405 - head/sys/ufs/ffs

2015-10-16 Thread Slawa Olhovchenkov
On Fri, Oct 16, 2015 at 03:06:02AM +, Warner Losh wrote:

> Author: imp
> Date: Fri Oct 16 03:06:02 2015
> New Revision: 289405
> URL: https://svnweb.freebsd.org/changeset/base/289405
> 
> Log:
>   Do not relocate extents to make them contiguous if the underlying drive can 
> do
>   deletions. Ability to do deletions is a strong indication that this
>   optimization will not help performance. It will only generate extra write
>   traffic. These devices are typically flash based and have a limited number 
> of
>   write cycles. In addition, making the file contiguous in LBA space doesn't
>   improve the access times from flash devices because they have no seek time.

In reality, flash devices have seek time, about 0.1ms.
Many flash devices can do 8 simultaneously "seek" (I think NVMe can do
more).
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"


Re: svn commit: r289405 - head/sys/ufs/ffs

2015-10-16 Thread Pedro Giffuni



On 10/16/15 03:53, Bruce Evans wrote:

On Fri, 16 Oct 2015, Hans Petter Selasky wrote:


On 10/16/15 08:21, Bruce Evans wrote:

[Bruce Evans didn't write:]
In addition, making the file contiguous in LBA space doesn't
  improve the access times from flash devices because they have no seek
time.


This is not exactly true, like Bruce pointed out too. Maybe there
should be a check, that if the block is too small reallocate it, else
leave it for the sake of the flash. Doing 1K accesses versus 64K
accesses will typically show up in the performance benchmark
regardless of how fast the underlying medium is.


Now I don't unerstand the whole point of the change.  Anything that reduces
i/o's is good, but AFAIK ffs_doreallocblks() is all in software.  Writes
should be delayed so that it doesn't have to do extra i/o's to back out of
committed writes.  Often it reduces the number of writes and increases
their size by making blocks contiguous so that the write can be clustered.
Increasing the write size is especially good for flash devices, but maybe
ffs's default block size is already large enough.



I agree with Bruce: reallocation (which our ext2fs also does) happens
in memory, before it hits the disk.

By the nature of their load, Netflix doesn't care about fragmentation,
but even in that case reallocblk doesn't hurt, and I don't see anything
inherent in SSDs that makes fragmentation desirable.

Of course, no one understands reallocblk better than Kirk, and Warner
knows SSD's pretty well so I must be missing something. :).

Pedro.
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"


svn commit: r289421 - in head/etc: . mtree ntp

2015-10-16 Thread Cy Schubert
Author: cy
Date: Fri Oct 16 14:04:16 2015
New Revision: 289421
URL: https://svnweb.freebsd.org/changeset/base/289421

Log:
  Add default leap-seconds file. This should help ntp networks get the
  leap second date correct
  
  Updates to the file can be obtained from ftp://time.nist.gov/pub/ or
  ftp://tycho.usno.navy.mil/pub/ntp/.
  
  Suggested by: dwmalone
  Reviewed by:  roberto, dwmalone, delphij
  Approved by:  roberto
  MFC after:1 week

Added:
  head/etc/ntp/
  head/etc/ntp/Makefile   (contents, props changed)
  head/etc/ntp/leap-seconds   (contents, props changed)
Modified:
  head/etc/Makefile
  head/etc/mtree/BSD.var.dist
  head/etc/ntp.conf

Modified: head/etc/Makefile
==
--- head/etc/Makefile   Fri Oct 16 12:53:22 2015(r289420)
+++ head/etc/Makefile   Fri Oct 16 14:04:16 2015(r289421)
@@ -240,6 +240,9 @@ distribution:
${_+_}cd ${.CURDIR}/defaults; ${MAKE} install
${_+_}cd ${.CURDIR}/devd; ${MAKE} install
${_+_}cd ${.CURDIR}/gss; ${MAKE} install
+.if ${MK_NTP} != "no"
+   ${_+_}cd ${.CURDIR}/ntp; ${MAKE} install
+.endif
${_+_}cd ${.CURDIR}/periodic; ${MAKE} install
 .if ${MK_PKGBOOTSTRAP} != "no"
${_+_}cd ${.CURDIR}/pkg; ${MAKE} install

Modified: head/etc/mtree/BSD.var.dist
==
--- head/etc/mtree/BSD.var.dist Fri Oct 16 12:53:22 2015(r289420)
+++ head/etc/mtree/BSD.var.dist Fri Oct 16 14:04:16 2015(r289421)
@@ -46,6 +46,8 @@
 ..
 ipf mode=0700
 ..
+ntp mode=0700
+..
 pkg
 ..
 ports

Modified: head/etc/ntp.conf
==
--- head/etc/ntp.conf   Fri Oct 16 12:53:22 2015(r289420)
+++ head/etc/ntp.conf   Fri Oct 16 14:04:16 2015(r289421)
@@ -77,3 +77,8 @@ restrict 127.127.1.0
 #
 #server 127.127.1.0
 #fudge 127.127.1.0 stratum 10
+
+# See http://support.ntp.org/bin/view/Support/ConfiguringNTP#Section_6.14.
+# for documentation regarding leapfile. Updates to the file can be obtained
+# from ftp://time.nist.gov/pub/ or ftp://tycho.usno.navy.mil/pub/ntp/.
+leapfile "/etc/ntp/leap-seconds"

Added: head/etc/ntp/Makefile
==
--- /dev/null   00:00:00 1970   (empty, because file is newly added)
+++ head/etc/ntp/Makefile   Fri Oct 16 14:04:16 2015(r289421)
@@ -0,0 +1,10 @@
+# $FreeBSD$
+
+NO_OBJ=
+
+FILES= leap-seconds
+
+FILESDIR=  /etc/ntp
+FILESMODE= 644
+
+.include 

Added: head/etc/ntp/leap-seconds
==
--- /dev/null   00:00:00 1970   (empty, because file is newly added)
+++ head/etc/ntp/leap-seconds   Fri Oct 16 14:04:16 2015(r289421)
@@ -0,0 +1,119 @@
+#
+# $FreeBSD$
+#
+#  ATOMIC TIME.
+#  The Coordinated Universal Time (UTC) is the reference time scale 
derived 
+#  from The "Temps Atomique International" (TAI) calculated by the Bureau 
+#  International des Poids et Mesures (BIPM) using a worldwide network of 
atomic 
+#  clocks. UTC differs from TAI by an integer number of seconds; it is the 
basis 
+#  of all activities in the world. 
+#
+#
+#  ASTRONOMICAL TIME (UT1) is the time scale based on the rate of rotation 
of the earth. 
+#  It is now mainly derived from Very Long Baseline Interferometry (VLBI). 
The various 
+#  irregular fluctuations progressively detected in the rotation rate of 
the Earth lead 
+#  in 1972 to the replacement of UT1 by UTC as the reference time scale. 
+#
+#
+#  LEAP SECOND
+#  Atomic clocks are more stable than the rate of the earth rotatiob since 
the later 
+#  undergoes a full range of geophysical perturbations at various time 
scales (lunisolar 
+#  and core-mantle torques,atmospheric and oceanic effetcs, ...)
+#  Leap seconds are needed to keep the two time scales in agreement, i.e. 
UT1-UTC smaller 
+#  than 0.9 second. So, when necessary a "leap second" is introduced in 
UTC.
+#  Since the adoption of this system in 1972 it has been necessary to add 
26 seconds to UTC, 
+#  firstly due to the initial choice of the value of the second (1/86400 
mean solar day of 
+#  the year 1820) and secondly to the general slowing down of the Earth's 
rotation. It is 
+#  theorically possible to have a negative leap second (a second removed 
from UTC), but so far, 
+#  all leap seconds have been positive (a second has been added to UTC). 
Based on what we know about the earth's rotation, 
+#  it is unlikely that we will ever have a negative leap second.
+#
+#
+#  HISTORY
+#  The first leap second was added on June 30, 1972. Until 2000, it was 
necessary in average to add a leap second at a rate 
+#  of 1 t

svn commit: r289422 - in head: cddl/contrib/opensolaris/lib/libzfs/common sys/cddl/contrib/opensolaris/common/zfs sys/cddl/contrib/opensolaris/uts/common/fs/zfs sys/cddl/contrib/opensolaris/uts/com...

2015-10-16 Thread Alexander Motin
Author: mav
Date: Fri Oct 16 14:45:21 2015
New Revision: 289422
URL: https://svnweb.freebsd.org/changeset/base/289422

Log:
  MFV r289310:
  4185 add new cryptographic checksums to ZFS: SHA-512, Skein, Edon-R
  
  Reviewed by: George Wilson 
  Reviewed by: Prakash Surya 
  Reviewed by: Saso Kiselkov 
  Reviewed by: Richard Lowe 
  Approved by: Garrett D'Amore 
  Author: Matthew Ahrens 
  
  illumos/illumos-gate@45818ee124adeaaf947698996b4f4c722afc6d1f
  
  This is only a partial merge of respective ZFS infrastructure changes.
  At this moment FreeBSD kernel has no those crypto algorithms, so the
  parts of the code to enable them are commented out.  When they are
  implemented, it will be trivial to plug them in.

Added:
  head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/edonr_zfs.c
 - copied unchanged from r289310, 
vendor-sys/illumos/dist/uts/common/fs/zfs/edonr_zfs.c
  head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/skein_zfs.c
 - copied unchanged from r289310, 
vendor-sys/illumos/dist/uts/common/fs/zfs/skein_zfs.c
Modified:
  head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_dataset.c
  head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_sendrecv.c
  head/sys/cddl/contrib/opensolaris/common/zfs/zfeature_common.c
  head/sys/cddl/contrib/opensolaris/common/zfs/zfeature_common.h
  head/sys/cddl/contrib/opensolaris/common/zfs/zfs_fletcher.c
  head/sys/cddl/contrib/opensolaris/common/zfs/zfs_fletcher.h
  head/sys/cddl/contrib/opensolaris/common/zfs/zfs_prop.c
  head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c
  head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/ddt.c
  head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu.c
  head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_send.c
  head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_dataset.c
  head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sha256.c
  head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c
  head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa_misc.c
  head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/dmu.h
  head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/spa.h
  head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/spa_impl.h
  head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zio.h
  head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zio_checksum.h
  head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_ioctl.c
  head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c
  head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio_checksum.c
  head/sys/cddl/contrib/opensolaris/uts/common/sys/debug.h
Directory Properties:
  head/cddl/contrib/opensolaris/   (props changed)
  head/cddl/contrib/opensolaris/lib/libzfs/   (props changed)
  head/sys/cddl/contrib/opensolaris/   (props changed)

Modified: head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_dataset.c
==
--- head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_dataset.cFri Oct 
16 14:04:16 2015(r289421)
+++ head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_dataset.cFri Oct 
16 14:45:21 2015(r289422)
@@ -1493,6 +1493,12 @@ zfs_setprop_error(libzfs_handle_t *hdl, 
"property setting is not allowed on "
"bootable datasets"));
(void) zfs_error(hdl, EZFS_NOTSUP, errbuf);
+   } else if (prop == ZFS_PROP_CHECKSUM ||
+   prop == ZFS_PROP_DEDUP) {
+   (void) zfs_error_aux(hdl, dgettext(TEXT_DOMAIN,
+   "property setting is not allowed on "
+   "root pools"));
+   (void) zfs_error(hdl, EZFS_NOTSUP, errbuf);
} else {
(void) zfs_standard_error(hdl, err, errbuf);
}

Modified: head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_sendrecv.c
==
--- head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_sendrecv.c   Fri Oct 
16 14:04:16 2015(r289421)
+++ head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_sendrecv.c   Fri Oct 
16 14:45:21 2015(r289422)
@@ -1423,7 +1423,7 @@ zfs_send_resume_token_to_nvlist(libzfs_h
 
/* verify checksum */
zio_cksum_t cksum;
-   fletcher_4_native(compressed, len, &cksum);
+   fletcher_4_native(compressed, len, NULL, &cksum);
if (cksum.zc_word[0] != checksum) {
free(compressed);
zfs_error_aux(hdl, dgettext(TEXT_DOMAIN,

Modified: head/sys/cddl/contrib/opensolaris/common/zfs/zfeature_common.c
==
--- head/sys/cddl/contrib/opensolaris/common/zfs/zfeature_common.c  Fri Oct 
16 14:04:16 2015(r289421)
+++ head/sys/cddl/contrib/opensolaris/common/zfs/zfeature_common.c  Fri Oct 
16 14:45:21 20

svn commit: r289423 - head/sys/kern

2015-10-16 Thread Konstantin Belousov
Author: kib
Date: Fri Oct 16 14:55:39 2015
New Revision: 289423
URL: https://svnweb.freebsd.org/changeset/base/289423

Log:
  If falloc_caps() failed, cleanup needs to be performed.  This is a bug
  in r289026.
  
  Sponsored by: The FreeBSD Foundation

Modified:
  head/sys/kern/kern_fork.c

Modified: head/sys/kern/kern_fork.c
==
--- head/sys/kern/kern_fork.c   Fri Oct 16 14:45:21 2015(r289422)
+++ head/sys/kern/kern_fork.c   Fri Oct 16 14:55:39 2015(r289423)
@@ -848,7 +848,7 @@ fork1(struct thread *td, int flags, int 
if (flags & RFPROCDESC) {
error = falloc_caps(td, &fp_procdesc, procdescp, 0, fcaps);
if (error != 0)
-   return (error);
+   goto fail2;
}
 
mem_charged = 0;
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"


svn commit: r289424 - head/sbin/newfs_msdos

2015-10-16 Thread Ed Maste
Author: emaste
Date: Fri Oct 16 15:19:36 2015
New Revision: 289424
URL: https://svnweb.freebsd.org/changeset/base/289424

Log:
  newfs_msdos: prefer snprintf to sprintf
  
  Obtained from:NetBSD
  Sponsored by: The FreeBSD Foundation

Modified:
  head/sbin/newfs_msdos/mkfs_msdos.c

Modified: head/sbin/newfs_msdos/mkfs_msdos.c
==
--- head/sbin/newfs_msdos/mkfs_msdos.c  Fri Oct 16 14:55:39 2015
(r289423)
+++ head/sbin/newfs_msdos/mkfs_msdos.c  Fri Oct 16 15:19:36 2015
(r289424)
@@ -585,7 +585,7 @@ int mkfs_msdos(const char *fname, const 
  (u_int)tm->tm_min));
mk4(bsx->exVolumeID, x);
mklabel(bsx->exVolumeLabel, o.volume_label ? o.volume_label : 
"NO NAME");
-   sprintf(buf, "FAT%u", fat);
+   snprintf(buf, sizeof(buf), "FAT%u", fat);
setstr(bsx->exFileSysType, buf, sizeof(bsx->exFileSysType));
if (!o.bootstrap) {
x1 += sizeof(struct bsx);
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"


Re: svn commit: r289422 - in head: cddl/contrib/opensolaris/lib/libzfs/common sys/cddl/contrib/opensolaris/common/zfs sys/cddl/contrib/opensolaris/uts/common/fs/zfs sys/cddl/contrib/opensolaris/uts/co

2015-10-16 Thread Andriy Gapon
On 16/10/2015 16:45, Alexander Motin wrote:
> Author: mav
> Date: Fri Oct 16 14:45:21 2015
> New Revision: 289422
> URL: https://svnweb.freebsd.org/changeset/base/289422
> 
> Log:
>   MFV r289310:
>   4185 add new cryptographic checksums to ZFS: SHA-512, Skein, Edon-R
>   
>   Reviewed by: George Wilson 
>   Reviewed by: Prakash Surya 
>   Reviewed by: Saso Kiselkov 
>   Reviewed by: Richard Lowe 
>   Approved by: Garrett D'Amore 
>   Author: Matthew Ahrens 
>   
>   illumos/illumos-gate@45818ee124adeaaf947698996b4f4c722afc6d1f
>   
>   This is only a partial merge of respective ZFS infrastructure changes.
>   At this moment FreeBSD kernel has no those crypto algorithms, so the
>   parts of the code to enable them are commented out.  When they are
>   implemented, it will be trivial to plug them in.

BTW, do we need to add SHA-512 support to the boot code?
Toomas Soome has already written some code for that:
https://github.com/tsoome/illumos-gate/blob/loader/usr/src/boot/sys/cddl/boot/zfs/sha256.c

> Added:
>   head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/edonr_zfs.c
>  - copied unchanged from r289310, 
> vendor-sys/illumos/dist/uts/common/fs/zfs/edonr_zfs.c
>   head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/skein_zfs.c
>  - copied unchanged from r289310, 
> vendor-sys/illumos/dist/uts/common/fs/zfs/skein_zfs.c
> Modified:
>   head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_dataset.c
>   head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_sendrecv.c
>   head/sys/cddl/contrib/opensolaris/common/zfs/zfeature_common.c
>   head/sys/cddl/contrib/opensolaris/common/zfs/zfeature_common.h
>   head/sys/cddl/contrib/opensolaris/common/zfs/zfs_fletcher.c
>   head/sys/cddl/contrib/opensolaris/common/zfs/zfs_fletcher.h
>   head/sys/cddl/contrib/opensolaris/common/zfs/zfs_prop.c
>   head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c
>   head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/ddt.c
>   head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu.c
>   head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_send.c
>   head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_dataset.c
>   head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sha256.c
>   head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c
>   head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa_misc.c
>   head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/dmu.h
>   head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/spa.h
>   head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/spa_impl.h
>   head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zio.h
>   head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zio_checksum.h
>   head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_ioctl.c
>   head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c
>   head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio_checksum.c
>   head/sys/cddl/contrib/opensolaris/uts/common/sys/debug.h
> Directory Properties:
>   head/cddl/contrib/opensolaris/   (props changed)
>   head/cddl/contrib/opensolaris/lib/libzfs/   (props changed)
>   head/sys/cddl/contrib/opensolaris/   (props changed)
> 


-- 
Andriy Gapon
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"


Re: svn commit: r289422 - in head: cddl/contrib/opensolaris/lib/libzfs/common sys/cddl/contrib/opensolaris/common/zfs sys/cddl/contrib/opensolaris/uts/common/fs/zfs sys/cddl/contrib/opensolaris/uts/co

2015-10-16 Thread Allan Jude
On 2015-10-16 12:32, Andriy Gapon wrote:
> On 16/10/2015 16:45, Alexander Motin wrote:
>> Author: mav
>> Date: Fri Oct 16 14:45:21 2015
>> New Revision: 289422
>> URL: https://svnweb.freebsd.org/changeset/base/289422
>>
>> Log:
>>   MFV r289310:
>>   4185 add new cryptographic checksums to ZFS: SHA-512, Skein, Edon-R
>>   
>>   Reviewed by: George Wilson 
>>   Reviewed by: Prakash Surya 
>>   Reviewed by: Saso Kiselkov 
>>   Reviewed by: Richard Lowe 
>>   Approved by: Garrett D'Amore 
>>   Author: Matthew Ahrens 
>>   
>>   illumos/illumos-gate@45818ee124adeaaf947698996b4f4c722afc6d1f
>>   
>>   This is only a partial merge of respective ZFS infrastructure changes.
>>   At this moment FreeBSD kernel has no those crypto algorithms, so the
>>   parts of the code to enable them are commented out.  When they are
>>   implemented, it will be trivial to plug them in.
> 
> BTW, do we need to add SHA-512 support to the boot code?
> Toomas Soome has already written some code for that:
> https://github.com/tsoome/illumos-gate/blob/loader/usr/src/boot/sys/cddl/boot/zfs/sha256.c
> 
>> Added:
>>   head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/edonr_zfs.c
>>  - copied unchanged from r289310, 
>> vendor-sys/illumos/dist/uts/common/fs/zfs/edonr_zfs.c
>>   head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/skein_zfs.c
>>  - copied unchanged from r289310, 
>> vendor-sys/illumos/dist/uts/common/fs/zfs/skein_zfs.c
>> Modified:
>>   head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_dataset.c
>>   head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_sendrecv.c
>>   head/sys/cddl/contrib/opensolaris/common/zfs/zfeature_common.c
>>   head/sys/cddl/contrib/opensolaris/common/zfs/zfeature_common.h
>>   head/sys/cddl/contrib/opensolaris/common/zfs/zfs_fletcher.c
>>   head/sys/cddl/contrib/opensolaris/common/zfs/zfs_fletcher.h
>>   head/sys/cddl/contrib/opensolaris/common/zfs/zfs_prop.c
>>   head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c
>>   head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/ddt.c
>>   head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu.c
>>   head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_send.c
>>   head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_dataset.c
>>   head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sha256.c
>>   head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c
>>   head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa_misc.c
>>   head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/dmu.h
>>   head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/spa.h
>>   head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/spa_impl.h
>>   head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zio.h
>>   head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zio_checksum.h
>>   head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_ioctl.c
>>   head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c
>>   head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio_checksum.c
>>   head/sys/cddl/contrib/opensolaris/uts/common/sys/debug.h
>> Directory Properties:
>>   head/cddl/contrib/opensolaris/   (props changed)
>>   head/cddl/contrib/opensolaris/lib/libzfs/   (props changed)
>>   head/sys/cddl/contrib/opensolaris/   (props changed)
>>
> 
> 

I am working on a SHA512/256 implementation for FreeBSD, it is almost
finished. I will get it in ASAP, and I can also handle adding the
support to our gptzfsboot.

-- 
Allan Jude



signature.asc
Description: OpenPGP digital signature


Re: svn commit: r289422 - in head: cddl/contrib/opensolaris/lib/libzfs/common sys/cddl/contrib/opensolaris/common/zfs sys/cddl/contrib/opensolaris/uts/common/fs/zfs sys/cddl/contrib/opensolaris/uts/co

2015-10-16 Thread Andriy Gapon
On 16/10/2015 18:41, Allan Jude wrote:
> I am working on a SHA512/256 implementation for FreeBSD, it is almost
> finished. I will get it in ASAP, and I can also handle adding the
> support to our gptzfsboot.

Sounds great.  Thank you!

-- 
Andriy Gapon
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"


svn commit: r289425 - head/usr.bin/ldd

2015-10-16 Thread Jung-uk Kim
Author: jkim
Date: Fri Oct 16 17:50:36 2015
New Revision: 289425
URL: https://svnweb.freebsd.org/changeset/base/289425

Log:
  Add support for ARM EABI.
  
  MFC after:1 week

Modified:
  head/usr.bin/ldd/ldd.c

Modified: head/usr.bin/ldd/ldd.c
==
--- head/usr.bin/ldd/ldd.c  Fri Oct 16 15:19:36 2015(r289424)
+++ head/usr.bin/ldd/ldd.c  Fri Oct 16 17:50:36 2015(r289425)
@@ -386,9 +386,20 @@ is_executable(const char *fname, int fd,
return (0);
}
if (hdr.elf.e_type == ET_DYN) {
-   if (hdr.elf.e_ident[EI_OSABI] == ELFOSABI_FREEBSD) {
+   switch (hdr.elf.e_ident[EI_OSABI]) {
+   case ELFOSABI_FREEBSD:
*is_shlib = 1;
return (1);
+#ifdef __ARM_EABI__
+   case ELFOSABI_NONE:
+   if (hdr.elf.e_machine != EM_ARM)
+   break;
+   if (EF_ARM_EABI_VERSION(hdr.elf.e_flags) <
+   EF_ARM_EABI_FREEBSD_MIN)
+   break;
+   *is_shlib = 1;
+   return (1);
+#endif
}
warnx("%s: not a FreeBSD ELF shared object", fname);
return (0);
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"


svn commit: r289426 - in head/sys/dev: mpr mps

2015-10-16 Thread Scott Long
Author: scottl
Date: Fri Oct 16 17:56:43 2015
New Revision: 289426
URL: https://svnweb.freebsd.org/changeset/base/289426

Log:
  Remove _FreeBSD_version check for something that was only an issue with
  9-CURRENT.
  
  Obtained from:  Netlfix, Inc
  MFC after:  3 days

Modified:
  head/sys/dev/mpr/mpr.c
  head/sys/dev/mpr/mprvar.h
  head/sys/dev/mps/mps.c
  head/sys/dev/mps/mps_sas.h
  head/sys/dev/mps/mpsvar.h

Modified: head/sys/dev/mpr/mpr.c
==
--- head/sys/dev/mpr/mpr.c  Fri Oct 16 17:50:36 2015(r289425)
+++ head/sys/dev/mpr/mpr.c  Fri Oct 16 17:56:43 2015(r289426)
@@ -1471,11 +1471,9 @@ mpr_setup_sysctl(struct mpr_softc *sc)
OID_AUTO, "enable_ssu", CTLFLAG_RW, &sc->enable_ssu, 0,
"enable SSU to SATA SSD/HDD at shutdown");
 
-#if __FreeBSD_version >= 900030
SYSCTL_ADD_UQUAD(sysctl_ctx, SYSCTL_CHILDREN(sysctl_tree),
OID_AUTO, "chain_alloc_fail", CTLFLAG_RD,
&sc->chain_alloc_fail, "chain allocation failures");
-#endif //FreeBSD_version >= 900030
 
SYSCTL_ADD_INT(sysctl_ctx, SYSCTL_CHILDREN(sysctl_tree),
OID_AUTO, "spinup_wait_time", CTLFLAG_RD,

Modified: head/sys/dev/mpr/mprvar.h
==
--- head/sys/dev/mpr/mprvar.h   Fri Oct 16 17:50:36 2015(r289425)
+++ head/sys/dev/mpr/mprvar.h   Fri Oct 16 17:56:43 2015(r289426)
@@ -265,9 +265,7 @@ struct mpr_softc {
int chain_free_lowwater;
u_int   enable_ssu;
int spinup_wait_time;
-#if __FreeBSD_version >= 900030
uint64_tchain_alloc_fail;
-#endif
struct sysctl_ctx_list  sysctl_ctx;
struct sysctl_oid   *sysctl_tree;
charfw_version[16];
@@ -451,11 +449,8 @@ mpr_alloc_chain(struct mpr_softc *sc)
sc->chain_free--;
if (sc->chain_free < sc->chain_free_lowwater)
sc->chain_free_lowwater = sc->chain_free;
-   }
-#if __FreeBSD_version >= 900030
-   else
+   } else
sc->chain_alloc_fail++;
-#endif
return (chain);
 }
 

Modified: head/sys/dev/mps/mps.c
==
--- head/sys/dev/mps/mps.c  Fri Oct 16 17:50:36 2015(r289425)
+++ head/sys/dev/mps/mps.c  Fri Oct 16 17:56:43 2015(r289426)
@@ -1468,11 +1468,9 @@ mps_setup_sysctl(struct mps_softc *sc)
OID_AUTO, "enable_ssu", CTLFLAG_RW, &sc->enable_ssu, 0,
"enable SSU to SATA SSD/HDD at shutdown");
 
-#if __FreeBSD_version >= 900030
SYSCTL_ADD_UQUAD(sysctl_ctx, SYSCTL_CHILDREN(sysctl_tree),
OID_AUTO, "chain_alloc_fail", CTLFLAG_RD,
&sc->chain_alloc_fail, "chain allocation failures");
-#endif //FreeBSD_version >= 900030
 
SYSCTL_ADD_INT(sysctl_ctx, SYSCTL_CHILDREN(sysctl_tree),
OID_AUTO, "spinup_wait_time", CTLFLAG_RD,

Modified: head/sys/dev/mps/mps_sas.h
==
--- head/sys/dev/mps/mps_sas.h  Fri Oct 16 17:50:36 2015(r289425)
+++ head/sys/dev/mps/mps_sas.h  Fri Oct 16 17:56:43 2015(r289426)
@@ -58,7 +58,6 @@ struct mpssas_target {
 
uint16_ttid;
SLIST_HEAD(, mpssas_lun) luns;
-   TAILQ_HEAD(, mps_command) commands;
struct mps_command *tm;
TAILQ_HEAD(, mps_command) timedout_commands;
uint16_texp_dev_handle;

Modified: head/sys/dev/mps/mpsvar.h
==
--- head/sys/dev/mps/mpsvar.h   Fri Oct 16 17:50:36 2015(r289425)
+++ head/sys/dev/mps/mpsvar.h   Fri Oct 16 17:56:43 2015(r289426)
@@ -283,9 +283,7 @@ struct mps_softc {
int chain_free_lowwater;
u_int   enable_ssu;
int spinup_wait_time;
-#if __FreeBSD_version >= 900030
uint64_tchain_alloc_fail;
-#endif
struct sysctl_ctx_list  sysctl_ctx;
struct sysctl_oid   *sysctl_tree;
charfw_version[16];
@@ -483,11 +481,8 @@ mps_alloc_chain(struct mps_softc *sc)
sc->chain_free--;
if (sc->chain_free < sc->chain_free_lowwater)
sc->chain_free_lowwater = sc->chain_free;
-   }
-#if __FreeBSD_version >= 900030
-   else
+   } else
sc->chain_alloc_fail++;
-#endif
return (chain);
 }
 
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn

Re: svn commit: r289415 - in head: . sys/sys

2015-10-16 Thread Bryan Drewery
On 10/16/2015 1:57 AM, Alexander Motin wrote:
> Author: mav
> Date: Fri Oct 16 08:57:11 2015
> New Revision: 289415
> URL: https://svnweb.freebsd.org/changeset/base/289415
> 
> Log:
>   Bump version and add notice about incompatibility introduced by resumable
>   send/receive support in ZFS.
> 
> Modified:
>   head/UPDATING
>   head/sys/sys/param.h
> 
> Modified: head/UPDATING
> ==
> --- head/UPDATING Fri Oct 16 08:22:21 2015(r289414)
> +++ head/UPDATING Fri Oct 16 08:57:11 2015(r289415)
> @@ -31,6 +31,12 @@ NOTE TO PEOPLE WHO THINK THAT FreeBSD 11
>   disable the most expensive debugging functionality run
>   "ln -s 'abort:false,junk:false' /etc/malloc.conf".)
>  
> +20151015:
> + Added ZFS support for resumable send/receive changed respective
> + IOCTL API/ABI.  Compatibility ABI shims were provided for other
> + functionality, while receive require version match between world
> + and kernel.

This means for example that 'zfs recv' in a 10.2 jail running on a 11.0
kernel will not work? Can compatibility be added for it?

-- 
Regards,
Bryan Drewery



signature.asc
Description: OpenPGP digital signature


Re: svn commit: r289415 - in head: . sys/sys

2015-10-16 Thread Alexander Motin
On 16.10.2015 21:17, Bryan Drewery wrote:
> On 10/16/2015 1:57 AM, Alexander Motin wrote:
>> Author: mav
>> Date: Fri Oct 16 08:57:11 2015
>> New Revision: 289415
>> URL: https://svnweb.freebsd.org/changeset/base/289415
>>
>> Log:
>>   Bump version and add notice about incompatibility introduced by resumable
>>   send/receive support in ZFS.
>>
>> Modified:
>>   head/UPDATING
>>   head/sys/sys/param.h
>>
>> Modified: head/UPDATING
>> ==
>> --- head/UPDATINGFri Oct 16 08:22:21 2015(r289414)
>> +++ head/UPDATINGFri Oct 16 08:57:11 2015(r289415)
>> @@ -31,6 +31,12 @@ NOTE TO PEOPLE WHO THINK THAT FreeBSD 11
>>  disable the most expensive debugging functionality run
>>  "ln -s 'abort:false,junk:false' /etc/malloc.conf".)
>>  
>> +20151015:
>> +Added ZFS support for resumable send/receive changed respective
>> +IOCTL API/ABI.  Compatibility ABI shims were provided for other
>> +functionality, while receive require version match between world
>> +and kernel.
> 
> This means for example that 'zfs recv' in a 10.2 jail running on a 11.0
> kernel will not work? Can compatibility be added for it?

The structures passed as argument for receive are large and completely
different. Unfortunately I don't know enough about that code to say
whether new one can be forged from the old one. I'll take another look,
but can not promise anything.

-- 
Alexander Motin
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"


Re: svn commit: r289405 - head/sys/ufs/ffs

2015-10-16 Thread Warner Losh

> On Oct 16, 2015, at 2:12 AM, Hans Petter Selasky  wrote:
> 
> On 10/16/15 08:21, Bruce Evans wrote:
>> In addition, making the file contiguous in LBA space doesn't
>>  improve the access times from flash devices because they have no seek
>> time.
> 
> Hi,
> 
> This is not exactly true, like Bruce pointed out too. Maybe there should be a 
> check, that if the block is too small reallocate it, else leave it for the 
> sake of the flash. Doing 1K accesses versus 64K accesses will typically show 
> up in the performance benchmark regardless of how fast the underlying medium 
> is.

But that’s not what this does. It isn’t the defrag code that takes the 2-8k 
fragments and squashes them into 16-64k block size for the device. This takes 
the larger blocks and makes sure they are next to each other. This takes large 
contiguous space (like 2MB) and puts as much as possible in a cylinder group. 
That’s totally useless on a flash drive.

Since the sizes of the blocks are so large, moving them won’t change any 
benchmarks.

Warner


signature.asc
Description: Message signed with OpenPGP using GPGMail


Re: svn commit: r289405 - head/sys/ufs/ffs

2015-10-16 Thread Warner Losh

> On Oct 16, 2015, at 7:19 AM, Slawa Olhovchenkov  wrote:
> 
> On Fri, Oct 16, 2015 at 03:06:02AM +, Warner Losh wrote:
> 
>> Author: imp
>> Date: Fri Oct 16 03:06:02 2015
>> New Revision: 289405
>> URL: https://svnweb.freebsd.org/changeset/base/289405
>> 
>> Log:
>>  Do not relocate extents to make them contiguous if the underlying drive can 
>> do
>>  deletions. Ability to do deletions is a strong indication that this
>>  optimization will not help performance. It will only generate extra write
>>  traffic. These devices are typically flash based and have a limited number 
>> of
>>  write cycles. In addition, making the file contiguous in LBA space doesn't
>>  improve the access times from flash devices because they have no seek time.
> 
> In reality, flash devices have seek time, about 0.1ms.
> Many flash devices can do 8 simultaneously "seek" (I think NVMe can do
> more).

That’s just not true. tREAD for most flash is a few tens of microseconds. The
streaming time is at most 10 microseconds. There’s no “seek” time in the classic
sense. Once you get the data, you have it. There’s no extra “read time” in
the NAND flash parts.

And the number of simultaneous reads depends a lot on how the flash vendor
organized the flash. Many of today’s designs use 8 or 16 die parts that have 2
to 4 planes on them, giving a parallelism in the 16-64 range. And that’s before
we get into innovative strategies that use partial page reads to decrease tREAD
time and novel data striping methods.

Seek time, as a separate operation, simply doesn’t exist.

Furthermore, NAND-based devices are log-structured with garbage collection
for both retention and to deal with retired blocks in the underlying NAND. The
relationship between LBA ranges and where the data is at any given time on
the NAND is almost uncorrelated.

So, rearranging data so that it is in LBA contiguous ranges doesn’t help once
you’re above the FFS block level.

Warner



signature.asc
Description: Message signed with OpenPGP using GPGMail


Re: svn commit: r289405 - head/sys/ufs/ffs

2015-10-16 Thread Warner Losh

> On Oct 16, 2015, at 8:03 AM, Pedro Giffuni  wrote:
> 
> 
> 
> On 10/16/15 03:53, Bruce Evans wrote:
>> On Fri, 16 Oct 2015, Hans Petter Selasky wrote:
>> 
>>> On 10/16/15 08:21, Bruce Evans wrote:
 [Bruce Evans didn't write:]
 In addition, making the file contiguous in LBA space doesn't
  improve the access times from flash devices because they have no seek
 time.
>>> 
>>> This is not exactly true, like Bruce pointed out too. Maybe there
>>> should be a check, that if the block is too small reallocate it, else
>>> leave it for the sake of the flash. Doing 1K accesses versus 64K
>>> accesses will typically show up in the performance benchmark
>>> regardless of how fast the underlying medium is.
>> 
>> Now I don't unerstand the whole point of the change.  Anything that reduces
>> i/o's is good, but AFAIK ffs_doreallocblks() is all in software.  Writes
>> should be delayed so that it doesn't have to do extra i/o's to back out of
>> committed writes.  Often it reduces the number of writes and increases
>> their size by making blocks contiguous so that the write can be clustered.
>> Increasing the write size is especially good for flash devices, but maybe
>> ffs's default block size is already large enough.
>> 
> 
> I agree with Bruce: reallocation (which our ext2fs also does) happens
> in memory, before it hits the disk.
> 
> By the nature of their load, Netflix doesn't care about fragmentation,
> but even in that case reallocblk doesn't hurt, and I don't see anything
> inherent in SSDs that makes fragmentation desirable.
> 
> Of course, no one understands reallocblk better than Kirk, and Warner
> knows SSD's pretty well so I must be missing something. :).

This has nothing to do with fragments at all. This is all about rearranging
FFS blocks arbitrarily, which isn’t all in memory before the writes happen.
The data can and does move, which generates TRIM operations for
the old data placement, and real writes for the new data placement. These
optimizations do help.

Warner


signature.asc
Description: Message signed with OpenPGP using GPGMail


svn commit: r289429 - head/sys/dev/mps

2015-10-16 Thread Scott Long
Author: scottl
Date: Fri Oct 16 20:18:12 2015
New Revision: 289429
URL: https://svnweb.freebsd.org/changeset/base/289429

Log:
  Revert an extra hunk that crept into the last commit.
  
  Submitted by: emax
  Obtained from:Netflix, Inc.
  MFC after:3 days

Modified:
  head/sys/dev/mps/mps_sas.h

Modified: head/sys/dev/mps/mps_sas.h
==
--- head/sys/dev/mps/mps_sas.h  Fri Oct 16 19:13:33 2015(r289428)
+++ head/sys/dev/mps/mps_sas.h  Fri Oct 16 20:18:12 2015(r289429)
@@ -58,6 +58,7 @@ struct mpssas_target {
 
uint16_ttid;
SLIST_HEAD(, mpssas_lun) luns;
+   TAILQ_HEAD(, mps_command) commands;
struct mps_command *tm;
TAILQ_HEAD(, mps_command) timedout_commands;
uint16_texp_dev_handle;
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"


Re: svn commit: r289405 - head/sys/ufs/ffs

2015-10-16 Thread Slawa Olhovchenkov
On Fri, Oct 16, 2015 at 01:22:44PM -0600, Warner Losh wrote:

> 
> > On Oct 16, 2015, at 7:19 AM, Slawa Olhovchenkov  wrote:
> > 
> > On Fri, Oct 16, 2015 at 03:06:02AM +, Warner Losh wrote:
> > 
> >> Author: imp
> >> Date: Fri Oct 16 03:06:02 2015
> >> New Revision: 289405
> >> URL: https://svnweb.freebsd.org/changeset/base/289405
> >> 
> >> Log:
> >>  Do not relocate extents to make them contiguous if the underlying drive 
> >> can do
> >>  deletions. Ability to do deletions is a strong indication that this
> >>  optimization will not help performance. It will only generate extra write
> >>  traffic. These devices are typically flash based and have a limited 
> >> number of
> >>  write cycles. In addition, making the file contiguous in LBA space doesn't
> >>  improve the access times from flash devices because they have no seek 
> >> time.
> > 
> > In reality, flash devices have seek time, about 0.1ms.
> > Many flash devices can do 8 simultaneously "seek" (I think NVMe can do
> > more).
> 
> That's just not true. tREAD for most flash is a few tens of microseconds. The
> streaming time is at most 10 microseconds. There's no "seek" time in the 
> classic
> sense. Once you get the data, you have it. There's no extra "read time" in
> the NAND flash parts.
> 
> And the number of simultaneous reads depends a lot on how the flash vendor
> organized the flash. Many of today's designs use 8 or 16 die parts that have 2
> to 4 planes on them, giving a parallelism in the 16-64 range. And that's 
> before
> we get into innovative strategies that use partial page reads to decrease 
> tREAD
> time and novel data striping methods.
> 
> Seek time, as a separate operation, simply doesn't exist.
> 
> Furthermore, NAND-based devices are log-structured with garbage collection
> for both retention and to deal with retired blocks in the underlying NAND. The
> relationship between LBA ranges and where the data is at any given time on
> the NAND is almost uncorrelated.
> 
> So, rearranging data so that it is in LBA contiguous ranges doesn't help once
> you're above the FFS block level.

Stream of random reads 512-4096 bytes from most flash SATA drives in one
thread give about 10K IOPS. This is only 40Mbit/s from 6*0.8 Gbit/s
SATA bandwidth. You may decompose 0.1ms to different, real delay (bank
select, command process and etc.) or give 0.1ms seek time for all
practical purpose.

___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"


svn commit: r289430 - head

2015-10-16 Thread Bryan Drewery
Author: bdrewery
Date: Fri Oct 16 20:23:04 2015
New Revision: 289430
URL: https://svnweb.freebsd.org/changeset/base/289430

Log:
  Remove .MAKE from targets that do more than just run sub-makes, such as
  calling rm or mtree.
  
  MFC after:1 week
  Sponsored by: EMC / Isilon Storage Division

Modified:
  head/Makefile.inc1

Modified: head/Makefile.inc1
==
--- head/Makefile.inc1  Fri Oct 16 20:18:12 2015(r289429)
+++ head/Makefile.inc1  Fri Oct 16 20:23:04 2015(r289430)
@@ -1063,7 +1063,7 @@ INSTALLKERNEL= ${_kernel}
 .endif
 .endfor
 
-buildkernel ${WMAKE_TGTS:N_worldtmp} ${.ALLTARGETS:M_*:N_worldtmp}: .MAKE 
.PHONY
+buildkernel ${WMAKE_TGTS:N_worldtmp:Nbuild32} ${.ALLTARGETS:M_*:N_worldtmp}: 
.MAKE .PHONY
 
 #
 # buildkernel
@@ -1493,7 +1493,7 @@ build-tools: .MAKE
 #
 # kernel-tools: Build kernel-building tools
 #
-kernel-tools: .MAKE
+kernel-tools:
mkdir -p ${MAKEOBJDIRPREFIX}/usr
mtree -deU -f ${.CURDIR}/etc/mtree/BSD.usr.dist \
-p ${MAKEOBJDIRPREFIX}/usr >/dev/null
@@ -1593,7 +1593,7 @@ NXBMAKE=  ${NXBENV} ${MAKE} \
 # For non-clang enabled targets that are still using the in tree gcc
 # we must build a gperf binary for one instance of its Makefiles.  On
 # clang-enabled systems, the gperf binary is obsolete.
-native-xtools: .MAKE
+native-xtools:
 .if ${MK_GCC_BOOTSTRAP} != "no"
mkdir -p ${OBJTREE}/gperf_for_gcc/usr/bin
${_+_}@${ECHODIR} "===> ${_gperf} (obj,depend,all,install)"; \
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"


svn commit: r289432 - head

2015-10-16 Thread Bryan Drewery
Author: bdrewery
Date: Fri Oct 16 20:59:03 2015
New Revision: 289432
URL: https://svnweb.freebsd.org/changeset/base/289432

Log:
  Tweak the guard more to suggest 'all' if SUBDIR_OVERRIDE is specified.  In 
that
  case 'all' does make sense.
  
  MFC after:2 weeks
  X-MFC-With:   r289411
  Sponsored by: EMC / Isilon Storage Division

Modified:
  head/Makefile

Modified: head/Makefile
==
--- head/Makefile   Fri Oct 16 20:51:25 2015(r289431)
+++ head/Makefile   Fri Oct 16 20:59:03 2015(r289432)
@@ -261,7 +261,7 @@ ${TGTS}: .PHONY
 
 _guard: .PHONY
@echo
-   @echo "Explicit target required.  Likely \"buildworld\" is wanted.  See 
build(7)."
+   @echo "Explicit target required.  Likely 
\"${SUBDIR_OVERRIDE:Dall:Ubuildworld}\" is wanted.  See build(7)."
@echo
@false
 
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"


Re: svn commit: r289405 - head/sys/ufs/ffs

2015-10-16 Thread Warner Losh

> On Oct 16, 2015, at 2:18 PM, Slawa Olhovchenkov  wrote:
> 
> On Fri, Oct 16, 2015 at 01:22:44PM -0600, Warner Losh wrote:
> 
>> 
>>> On Oct 16, 2015, at 7:19 AM, Slawa Olhovchenkov  wrote:
>>> 
>>> On Fri, Oct 16, 2015 at 03:06:02AM +, Warner Losh wrote:
>>> 
 Author: imp
 Date: Fri Oct 16 03:06:02 2015
 New Revision: 289405
 URL: https://svnweb.freebsd.org/changeset/base/289405
 
 Log:
 Do not relocate extents to make them contiguous if the underlying drive 
 can do
 deletions. Ability to do deletions is a strong indication that this
 optimization will not help performance. It will only generate extra write
 traffic. These devices are typically flash based and have a limited number 
 of
 write cycles. In addition, making the file contiguous in LBA space doesn't
 improve the access times from flash devices because they have no seek time.
>>> 
>>> In reality, flash devices have seek time, about 0.1ms.
>>> Many flash devices can do 8 simultaneously "seek" (I think NVMe can do
>>> more).
>> 
>> That's just not true. tREAD for most flash is a few tens of microseconds. The
>> streaming time is at most 10 microseconds. There's no "seek" time in the 
>> classic
>> sense. Once you get the data, you have it. There's no extra "read time" in
>> the NAND flash parts.
>> 
>> And the number of simultaneous reads depends a lot on how the flash vendor
>> organized the flash. Many of today's designs use 8 or 16 die parts that have 
>> 2
>> to 4 planes on them, giving a parallelism in the 16-64 range. And that's 
>> before
>> we get into innovative strategies that use partial page reads to decrease 
>> tREAD
>> time and novel data striping methods.
>> 
>> Seek time, as a separate operation, simply doesn't exist.
>> 
>> Furthermore, NAND-based devices are log-structured with garbage collection
>> for both retention and to deal with retired blocks in the underlying NAND. 
>> The
>> relationship between LBA ranges and where the data is at any given time on
>> the NAND is almost uncorrelated.
>> 
>> So, rearranging data so that it is in LBA contiguous ranges doesn't help once
>> you're above the FFS block level.
> 
> Stream of random reads 512-4096 bytes from most flash SATA drives in one
> thread give about 10K IOPS. This is only 40Mbit/s from 6*0.8 Gbit/s
> SATA bandwidth. You may decompose 0.1ms to different, real delay (bank
> select, command process and etc.) or give 0.1ms seek time for all
> practical purpose.

I strongly disagree. That’s not seek time in the classic sense. All of those 
100us
are the delay from reading the data from the flash. The reason I’m so adamant
is that adjacent pages read have exactly the same cost. In a spinning disk,
adjacent sectors read have a tiny cost compared to moving the head (seeking).

Then again, I spent almost three years building a PCIe NAND-based flash
drive, so maybe I’m biased by that experience...

Warner



signature.asc
Description: Message signed with OpenPGP using GPGMail


svn commit: r289433 - head

2015-10-16 Thread Bryan Drewery
Author: bdrewery
Date: Fri Oct 16 21:09:15 2015
New Revision: 289433
URL: https://svnweb.freebsd.org/changeset/base/289433

Log:
  Correct a bitrotted comment about installworld order requirements.
  
  The case of make(1) using a new /bin/sh issue was fixed in r173219 when ITOOLS
  was introduced.
  
  There are still issues with mid-install errors leaving a system unusable that
  are currently non-trivial to solve.  The safest ordering requires installing
  rtld, libc and libthr (in that order) before anything else.  We don't do that
  now though.  Much improvement is needed here still.
  
  Discussed with:   kip and kan (rtld/library ordering)
  MFC after:1 week
  Sponsored by: EMC / Isilon Storage Division

Modified:
  head/Makefile.inc1

Modified: head/Makefile.inc1
==
--- head/Makefile.inc1  Fri Oct 16 20:59:03 2015(r289432)
+++ head/Makefile.inc1  Fri Oct 16 21:09:15 2015(r289433)
@@ -50,11 +50,14 @@
 .include 
 .include 
 
-# We must do lib/ and libexec/ before bin/, because if installworld
-# installs a new /bin/sh, the 'make' command will *immediately*
-# use that new version.  And the new (dynamically-linked) /bin/sh
-# will expect to find appropriate libraries in /lib and /libexec.
-#
+# We must do lib/ and libexec/ before bin/ in case of a mid-install error to
+# keep the users system reasonably usable.  For static->dynamic root upgrades,
+# we don't want to install a dynamic binary without rtld and the needed
+# libraries.  More commonly, for dynamic root, we don't want to install a
+# binary that requires a newer library version that hasn't been installed yet.
+# This ordering is not a guarantee though.  The only guarantee of a working
+# system here would require fine-grained ordering of all components based
+# on their dependencies.
 SRCDIR?=   ${.CURDIR}
 .if defined(SUBDIR_OVERRIDE)
 SUBDIR=${SUBDIR_OVERRIDE}
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"


Re: svn commit: r289433 - head

2015-10-16 Thread Bryan Drewery
On 10/16/2015 2:09 PM, Bryan Drewery wrote:
>   Discussed with: kip and kan (rtld/library ordering)

Ugh, sorry. This was *kib*.

I know a "kip" outside of the community as well that was I talking to
earlier which lead to this.

-- 
Regards,
Bryan Drewery



signature.asc
Description: OpenPGP digital signature


Re: svn commit: r289405 - head/sys/ufs/ffs\

2015-10-16 Thread Slawa Olhovchenkov
On Fri, Oct 16, 2015 at 03:00:50PM -0600, Warner Losh wrote:

>  Do not relocate extents to make them contiguous if the underlying drive 
>  can do
>  deletions. Ability to do deletions is a strong indication that this
>  optimization will not help performance. It will only generate extra write
>  traffic. These devices are typically flash based and have a limited 
>  number of
>  write cycles. In addition, making the file contiguous in LBA space 
>  doesn't
>  improve the access times from flash devices because they have no seek 
>  time.
> >>> 
> >>> In reality, flash devices have seek time, about 0.1ms.
> >>> Many flash devices can do 8 simultaneously "seek" (I think NVMe can do
> >>> more).
> >> 
> >> That's just not true. tREAD for most flash is a few tens of microseconds. 
> >> The
> >> streaming time is at most 10 microseconds. There's no "seek" time in the 
> >> classic
> >> sense. Once you get the data, you have it. There's no extra "read time" in
> >> the NAND flash parts.
> >> 
> >> And the number of simultaneous reads depends a lot on how the flash vendor
> >> organized the flash. Many of today's designs use 8 or 16 die parts that 
> >> have 2
> >> to 4 planes on them, giving a parallelism in the 16-64 range. And that's 
> >> before
> >> we get into innovative strategies that use partial page reads to decrease 
> >> tREAD
> >> time and novel data striping methods.
> >> 
> >> Seek time, as a separate operation, simply doesn't exist.
> >> 
> >> Furthermore, NAND-based devices are log-structured with garbage collection
> >> for both retention and to deal with retired blocks in the underlying NAND. 
> >> The
> >> relationship between LBA ranges and where the data is at any given time on
> >> the NAND is almost uncorrelated.
> >> 
> >> So, rearranging data so that it is in LBA contiguous ranges doesn't help 
> >> once
> >> you're above the FFS block level.
> > 
> > Stream of random reads 512-4096 bytes from most flash SATA drives in one
> > thread give about 10K IOPS. This is only 40Mbit/s from 6*0.8 Gbit/s
> > SATA bandwidth. You may decompose 0.1ms to different, real delay (bank
> > select, command process and etc.) or give 0.1ms seek time for all
> > practical purpose.
> 
> I strongly disagree. That's not seek time in the classic sense. All of those 
> 100us
> are the delay from reading the data from the flash. The reason I'm so adamant
> is that adjacent pages read have exactly the same cost. In a spinning disk,
> adjacent sectors read have a tiny cost compared to moving the head (seeking).
> 
> Then again, I spent almost three years building a PCIe NAND-based flash
> drive, so maybe I'm biased by that experience...

For internal view you right.
For external view this delay like seek time.
For general HDD total_time = seek_time + transfer_time.
seek_time independed of block_size.
transfer_time depended of block_size and is block_size/transfer_speed.

http://tweakers.net/benchdb/testcombo/3817
https://docs.google.com/spreadsheets/d/1BJ-XY0xwc1JvHJxfOGbcW2Be-q5GQbATcMOh9W0d1-U/edit?usp=sharing

This is very closed aproximated by 0.11+block_size/461456. As for HDD.
Yes, no real seek. But in all models this is like seek time,
regardless of real cause.
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"


svn commit: r289434 - head

2015-10-16 Thread Bryan Drewery
Author: bdrewery
Date: Fri Oct 16 22:41:31 2015
New Revision: 289434
URL: https://svnweb.freebsd.org/changeset/base/289434

Log:
  Remove lockf as an ITOOL.
  
  It was added in r152006 to handle serializing access of info/dir when
  installing INFO files.  We no longer support INFO files since r276551
  though.
  
  Sponsored by: EMC / Isilon Storage Division

Modified:
  head/Makefile.inc1

Modified: head/Makefile.inc1
==
--- head/Makefile.inc1  Fri Oct 16 21:09:15 2015(r289433)
+++ head/Makefile.inc1  Fri Oct 16 22:41:31 2015(r289434)
@@ -823,7 +823,7 @@ _zoneinfo=  zic tzsetup
 
 ITOOLS=[ awk cap_mkdb cat chflags chmod chown cmp cp \
date echo egrep find grep id install ${_install-info} \
-   ln lockf make mkdir mtree mv pwd_mkdb \
+   ln make mkdir mtree mv pwd_mkdb \
rm sed services_mkdb sh strip sysctl test true uname wc ${_zoneinfo} \
${LOCAL_ITOOLS}
 
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"


svn commit: r289435 - head

2015-10-16 Thread Bryan Drewery
Author: bdrewery
Date: Fri Oct 16 23:53:37 2015
New Revision: 289435
URL: https://svnweb.freebsd.org/changeset/base/289435

Log:
  Fix adding manpages installed by LOCAL_DIRS to whatis file.
  
  The ordering of 'etc' in the install has a long history dating back to the
  first time it was realized it needed to be "last" in r4486.  That commit
  still left it before LOCAL_DIRS though.  By having it before LOCAL_DIRS
  any manpages they install were not being added to the whatis database in the
  install image.  They would likely show up in the file after a periodic
  rebuild of the file though.
  
  Currently the whatis file is built by an 'afterinstall' hook in etc/Makefile
  that calls share/man's 'makedb' target.
  
  MFC after:2 weeks
  Sponsored by: EMC / Isilon Storage Division

Modified:
  head/Makefile.inc1

Modified: head/Makefile.inc1
==
--- head/Makefile.inc1  Fri Oct 16 22:41:31 2015(r289434)
+++ head/Makefile.inc1  Fri Oct 16 23:53:37 2015(r289435)
@@ -88,10 +88,6 @@ SUBDIR+= tests
 .if ${MK_OFED} != "no"
 SUBDIR+=contrib/ofed
 .endif
-#
-# We must do etc/ last for install/distribute to work.
-#
-SUBDIR+=etc
 
 # Local directories are last, since it is nice to at least get the base
 # system rebuilt before you do them.
@@ -115,6 +111,12 @@ SUBDIR+=   ${_DIR}
 .endfor
 .endif
 
+# We must do etc/ last as it hooks into building the man whatis file
+# by calling 'makedb' in share/man.  This is only relevant for
+# install/distribute so they build the whatis file after every manpage is
+# installed.
+SUBDIR+=etc
+
 .if defined(NOCLEAN)
 NO_CLEAN=  ${NOCLEAN}
 .endif
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"


svn commit: r289438 - head

2015-10-16 Thread Bryan Drewery
Author: bdrewery
Date: Sat Oct 17 03:51:50 2015
New Revision: 289438
URL: https://svnweb.freebsd.org/changeset/base/289438

Log:
  Rework the world subdir build targets to use the standard SUBDIR_PARALLEL 
mechanism.
  
  Back in r30113, the 'par-*' targets were added to parallelize portions of
  the build in a very similar fashion as the SUBDIR_PARALLEL feature used in
  r263778.  Calling a target without 'par-' (for 'parallel') resulted in the
  standard bsd.subdir.mk handling without parallelization.  Given we have
  SUBDIR_PARALLEL now there is no reason to duplicate the handling here.
  
  In build logs this will result in the ${dir}.${target}__D targets now showing
  as the normal ${target}_subdir_${dir} targets.
  
  I audited all of the uses of Makefile.inc1 and Makefile's targets that use
  bsd.subdir.mk and found that all but 'all' and 'install' were fine to use
  as always parallel.
- For 'install' (from installworld -j) the ordering of lib/ and libexec/
  before the rest of the system (described in r289433), and etc/ being last
  (described in r289435), is all that matters.  So now a .WAIT is added in
  the proper places when invoking any 'install*' target.  A parallel
  installworld does work and took 46% of the time a non-parallel
  install would take on my system with -j15 to ZFS.
- For 'all' I left the default handling for this to not run in parallel.  A
  'par-all' target is still used by the 'everything' stage of buildworld
  to continue building in parallel as it already has been.  This works
  because most of the dependencies are handled by the early bootstrap
  phases as well as 'libraries' and 'includes' phases.  This lets
  all of the SUBDIR build in parallel fine, such as bin/ and lib/.  This
  will not work if the user invokes 'all' though as we have dependencies
  spread all over the system with no way to depend between them (except
  for the dirdeps feature in the META_MODE build).  Calling 'make all'
  from the top-level is still useful at least when using SUBDIR_OVERRIDE.
  
  MFC after:2 weeks
  Sponsored by: EMC / Isilon Storage Division

Modified:
  head/Makefile.inc1

Modified: head/Makefile.inc1
==
--- head/Makefile.inc1  Sat Oct 17 03:13:22 2015(r289437)
+++ head/Makefile.inc1  Sat Oct 17 03:51:50 2015(r289438)
@@ -63,6 +63,10 @@ SRCDIR?= ${.CURDIR}
 SUBDIR=${SUBDIR_OVERRIDE}
 .else
 SUBDIR=lib libexec
+.if make(install*)
+# Ensure libraries are installed before progressing.
+SUBDIR+=.WAIT
+.endif
 SUBDIR+=bin
 .if ${MK_CDDL} != "no"
 SUBDIR+=cddl
@@ -115,6 +119,9 @@ SUBDIR+=${_DIR}
 # by calling 'makedb' in share/man.  This is only relevant for
 # install/distribute so they build the whatis file after every manpage is
 # installed.
+.if make(install*)
+SUBDIR+=.WAIT
+.endif
 SUBDIR+=etc
 
 .if defined(NOCLEAN)
@@ -589,9 +596,9 @@ _cleanobj:
@echo "--"
@echo ">>> stage 2.1: cleaning up the object tree"
@echo "--"
-   ${_+_}cd ${.CURDIR}; ${WMAKE} ${CLEANDIR:S/^/par-/}
+   ${_+_}cd ${.CURDIR}; ${WMAKE} ${CLEANDIR}
 .if defined(LIB32TMP)
-   ${_+_}cd ${.CURDIR}; ${LIB32WMAKE} -f Makefile.inc1 
${CLEANDIR:S/^/par-/}
+   ${_+_}cd ${.CURDIR}; ${LIB32WMAKE} -f Makefile.inc1 ${CLEANDIR}
 .endif
 .endif
 _obj:
@@ -599,7 +606,7 @@ _obj:
@echo "--"
@echo ">>> stage 2.2: rebuilding the object tree"
@echo "--"
-   ${_+_}cd ${.CURDIR}; ${WMAKE} par-obj
+   ${_+_}cd ${.CURDIR}; ${WMAKE} obj
 _build-tools:
@echo
@echo "--"
@@ -618,7 +625,7 @@ _includes:
@echo "--"
@echo ">>> stage 4.1: building includes"
@echo "--"
-   ${_+_}cd ${.CURDIR}; ${WMAKE} SHARED=symlinks par-includes
+   ${_+_}cd ${.CURDIR}; ${WMAKE} SHARED=symlinks includes
 _libraries:
@echo
@echo "--"
@@ -632,7 +639,7 @@ _depend:
@echo "--"
@echo ">>> stage 4.3: make dependencies"
@echo "--"
-   ${_+_}cd ${.CURDIR}; ${WMAKE} par-depend
+   ${_+_}cd ${.CURDIR}; ${WMAKE} depend
 everything:
@echo
@echo "--"
@@ -1944,22 +1951,18 @@ _startup_libs: ${_startup_libs:S/$/__L/}
 _prebuild_libs: ${_prebuild_libs:S/$/__L/}
 _g

svn commit: r289439 - head

2015-10-16 Thread Bryan Drewery
Author: bdrewery
Date: Sat Oct 17 04:03:53 2015
New Revision: 289439
URL: https://svnweb.freebsd.org/changeset/base/289439

Log:
  Export OSRELDATE so sub-makes don't look it up again.
  
  We pass BOOTSTRAPPING=${OSRELDATE} to some of the sub-makes.  Rather than
  chase every ${MAKE} invokation, just export it as bmake lets us.
  
  Sponsored by: EMC / Isilon Storage Division

Modified:
  head/Makefile.inc1

Modified: head/Makefile.inc1
==
--- head/Makefile.inc1  Sat Oct 17 03:51:50 2015(r289438)
+++ head/Makefile.inc1  Sat Oct 17 04:03:53 2015(r289439)
@@ -148,6 +148,7 @@ OSRELDATE!= awk '/^\#define[[:space:]]*_
 .else
 OSRELDATE= 0
 .endif
+.export OSRELDATE
 .endif
 
 .if !defined(VERSION)
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"


Re: svn commit: r289405 - head/sys/ufs/ffs

2015-10-16 Thread Bruce Evans

On Fri, 16 Oct 2015, Warner Losh wrote:


On Oct 16, 2015, at 2:12 AM, Hans Petter Selasky  wrote:

On 10/16/15 08:21, Bruce Evans wrote:

In addition, making the file contiguous in LBA space doesn't
 improve the access times from flash devices because they have no seek
time.


This is not exactly true, like Bruce pointed out too. Maybe there should be a 
check, that if the block is too small reallocate it, else leave it for the sake 
of the flash. Doing 1K accesses versus 64K accesses will typically show up in 
the performance benchmark regardless of how fast the underlying medium is.


But that?s not what this does. It isn?t the defrag code that takes the 2-8k 
fragments and squashes them into 16-64k block size for the device. This takes 
the larger blocks and makes sure they are next to each other. This takes large 
contiguous space (like 2MB) and puts as much as possible in a cylinder group. 
That?s totally useless on a flash drive.

Since the sizes of the blocks are so large, moving them won?t change any 
benchmarks.


Not true.  First, the fragment size is actually 512-4K and the block size
is 4K-64K and the block size is that of ffs, not of the device (but the
ffs size should be chosen to be a multiple of the device size).

ffs_doreallocblks() doesn't take large (2MB) blocks and put them in in
a cylinder group.  It takes not very large (4K-64K) ffs blocks and tries
to make them contiguous.  Sometimes this gives large (2MB) contiguous
sets of blocks).  Cylinder groups are irrelevant except that contiguity
is impossible across them.

The pessimization that I was talking about was expanding the number of
i/o's by a large (anything more than 1%) factor by not grouping ffs
blocks.

I know too much about this from benchmarking and fixing the 10-20%
performance loss in soft updates for _reading_ of medium-size files
from pessimizal block allocation.  The loss was larger for soft
updates than for other cases because ffs used its delayed allocation
to perfectly pessimize the location of the indirect block.  10%-20%
is for files in /usr/src with an ffs block size smaller than its
current default of 32K, on a not very fast hard disks.  Most of
these files don't need an indirect block when the ffs block size is
32K or even 16K.  It is surprising that the effect is so large when
the block size is even 16K.  Then only files larger than 192K need
an indirect block.  The size of the effect also depends on the
amount of caching in disks.  With none, any seek kills performance
by much more than 10-20%.  With some, small seeks are almost free.
They just take another i/o to access the disk's cache provided the
disk does good read-ahead.

The bug seems to be that ffs_doreallocblks() does trimming for _all_
blocks that it looks at although most of them were just written with
a delayed write and never reached the disk.  Even the extra i/o's
for this are not good.  Hopefully the disk does nothing for trims
on unwritten blocks.

Bruce___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r289253 - head/share/mk

2015-10-16 Thread NGie Cooper

> On Oct 14, 2015, at 13:16, Bryan Drewery  wrote:

...

> I think what is being seen here is because I removed 'cleanobj' from
> bsd.progs.mk (in r288201) to let bsd.obj.mk handle it. It sometimes
> recurses clean when needed. If there's an obj dir 'cleanobj' will just
> remove that and avoid 'clean'.
> 
> For example (without bsd.progs.mk)
> 
> ~/svn/clean2/usr.bin/vi # make obj
> /usr/obj/root/svn/clean2/usr.bin/vi created for /root/svn/clean2/usr.bin/vi
> ===> catalog (obj)
> /usr/obj/root/svn/clean2/usr.bin/vi/catalog created for
> /root/svn/clean2/usr.bin/vi/catalog
> 
> 
> ~/svn/clean2/usr.bin/vi # make cleanobj
> ===> catalog (cleanobj)
> rm -f dump dutch english french german polish ru_RU.KOI8-R spanish
> swedish  uk_UA.KOI8-U zh_CN.GB2312 dutch.UTF-8 french.UTF-8 german.UTF-8
> polish.UTF-8 spanish.UTF-8 swedish.UTF-8 zh_CN.UTF-8 ru_RU.UTF-8
> uk_UA.UTF-8 english.base *.check __ck1 __ck2
> 
> Next call finds no obj dir and tries cleaning in local dirs:
> 
> ~/svn/clean2/usr.bin/vi # make cleanobj
> rm -f nex nvi nvi.full nvi.debug cl_funcs.o cl_main.o cl_read.o
> cl_screen.o cl_term.o conv.o cut.o delete.o encoding.o exf.o key.o
> line.o log.o main.o mark.o msg.o options.o options_f.o put.o recover.o
> screen.o search.o seq.o util.o ex.o ex_abbrev.o ex_append.o ex_args.o
> ex_argv.o ex_at.o ex_bang.o ex_cd.o ex_cmd.o ex_cscope.o ex_delete.o
> ex_display.o ex_edit.o ex_equal.o ex_file.o ex_filter.o ex_global.o
> ex_init.o ex_join.o ex_map.o ex_mark.o ex_mkexrc.o ex_move.o ex_open.o
> ex_preserve.o ex_print.o ex_put.o ex_quit.o ex_read.o ex_screen.o
> ex_script.o ex_set.o ex_shell.o ex_shift.o ex_source.o ex_stop.o
> ex_subst.o ex_tag.o ex_txt.o ex_undo.o ex_usage.o ex_util.o ex_version.o
> ex_visual.o ex_write.o ex_yank.o ex_z.o getc.o v_at.o v_ch.o v_cmd.o
> v_delete.o v_ex.o v_increment.o v_init.o v_itxt.o v_left.o v_mark.o
> v_match.o v_paragraph.o v_put.o v_redraw.o v_replace.o v_right.o
> v_screen.o v_scroll.o v_search.o v_section.o v_sentence.o v_status.o
> v_txt.o v_ulcase.o v_undo.o v_util.o v_word.o v_xchar.o v_yank.o v_z.o
> v_zexit.o vi.o vs_line.o vs_msg.o vs_refresh.o vs_relative.o vs_smap.o
> vs_split.o regcomp.o regerror.o regexec.o regfree.o vi.1.gz vi.1.cat.gz
> ===> catalog (clean)
> rm -f dump dutch english french german polish ru_RU.KOI8-R spanish
> swedish  uk_UA.KOI8-U zh_CN.GB2312 dutch.UTF-8 french.UTF-8 german.UTF-8
> polish.UTF-8 spanish.UTF-8 swedish.UTF-8 zh_CN.UTF-8 ru_RU.UTF-8
> uk_UA.UTF-8 english.base *.check __ck1 __ck2
> rm -f .depend tags
> ===> catalog (cleandepend)
> ===> catalog (cleanobj)
> rm -f dump dutch english french german polish ru_RU.KOI8-R spanish
> swedish  uk_UA.KOI8-U zh_CN.GB2312 dutch.UTF-8 french.UTF-8 german.UTF-8
> polish.UTF-8 spanish.UTF-8 swedish.UTF-8 zh_CN.UTF-8 ru_RU.UTF-8
> uk_UA.UTF-8 english.base *.check __ck1 __ck2
> 
> 
> Similar behavior will happen with bsd.progs.mk usage now. 'cleandir'
> also has this flip-flopping behavior.

That could be a part of it. Part of the other unintuitive legacy behavior for 
make cleandir is that you have to call it twice in order for it to completely 
invoke all of the clean targets that its dependent on:

$ uname -a
FreeBSD fbsd11 11.0-CURRENT FreeBSD 11.0-CURRENT #6 
r288943+b94810a(isilon-atf): Tue Oct  6 14:54:34 PDT 2015 
ngie@fbsd11:/usr/obj/usr/src/git/sys/GENERIC-NODEBUG  amd64
$ make cleandir
(cd /usr/src/svn/tests/sys/posixshm &&  DEPENDFILE=.depend.posixshm_test  make 
-f /usr/src/svn/tests/sys/posixshm/Makefile _RECURSING_PROGS=  SUBDIR= 
PROG=posixshm_test  cleandir)
rm -f  posixshm_test posixshm_test.o
rm -f .depend.posixshm_test GPATH GRTAGS GSYMS GTAGS
[ngie@fbsd11 /usr/src/svn/tests/sys/posixshm]$ make cleandir
(cd /usr/src/svn/tests/sys/posixshm &&  DEPENDFILE=.depend.posixshm_test  make 
-f Makefile _RECURSING_PROGS=  SUBDIR= PROG=posixshm_test  clean)
rm -f  posixshm_test posixshm_test.o
rm -f Kyuafile.auto Kyuafile.auto.tmp
(cd /usr/src/svn/tests/sys/posixshm &&  DEPENDFILE=.depend.posixshm_test  make 
-f Makefile _RECURSING_PROGS=  SUBDIR= PROG=posixshm_test  cleandepend)
rm -f .depend.posixshm_test GPATH GRTAGS GSYMS GTAGS
(cd /usr/src/svn/tests/sys/posixshm &&  DEPENDFILE=.depend.posixshm_test  make 
-f Makefile _RECURSING_PROGS=  SUBDIR= PROG=posixshm_test  cleandir)
rm -f  posixshm_test posixshm_test.o
rm -f .depend.posixshm_test GPATH GRTAGS GSYMS GTAGS

This might be some of what you were observing above. It looks better now after 
the last few days, but it was definitely less functional on head between the 
12th and now on head.

Thanks!
-NGie
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"


svn commit: r289441 - in head: etc/mtree tests/sys tests/sys/posixshm tools/regression/posixshm tools/test/posixshm

2015-10-16 Thread Garrett Cooper
Author: ngie
Date: Sat Oct 17 04:32:21 2015
New Revision: 289441
URL: https://svnweb.freebsd.org/changeset/base/289441

Log:
  Integrate tools/test/posixshm and tools/regression/posixshm into the FreeBSD
  test suite as tests/sys/posixshm
  
  Some other highlights:
  - Convert the testcases over to ATF
  - Don't use hardcoded paths to /tmp (which violate the ATF/kyua samdbox); use
mkstemp to generate temporary paths for non-SHM_ANON shm objects.
  
  MFC after: 2 weeks
  Sponsored by: EMC / Isilon Storage Division

Added:
  head/tests/sys/posixshm/
 - copied from r289223, user/ngie/more-tests2/tests/sys/posixshm/
  head/tests/sys/posixshm/posixshm_test.c
 - copied, changed from r289437, 
user/ngie/more-tests2/tests/sys/posixshm/posixshm_test.c
Deleted:
  head/tests/sys/posixshm/posixshm.c
  head/tests/sys/posixshm/shm_test.c
  head/tests/sys/posixshm/test.c
  head/tests/sys/posixshm/test.h
  head/tools/regression/posixshm/
  head/tools/test/posixshm/
Modified:
  head/etc/mtree/BSD.tests.dist
  head/tests/sys/Makefile
  head/tests/sys/posixshm/Makefile
Directory Properties:
  head/   (props changed)

Modified: head/etc/mtree/BSD.tests.dist
==
--- head/etc/mtree/BSD.tests.dist   Sat Oct 17 04:07:41 2015
(r289440)
+++ head/etc/mtree/BSD.tests.dist   Sat Oct 17 04:32:21 2015
(r289441)
@@ -410,6 +410,8 @@
 unlink
 ..
 ..
+posixshm
+..
 vfs
 ..
 vm

Modified: head/tests/sys/Makefile
==
--- head/tests/sys/Makefile Sat Oct 17 04:07:41 2015(r289440)
+++ head/tests/sys/Makefile Sat Oct 17 04:32:21 2015(r289441)
@@ -12,6 +12,7 @@ TESTS_SUBDIRS+=   kqueue
 TESTS_SUBDIRS+=mqueue
 TESTS_SUBDIRS+=netinet
 TESTS_SUBDIRS+=opencrypto
+TESTS_SUBDIRS+=posixshm
 TESTS_SUBDIRS+=vfs
 TESTS_SUBDIRS+=vm
 

Modified: head/tests/sys/posixshm/Makefile
==
--- user/ngie/more-tests2/tests/sys/posixshm/Makefile   Tue Oct 13 16:50:12 
2015(r289223)
+++ head/tests/sys/posixshm/MakefileSat Oct 17 04:32:21 2015
(r289441)
@@ -2,10 +2,6 @@
 
 TESTSDIR=  ${TESTSBASE}/sys/posixshm
 
-TAP_TESTS_C+=  posixshm_test
-PLAIN_TESTS_C+=posixshm_test2
-
-SRCS.posixshm_test=posixshm.c test.c
-SRCS.posixshm_test2=   shm_test.c
+ATF_TESTS_C+=  posixshm_test
 
 .include 

Copied and modified: head/tests/sys/posixshm/posixshm_test.c (from r289437, 
user/ngie/more-tests2/tests/sys/posixshm/posixshm_test.c)
==
--- user/ngie/more-tests2/tests/sys/posixshm/posixshm_test.cSat Oct 17 
03:13:22 2015(r289437, copy source)
+++ head/tests/sys/posixshm/posixshm_test.c Sat Oct 17 04:32:21 2015
(r289441)
@@ -110,18 +110,18 @@ scribble_object(void)
fd = shm_open(test_path, O_CREAT | O_EXCL | O_RDWR, 0777);
}
if (fd < 0)
-   atf_tc_fail("shm_open");
+   atf_tc_fail("shm_open failed; errno=%d", errno);
if (ftruncate(fd, getpagesize()) < 0)
-   atf_tc_fail("ftruncate");
+   atf_tc_fail("ftruncate failed; errno=%d", errno);
 
page = mmap(0, getpagesize(), PROT_READ | PROT_WRITE, MAP_SHARED, fd,
0);
if (page == MAP_FAILED)
-   atf_tc_fail("mmap");
+   atf_tc_fail("mmap failed; errno=%d", errno);
 
page[0] = '1';
if (munmap(page, getpagesize()) < 0)
-   atf_tc_fail("munmap");
+   atf_tc_fail("munmap failed; errno=%d", errno);
 
return (fd);
 }
@@ -136,32 +136,18 @@ ATF_TC_BODY(remap_object, tc)
 
page = mmap(0, getpagesize(), PROT_READ | PROT_WRITE, MAP_SHARED, fd,
0);
-   if (page == MAP_FAILED) {   
-   atf_tc_fail("mmap(2)");
-   close(fd);
-   shm_unlink(test_path);
-   return;
-   }
+   if (page == MAP_FAILED)
+   atf_tc_fail("mmap(2) failed; errno=%d", errno);
 
-   if (page[0] != '1') {   
-   atf_tc_fail("missing data");
-   close(fd);
-   shm_unlink(test_path);
-   return;
-   }
+   if (page[0] != '1')
+   atf_tc_fail("missing data ('%c' != '1')", page[0]);
 
close(fd);
-   if (munmap(page, getpagesize()) < 0) {
-   atf_tc_fail("munmap");
-   shm_unlink(test_path);
-   return;
-   }
-
-   if (shm_unlink(test_path) < 0) {
-   atf_tc_fail("shm_unlink");
-   return;
-   }
+   if (munmap(page, getpagesize()) < 0)
+   

svn commit: r289442 - head/sys/conf

2015-10-16 Thread Bryan Drewery
Author: bdrewery
Date: Sat Oct 17 05:49:07 2015
New Revision: 289442
URL: https://svnweb.freebsd.org/changeset/base/289442

Log:
  Do as r289391 did for share/mk and make installing to a non-existent directory
  an error.
  
  Most of these do a 'mkdir -p' or 'install -d' before installing, but add
  the trailing / here for consistency with the userland install.
  
  MFC after:2 weeks
  X-MFC-With:   r289391
  Sponsored by: EMC / Isilon Storage Division

Modified:
  head/sys/conf/dtb.mk
  head/sys/conf/kern.post.mk
  head/sys/conf/kmod.mk

Modified: head/sys/conf/dtb.mk
==
--- head/sys/conf/dtb.mkSat Oct 17 04:32:21 2015(r289441)
+++ head/sys/conf/dtb.mkSat Oct 17 05:49:07 2015(r289442)
@@ -69,7 +69,7 @@ _dtbinstall:
test -d ${DESTDIR}${DTBDIR} || ${INSTALL} -d -o ${DTBOWN} -g ${DTBGRP} 
${DESTDIR}${DTBDIR}
 .for _dtb in ${DTB}
${INSTALL} -o ${DTBOWN} -g ${DTBGRP} -m ${DTBMODE} \
-   ${_INSTALLFLAGS} ${_dtb} ${DESTDIR}${DTBDIR}
+   ${_INSTALLFLAGS} ${_dtb} ${DESTDIR}${DTBDIR}/
 .endfor
 .endif # !target(realinstall)
 .endif # !target(install)

Modified: head/sys/conf/kern.post.mk
==
--- head/sys/conf/kern.post.mk  Sat Oct 17 04:32:21 2015(r289441)
+++ head/sys/conf/kern.post.mk  Sat Oct 17 05:49:07 2015(r289442)
@@ -270,22 +270,22 @@ kernel-install:
fi
 .endif
mkdir -p ${DESTDIR}${KODIR}
-   ${INSTALL} -p -m 555 -o ${KMODOWN} -g ${KMODGRP} ${KERNEL_KO} 
${DESTDIR}${KODIR}
+   ${INSTALL} -p -m 555 -o ${KMODOWN} -g ${KMODGRP} ${KERNEL_KO} 
${DESTDIR}${KODIR}/
 .if defined(DEBUG) && !defined(INSTALL_NODEBUG) && ${MK_KERNEL_SYMBOLS} != "no"
mkdir -p ${DESTDIR}${KERN_DEBUGDIR}${KODIR}
-   ${INSTALL} -p -m 555 -o ${KMODOWN} -g ${KMODGRP} ${KERNEL_KO}.debug 
${DESTDIR}${KERN_DEBUGDIR}${KODIR}
+   ${INSTALL} -p -m 555 -o ${KMODOWN} -g ${KMODGRP} ${KERNEL_KO}.debug 
${DESTDIR}${KERN_DEBUGDIR}${KODIR}/
 .endif
 .if defined(KERNEL_EXTRA_INSTALL)
-   ${INSTALL} -p -m 555 -o ${KMODOWN} -g ${KMODGRP} 
${KERNEL_EXTRA_INSTALL} ${DESTDIR}${KODIR}
+   ${INSTALL} -p -m 555 -o ${KMODOWN} -g ${KMODGRP} 
${KERNEL_EXTRA_INSTALL} ${DESTDIR}${KODIR}/
 .endif
 
 
 
 kernel-reinstall:
@-chflags -R noschg ${DESTDIR}${KODIR}
-   ${INSTALL} -p -m 555 -o ${KMODOWN} -g ${KMODGRP} ${KERNEL_KO} 
${DESTDIR}${KODIR}
+   ${INSTALL} -p -m 555 -o ${KMODOWN} -g ${KMODGRP} ${KERNEL_KO} 
${DESTDIR}${KODIR}/
 .if defined(DEBUG) && !defined(INSTALL_NODEBUG) && ${MK_KERNEL_SYMBOLS} != "no"
-   ${INSTALL} -p -m 555 -o ${KMODOWN} -g ${KMODGRP} ${KERNEL_KO}.debug 
${DESTDIR}${KERN_DEBUGDIR}${KODIR}
+   ${INSTALL} -p -m 555 -o ${KMODOWN} -g ${KMODGRP} ${KERNEL_KO}.debug 
${DESTDIR}${KERN_DEBUGDIR}${KODIR}/
 .endif
 
 config.o env.o hints.o vers.o vnode_if.o:

Modified: head/sys/conf/kmod.mk
==
--- head/sys/conf/kmod.mk   Sat Oct 17 04:32:21 2015(r289441)
+++ head/sys/conf/kmod.mk   Sat Oct 17 05:49:07 2015(r289442)
@@ -296,10 +296,10 @@ realinstall: _kmodinstall
 .ORDER: beforeinstall _kmodinstall
 _kmodinstall:
${INSTALL} -o ${KMODOWN} -g ${KMODGRP} -m ${KMODMODE} \
-   ${_INSTALLFLAGS} ${PROG} ${DESTDIR}${KMODDIR}
+   ${_INSTALLFLAGS} ${PROG} ${DESTDIR}${KMODDIR}/
 .if defined(DEBUG_FLAGS) && !defined(INSTALL_NODEBUG) && ${MK_KERNEL_SYMBOLS} 
!= "no"
${INSTALL} -o ${KMODOWN} -g ${KMODGRP} -m ${KMODMODE} \
-   ${_INSTALLFLAGS} ${PROG}.debug ${DESTDIR}${KERN_DEBUGDIR}${KMODDIR}
+   ${_INSTALLFLAGS} ${PROG}.debug ${DESTDIR}${KERN_DEBUGDIR}${KMODDIR}/
 .endif
 
 .include 
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"


svn commit: r289443 - head

2015-10-16 Thread Bryan Drewery
Author: bdrewery
Date: Sat Oct 17 05:55:45 2015
New Revision: 289443
URL: https://svnweb.freebsd.org/changeset/base/289443

Log:
  Always export VERSION to the environment to avoid looking it up again in
  sub-makes.
  
  Some of the world phases that used plain '${MAKE} -f Makefile.inc1' were not
  passing this variable along which caused them to look it up again.  By
  using bmake's .export we can remove it from all of the other environment
  lines.
  
  Add a comment about the usage for VERSION for ctfmerge.
  
  Sponsored by: EMC / Isilon Storage Division

Modified:
  head/Makefile.inc1

Modified: head/Makefile.inc1
==
--- head/Makefile.inc1  Sat Oct 17 05:49:07 2015(r289442)
+++ head/Makefile.inc1  Sat Oct 17 05:55:45 2015(r289443)
@@ -151,12 +151,14 @@ OSRELDATE=0
 .export OSRELDATE
 .endif
 
+# Set VERSION for CTFMERGE to use via the default CTFFLAGS=-L VERSION.
 .if !defined(VERSION)
 REVISION!= ${MAKE} -C ${SRCDIR}/release -V REVISION
 BRANCH!=   ${MAKE} -C ${SRCDIR}/release -V BRANCH
 SRCRELDATE!=   awk '/^\#define[[:space:]]*__FreeBSD_version/ { print $$3 }' \
${SRCDIR}/sys/sys/param.h
 VERSION=   FreeBSD ${REVISION}-${BRANCH:C/-p[0-9]+$//} ${TARGET_ARCH} 
${SRCRELDATE}
+.export VERSION
 .endif
 
 KNOWN_ARCHES?= aarch64/arm64 amd64 arm armeb/arm armv6/arm armv6hf/arm i386 
i386/pc98 mips mipsel/mips mips64el/mips mips64/mips mipsn32el/mips 
mipsn32/mips powerpc powerpc64/powerpc sparc64
@@ -265,7 +267,6 @@ CROSSENV+=  ${TARGET_CFLAGS}
 BMAKEENV=  INSTALL="sh ${.CURDIR}/tools/install.sh" \
PATH=${BPATH}:${PATH} \
WORLDTMP=${WORLDTMP} \
-   VERSION="${VERSION}" \
MAKEFLAGS="-m ${.CURDIR}/tools/build/mk ${.MAKEFLAGS}"
 # need to keep this in sync with targets/pseudo/bootstrap-tools/Makefile
 BSARGS=DESTDIR= \
@@ -302,8 +303,7 @@ XMAKE=  TOOLS_PREFIX=${WORLDTMP} ${BMAKE
 # kernel-tools stage
 KTMAKEENV= INSTALL="sh ${.CURDIR}/tools/install.sh" \
PATH=${BPATH}:${PATH} \
-   WORLDTMP=${WORLDTMP} \
-   VERSION="${VERSION}"
+   WORLDTMP=${WORLDTMP}
 KTMAKE=TOOLS_PREFIX=${WORLDTMP} MAKEOBJDIRPREFIX=${WORLDTMP} \
${KTMAKEENV} ${MAKE} ${WORLD_FLAGS} -f Makefile.inc1 \
DESTDIR= \
@@ -316,7 +316,6 @@ KTMAKE= TOOLS_PREFIX=${WORLDTMP} MAKEOB
 # world stage
 WMAKEENV=  ${CROSSENV} \
_LDSCRIPTROOT= \
-   VERSION="${VERSION}" \
INSTALL="sh ${.CURDIR}/tools/install.sh" \
PATH=${TMPPATH}
 
@@ -459,7 +458,6 @@ LIB32FLAGS+=--sysroot=${WORLDTMP}
 # Yes, the flags are redundant.
 LIB32WMAKEENV+=MAKEOBJDIRPREFIX=${LIB32_OBJTREE} \
_LDSCRIPTROOT=${LIB32TMP} \
-   VERSION="${VERSION}" \
INSTALL="sh ${.CURDIR}/tools/install.sh" \
PATH=${TMPPATH} \
LIBDIR=/usr/lib32 \
@@ -1587,7 +1585,6 @@ cross-tools: .MAKE
 
 NXBENV=MAKEOBJDIRPREFIX=${OBJTREE}/nxb \
INSTALL="sh ${.CURDIR}/tools/install.sh" \
-   VERSION="${VERSION}" \
PATH=${PATH}:${OBJTREE}/gperf_for_gcc/usr/bin
 NXBMAKE=   ${NXBENV} ${MAKE} \
TBLGEN=${OBJTREE}/nxb-bin/usr/bin/tblgen \
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"


svn commit: r289444 - head

2015-10-16 Thread Bryan Drewery
Author: bdrewery
Date: Sat Oct 17 05:57:29 2015
New Revision: 289444
URL: https://svnweb.freebsd.org/changeset/base/289444

Log:
  For 'buildenvvars' show any .exported variables as well to cover recent
  exporting of OSRELDATE and VERSION.  These already do export to 'buildenv'
  fine.
  
  Sponsored by: EMC / Isilon Storage Division

Modified:
  head/Makefile.inc1

Modified: head/Makefile.inc1
==
--- head/Makefile.inc1  Sat Oct 17 05:55:45 2015(r289443)
+++ head/Makefile.inc1  Sat Oct 17 05:57:29 2015(r289444)
@@ -747,7 +747,7 @@ buildworld_epilogue:
 # to quote multiword values.
 #
 buildenvvars:
-   @echo ${WMAKEENV:Q}
+   @echo ${WMAKEENV:Q} ${.MAKE.EXPORTED:@v@$v=\"${$v}\"@}
 
 .if ${.TARGETS:Mbuildenv}
 .if ${.MAKEFLAGS:M-j}
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"