Re: Interactive tool for installing packages
In the last episode (Nov 10), per...@pluto.rain.com said: > Marin Atanasov Nikolov wrote: > > in order to install the program, you need to: > > > > # git clone git://git.unix-heaven.org/public/pkg_add_it > ... > > Surely, there's room for improvement, but that's a start.. :) > > Dunno about anyone else, but from my standpoint it would be a _big_ > improvement to provide a more recent snapshot than the 6-month-old > pkg_add_it-1.2.tar.gz on ftp.freebsd.org so one doesn't have to install > git, with its boatload of dependencies*, to see the recent improvements. > > * The amount of stuff downloaded by > cd /usr/ports/devel/git ; make fetch-recursive > is, shall we say, impressive. I use the devel/hg-git port to pull all the git trees I need to access using mercurial. -- Dan Nelson dnel...@allantgroup.com ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: ugen claiming pcie device
In the last episode (Nov 13), Christopher Bowman said: > I have a Xilinx PCIe card installed in my machine and it appears that > ugen0 is claiming it. Why would a PCIe device even be offered to ugen? > > The message I get on boot up is: > ugen0 on uhub3 > Any way I can prevent this so my on kld driver can attach? > Regards If it's hanging off uhub3, it's a usb device :) Google says that is the Xilinx "USB platform cable" device. The ugen device attaches to any usb device that no other driver has claimed. Maybe the Xilinx card provides a virtual usb controller and device that you control the card with? -- Dan Nelson dnel...@allantgroup.com ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Namecache lock contention?
In the last episode (Jan 28), Ivan Voras said: > I have this situation on a PHP server: > > 36623 www 1 760 237M 30600K *Name 6 0:14 47.27% php-cgi > 36638 www 1 760 237M 30600K *Name 3 0:14 46.97% php-cgi > 36628 www 1 1050 237M 30600K *Name 2 0:14 46.88% php-cgi > 36627 www 1 1050 237M 30600K *Name 0 0:14 46.78% php-cgi > 36639 www 1 1050 237M 30600K *Name 5 0:14 46.58% php-cgi > 36643 www 1 1050 237M 30600K *Name 7 0:14 46.39% php-cgi > 36629 www 1 760 237M 30600K *Name 1 0:14 46.39% php-cgi > 36642 www 1 1050 237M 30600K *Name 2 0:14 46.39% php-cgi > 36626 www 1 1050 237M 30600K *Name 5 0:14 46.19% php-cgi > 36654 www 1 1050 237M 30600K *Name 7 0:13 46.19% php-cgi > 36645 www 1 1050 237M 30600K *Name 1 0:14 45.75% php-cgi > 36625 www 1 1050 237M 30600K *Name 0 0:14 45.56% php-cgi > 36624 www 1 1050 237M 30600K *Name 6 0:14 45.56% php-cgi > 36630 www 1 760 237M 30600K *Name 7 0:14 45.17% php-cgi > 36631 www 1 1050 237M 30600K RUN 4 0:14 45.17% php-cgi > 36636 www 1 1050 237M 30600K *Name 3 0:14 44.87% php-cgi > > It looks like periodically most or all of the php-cgi processes are > blocked in "*Name" for long enough that "top" notices, then continue, > probably in a "thundering herd" way. From grepping inside /sys the most > likely suspect seems to be something in the namecache, but I can't find > exactly a symbol named "Name" or string beginning with "Name" that would > be connected to a lock. My guess would be: kern/vfs_cache.c:151 static struct rwlock cache_lock; kern/vfs_cache.c:152 RW_SYSINIT(vfscache, &cache_lock, "Name Cache"); The CACHE_*LOCK() macros.c in vfs_cache use cache_lock, so you've got lots of possible contention points. procstat -ka and/or dtrace might help you determine exactly where. -- Dan Nelson dnel...@allantgroup.com ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: reverse of getchar() read() open() fopen() ?
In the last episode (Feb 11), Julian H. Stacey said: > Hi hackers@, > Do we have C libraries with reverse of getchar() [ & maybe read() ] & > fopen() [ & maybe open() ] etc, to read from end of file toward beginning > ? I dont see anything in the See Also sections. I'm not looking to > write, just read. I'm looking for something that returns last char in > file as first etc, I'm not interested in wchars etc, I could write some C > functions, with seek etc & probably will, if none exist, but no point if > they already exist ? tail -r does this on a line-by-line basis. Tail does it for regular files by mmaping and then reading backwards, but you could also read the block of data at (filesize MOD buffersize) and return its bytes in reverse order, then seek back buffersize bytes, read that block and print it reversed, etc. You might even be able to write functions that could be passed to funopen(). Then you'd have a regular FILE* that you could call with regular stdio functions. Getting the buffering right for good performance might get tricky, though. -- Dan Nelson dnel...@allantgroup.com ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: usertime and systime
In the last episode (Mar 16), Thiago Damas said:
> Hi,
> without procfs, there is a way to get usertime and systime from a
> running process?
Try applying the attached patch to ps. I've had it for a while but never
submitted a PR.
Heh. I've had it for a very long time.
http://lists.freebsd.org/pipermail/freebsd-hackers/2009-March/027918.html
--
Dan Nelson
dnel...@allantgroup.com
Index: ps.1
===
--- ps.1 (revision 219700)
+++ ps.1 (working copy)
@@ -587,6 +587,8 @@ symbolic process state (alias
saved gid from a setgid executable
.It Cm svuid
saved UID from a setuid executable
+.It Cm systime
+accumulated system CPU time
.It Cm tdaddr
thread address
.It Cm tdev
@@ -617,6 +619,8 @@ scheduling priority on return from system call (al
.Cm usrpri )
.It Cm user
user name (from UID)
+.It Cm usertime
+accumulated user CPU time
.It Cm vsz
virtual size in Kbytes (alias
.Cm vsize )
Index: keyword.c
===
--- keyword.c (revision 219700)
+++ keyword.c (working copy)
@@ -186,6 +186,7 @@ static VAR var[] = {
UINT, UIDFMT, 0},
{"svuid", "SVUID", NULL, 0, kvar, NULL, UIDLEN, KOFF(ki_svuid),
UINT, UIDFMT, 0},
+ {"systime", "SYSTIME", NULL, USER, systime, NULL, 9, 0, CHAR, NULL, 0},
{"tdaddr", "TDADDR", NULL, 0, kvar, NULL, sizeof(void *) * 2,
KOFF(ki_tdaddr), KPTR, "lx", 0},
{"tdev", "TDEV", NULL, 0, tdev, NULL, 5, 0, CHAR, NULL, 0},
@@ -207,6 +208,7 @@ static VAR var[] = {
KOFF(ki_paddr), KPTR, "lx", 0},
{"user", "USER", NULL, LJUST|DSIZ, uname, s_uname, USERLEN, 0, CHAR,
NULL, 0},
+ {"usertime", "USERTIME", NULL, USER, usertime, NULL, 9, 0, CHAR, NULL, 0},
{"usrpri", "", "upr", 0, NULL, NULL, 0, 0, CHAR, NULL, 0},
{"vsize", "", "vsz", 0, NULL, NULL, 0, 0, CHAR, NULL, 0},
{"vsz", "VSZ", NULL, 0, vsize, NULL, 5, 0, CHAR, NULL, 0},
Index: extern.h
===
--- extern.h (revision 219700)
+++ extern.h (working copy)
@@ -79,12 +79,14 @@ int s_uname(KINFO *);
void showkey(void);
void started(KINFO *, VARENT *);
void state(KINFO *, VARENT *);
+void systime(KINFO *, VARENT *);
void tdev(KINFO *, VARENT *);
void tdnam(KINFO *, VARENT *);
void tname(KINFO *, VARENT *);
void ucomm(KINFO *, VARENT *);
void uname(KINFO *, VARENT *);
void upr(KINFO *, VARENT *);
+void usertime(KINFO *, VARENT *);
void vsize(KINFO *, VARENT *);
void wchan(KINFO *, VARENT *);
__END_DECLS
Index: print.c
===
--- print.c (revision 219700)
+++ print.c (working copy)
@@ -588,6 +588,79 @@ cputime(KINFO *k, VARENT *ve)
}
void
+systime(KINFO *k, VARENT *ve)
+{
+ VAR *v;
+ long secs;
+ long psecs; /* "parts" of a second. first micro, then centi */
+ char obuff[128];
+ static char decimal_point;
+
+ if (decimal_point == '\0')
+ decimal_point = localeconv()->decimal_point[0];
+ v = ve->var;
+ if (!k->ki_valid) {
+ secs = 0;
+ psecs = 0;
+ } else {
+ /*
+ * This counts time spent handling interrupts. We could
+ * fix this, but it is not 100% trivial (and interrupt
+ * time fractions only work on the sparc anyway). XXX
+ */
+ secs = k->ki_p->ki_rusage.ru_stime.tv_sec;
+ psecs = k->ki_p->ki_rusage.ru_stime.tv_usec;
+ if (sumrusage) {
+ secs += k->ki_p->ki_childstime.tv_sec;
+ psecs += k->ki_p->ki_childstime.tv_usec;
+ }
+ /*
+ * round and scale to 100's
+ */
+ psecs = (psecs + 5000) / 1;
+ secs += psecs / 100;
+ psecs = psecs % 100;
+ }
+ (void)snprintf(obuff, sizeof(obuff), "%3ld:%02ld%c%02ld",
+ secs / 60, secs % 60, decimal_point, psecs);
+ (void)printf("%*s", v->width, obuff);
+}
+
+void
+usertime(KINFO *k, VARENT *ve)
+{
+ VAR *v;
+ long secs;
+ long psecs; /* "parts" of a second. first micro, then centi */
+ char obuff[128];
+ static char decimal_point;
+
+ if (decimal_point == '\0')
+ decimal_point = localeconv()->decimal_point[0];
+ v = ve->var;
+ if (!k->ki_valid) {
+ secs = 0;
+ psecs = 0;
+ } else {
+ secs = k->ki_p->ki_rusage.ru_utime.tv_sec;
+ psecs = k->ki_p->ki_rusage.ru_utime.tv_usec;
+ if (sumrusage) {
+ secs += k->ki_p->ki_childutime.tv_sec;
+ psecs += k->ki_p->ki_childutime.tv_usec;
+ }
+ /*
+ * round and scale to 100's
+ */
+ psecs = (psecs + 5000) / 1;
+ secs += psecs / 100;
+ psecs = psecs % 100;
+ }
+ (void)snprintf(obuff, sizeof(obuff), "%3ld:%02ld%c%02ld",
+ secs / 60, secs % 60, decimal_point, psecs);
+ (void)printf("%*s", v->width, obuff);
+}
+
+void
elapsed(KINFO *k, VARENT *ve)
{
VAR *v;
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: usertime and systime
In the last episode (Mar 16), Kostik Belousov said: > On Wed, Mar 16, 2011 at 12:56:14PM -0500, Dan Nelson wrote: > > In the last episode (Mar 16), Thiago Damas said: > > > Hi, > > > without procfs, there is a way to get usertime and systime from a > > > running process? > > > > Try applying the attached patch to ps. I've had it for a while but > > never submitted a PR. > > > > Heh. I've had it for a very long time. > > http://lists.freebsd.org/pipermail/freebsd-hackers/2009-March/027918.html > > Yes, apparently, this is often requested feature. > > I dislike the copying of the existing code, sincere up to the comment that > is not quite relevant (about interrupts in systime). I slightly > reorganized the patch to reduce the copy/paste part of it. > > Do you have comments ? I like it. -- Dan Nelson dnel...@allantgroup.com ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Why user time of the process depends on machine load?
In the last episode (Jun 15), Yuri said: > When I test performance of the code, I always observe dependency of CPU > user time on the presence of other CPU intense processes. Same CPU-only > deterministic process that on the quiet machine completes in 220 user > seconds in the presence of, for example, kde rebuild would complete in > 261, 266 or even 379 user seconds. I am talking about times shown by > time(1), not actual an execution time. It's the same time as getrusage(2) > returns in ru_utime field. > > Why time that process takes in user seconds depends on what other > processes are running? > > FreeBSD-8.2 STABLE on i7 CPU @ 9200 @ 2.67GHz. Some possible factors: o Intel Turbo Boost, which raises the clock rate of a single core if the other cores are idle. A single process on an idle system will run faster. o i7 chips have a shared L3 cache across all cores, so a single process on an idle system will tend to have more of its data in cache compared to a system with multiple processes, so it spends less time waiting for slower physical memory lookups. o Process accounting isn't exact. I may be wrong, but I don't think timestamps are taken every time a syscall is invoked and returns. Some time marked as "user" may actually be "system" time, in which case you may be seeing the effect of contention in the kernel as more processes are run. You may be able to disable Turbo Boot in your BIOS, which you can use to determine how much of the single-process speedup is due to that. Unrelated but still interesting note on your particular CPU: http://www.passmark.com/forum/showthread.php?t=2256 -- Dan Nelson dnel...@allantgroup.com ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Default value for UIDs
In the last episode (Jun 28), Chris Rees said: > Hi all, > > [crees@zeus]~% tail -n 2 /usr/ports/UIDs > dbxml:*:949:949::0:0:dbXML user:/nonexistent:/sbin/nologin > nobody:*:65534:65534::0:0:Unprivileged user:/nonexistent:/usr/sbin/nologin > [crees@zeus]~% grep crees /etc/passwd > crees:*:1001:1001:Chris Rees:/home/crees:/bin/tcsh > chris:*:1001:1001:Chris Rees:/home/crees:/bin/tcsh > [crees@zeus]~% > > I'm a little concerned at how close the ports UIDs are getting to the > username space... There are only 216 entries in UIDs, though, so if people are just using "last entry + 1" when adding new ones, they should probably start filling the gaps instead. The 100s and 200s are pretty dense, but 350-399 only has 5 entries, 400-499 has 4, 600-699 has 7, 700-799 has 3, etc. -- Dan Nelson dnel...@allantgroup.com ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Freebsd-7.4 + std gcc 4.2.1 fails to honour -march=i586
In the last episode (Jul 17), Julian H. Stacey said: > Hi all, > ENVIRONMENT: > Standard > gcc version 4.2.1 20070719 [FreeBSD] > that comes with > FreeBSD 7.4-RELEASE > on my 686 host with > CFLAGS += -march=i586 > in > /etc/make.conf > used with > cd /usr/src/bin/who ; make clean ; make cleandir ; make clean ; make > reports > Warning: Object directory not changed from original /usr/src/usr.bin/who > cc -O2 -fno-strict-aliasing -march=i586 -c /usr/src/usr.bin/who/who.c > cc -O2 -fno-strict-aliasing -march=i586-o who who.o > looking with > file who > reports > who: ELF 32-bit LSB executable, Intel 80386, version 1 > (FreeBSD), for FreeBSD 7.4, dynamically linked (uses shared > libs), FreeBSD-style, not stripped > > Problem: > Use it from a 586 7.4-RELEASE host (AMD+NFS) > file /host/sony/usr/src/usr.bin/who/who > /host/sony/usr/src/usr.bin/who/who: ELF 32-bit LSB executable, > Intel 80386, version 1 (FreeBSD), for FreeBSD 7.4, dynamically > linked (uses shared libs), FreeBSD-style, not stripped > /host/sony/usr/src/usr.bin/who/who > fails with > Illegal instruction Were the crt*.o files on your fast machine also compiled with -march=i586 ? If you run gdb on the core file, can you determine what function the bad instruction is in? -- Dan Nelson dnel...@allantgroup.com ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Recommended amount of swap
In the last episode (Sep 05), Sean Hamilton said: > What is the state of the art for the recommended amount of swap in > FreeBSD? Both "normal" systems with 512 MB - 8 GB of RAM, and large > database systems with around 128 - 256 GB. I suggest 2x RAM for systems less than 4gb or so. Anything more than 4GB of swap is probably never going to be used, and if it is used, you're just going to thrash your swap device. If you have 128GB of RAM and need to swap to disk, you desperately need more RAM, not swap :) -- Dan Nelson dnel...@allantgroup.com ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Recommended amount of swap
In the last episode (Sep 05), Eitan Adler said: > On Mon, Sep 5, 2011 at 3:48 PM, Dan Nelson wrote: > > In the last episode (Sep 05), Sean Hamilton said: > >> What is the state of the art for the recommended amount of swap in > >> FreeBSD? Both "normal" systems with 512 MB - 8 GB of RAM, and large > >> database systems with around 128 - 256 GB. > > Keep in mind that you need at least ram = swap to get a coredump. Not if debug.minidump is set to 1, which it is by default. In that case, only mapped memory gets dumped, which should ignore disk cache pages and be a lot smaller than your RAM size. Enabling ZFS may make your dumps bigger unless the minidump code is smart enough to not dump the ARC. -- Dan Nelson dnel...@allantgroup.com ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Does anyone use nscd?
In the last episode (Oct 04), Trond Endrestol said: > On Tue, 4 Oct 2011 18:51+0200, Dag-Erling Smorgrav wrote: > > Trond Endrestol writes: > > > It's in daily use at Gjovik Technical College (Fagskolen i Gjovik), > > > here in Norway. Both the mail and web servers authenticates our users > > > by LDAP, and nscd certainly speeds up the lookups. > > > > OK. No trouble with clients dying of SIGPIPE? I could never reproduce > > the bug, but both users who reported problems used ldap, and I don't > > have an LDAP server to test against, so I thought it might be specific > > to LDAP. > > Not in my (somewhat limited) experience. On a tangent, I also heavily recommend using the nss-pam-ldapd port instead of nss_ldap. It includes a daemon called nslcd which is the only process that links to the ldap libary. The nss module is a tiny plug that talks to nslcd using a simple protocol. It really reduces the socket count to your ldap server, and removes the potential namespace problems caused by dlopening libldap.so in every process. -- Dan Nelson dnel...@allantgroup.com ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Measuring memory footprint in C/C++ code on FreeBSD
In the last episode (Oct 21), Razmig K said: > Le 21.10.2011 10:44, Peter Jeremy a écrit : > > On 2011-Oct-20 19:57:31 +0200, Razmig K wrote: > > It's not clear whether the program is attempting to determine it's own > > (or a child's) memory footprint, or that of an arbitrary process. In > > the former case, getrusage() is the obvious choice. This as a portable > > interface. > > The program has to determine its own memory footprint. It has no children. > > > If you want to examine arbitrary processes, the best interface on > > FreeBSD would be kvm_getprocs(3). > > > > BTW, since you mention heap objects, I presume you are aware that > > malloc() uses mmap(), rather than sbrk() to obtain memory. > > No I wasn't aware of that. > > In few words, the program needs to obtain and report information > reported by the SIZE column of top, since it is going to be run many > times, and it is impractical to watch top for this purpose. top also uses kvm_getprocs, so you can use that as a template (see the manpage and source at usr/src/usr.bin/top/machine.c). If you call it with the KERN_PROC_PID flag, you can get the stats for a single processs by pid. If you want even more detail, you can look at the source to the procstat command, which uses some other calls to dump the vm map of processes. -- Dan Nelson dnel...@allantgroup.com ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: _SC_GETPW_R_SIZE_MAX undefined in sysconf.c, what is correct value?
In the last episode (Oct 23), Christopher J. Ruwe said:
> I need to get the maximum size of an pwd-entry to determine the correct
> buffersize for calling getpwnam_r("uname",&pwd, buf, bufsize, &pwdp). I
> would like to use sysconf(_SC_GETPW_R_SIZE_MAX) to determine bufsize,
> which unfornutately fails (returns -1). Currently, I used 16384, which
> seems to be too much, bit works for the time being.
>
> From recent mails I get that _SC_GETPW_R_SIZE_MAX is not available on
> FreeBSD, e.g.,
> http://lists.freebsd.org/pipermail/freebsd-ports-bugs/2009-September/173081.html
> and http://www.redhat.com/archives/libvir-list/2011-May/msg01013.html.
> This assertion seems to be backed by /usr/srclib/libc/gen/sysconf.c, line
> 374.
>
> From Stevens & Rago, Adavanced Programming in the UNIX Environment, I can
> get that FreeBSD supports all possible members in the passwd structure,
> but where can I determine the maximum size of these so that I can
> calculate the ax size of the pwd struct therefrom? Does anybody know or
> know where to look what maximum size a pwd-entry can have?
>
> Knowing the maximum size a pwd structure can have, it seems like to be an
> idea to define the correct value in sysconf.c. As that is not done though
> suggested in the code, are there any obstacles in defining that value so
> nobody has done that so far?
>From looking at the libc/gen/getpwent.c file, it looks like a maximum size
might be 1MB. The wrapper functions that convert getpw*_r functions into
ones that simply return a pointer to malloced data all use the getpw()
helper function, which starts with a 1k buffer and keeps doubling its size
until the data fits or it hits PWD_STORAGE_MAX (1MB). PWD_STORAGE_MAX is
only checked within that getpw() function, though, so it's possible that an
nss library might return an even longer string to a get*_r call. It's up to
you to decide what your own limit is :)
--
Dan Nelson
dnel...@allantgroup.com
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: _SC_GETPW_R_SIZE_MAX undefined in sysconf.c, what is correct value?
In the last episode (Oct 24), Christopher J. Ruwe said:
> On Sun, 23 Oct 2011 19:10:34 -0500
> Dan Nelson wrote:
> > In the last episode (Oct 23), Christopher J. Ruwe said:
> > > I need to get the maximum size of an pwd-entry to determine the
> > > correct buffersize for calling getpwnam_r("uname",&pwd, buf, bufsize,
> > > &pwdp). I would like to use sysconf(_SC_GETPW_R_SIZE_MAX) to
> > > determine bufsize, which unfornutately fails (returns -1). Currently,
> > > I used 16384, which seems to be too much, bit works for the time
> > > being.
[..]
> > From looking at the libc/gen/getpwent.c file, it looks like a maximum
> > size might be 1MB. The wrapper functions that convert getpw*_r
> > functions into ones that simply return a pointer to malloced data all
> > use the getpw() helper function, which starts with a 1k buffer and keeps
> > doubling its size until the data fits or it hits PWD_STORAGE_MAX (1MB).
> > PWD_STORAGE_MAX is only checked within that getpw() function, though, so
> > it's possible that an nss library might return an even longer string to
> > a get*_r call. It's up to you to decide what your own limit is :)
>
> Uh ... it's just that I hoped I had not to decide ;-)
>
> However, 1M seems to be rather large to me. Let's see (pwd.h):
>
> 116 struct passwd {
> 117 char*pw_name; /* user name */
> 118 char*pw_passwd; /* encrypted password */
> 119 uid_t pw_uid; /* user uid */
> 120 gid_t pw_gid; /* user gid */
> 121 time_t pw_change; /* password change time */
> 122 char*pw_class; /* user access class */
> 123 char*pw_gecos; /* Honeywell login info */
> 124 char*pw_dir;/* home directory */
> 125 char*pw_shell; /* default shell */
> 126 time_t pw_expire; /* account expiration */
> 127 int pw_fields; /* internal: fields filled in */
> 128 };
>
> So pw_name -> MAXLOGNAME (from param.h) = 17. pw_passwd ->
> http://www.freebsd.org/doc/handbook/one-time-passwords.html = 129. pw_uid
> & pw_gid each sizeof(__uint32_t) ?= 32b. time_t -> sizeof(__int64_t) ?=
> 64b.
>
> At some point, I would just sum it up and reach some size which might be
> machine dependant, but should be somewhere (guessing/estimating now)
> between 4k and 16k. I am short on time just now, am I on the right track
> or am I missing something which should be obvious to someone with
> experience, but is not to me (lacking experience)?
The getpwnam_r function needs enough space to store the "struct passwd"
itself (which has a constant size) plus the strings pointed to by pw_name,
pw_class, pw_gecos, pw_dir, and pw_shell. If you have enough control over
your environment that you can guarantee that the sum of those strings won't
be larger than 4k, then you can just used a fixed buffer of that size. Even
1k is probably large enough for 99.999% of all systems. That's a really
long home directory or shell path :) On the other hand, the GECOS field is
theoretially free-form and could contain a lot of data. I've never see it
hold more than an office number myself, though.
--
Dan Nelson
dnel...@allantgroup.com
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: _SC_GETPW_R_SIZE_MAX undefined in sysconf.c, what is correct value?
In the last episode (Oct 25), Christopher J. Ruwe said:
> On Mon, 24 Oct 2011 15:42:10 -0500
> Dan Nelson wrote:
> > In the last episode (Oct 24), Christopher J. Ruwe said:
> > > On Sun, 23 Oct 2011 19:10:34 -0500
> > > Dan Nelson wrote:
> > > > In the last episode (Oct 23), Christopher J. Ruwe said:
> > > > > I need to get the maximum size of an pwd-entry to determine the
> > > > > correct buffersize for calling getpwnam_r("uname",&pwd, buf,
> > > > > bufsize, &pwdp). I would like to use
> > > > > sysconf(_SC_GETPW_R_SIZE_MAX) to determine bufsize, which
> > > > > unfornutately fails (returns -1). Currently, I used 16384,
> > > > > which seems to be too much, bit works for the time being.
> > [..]
> > > > From looking at the libc/gen/getpwent.c file, it looks like a
> > > > maximum size might be 1MB. The wrapper functions that convert
> > > > getpw*_r functions into ones that simply return a pointer to
> > > > malloced data all use the getpw() helper function, which starts with
> > > > a 1k buffer and keeps doubling its size until the data fits or it
> > > > hits PWD_STORAGE_MAX (1MB). PWD_STORAGE_MAX is only checked within
> > > > that getpw() function, though, so it's possible that an nss library
> > > > might return an even longer string to a get*_r call. It's up to you
> > > > to decide what your own limit is :)
> > >
> > > Uh ... it's just that I hoped I had not to decide ;-)
> >
> > The getpwnam_r function needs enough space to store the "struct passwd"
> > itself (which has a constant size) plus the strings pointed to by
> > pw_name, pw_class, pw_gecos, pw_dir, and pw_shell. If you have enough
> > control over your environment that you can guarantee that the sum of
> > those strings won't be larger than 4k, then you can just used a fixed
> > buffer of that size. Even 1k is probably large enough for 99.999% of
> > all systems. That's a really long home directory or shell path :) On
> > the other hand, the GECOS field is theoretially free-form and could
> > contain a lot of data. I've never see it hold more than an office
> > number myself, though.
> >
>
> Thanks for your help so far. Just assuming (I am not sufficiently clear
> about myself and my own intents) I want to be precise and am afraid of
> guessing: Can I assume that the gecos field is an entry in /etc/passwd and
> can therefore never exceed LINE_MAX, i.e., 2048B (limits.h, line 72)? Or,
> more precisely, ( 2048B - sum( lenght(all fields except passwd) ) )?
> Would that be an acceptable limit to set the getpwnam_r( ... ) buffer to
> and/or would that be an acceptable value to replace the following bit from
> sysconf.c?
>
> 372 #if _POSIX_THREAD_SAFE_FUNCTIONS > -1
> 373 case _SC_GETGR_R_SIZE_MAX:
> 374 case _SC_GETPW_R_SIZE_MAX:
> 375 #error "somebody needs to implement this"
> 376#endif
If your nsswitch.conf has "passwd: files" in it, then yes, you can assume
that the 2048-byte limit applies. However, if you are using nss_ldap,
nss_mysql, nss_winbind, or some other nsswitch module that provides user
info, that backend user system may be capable of returning longer strings.
If you want to be able to handle any struct passwd that might be thrown at
you, you should implement a "retry with doubling" loop similar to the one in
libc/gen/getpwent.c:getpw() .
--
Dan Nelson
dnel...@allantgroup.com
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: What is going on with ash / sh
In the last episode (Nov 02), Mark Saad said: > Hackers > What is going on here, if I run the following shell script, what is > the expected output . The script is named xxx > > #!/bin/sh > ps -ax | grep -v grep | grep xxx > > Here is what I see > > # sh xxx > 88318 p0 S+ 0:00.00 sh xxx > 88320 p0 R+ 0:00.00 sh xxx > 88321 p0 R+ 0:00.00 sh xxx > > Can someone explain this ? What does your script do? If it contains subshells or pipelines, the main process will fork child processes to handle those. -- Dan Nelson dnel...@allantgroup.com ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: What is going on with ash / sh
In the last episode (Nov 02), Dan Nelson said: > In the last episode (Nov 02), Mark Saad said: > > Hackers > > What is going on here, if I run the following shell script, what is > > the expected output . The script is named xxx > > > > #!/bin/sh > > ps -ax | grep -v grep | grep xxx > > > > Here is what I see > > > > # sh xxx > > 88318 p0 S+ 0:00.00 sh xxx > > 88320 p0 R+ 0:00.00 sh xxx > > 88321 p0 R+ 0:00.00 sh xxx > > > > Can someone explain this ? > > What does your script do? If it contains subshells or pipelines, the main > process will fork child processes to handle those. Sorry; I misread your original post. Yes, that script forks off two subshells to handle the pipeline, and the ps command caught the state where the subshells had been created but had not yet exec'ed their grep commands. -- Dan Nelson dnel...@allantgroup.com ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: [clang] Build error on r232474 (and a few before, don't know exactly which)
In the last episode (Mar 03), Brandon Falk said:
> I'm trying to build r232474 with clang (build environment is 10.0-CURRENT
> r231589 amd64 with clang), and I fail on `make -j16 buildworld`. I've
> tried with and without threads. I've built so many builds of clang that I
> can't even count, so I'm confident my environment is set up properly. I'm
> building under a virtual machine, although I've never had issues with that
> before.
You didn't actually paste an error at all below, but the fact that the
top-level make reported an error from one of the sub-makes. You'll need to
either capture the entire build log and scroll through it from the bottom up
to find the error message, or build without -j16 so the error is at the
bottom of the output.
> error
>
> ===> gnu/usr.bin/texinfo/doc (all)
> makeinfo --no-split -I /root/src/gnu/usr.bin/texinfo/doc -I
> /root/src/gnu/usr.bin/texinfo/doc/../../../../contrib/texinfo/doc
> /root/src/gnu/usr.bin/texinfo/doc/../../../../contrib/texinfo/doc/info.texi
> -o info.info
> makeinfo --no-split -I /root/src/gnu/usr.bin/texinfo/doc -I
> /root/src/gnu/usr.bin/texinfo/doc/../../../../contrib/texinfo/doc
> /root/src/gnu/usr.bin/texinfo/doc/../../../../contrib/texinfo/doc/info-stnd.texi
> -o info-stnd.info
> ln -fs
> /root/src/gnu/usr.bin/texinfo/doc/../../../../contrib/texinfo/doc/texinfo.txi
> texinfo.texi
> makeinfo --no-split -I /root/src/gnu/usr.bin/texinfo/doc -I
> /root/src/gnu/usr.bin/texinfo/doc/../../../../contrib/texinfo/doc
> texinfo.texi -o texinfo.info
> gzip -cn info.info> info.info.gz
> gzip -cn info-stnd.info> info-stnd.info.gz
> gzip -cn texinfo.info> texinfo.info.gz
> 1 error
> *** [everything] Error code 2
> 1 error
> *** [buildworld] Error code 2
> 1 error
>
> END error
>
> Make.conf
>
> .if !defined(CC) || ${CC} == "cc"
> CC=clang
> .endif
> .if !defined(CXX) || ${CXX} == "c++"
> CXX=clang++
> .endif
> .if !defined(CPP) || ${CPP} == "cpp"
> CPP=clang-cpp
> .endif
>
> NO_WERROR=
> WERROR=
> NO_FSCHG=
>
> # added by use.perl 2012-03-03 16:12:59
> PERL_VERSION=5.12.4
>
> END Make.conf
>
> -Brandon
>
>
> ___
> freebsd-hackers@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
> To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
--
Dan Nelson
dnel...@allantgroup.com
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Regarding core disable in FreeBSD 9
In the last episode (Apr 13), Florian Smeets said: > On 13.04.12 19:34, Alexander Best wrote: > > On Sat Apr 14 12, Mahesh Babu wrote: > >> How to disable a particular core in FreeBSD 9? How to enable it again? > > > > i don't think it's possible to do that in freebsd. what you can do is to > > disable SMP oder hyperthreading. alternatively you can assign a certain > > process to a certain core. > > > > i think there's a project to disable and enable specific cores on the > > fly. freebsd is pretty far behind regarding this feature. beos was > > able to do this anno 1998 or so afair. > > You can set the following in /boot/loader.conf > > hint.lapic.128.disabled=1 > hint.lapic.130.disabled=1 > > Where 128 and 130 are IDs of cores you want to disable, you can find the > IDs for your CPUs/cores in > > dmesg or /var/log/messages e.g. > > cpu0 (BSP): APIC ID: 0 > cpu1 (AP): APIC ID: 2 > cpu2 (AP): APIC ID: 4 > cpu3 (AP): APIC ID: 16 > > Enabling and disabling on the fly is not possible. You can't completely disable a core, but you can tell the scheduler not to use it, via the cpuset command. For example, "cpuset -s 1 -l 0,1" will change the mask for cpuset 1 (the default set) to only allow cpus 0 and 1. -- Dan Nelson dnel...@allantgroup.com ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Threaded 6.4 code compiled under 9.0 uses a lot more memory?..
In the last episode (Oct 31), Karl Pielorz said: > --On 31 October 2012 16:06 +0200 Konstantin Belousov > wrote: > > Since you neglected to provide the verbatim output of procstat, nothing > > conclusive can be said. Obviously, you can make an investigation on > > your own. > > Sorry - when I ran it this morning the output was several hundred lines - > I didn't want to post all of that to the list 99% of the lines are very > similar. I can email it you off-list if having the whole lot will help? > > >> Then there's a bunch of 'large' blocks e.g.. > >> > >> PID STARTEND PRT RES PRES REF SHD FL TP > >> PATH 20100x801c00x80280 rw- 28690 4 0 > >> df 20100x802800x80340 rw- 18800 1 0 > > > > Most likely, these are malloc arenas. > > Ok, that's the heaviest usage. > > >> Then lots of 'little' blocks, > >> > >> 2010 0x70161000 0x70181000 rw- 160 1 0 ---D df > > > > And those are thread stacks. > > Ok, lots of those (lots of threads going on) - but they're all pretty > small. > > My code only has a single call to malloc, which allocates around 20k per > thread. > > Obviously there's other libraries and stuff running with the code - so > would I be correct in guessing that they are more than likely for most of > these large blocks? Note that libmilter may do a lot of mallocs on its own, especially if you are examining the message body. There are also jemalloc tuning options that may lower total meory usage if you are using a lot of threads. I'd take a look at the G and R flags first. -- Dan Nelson dnel...@allantgroup.com ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: lib for working with graphs
In the last episode (Nov 28), Andriy Gapon said: > on 28/11/2012 16:31 David Wolfskill said the following: > > On Wed, Nov 28, 2012 at 04:20:28PM +0200, Andriy Gapon wrote: > >> > >> Does anyone know a light-weight BSD-licensed (or analogous) library / > >> piece of code for doing useful things with graphs? Thank you. > >> > > > > Errr "graphs" is fairly ambiguous, and "things with graphs" covers a > > very wide range of activities. > > Graphs as in vertices, edges, etc :) And things like graph basics: BFS, > DFS, connected components, topological sort, etc Graphviz would be the most popular package for stuff like this, I think, and it includes a C API. It's licensed under the Eclipse Public License. http://www.graphviz.org/ http://www.graphviz.org/Gallery.php http://www.graphviz.org/doc/libguide/libguide.pdf -- Dan Nelson dnel...@allantgroup.com ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: /bin/sh => STDIN & functions, var scope messing
In the last episode (May 31), Reid Linnemann said: > On Fri, May 31, 2013 at 1:12 PM, Teske, Devin > wrote: > > If you're arguing we have to change sh's behavior to be more compliant, > > jilles already quoted XCU 2.12 (our shell is well within its right to > > run any/all lvalue/rvalue operands of a pipe in a sub-shell without > > contradicting the guidelines). > > > > But if you're arguing that it has to change to make things better or > > easier... I don't know about that. Might just make people lulled into > > using a style that's non-portable. I'd like to keep things the way they > > are so that if you program for FreeBSD, you're inherently going to > > program in a fashion that is more portable. > > FWIW bash (invoked as sh) on RHEL-based linux systems exhibits the same > behavior- > > sh-3.2$ var=1 > sh-3.2$ yes|var=2 > sh-3.2$ echo $var > 1 > sh-3.2$ > > If my opinion matters at all, I would agree that for the sake of > portability that behavior be consistent with the majority of sh > implementations rather than "right" according to arbitrary ruling. On the other hand, zsh runs the last component of a pipeline in the parent shell. The usual model is "do work in pipeline, process results in final component", and being able to simply set variables there that can be used in the rest of the script is very elegant. -- Dan Nelson dnel...@allantgroup.com ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: NFS based /usr prevents normal startup due to slow net init
In the last episode (Mar 02), Steven Hartland said: > Another observation from my recent dealings with using > NFS based /usr is that the remote critical mounts via > nfs dont always give the network enough time to > initialise before running. The first error displayed > is: > Mounting NFS file systems:mount_nfs: nfs1: hostname nor servname provided, > or not known > > This is particularly noticeable when the machine is > connected to Cisco equipment as they take quite a > while link to the connected host after initialisation. That can usually be fixed by making sure you have portfast enabled on the Cisco for any non-switch ports, btw. Takes the port setup time down from 30 seconds to under 5. -- Dan Nelson [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Problem with test(1)
In the last episode (Apr 05), Joe Marcus Clarke said: > I noticed something weird with test(1) when I ran across a problem port > Makefile. Our test(1) doesn't properly check to make sure there is an > operand argument to unary operators like -f. For example: > > test -f > > Will print "TRUE" on FreeBSD. On Solaris, it will die: > > /usr/bin/test[8]: test: argument expected > > I think this patch is correct in that it does fix the problem, and the > TEST.sh and TEST.csh regression scripts report the same results pre and > post-patch. Comments? If you follow POSIX's description of test, FreeBSD's current behaviour is valid and Solaris isn't: http://www.opengroup.org/onlinepubs/009695399/utilities/test.html The algorithm for determining the precedence of the operators and the return value that shall be generated is based on the number of arguments presented to test. (However, when using the "[...]" form, the right-bracket final argument shall not be counted in this algorithm.) In the following list, $1, $2, $3, and $4 represent the arguments presented to test: 0 arguments: Exit false (1). 1 argument: Exit true (0) if $1 is not null; otherwise, exit false. ... Unary operators shouldn't get parsed as such unless there are two arguments. > http://www.marcuscom.com/downloads/test.c.diff -- Dan Nelson [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Discovering list of open files from "kernel level" without using utils like lsof
In the last episode (Apr 08), Garrett Cooper said: > I'm trying to see whether it's possible to grab the list of > files open from a kernel level on FreeBSD, using a userland library > interface as opposed to lsof. > I'm trying to see if there's a simple tool that I could code in > C/C++ if necessary to spin down disks automatically to save power and > disk life. Plus, I think that lsof actually would probe the devices > and 'wake them up' instead of keeping them as-is. However, I could be > wrong so if I am please let me know. Take a look at how /usr/bin/fstat does it. There is apparently a "kern.file" sysctl that holds the open file table, but fstat digs through kernel memory. -- Dan Nelson [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: More questions about BDB
In the last episode (May 20), Sean Bryant said:
> Just a personal curiosity. Is there a particular reason why FreeBSD
> is holding on to BDB 1.85?
All later versions have a non-BSD license (a source redistribution
requirement was added), which means it can't go in the base system.
BDB is built into libc and is used for the hashed passwd & termcap
databases.
--
Dan Nelson
[EMAIL PROTECTED]
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: nss_ldap without nscd or cached ?
In the last episode (May 24), Mohacsi Janos said: > I think there is a some architectural issues with the current > implementation of nsswitch or nsdispatch(3). Let's assume you want > to authenticate against an LDAP database. You will install nss_ldap > from port. You configure nss_ldap.conf with binddn and its bindpw. > Here comes the problem: > > 1. If permission of nss_ldap.conf is 0400 since it contains the > clear text password of the binddn, then an ordinary user cannot bind > to the database and cannot get UID->name information from LDAP > database. See output: > > [EMAIL PROTECTED]> ls -l /home > total 6 > drwxr-xr-x 3 9027 wheel 512 May 23 17:57 user1 > drwxrwxr-x 3 root 9030 512 May 23 15:14 documents > drwxr-xr-x 2 9013 9013 512 May 23 15:13 user2 > You should be able to grant the anonymous user read access to user/group names and group membership attributes. That way you can do simple things like name->uid lookups without having to bind at all. -- Dan Nelson [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Using userland library in Kernel
In the last episode (Aug 08), Biks N said: > I am new to FreeBSD kernel programming and I am trying to use > userland library (zlib) in FreeBSD kernel. But I am not sure if zlib > library is linkable from the kernel. It isn't. However, there is a zlib implementation in the kernel already. It's hidden under the PPP_DEFLATE kernel option (the source is in sys/net/ppp_deflate.c). The functions are all prefixed with z_, but apart from that it works the same as userland zlib. -- Dan Nelson [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: memory pool, rfc
In the last episode (Nov 01), Eduardo Morras said: > I have some free time and want to do an memory pool. The idea is to > have a memory zone of N KB (or several MB) compressed in memory. I > have fast compression algorithms now that can release under BSD > licence that are faster than hd i/o, so it take less > compress/decompress a memory zone than read/write it to disk. I don't > know if it already exist for FreeBSD, so if it's already done i'll > try to improve it. [...] > For what can be used? > > - Memory pools in applications (like malloc) > - Ram disks > - Disk Cache (permit bigger disk cache) > - 'On the fly' filesystem compression (and it takes less read/write > compressed data than non-compressed) zfs already has modular compression algorithms; it would be rather easy to add a mozule for your method and compare it to the existing gzip and lzjb algorithms. -- Dan Nelson [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Some FreeBSD performance Issues
In the last episode (Nov 08), Randall Hyde said: > It appears that character-at-a-time file I/O is *exceptionally* slow. > Yes, I realize that when processing large files I really ought to be > doing block/buffered I/O to get the best performance, but for certain > library routines I've written it's been far more convenient to do > character-at-a-time I/O rather than deal with all the buffering > issues. In the past, while slower, this character-at-a-time paradigm > has provided reasonable, though not stellar, performance under > Windows and Linux. However, with the port to FreeBSD I'm seeing a > three-orders-of-magnitude performance loss. Here's my little test > program: [...] > The "fileio.open" call is basically a bsd.open( "socket.h", bsd.O_RDONLY ); > API call. The socket.h file is about 19K long (it's from the FreeBSD > include file set). In particular, I would draw your attention to the first > two tests that do character-at-a-time I/O. The difference in performance What timings does "dd if=/usr/include/sys/socket.h of=/dev/null ibs=1 obs=64k" report? It takes about .4 sec on my non-idle dual pIII-900 system. Try truss'ing your program as it runs; maybe the program is doing some extra syscalls you aren't aware of? -- Dan Nelson [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: select
In the last episode (Jan 03), Metin KAYA said: > Hi all, > > How select(2) will behave if I give the "utimeout" parameter as > NULL? >From the man page: If timeout is a null pointer, the select blocks indefinitely. http://www.freebsd.org/cgi/man.cgi?query=select -- Dan Nelson [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: A (perhaps silly) kqueue question
In the last episode (Mar 14), Vlad GALU said:
> On 3/14/08, Vlad GALU <[EMAIL PROTECTED]> wrote:
> > On 3/8/08, Vlad GALU <[EMAIL PROTECTED]> wrote:
> > > On 3/8/08, Robert Watson <[EMAIL PROTECTED]> wrote:
> > > > On Fri, 7 Mar 2008, Vlad GALU wrote:
> > > > > I see an unusual symptom with one of our in-house
> > > > > applications. The main I/O loop calls kevent(), which in turn
> > > > > returns two events with EV_EOF error set, always for the same
> > > > > descriptors (they're both socket descriptors). As the man
> > > > > page is not pretty clear about it and I don't have my UNP
> > > > > copy at hand, I would like to ask the list whether the error
> > > > > events are supposed to be one-shot or not.
> > > >
> > > > I wonder if it's returning one event for the read socket
> > > > buffer, and one event for the write socket buffer, since there
> > > > are really two event sources for each socket? Not that this
> > > > is desirable behavior, but it might explain it. If you
> > > > shutdown() only read, do you get back one EOF kevent and one
> > > > writable kevent?
> > >
> > > I'll try that and see. The only issue being the low frequency
> > > this symptom appears at. I'll get back to the list once I have
> > > more info.
> >
> > Haven't gotten to the point of testing shutdown() behavior, but
> > here's a truss excerpt of the symptom:
> >
> > -- cut here --
> > kevent(3,0x0,0,{0x7,EVFILT_WRITE,EV_EOF,54,0x832c,0x800d08080
> > 0x7,EVFILT_READ,EV_EOF,54,0x2a,0x800d08080},1024,0x0) = 2 (0x2)
> > kevent(3,0x0,0,{0x7,EVFILT_WRITE,EV_EOF,54,0x832c,0x800d08080
> > 0x7,EVFILT_READ,EV_EOF,54,0x2a,0x800d08080},1024,0x0) = 2 (0x2)
> > kevent(3,0x0,0,{0x7,EVFILT_WRITE,EV_EOF,54,0x832c,0x800d08080
> > 0x7,EVFILT_READ,EV_EOF,54,0x2a,0x800d08080},1024,0x0) = 2 (0x2)
> > -- and here --
> >
> > So two EOF are returrned for descriptor 7, and errno would be
> > ECONNRESET. The question is now, why isn't it oneshot?
>
> Ah one more thing. When EOF is caught, a handler which forcibly
> removes the event is called, but it keeps poping up again and again.
Are you sure the event is being removed? I used to have a hack that
made the kernel return its current eventlist for a kqueue when you
called kevent() with nchanges set to -1 (handy for placing in a program
and using truss to print the result), but it has rotted. I'm attaching
it in case anyone wants to make it work again.
Since you got EOF status for both the read and write halves of the
socket, why not just close the fd? From my reading of the manpages,
unless you specified EV_ONESHOT when you added the event, events will
fire until you remove them or the condition that triggers them stops.
--
Dan Nelson
[EMAIL PROTECTED]
Index: kern_event.c
===
RCS file: /home/ncvs/src/sys/kern/kern_event.c,v
retrieving revision 1.113
diff -u -r1.113 kern_event.c
--- kern_event.c 14 Jul 2007 21:23:30 - 1.113
+++ kern_event.c 17 Jul 2007 18:10:47 -
@@ -659,6 +659,41 @@
nerrors = 0;
+#if 0 /* 1.92 broke this */
+ if (nchanges == -1) {
+ /* dump our eventlist into k_ops->arg */
+ int i;
+ int count = 0;
+ struct knote *kn;
+ error = 0;
+ KQ_LOCK(kq);
+
+ /* Walk our filedescriptor lists */
+ for (i = 0; i < kq->kq_knlistsize && count < nevents; i++) {
+ SLIST_FOREACH(kn, &kq->kq_knlist[i], kn_link) {
+copyout(&kn->kn_kevent, &(struct kevent)k_ops->arg[count], sizeof(kn->kn_kevent));
+count++;
+if (count >= nevents)
+ break;
+ }
+ }
+
+ /* Walk our hash tables */
+ if (kq->kq_knhashmask != 0) {
+ for (i = 0; i <= kq->kq_knhashmask && count < nevents; i++) {
+SLIST_FOREACH(kn, &kq->kq_knhash[i], kn_link) {
+ copyout(&kn->kn_kevent, &(struct kevent)k_ops->arg[count], sizeof(kn->kn_kevent));
+ count++;
+ if (count >= nevents)
+ break;
+}
+ }
+ }
+ KQ_UNLOCK(kq);
+ td->td_retval[0] = count;
+ goto done;
+ }
+#endif
while (nchanges > 0) {
n = nchanges > KQ_NEVENTS ? KQ_NEVENTS : nchanges;
error = k_ops->k_copyin(k_ops->arg, keva, n);
@@ -961,10 +996,12 @@
if ((kev->flags & EV_DISABLE) &&
((kn->kn_status & KN_DISABLED) == 0)) {
kn->kn_status |= KN_DISABLED;
+ kn->kn_kevent.flags |= EV_DISABLE;
}
if ((kev->flags & EV_ENABLE) && (kn->kn_status & KN_DISABLED)) {
kn->kn_status &= ~KN_DISABLED;
+ kn->kn_kevent.flags &= ~EV_DISABLE;
if ((kn->kn_status & KN_ACTIVE) &&
((kn->kn_status & KN_QUEUED) == 0))
knote_enqueue(kn);
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: OpenBSD sdiff Question
In the last episode (Mar 17), Steven Kreuzer said: > On Sat, Mar 15, 2008 at 11:29:19PM -0700, David O'Brien wrote: > > On Sat, Mar 15, 2008 at 03:21:01PM -0700, Bert JW Regeer wrote: > > > Even if BSD has no tradition to keep a separate program version, it is > > > still very handy to be able to give this data to other developers if > > > something is failing. > > > > $ ident failing-binary is the output that means something. A version > > string will not. > > > > > > > Programs that don't have a -v or --version switch are frustrating to > > > > Anyone used to working on BSD will not expect a -v switch. It > > isn't part of BSD tradition. The simple fact there is no obivous > > "version" to print just shows that in a OS that is developed and > > built as a whole, having a version on the util is meaningless. > > > > > Dropping -v would be a bad thing, and make the tools not > > > compatible, thus breaking many scripts that do expect a -v. > > > > Come on, how many scripts do you write that do "sdiff -v" today? > > I have to agree with this. > > I will submit the port without -v/--version > and worse comes to worse, add it in later if enough people complain. On the other hand, some programs that are contributed sources or are developed outside the FreeBSD cvs tree do have versions of their own. awk and tar, for example both recognize the --version flag (but not -v since that is already used). -- Dan Nelson [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: binary file within a shell script
In the last episode (May 08), Mathieu Prevot said: > Hi there, > > I would like to use one exec file from a shellscript but I would like > it to be incorporated in the same file, like Nvidia do for its FreeBSD > drivers. How can I do this in a convenient way ? Take a look at the file generated by /usr/bin/gzexe; that's one way to do it (basically, determine the number of lines in your shell script, append your binary file to the end of the script, and use tail to extract only the binary file to a tempfile). -- Dan Nelson [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Overcommit and calloc()
In the last episode (Jul 19), Dag-Erling Smorgrav said: > "Kelly Yancey" writes: > > Ahh...but wouldn't the bzero() touch all of the memory just > > allocated functionally making it non-overcommit? > > No. If it were an "non-overcomitting malloc", it would return NULL > and set errno to ENOMEM, instead of dumping core. It should be possible to modify calloc to trap signals, then bzero. If bzero faults, you free the memory and return NULL. No, wait. You can't trap SIGKILL. How about this. mmap() some anonymous memory MAP_SHARED, fork a child to bzero it. If the child dies, unmmap and return NULL. If the child succeeds, use the memory. This memory won't be freeable with malloc(), though. -Dan Nelson dnel...@emsphone.com To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message
Re: file(1) Magdir candidate: wintendo
In the last episode (Jul 26), Sheldon Hearn said: > On Mon, 26 Jul 1999 23:30:08 +0200, Mark Murray wrote: > > You could start with "WinEXE". > > Save game file formats, not game executables. You think WinEXE tops > my pcgames suggestion? What about multi-OS games, like Quake? How about just calling it "games" ? -- Dan Nelson dnel...@emsphone.com To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message
Re: bftp(1)
In the last episode (Aug 17), Alexey M. Zelkin said: > Man page for telnetd(8) references to bftp(1) which will be installed > into /usr/ucb/bftp and of course does not exists. Can someone > describe in few words is this staff still supported ? If so, there > bftp(1) staff can be received/downloaded ? Otherwise I think it's > good idea to remove this staff from manpage and probably from source > code. I did an Altavista search for bftp and it looks like it's a background FTP client. Original source is at http://astro.temple.edu/~yxue , and a '97 paper re-implementing it is at http://renoir.vill.edu/~yhang . It looks like ports/ftp/ncftp3 has all the features of bftp (scheduled background transfers, auto-resume, multiple file xfers) except it doens't email the user then the transfer is done. -- Dan Nelson dnel...@emsphone.com To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message
Re: font edit tools
In the last episode (Aug 22), Sergey Babkin said: > Alexey M. Zelkin wrote: > > > > hi, > > > > Which tools can be used to edit syscons fonts ? > > Any of the tools you use to edit the DOS fonts. > My favorite one it Evafont by Pete Kvitek. But > there were a lot of tools floating around. I like one called Font Mania!; the author doesn't seem to have a web presence, but an URL is http://jconroy.dragonfire.net/zzt/utilities/Fm.zip -- Dan Nelson dnel...@emsphone.com To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message
Re: X mailers (was Re: ANNOUNCE: Linux ABI/SDK standards for OpenGL/Mesa)
In the last episode (Sep 10), Andrew Reilly said: > On Thu, Sep 09, 1999 at 01:21:09PM +0200, Markus Stumpf wrote: > > On Thu, Sep 09, 1999 at 12:08:01PM +1000, Andrew Reilly wrote: > > > really easy, with a shell script that's just a case $SENDER > > > > It's even "easier" :-) > > I subscribe new mailing lists (and resubscribed old ones) as > > maex-listn...@space.net > > Well, that's arguably the way qmail wants it to be, but not helpful > if you want your mailing list traffic to come through the one > ISP-provided pop account. (Costs less that way, with my current > ISP.) If your ISP runs sendmail (possibly other MTAs), you can use the user+det...@host.com syntax. All mail is sent to the "user" mailbox, but filters like procmail see the "detail" portion too, and can filter on it. -- Dan Nelson dnel...@emsphone.com To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message
Re: Bug in dd seeking beyond 2G
In the last episode (Sep 15), Bakul Shah said: > PR bin/6509 (submitted in May 1998) already has a patch to fix this > but it was rejected because off_t was assumed by the bug > fixer/submitter to be a quat (int64_t). I can't even get an IDE disk > below 2G byte easily! And we are still years away from zettabyte > disks. So I don't see the point of blocking a _useful_ change that > *considerably* improves the situation just because it is not done the > `right way'. RCS file: /home/ncvs/src/bin/dd/dd.c,v revision 1.17 date: 1999/06/19 19:49:32; author: green; state: Exp; lines: +25 -21 Miscellaneous dd(1) changes: mainly fixing variable types (size_t, ssize_t, off_t, int, u_int64_t, etc.). dd(1) should now work properly with REALLY big amounts of data. Should be a -stable candidate by now (3 months of testing?) -- Dan Nelson dnel...@emsphone.com To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message
Re: How to get a thread ID?
In the last episode (Jun 03), Václav Haisman said: > is it possible to obtain some sort of a thread ID that identifies a thread > within a process other than pthread_self()? Something like gettid() on > Linux? Apparently, on FreeBSD the pthread_t is a pointer type and does > not identify the thread well enough. GDB on FreeBSD seems to know about > threads and does not seem to use the same ID as is pthread_t. The return value of pthread_self() is a pointer to the (private) "struct pthread" for the current thread, and should uniquely identify a thread. Do you have a testcase that shows otherwise? GDB might just enumerate the currently active threads starting from 1. -- Dan Nelson dnel...@allantgroup.com ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: sysctl way too slow
In the last episode (Jul 14), Joerg Sonnenberger said: > On Wed, Jul 14, 2010 at 11:49:07PM +1200, Atom Smasher wrote: > > the same info is available on linux via /sys and /proc and on comparable > > hardware, i can get the info about 100x faster. > > Are you sure that Linux is not just caching the data? I know of at least > one system where it takes more than 100ms to query the battery state due > to extremely slow hardware, I wouldn't be surprised if you can do worse. I have an old Dell laptop where it takes almost a full second to fetch battery data via ACPI. -- Dan Nelson dnel...@allantgroup.com ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: adding sysctls
In the last episode (Jun 18), Zane C.B. said:
> Any one know of any recent documentation for adding a sysctl to a
> kernel module for FreeBSD 6 and 7?
man 9 sysctl
--
Dan Nelson
[EMAIL PROTECTED]
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Securelevels
In the last episode (Jun 29), Ivaylo Mateev said:
> I think I found a bug.
>
> [EMAIL PROTECTED] /usr/home/strato]$ sudo sysctl kern.securelevel
> kern.securelevel: 2
> [EMAIL PROTECTED] /usr/home/strato]$ kgdb
> kgdb: /dev/mem: Permission denied
> [EMAIL PROTECTED] /usr/home/strato]$ sudo kgdb
> [GDB will not be able to debug user-mode threads: /usr/lib/libthread_db.so:
> Undefined symbol "ps_pglobal_lookup"]
> GNU gdb 6.1.1 [FreeBSD]
>
> I am running in securelevel 2. That means nithing can have direct access
> to /dev/mem, acording to man security:
>
> 1 Secure mode - the system immutable and system append-only flags may
> not be turned off; disks for mounted file systems, /dev/mem and
> /dev/kmem may not be opened for writing; /dev/io (if your platform
> has it) may not be opened at all; kernel modules (see kld(4)) may
> not be loaded or unloaded.
>
> 2 Highly secure mode - same as secure mode, plus disks may not be
> opened for writing (except by mount(2)) whether mounted or not.
> This level precludes tampering with file systems by unmounting
> them, but also inhibits running newfs(8) while the system is multi-
> user.
# truss kgdb < /dev/null |& grep /dev/mem
open("/dev/mem",O_RDONLY,00) = 4 (0x4)
#
Read-only opens of /dev/mem are allowed. "kgdb -w" should fail,
however.
--
Dan Nelson
[EMAIL PROTECTED]
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: IPFW uid logging...
In the last episode (Sep 08), Dan Mahoney, System Admin said: > I have the following rule set up in ipfw to limit the exposure of bad > php scripts and trojans that try to send mail directly. > > allow tcp from any to any dst-port 25 uid root > deny log tcp from any to any dst-port 25 out > > However, the log messages I get look like this: > > Sep 8 13:21:11 prime kernel: ipfw: 610 Deny TCP > 72.9.101.130:58117 209.85.133.114:25 out via em0 > Sep 8 13:21:16 prime kernel: ipfw: 610 Deny TCP > 72.9.101.130:56672 202.12.31.144:25 out via em0 > > Which is to say, they don't include the UID -- and I have several hundred > sites, each with its own UID. > > Yes, I could go ahead and set up a thousand "deny" rules, one for > each UID -- but being able to log this info (since it IS being > checked) would be great. It should be possible to add a couple more arguments to ipfw_log() so that ipfw_chk() can pass it the ugid_lookup flag and a pointer to the fw_ugid_cache struct. Then you can edit ipfw_log to print the contents of that struct if ugid_lookup==1. That would result in the logging of uid for any failed packet that had to go through a uid check on the way to the deny rule. -- Dan Nelson [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: IPFW uid logging...
In the last episode (Sep 09), Daan Vreeken said: > On Monday 08 September 2008 22:03:29 Dan Mahoney, System Admin wrote: > > On Mon, 8 Sep 2008, Dan Nelson wrote: > > > In the last episode (Sep 08), Dan Mahoney, System Admin said: > > >> I have the following rule set up in ipfw to limit the exposure > > >> of bad php scripts and trojans that try to send mail directly. > > >> > > >> allow tcp from any to any dst-port 25 uid root > > >> deny log tcp from any to any dst-port 25 out > > >> > > >> However, the log messages I get look like this: > > >> > > >> Sep 8 13:21:11 prime kernel: ipfw: 610 Deny TCP > > >> 72.9.101.130:58117 209.85.133.114:25 out via em0 > > >> Sep 8 13:21:16 prime kernel: ipfw: 610 Deny TCP > > >> 72.9.101.130:56672 202.12.31.144:25 out via em0 > > >> > > >> Which is to say, they don't include the UID -- and I have > > >> several hundred sites, each with its own UID. > > >> > > >> Yes, I could go ahead and set up a thousand "deny" rules, one > > >> for each UID -- but being able to log this info (since it IS > > >> being checked) would be great. > > > > > > It should be possible to add a couple more arguments to > > > ipfw_log() so that ipfw_chk() can pass it the ugid_lookup flag > > > and a pointer to the fw_ugid_cache struct. Then you can edit > > > ipfw_log to print the contents of that struct if ugid_lookup==1. > > > That would result in the logging of uid for any failed packet > > > that had to go through a uid check on the way to the deny rule. > > > > Okay, so if it's fairly easy to do, the question would be "since I > > don't feel right hacking in this change myself -- how could I > > propose this as a feature?" It's not a BUG per-se, but I think it > > could be useful to others as well. > > Hi Dan, Dan and the list, > > I own a webhosting company and here too every domain gets it's own > user, so I like this proposal. I've hacked together a first try, > which seems to be working. A patch (against -HEAD) can be found here: > > http://vehosting.nl/pub_diffs/ip_fw2.c_uid_2008_09_09.diff > > Improvements / tips / comments are welcome ;-) I like it. Maybe print gid as well, since there's an ipfw rule for that too. -- Dan Nelson [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Error: Can't find libjava.so
In the last episode (Sep 14), Marcel Grandemange said: > I do realize this is probably better suited for freebsd-questions , > however haven't received any response and was simply hoping someone > would be kind enough. > > I recently obtained a very decent ups, however it is not supported by > NUT. > > It does however come with winpower software that does run on FreeBSD. > > However it rewuired java. > > So installed from ports > > And was presented with following error: > > Error: can't find libjava.so > > This is on system in folder "/usr/local/Diablo-jre1.6.0/lib/amd64/libjava.so Are you running an amd64 winpower binary? If not, you'll probably need to install an x86 java. You can't mix libraries for different architectures. -- Dan Nelson [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: FreeBSD and ISCSI, Strange Problem
In the last episode (Oct 17), Daniel Dias Gonçalves said: > Thanks Danny, > > The patch work fine, but i have new problem: > > Have two servers 7.0R with iscsi-2.1. > They are mounted the same directory way iSCSI. When I create an archive > inside of this directory in the server A, the server B don't show the > archive, if in the server B to unmount and mount the directory again, > the archive created in the server A appears It. > > Where it is the problem? You can't mount the same UFS filesystem on two servers at once. You'll end up with a horribly-corrupted filesystem, since neither one is aware of changes the other makes. You would need a shared-storage cluster filessytem to be able to do that (or mount the volume read-only on both servers). Mount the filesystem on one server only, then access it via NFS from the other. -- Dan Nelson [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Pipes, cat buffer size
In the last episode (Oct 18), Ivan Voras said: > I'm working on a program that's intended to be used as a "filter", as > in "something | myprogram > file". I'm trying it with cat and I'm > seeing my read()s return small blocks, 64 kB in size. I suppose this > is because cat writes in 64 kB blocks. So: > > a) Is there a way to programatically, per-process, set the pipe buffer > size? The program in question is a compressor and it's particularly > inefficient when given small blocks and I'm wondering if the system can > buffer enough data for it. Why not keep reading until you reach your desired compression block size? Bzip2's default blocksize is 900k, for example. > b) Is there any objection to the following patch to cat: It might be simpler to just use "dd if=myfile obs=1m" instead of patching cat. -- Dan Nelson [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Pipes, cat buffer size
In the last episode (Oct 19), Ivan Voras said: > Dan Nelson wrote: > > In the last episode (Oct 18), Ivan Voras said: > >> I'm working on a program that's intended to be used as a "filter", > >> as in "something | myprogram > file". I'm trying it with cat and > >> I'm seeing my read()s return small blocks, 64 kB in size. I > >> suppose this is because cat writes in 64 kB blocks. So: > >> > >> a) Is there a way to programatically, per-process, set the pipe buffer > >> size? The program in question is a compressor and it's particularly > >> inefficient when given small blocks and I'm wondering if the system can > >> buffer enough data for it. > > > > Why not keep reading until you reach your desired compression block > > size? Bzip2's default blocksize is 900k, for example. > > Of course. But that's not the point :) From what I see (didn't look at > the code), Linux for example does some kind of internal buffering that > decouples how the reader and the writer interact. I think that with > FreeBSD's current behaviour the writer could write 1-byte buffers and > the reader will be forced to read each byte individually. I don't know > if there's some ulterior reason for this. No; take a look at /sys/kern/sys_pipe.c . Depending on how much data is in the pipe, it switches between async in-kernel buffering (<8192 bytes), and direct page wiring between sender and receiver (basically zero-copy). > >> b) Is there any objection to the following patch to cat: > > > > It might be simpler to just use "dd if=myfile obs=1m" instead of > > patching cat. > > I believe patching cat to bring its block size into the century of the > fruitbat has its own benefits. -- Dan Nelson [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Pipes, cat buffer size
In the last episode (Oct 19), Ivan Voras said: > 2008/10/19 Dan Nelson <[EMAIL PROTECTED]>: > > In the last episode (Oct 19), Ivan Voras said: > >> Of course. But that's not the point :) From what I see (didn't > >> look at the code), Linux for example does some kind of internal > >> buffering that decouples how the reader and the writer interact. I > >> think that with FreeBSD's current behaviour the writer could write > >> 1-byte buffers and the reader will be forced to read each byte > >> individually. I don't know if there's some ulterior reason for > >> this. > > > > No; take a look at /sys/kern/sys_pipe.c . Depending on how much > > data is in the pipe, it switches between async in-kernel buffering > > (<8192 bytes), and direct page wiring between sender and receiver > > (basically zero-copy). > > Ok, maybe it's just not behaving as I thought it should. See this > test program: [ program that prints the amount of data in each read() ] > and this command line: > > > dd bs=1 if=/dev/zero| ./reader > > The output of this on RELENG_7 is: > > read 8764 bytes > read 1 bytes [..] > read 1 bytes > read 1 bytes > ... > > The first value puzzles me - so it actually is doing some kind of > buffering. Linux isn't actually much better, but the intention is > there: > > $ dd if=/dev/zero bs=1 | ./bla > read 1 bytes > read 38 bytes > read 8 bytes > read 2 bytes [..] > read 2 bytes > read 3 bytes > read 3 bytes > read 112 bytes > read 2 bytes > read 2 bytes > ... > > Maybe FreeBSD switches between the writer and the reader too soon so > the buffer doesn't get filled? If your reader isn't doing any real work between reads, it is always reading, so the pipe will never fill up. The delay in FreeBSD was probably due to the shell spawning the writer first, so it buffered up 8k of data before the reader was ready. After that, the reader was able to pull data as fast as the writer pushed. > Using cat (which started all this), FreeBSD consistently processes > 4096 byte buffers, while Linux's sizes are all over the place - from > 4 kB to 1 MB, randomly fluctuating. My goal would be (if it's > possible - it might not be) to maximize coalescing in an environment > where the reader does something with the data (e.g. compression) so > there should be a reasonable amount of backlogged input data. Remember that increasing coelescing also increases latency and decreases the parallelism between reader and writer (since if you coalesce you cause the reader to wait for data that's already been writen, in the hopes that the writer will write again soon). > But if it works in general, it may simply be that it isn't really > applicable to my purpose (and I should modify the reader to read > multiple blocks). That's my suggestion, yes. That way your program would also work when passed data from an internet socket (where you will get varying read() sizes too). It wouldn't add more than 10 lines to wrap your read in a loop that exits when your preferred size has been reached. > Though it won't help me, I still think that modifying cat is worth it :) -- Dan Nelson [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Why does adding /usr/lib32 to LD_LIBRARY_PATH break 64-bit binaries?
In the last episode (Oct 23), Alexander Sack said: > On Thu, Oct 23, 2008 at 10:09 PM, Alexander Kabaev <[EMAIL PROTECTED]> wrote: > > LD_LIBRARY_PATH is for native 64bit rtld. If you want a specific > > path added for use by 32-bit ld-elf.so.1 only, use > > LD_32_LIBRARY_PATH. > > > > Said that, your problem is likely caused by the fact that there is > > no /lib32, only /usr/lib32. So if 64-bit library lives in /lib, > > your LD_LIBRARY_PATH will cause loader to find its 32-bit > > equivalent in /usr/lib32 first. > > > > Try LD_LIBRARY_PATH=/lib:/usr/lib:/usr/lib32:/usr/lib64 for better > > results. > > Yes I figured that out on my own but my question still exists, why > isn't /usr/lib similar in format to /usr/lib32 though with respect to > major numbers? Ever since the switch from static to dynamic-linked /bin and /sbin, some shared libraries are needed during the boot process. Those libraries live in /lib, and since there are no 32-bit binaries required to boot a 64-bit system, there is no need for a /lib32. -- Dan Nelson [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: zfs & waiting on zio->io_cv
In the last episode (Oct 24), Danny Braniss said: > there is a big delay (probably more than 1 sec.) when doing simple tasks > on this zfs, like ls(1), or 'zfs list', long enough to hit ^T > and get the same [zio->io_cv)], any hints? > > store-01# zfs list > (hitting ^T)load: 0.00 cmd: zfs 88376 [zio->io_cv)] 0.00u 0.00s 0% 1672k > (hitting ^T)load: 0.00 cmd: zfs 88376 [zio->io_cv)] 0.00u 0.00s 0% 1684k > NAME USED AVAIL REFER MOUNTPOINT > h 472G 11.2T23K /h > h/home466G 11.2T 466G /h/home > h/[EMAIL PROTECTED]54K - 466G - > h/root 18K 11.2T18K /h/root > h/src 18K 11.2T18K /h/src > h/system 5.64G 11.2T 5.64G /h/system That's sort of the equivalent to waiting in "biord" on a UFS filesystem, I think. ZFS is just waiting for the disk to return a block. If you happen to do something during the window where ZFS is commiting its transaction group, it has to wait until the sync finishes. If some other process is doing a lot of writes, or you only have one disk in your zpool, or your pool is close to full, it may take a couple seconds to sync. There's a couple of things you can try to improve interactive performance. Raising zfs's arc_max is the easiest to do, and will let ZFS cache more stuff, increasing the likelyhood that an "ls" will be able to read from cache instead of having to go to disk. Setting it at 1/4 your physical RAM is probably as high as you can go without causing panics. Raising txg_time ( in /sys/cddl/.../zfs/txg.c ) from 5 to say 30 will tell zfs to sync less often, which can be a win if you don't actually do that much writing. With a single spindle, it may take a substantial fraction of a second just to sync a tiny txg due to the number of copies of metadata ZFS writes for redundancy. If you do a lot of writing, lowering zfs_vdev_max_pending ( in /sys/cddl/.../zfs/vdev_queue.c ) from 35 down to 16 or less will reduce the number of simultaneous I/Os ZFS will try to send to each disk, which will let your reads compete a little better with other I/O. On ATA or SATA disks, you might want to set it to 2. -- Dan Nelson [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Is chflags' "nodump + sunlnk" = "uchg"
In the last episode (Nov 12), Charles Darwin said: > Hi all, > > Title is the question actually: Is chflags' "nodump + sunlnk" = "uchg" No; why would it be? From /usr/include/sys/stat.h: #define UF_NODUMP 0x0001 /* do not dump file */ #define SF_NOUNLINK 0x0010 /* file may not be removed or renamed */ #define UF_IMMUTABLE0x0002 /* file may not be changed */ nodump+sunlnk would be 0x0011, while uchg is 0x0002 . # touch a b # chflags nodump,sunlnk a # chflags uchg b # ls -lo a b -rw-r--r-- 1 root wheel sunlnk,nodump 0 Nov 12 17:42 a -rw-r--r-- 1 root wheel uchg 0 Nov 12 17:42 b # -- Dan Nelson [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Is chflags' "nodump + sunlnk" = "uchg"
In the last episode (Nov 13), Charles Darwin said: > On 12-Nov-08, at 6:43 PM, Dan Nelson wrote: > > In the last episode (Nov 12), Charles Darwin said: > >> Hi all, > >> > >> Title is the question actually: Is chflags' "nodump + sunlnk" = > >> "uchg" > > > > No; why would it be? > > I mean as far as their effect on a directory; doesn't "Don't change" > for a directory mean "Don't add + Don't remove"? Ok, that's sort of a different question. "nodump" is only checked by /sbin/dump, so we'll skip that. "sunlnk" is a root-only flag, while "uchg" can be set or reset by the file's owner as well. So let's look at "uunlnk" and "uchg": $ mkdir lnk chg $ touch lnk/file1 chg/file1 $ chflags uunlnk lnk ; chflags uchg chg $ ls -lo total 8 drwxr-xr-x 4 dan wheel - 512 Nov 13 12:59 ./ drwxrwxrwt 24 root wheel - 1024 Nov 13 12:59 ../ drwxr-xr-x 2 dan wheel uchg512 Nov 13 12:59 chg/ drwxr-xr-x 2 dan wheel uunlnk 512 Nov 13 12:59 lnk/ $ rm chg/file1 rm: chg/file1: Operation not permitted $ rm lnk/file1 $ touch chg/file2 touch: chg/file2: Operation not permitted $ touch lnk/file2 $ rm lnk/file2 $ mv chg chg1 mv: rename chg to chg1: Operation not permitted $ mv lnk lnk1 mv: rename lnk to lnk1: Operation not permitted $ rmdir chg rmdir: chg: Operation not permitted $ rmdir lnk rmdir: lnk: Operation not permitted So as far as directories are concerned, the only difference between uchg and uunlnk is that uchg prohibits creation and deletion of files inside it, and uunlnk allows it. You can also look at /sys/ufs/ufs/ufs_vnops.c and see what code looks at which flags. -- Dan Nelson [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: How process size is calculated? Is it always based on the current highest available address in memory space?
In the last episode (Dec 28), Yuri said:
> malloc(3) can be controlled by MALLOC_OPTIONS to use mmap-based
> allocation as opposed to sbrk-based allocation. But
> allocations/deallocations with mmaps can eventually lead to
> non-continuously mmapped memory (having some non-mmapped gaps).
>
> Are these gaps excluded from the process size or size is always
> linked to the current highest available address in memory space?
It looks like only mapped memory is counted in process size. The test
program below shows that the reported size goes down, even if a memory
range inbetween two others is unmapped:
$ ./mmap
Before mmap:
UID PID PPID CPU PRI NI VSZ RSS MWCHAN STAT TT TIME COMMAND
1000 48058 62165 0 8 0 1928 824 wait S+ph0:00.01 ./mmap
mmap 64MB A=0x2828
UID PID PPID CPU PRI NI VSZ RSS MWCHAN STAT TT TIME COMMAND
1000 48058 62165 0 8 0 67464 824 wait S+ph0:00.01 ./mmap
mmap 64MB B=0x2c28
UID PID PPID CPU PRI NI VSZ RSS MWCHAN STAT TT TIME COMMAND
1000 48058 62165 0 8 0 133000 824 wait S+ph0:00.01 ./mmap
mmap 64MB C=0x3028
UID PID PPID CPU PRI NI VSZ RSS MWCHAN STAT TT TIME COMMAND
1000 48058 62165 0 8 0 198536 824 wait S+ph0:00.01 ./mmap
munmap B
UID PID PPID CPU PRI NI VSZ RSS MWCHAN STAT TT TIME COMMAND
1000 48058 62165 0 8 0 133000 824 wait S+ph0:00.01 ./mmap
#include
#include
#include
#include
int main(void)
{
char *cmd;
void *a, *b, *c;
asprintf(&cmd, "ps axlp %d", getpid());
printf("Before mmap:\n");
system(cmd);
a = mmap(NULL, 64 * 1024 * 1024, PROT_READ|PROT_WRITE, MAP_ANON, -1, 0);
printf("mmap 64MB A=%p\n", a);
system(cmd);
b = mmap(NULL, 64 * 1024 * 1024, PROT_READ|PROT_WRITE, MAP_ANON, -1, 0);
printf("mmap 64MB B=%p\n", b);
system(cmd);
c = mmap(NULL, 64 * 1024 * 1024, PROT_READ|PROT_WRITE, MAP_ANON, -1, 0);
printf("mmap 64MB C=%p\n", c);
system(cmd);
printf("munmap B\n");
munmap(b, 64 * 1024 * 1024);
system(cmd);
return 0;
}
--
Dan Nelson
dnel...@allantgroup.com
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: threaded, forked, rethreaded processes will deadlock
In the last episode (Jan 08), Daniel Eischen said: > On Thu, 8 Jan 2009, Brian Fundakowski Feldman wrote: > > It appears that the post-fork hooks for malloc(3) are somewhat > > broken such that when a threaded program forks, and then its child > > attempts to go threaded, it deadlocks because it already appears to > > have locks held. I am not familiar enough with the current > > libthr/libc/rtld-elf interaction that I've been able to fix it > > myself, unfortunately. > > There's really nothing to fix - according to POSIX you are only > allowed to call async-signal-safe functions in the child forked from > a threaded process. If you are trying to do anything other than > that, it may or may not work on FreeBSD, but it is not guaranteed and > is not portable. The Rationale section of the pthread_atfork() page is a good read here, too. -- Dan Nelson dnel...@allantgroup.com ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: portupgrade spurious skips
In the last episode (Feb 26), Nate Eldredge said: > In the past few months I've noticed a bug in portupgrade. When I update > my ports tree and do `portupgrade -a', often a few ports will be skipped, > supposedly because another port on which they depend failed to install. > However, the apparently failed port actually did not fail, and if I rerun > `portupgrade -a', some of the skipped ports will install successfully > without complaint. After enough iterations I can eventually get all of > them. > > I'd like to file a PR about this, but it's a little bit tricky coming up > with a test case, since the behavior depends on having outdated packages > installed, and on the dependencies between them. Moreover, after I run > `portupgrade -a' and notice the problem, the state of the installed > packages has changed and the same packages aren't skipped the next time. > So my question is whether anyone has ideas about how to construct a > reasonable test case that could help me make this reproducible and easier > to investigate. Any thoughts? "me too".. It seems to happen frequently when doing > 20 or so ports. Every week or so I upgrade old ports and don't have problems, but after a gnome or xorg update that ends up bumping the portrevision for half my installed ports, it always takes three or four "portupgrade -a" loops for everything to buid. If you've got ZFS, you can snapshot your filesystems, and if portupgrade fails, roll back to the snapshot and do it again to see if it happens on the same port a second time. Or if you know ruby, you could instrument the code that checks for port build errors and see if it's got a bug in it... -- Dan Nelson dnel...@allantgroup.com ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: CPU user/kernel time given the PID
In the last episode (Mar 08), Jay Loden said:
> Oliver Fromme wrote:
> > ps(1) and top(1) both use ki_pctcpu, see the getpcpu() function in
> > src/bin/ps/print.c and format_next_process() in
> > src/usr.bin/top/machine.c
>
> Hi Oliver, thanks for the reply. I noticed the same after some digging
> through the source code for ps and top. While CPU usage % is a useful
> number also, I was hoping to be able to get CPU time(s). Possibly that
> information simply isn't available on FreeBSD like it is for other OSes.
I was wondering why you were having so much trouble finding what you were
looking for, and then I realized I have a patch that I have never submitted
a PR for: the addition of "systime" and "usertime" ps keywords :) It simply
reads the rusage struct, and returns the same values that getrusage() does.
--
Dan Nelson
dnel...@allantgroup.com
Index: extern.h
===
RCS file: /home/ncvs/src/bin/ps/extern.h,v
retrieving revision 1.37
diff -u -p -r1.37 extern.h
--- extern.h23 Jun 2004 23:48:09 - 1.37
+++ extern.h7 Jan 2005 06:46:15 -
@@ -78,11 +78,13 @@ int s_uname(KINFO *);
voidshowkey(void);
voidstarted(KINFO *, VARENT *);
voidstate(KINFO *, VARENT *);
+voidsystime(KINFO *, VARENT *);
voidtdev(KINFO *, VARENT *);
voidtname(KINFO *, VARENT *);
voiducomm(KINFO *, VARENT *);
voiduname(KINFO *, VARENT *);
voidupr(KINFO *, VARENT *);
+voidusertime(KINFO *, VARENT *);
voidvsize(KINFO *, VARENT *);
voidwchan(KINFO *, VARENT *);
__END_DECLS
Index: keyword.c
===
RCS file: /home/ncvs/src/bin/ps/keyword.c,v
retrieving revision 1.76
diff -u -p -r1.76 keyword.c
--- keyword.c 6 Apr 2006 03:24:31 - 1.76
+++ keyword.c 2 Mar 2007 17:23:10 -
@@ -185,6 +185,7 @@ static VAR var[] = {
UINT, UIDFMT, 0},
{"svuid", "SVUID", NULL, 0, kvar, NULL, UIDLEN, KOFF(ki_svuid),
UINT, UIDFMT, 0},
+ {"systime", "SYSTIME", NULL, USER, systime, NULL, 9, 0, CHAR, NULL, 0},
{"tdev", "TDEV", NULL, 0, tdev, NULL, 4, 0, CHAR, NULL, 0},
{"time", "TIME", NULL, USER, cputime, NULL, 9, 0, CHAR, NULL, 0},
{"tpgid", "TPGID", NULL, 0, kvar, NULL, 4, KOFF(ki_tpgid), UINT,
@@ -203,6 +204,7 @@ static VAR var[] = {
"lx", 0},
{"user", "USER", NULL, LJUST|DSIZ, uname, s_uname, USERLEN, 0, CHAR,
NULL, 0},
+ {"usertime", "USERTIME", NULL, USER, usertime, NULL, 9, 0, CHAR, NULL,
0},
{"usrpri", "", "upr", 0, NULL, NULL, 0, 0, CHAR, NULL, 0},
{"vsize", "", "vsz", 0, NULL, NULL, 0, 0, CHAR, NULL, 0},
{"vsz", "VSZ", NULL, 0, vsize, NULL, 5, 0, CHAR, NULL, 0},
Index: print.c
===
RCS file: /home/ncvs/src/bin/ps/print.c,v
retrieving revision 1.95
diff -u -p -r1.95 print.c
--- print.c 17 Sep 2007 05:27:18 - 1.95
+++ print.c 11 Oct 2007 19:54:02 -
@@ -551,6 +551,79 @@ cputime(KINFO *k, VARENT *ve)
}
void
+systime(KINFO *k, VARENT *ve)
+{
+ VAR *v;
+ long secs;
+ long psecs; /* "parts" of a second. first micro, then centi */
+ char obuff[128];
+ static char decimal_point;
+
+ if (decimal_point == '\0')
+ decimal_point = localeconv()->decimal_point[0];
+ v = ve->var;
+ if (!k->ki_valid) {
+ secs = 0;
+ psecs = 0;
+ } else {
+ /*
+* This counts time spent handling interrupts. We could
+* fix this, but it is not 100% trivial (and interrupt
+* time fractions only work on the sparc anyway). XXX
+*/
+ secs = k->ki_p->ki_rusage.ru_stime.tv_sec;
+ psecs = k->ki_p->ki_rusage.ru_stime.tv_usec;
+ if (sumrusage) {
+ secs += k->ki_p->ki_childstime.tv_sec;
+ psecs += k->ki_p->ki_childstime.tv_usec;
+ }
+ /*
+* round and scale to 100's
+*/
+ psecs = (psecs + 5000) / 1;
+ secs += psecs / 100;
+ psecs = psecs % 100;
+ }
+ (void)snprintf(obuff, sizeof(obuff), "%3ld:%02ld%c%02ld",
+ secs / 60, secs % 60, decimal_point, psecs);
+ (void)printf("%*s", v->width, obuff);
+}
+
+void
+usertime(KINFO *k, VARENT *ve)
+{
+ VAR *v;
+ long secs;
+ long
Re: rebuilding libpcap
In the last episode (Mar 16), Alexej Sokolov said: > how to correctly rebuild only libpcap from /usr/src/contrib without > rebuilding the whole world ? I try to do in libpcap some changes, then > make; make install in > /usr/src/contrib/libpcap, > but the changes are not visible by calling changed functions :( > What I do wrong ? /usr/src/contrib is a repository of 3rd-party source trees, and they're not meant to be built from. Try running your "make ; make install" in /usr/src/lib/libpcap instead. -- Dan Nelson dnel...@allantgroup.com ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: working of syscall handling
In the last episode (Apr 08), Mehul Chadha said:
> In the program given below the function readlink gets called up when
> printf is executed and the program ends without any output.
>
> readlink is a system call (syscall number = 58) which is being made by the
> printf function, but according to my understanding of system call, it is
> made by putting the handler number in eax register and then interrupting
> the processor, so that it can enter the kernel mode and execute the
> required function, but in this case(dont know why) my readlink function
> gets called up which should not have happened.
Readlink is not only a syscall, but a POSIX library function. You are
overriding that, and FreeBSD's malloc function uses readlink to read the
/etc/malloc.conf settings file. printf calls malloc, so that's why your
program exits.
http://www.opengroup.org/onlinepubs/9699919799/functions/readlink.html
> I will be very thankful if you can help me with it.
>
> #include
>
> int readlink(void *a, void *b)
> {
> exit(0);
> }
>
> int main(int argc, char **argv)
> {
> printf("Hello World");
> }
--
Dan Nelson
dnel...@allantgroup.com
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Garbled kernel messages on shutdown
In the last episode (Apr 17), Bruce Cran said: > On Thu, 16 Apr 2009 14:14:19 -0400 Damian Gerow > wrote: > > Gary Jennejohn wrote: > > > [snip a whole bunch of stuff] > > > > This kernel output really looks bad: > > > > Wai > > > > tSiynngc i(nmga xd is6k0s ,s evcnoonddess) rfeomraisnyisntge.m. > > > > .pr0ocess `syncer' to stop...0 done > > > > > > I can't speak to the rest, but this is probably because you have SMP > > > and don't have `options PRINTF_BUFR_SIZE=128' in your kernel config. > > > > Ah, so that's what causes that. > > > > Any particular reason GENERIC has SMP, but doesn't set > > PRINTF_BUFR_SIZE=128? > > I think from previous discussions there might be some concern about > stack usage when it's enabled. I was going to ask if a mutex or other method could be used in the console driver somewhere to prevent interleaved writes (the same way two userland writes to a single fd aren't interleaved), but I looked at the kernel's kvprintf function, and it really does send a character at a time to its output callback function. Maybe a mutex can be added inside kvprintf if TOCONS is set in pca.flags? So instead of malloc'ing a buffer, just make the 2nd kvprintf call wait for the first to finish. -- Dan Nelson dnel...@allantgroup.com ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Monitoring throughput of PCIe lanes
In the last episode (Jun 01), Julian Elischer said: > Jason Chambers wrote: > > I'm wondering if there is currently a way to monitor the throughput of a > > PCIe lane or group of lanes associated with a device ? > > > > I've done a little exploring of the source and man pages but have yet to > > find anything that seems to relate in an obvious form. > > unfortunatly PCIe traffic occurs at a level below that at which the OS can > really monitor. > > there would have to be some hardware support I think. Solaris has a busstat command that can print a huge number of low-level counters, including PCI DMA counts. If it's supported under OpenSolaris it should be easy to check and see whether it's dependant on Sun hardware or works with any PC (just boot it up and run busstat -l). -- Dan Nelson dnel...@allantgroup.com ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: top shows that multithreaded program uses 19623.14% CPU
In the last episode (Jun 13), Yuri said: > I ran then program with 1500 threads and in top it looked the this: > > PID USERNAMETHR PRI NICE SIZERES STATE C TIMECPU COMMAND > 20382 yuri 1500 990 641M 462M umtxn 0 0:00 19623.14% > quicksort > > > Where can I read how CPU column is calculated? > > Some time ago I saw another weird fenomenon when CPU column: 5% CPU load > for the pocess that just cycles in CPU. Top doesn't seem to show the right %CPU at all for threaded apps unless you hit H to view each thread individually. Then you get accurate numbers (but you have to manually sum up the threads usage if you want to see the total %CPU for an entire process). -- Dan Nelson dnel...@allantgroup.com ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Can I bind POSIX thread to cpu core?
In the last episode (Jun 28):
> I have system with 4 core cpu. How can I bind POSIX thread to the one
> core? I mean that this thread can be executed on the fixed core.
See the cpuset(2) and cpuset_setaffinity(2) manpages. Something like this
should work:
#include
#include
#include
#include
int main(void)
{
int i;
cpuset_t myset;
/* Get CPU mask for the current thread */
if (cpuset_getaffinity(CPU_LEVEL_WHICH, CPU_WHICH_TID, -1,
sizeof(myset), &myset) == -1)
err(1, "getaffinity failed");
/* Find first available CPU - don't assume CPU0 is always available */
for (i = 0; i < CPU_SETSIZE; i++)
if (CPU_ISSET(i, &myset))
break;
if (i == CPU_SETSIZE)
err(1, "Not allowed to run on any CPUs? How did I print this,
then?");
/* Set new CPU mask */
printf ("Setting affinity to CPU %d\n", i);
CPU_ZERO(&myset);
CPU_SET(i, &myset);
if (cpuset_setaffinity(CPU_LEVEL_WHICH, CPU_WHICH_TID, -1,
sizeof(myset), &myset) == -1)
warn("setaffinity failed");
/* Do CPU-intensive stuff here */
return 0;
}
--
Dan Nelson
dnel...@allantgroup.com
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Getting running time of child
In the last episode (Nov 19), Koffie Yahoo said: > > It's not as portable as getrusage(2), but you could probably get the > > information you want using libkvm's kvm_getprocs(3) function. The > > information available is defined in the kinfo_proc structure in > > /usr/include/sys/user.h. > > Unfortunately, as far as I can see the kinfo_proc structure only contains > the sum of user time and system time and not the two values separately, > or have I missed something? Take a look at the the ki_rusage struct inside kinfo_proc. -- Dan Nelson dnel...@allantgroup.com ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: scp more perfectly fills the pipe than NFS/TCP
In the last episode (Dec 19), Zaphod Beeblebrox said: > Here's an interesting conundrum. I don't know what's different between > the TCP that scp uses from the TCP that NFS uses, but given the same two > FreeBSD machines, SCP fills the pipe with packets better. > > Examine the following graphic: http://www.eicat.ca/~dgilbert/example-mrtg.png > > The system doing the scp and the NFS server is FreeBSD-7.2-p1. The system > receiving the scp and the NFS client is FreeBSD-8.0-p1 > > The scp transfer is the left hand side of the graph and the NFS transfer > is on the right. > > The NFS is mounted with "-3 -T -b -l -i" and no other options. Files are > being moved over NFS with the system "mv" command. The files in each case > are large (50 to 500 meg files). If you increase the NFS blocksize (-r 32768 for example) you will get slightly better performance, but you will likely never match the scp results. They're doing two different things under the hood: scp is streaming the entire file in one operation, while NFS is performing many "read 8k at offset 0", "read 8k at offset 8k", etc requests one after another, so a high-latency connection will take a performance hit due to the latency in issuing each command. According to the mount_nfs manpage, it looks like there is some prefetching that can be enabled with the "-a ##" option. It doesn't say what the default is, though. -- Dan Nelson dnel...@allantgroup.com ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: scp more perfectly fills the pipe than NFS/TCP
In the last episode (Dec 21), Zaphod Beeblebrox said: > On Sun, Dec 20, 2009 at 12:27 AM, Dan Nelson wrote: > > In the last episode (Dec 19), Zaphod Beeblebrox said: > >> Here's an interesting conundrum. I don't know what's different between > >> the TCP that scp uses from the TCP that NFS uses, but given the same > >> two FreeBSD machines, SCP fills the pipe with packets better. > >> > >> Examine the following graphic: > >> http://www.eicat.ca/~dgilbert/example-mrtg.png > >> > >> The system doing the scp and the NFS server is FreeBSD-7.2-p1. The > >> system receiving the scp and the NFS client is FreeBSD-8.0-p1 > >> > >> The scp transfer is the left hand side of the graph and the NFS > >> transfer is on the right. > >> > >> The NFS is mounted with "-3 -T -b -l -i" and no other options. Files > >> are being moved over NFS with the system "mv" command. The files in > >> each case are large (50 to 500 meg files). > > > > If you increase the NFS blocksize (-r 32768 for example) you will get > > slightly better performance, but you will likely never match the scp > > results. They're doing two different things under the hood: scp is > > streaming the entire file in one operation, while NFS is performing many > > "read 8k at offset 0", "read 8k at offset 8k", etc requests one after > > another, so a high-latency connection will take a performance hit due to > > the latency in issuing each command. According to the mount_nfs > > manpage, it looks like there is some prefetching that can be enabled > > with the "-a ##" option. It doesn't say what the default is, though. > > While the link is slow, it is really directly connected with a latency > of 10ms or so. Isn't mv mmap()'ing large enough regions to cause > there to be a reasonable queue to transfer? I've never been impressed with FreeBSD's ability to detect sequential read patterns and prefetch blocks ahead of time, even on local ufs filesystems. -- Dan Nelson dnel...@allantgroup.com ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: pthread_{mutex,cond} & fifo/starvation/scheduling policy
In the last episode (Jan 19), Bernard van Gastel said:
> I'm curious to the exact scheduling policy of POSIX threads in relation to
> mutexes and conditions. If there are two threads (a & b), both with the
> following code:
>
> while (1) {
> pthread_mutex_lock(mutex);
> ...
> pthread_mutex_unlock(mutex);
> }
>
> What is the scheduling policy of the different thread libraries? Are both
> threads getting an equal amount of time? Are there no starvation issues
> (are they executed in alternating turns)? (a test program of mine
> indicates that libpthread and libthr both have starvation issues, in
> contrary to Mac OS X 10.6)
There's no guarantee of fairness when dealing with mutexes afaik. My guess
is that if thread "a" unlocks the mutex and still has time left in its
quantum, it'll be able to grab it again without even going to the kernel.
>From the POSIX docs on mutexes:
http://www.opengroup.org/onlinepubs/9699919799/functions/pthread_mutex_lock.html#tag_16_439_08
"Mutex objects are intended to serve as a low-level primitive from which
other thread synchronization functions can be built. As such, the
implementation of mutexes should be as efficient as possible, and this
has ramifications on the features available at the interface.
The mutex functions and the particular default settings of the mutex
attributes have been motivated by the desire to not preclude fast,
inlined implementations of mutex locking and unlocking."
The idea being that mutexes should be held for as little time as possible.
Is there a way to write your code so that most of the CPU-consuming activity
is done outside of the mutex? Perhaps use a job queue of some sort, and
only lock the mutex when pushing/popping elements. Then worker processes
can run without holding the mutex, and will be fairly scheduled by the
kernel.
--
Dan Nelson
dnel...@allantgroup.com
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: ps "time" field jumps backward
In the last episode (Feb 05), Linda Messerschmidt said: > For most of 7.2, on up to a 7.3-PRERELEASE built yesterday, I've noticed > that the "time" field reported by ps and top jumps around for some > processes. I've particularly noticed it with MySQL. > > Here are some repeated ps results (ps axo pid,time,wchan,comm) for the > same process over a few minutes: > > 1647 0:08.06 ucond mysqld > 1647 0:08.06 ucond mysqld > 1647 0:08.06 ucond mysqld > 1647 0:08.06 ucond mysqld > 1647 0:08.07 ucond mysqld > 1647 0:10.24 wdrain mysqld > 1647 0:08.08 ucond mysqld > 1647 0:09.25 - mysqld > > It's like when it starts working it builds up time, but as soon as it > goes back to ucond, it resets back to the earlier value. By default, ps only lists one thread when listing threaded processes, and if it's anything like top, the CPU column seems to be randomly picked from one thread. Try running "ps axHo pid,lwp,time,wchan,comm" instead (add H and lwp). That will print the individual threads, plus the thread id so you can track them across runs. You should see one thread with a CPU value of "8.08" (or so), and other values for the rest. Ideally, top and ps would total up all the per-thread CPU counts when displaying the per-process numbers, but it doesn't seem to. -- Dan Nelson dnel...@allantgroup.com ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: kernel malloc() and free()
In the last episode (Mar 01), Shrivatsan said: > I am looking at the code that allocates and frees kernel memory. I > understand that allocating kernel memory is quite different from the user > level mallocs. In case of user level mallocs, we allocate requested size > + 4 bytes and store the requested size in the additional 4 bytes. Actually FreeBSD's userland malloc hasn't ever had a 4-byte-per-element overhead like this. All BSD mallocs have been block-based, where all objects in a page are the same size. The original bsd malloc had power-of-2 size groupings (blocks holding the same size objects were linked in a list). phkmalloc was imported in 1995 which moves the block metadata into a "page directory", which is separated from the memory returned by malloc(). The current malloc (jemalloc) has a similar setup but scales much better on SMP systems. http://phk.freebsd.dk/pubs/malloc.pdf http://people.freebsd.org/~jasone/jemalloc/bsdcan2006/jemalloc.pdf > However, in the case of kernel, allocating an additional 4 bytes is a > overhead since the request might fall in the next bucket. I looked into > the code and the documentation in the file uma_int.h, but I don't quite > understand the relation between slabs, zones and keg. How do we determine > the size of the memory that we are trying to free from given the virtual > address? I'm not too familiar with the kernel malloc, but it looks like each keg holds objects of the same size (and a keg may hold multiple slabs), so once you get a pointer to the slab header with the vtoslab() macro, slab->us_keg->uk_size points to the size of the allocation. Hm. Even after some reading, I'm not sure how the slabs keep track of which elements are allocated or free. I expected to find a bitmap somewhere that malloc() sets and free() clears, but I don't see it. Maybe some kernel hacker can explain it better :) Regardless, the size of the allocation at this point isn't important, since you know all the items in the page are the same size. -- Dan Nelson dnel...@allantgroup.com ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: unable to offline a failing drive in a zfs RAIDZ
In the last episode (Mar 01), Aryeh Friedman said: > I have a raidz setup as per the handbook but when I attempt to "offline" a > failing drive it will not let me: > > kate# zpool status -c > invalid option 'c' > usage: > status [-vx] [pool] ... > kate# zpool status -v > pool: storage > state: ONLINE > scrub: resilver completed with 0 errors on Mon Mar 1 17:36:48 2010 > config: > > NAMESTATE READ WRITE CKSUM > storage ONLINE 0 0 0 > raidz1ONLINE 0 0 0 > ad7 ONLINE 0 0 0 > ad8 ONLINE 0 0 0 > ad9 ONLINE 0 0 0 > ad10ONLINE 0 0 0 > ad12ONLINE 0 0 0 > > errors: No known data errors > kate# zpool offline storage ad12 > cannot offline ad12: no valid replicas What version of FreeBSD are you running? This looks like a known bug. It originally worked for mirrors but not RAIDZ vdevs - "zpool offline is a bit too conservative": http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=2171359 , and works for me on a 7-stable kernel: (r...@studio) /root># uname -a FreeBSD studio.evoy.net 7.3-PRERELEASE FreeBSD 7.3-PRERELEASE #53: Tue Feb 2 17:19:46 CST 2010 z...@studio.evoy.net:/usr/src-7/sys/amd64/compile/STUDIO amd64 (r...@studio) /root># mdconfig -a -t swap -s 1g ; mdconfig -a -t swap -s 1g ; mdconfig -a -t swap -s 1g md1 md2 md3 (r...@studio) /root># zpool create dummy raidz md1 md2 md3 (r...@studio) /root># zpool offline dummy md2 (r...@studio) /root># zpool status dummy pool: dummy state: DEGRADED status: One or more devices has been taken offline by the administrator. Sufficient replicas exist for the pool to continue functioning in a degraded state. action: Online the device using 'zpool online' or replace the device with 'zpool replace'. scrub: none requested config: NAMESTATE READ WRITE CKSUM dummy DEGRADED 0 0 0 raidz1DEGRADED 0 0 0 md1 ONLINE 0 0 0 md2 OFFLINE 0 0 0 md3 ONLINE 0 0 0 -- Dan Nelson dnel...@allantgroup.com ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: VMDirectPath with FreeBSD 8 VM under ESXi 4.0
In the last episode (Mar 09), Christopher Smith said: > I wasn't 100% on where this should go, but -hackers seemed like a reasoned > starting point. > > My overall objective is to setup a ZFS fileserver VM, and for my first > attempt I am trying to use VMDirectPath (ie: PCI pass-through) with a > FreeBSD 8.0 VM under ESXi 4 to pass-through the moterboard chipset SATA > controller (and when I expand in the future, the SAS controller I get). > Unfortunately, whenever I add the mapped PCI device to the VM, it powers > itself off about halfway through the boot sequence. > > I have confirmed it's not a fundamental problem by trying Linux and > OpenSolaris VMs - both can see the PCI device (an Intel 3420 chipset SATA > controller) and the drives attached to it. This problem only occurs with > the FreeBSD 8.0 (and 7.3RC2) VMs. > > I've also tried booting up the FreeBSD installer DVD on the bare hardware, > to make sure it's not a problem with that particular controller. > > The relevant part of the vmware.log that is generated is: > Code: > > Sep 19 05:19:26.676: vcpu-0| PCIPassthru: 000:31.2 : barSize: 2048 is not > pgsize multiple > Sep 19 05:19:26.677: vcpu-0| PCIPassthru: 000:31.2 : barSize: 2048 is not > pgsize multiple > Sep 19 05:19:26.677: vcpu-0| ASSERT bora/vmcore/vmx/main/physMem.c:2148 > bugNr=254266 > Sep 19 05:19:30.295: vcpu-0| Backtrace: > Sep 19 05:19:30.295: vcpu-0| Backtrace[0] 0x5e521d88 eip 0xbbf58ed > Sep 19 05:19:30.295: vcpu-0| Backtrace[1] 0x5e5221c8 eip 0xb7f405c > Sep 19 05:19:30.295: vcpu-0| Backtrace[2] 0x5e522218 eip 0xb9cafca > Sep 19 05:19:30.295: vcpu-0| Backtrace[3] 0x5e522248 eip 0xb9b929e > Sep 19 05:19:30.295: vcpu-0| Backtrace[4] 0x5e5222a8 eip 0xb9e92fd > Sep 19 05:19:30.295: vcpu-0| Backtrace[5] 0x5e5222d8 eip 0xb9e9442 > Sep 19 05:19:30.295: vcpu-0| Backtrace[6] 0x5e5222e8 eip 0xb9b8c5d > Sep 19 05:19:30.295: vcpu-0| Backtrace[7] 0x5e5223c8 eip 0xb8efea1 > Sep 19 05:19:30.295: vcpu-0| Backtrace[8] 0x5e5224b8 eip 0x173a24fb > Sep 19 05:19:30.295: vcpu-0| Backtrace[9] eip 0x17489e3e > Sep 19 05:19:30.295: vcpu-0| SymBacktrace[0] 0x5e521d88 eip 0xbbf58ed in > function (null) in object /bin/vmx loaded at 0xb795000 > Sep 19 05:19:30.295: vcpu-0| SymBacktrace[1] 0x5e5221c8 eip 0xb7f405c in > function Panic in object /bin/vmx loaded at 0xb795000 > Sep 19 05:19:30.295: vcpu-0| SymBacktrace[2] 0x5e522218 eip 0xb9cafca in > function (null) in object /bin/vmx loaded at 0xb795000 > Sep 19 05:19:30.295: vcpu-0| SymBacktrace[3] 0x5e522248 eip 0xb9b929e in > function (null) in object /bin/vmx loaded at 0xb795000 > Sep 19 05:19:30.295: vcpu-0| SymBacktrace[4] 0x5e5222a8 eip 0xb9e92fd in > function (null) in object /bin/vmx loaded at 0xb795000 > Sep 19 05:19:30.295: vcpu-0| SymBacktrace[5] 0x5e5222d8 eip 0xb9e9442 in > function (null) in object /bin/vmx loaded at 0xb795000 > Sep 19 05:19:30.295: vcpu-0| SymBacktrace[6] 0x5e5222e8 eip 0xb9b8c5d in > function (null) in object /bin/vmx loaded at 0xb795000 > Sep 19 05:19:30.295: vcpu-0| SymBacktrace[7] 0x5e5223c8 eip 0xb8efea1 in > function (null) in object /bin/vmx loaded at 0xb795000 > Sep 19 05:19:30.295: vcpu-0| SymBacktrace[8] 0x5e5224b8 eip 0x173a24fb in > function (null) in object /lib/libpthread.so.0 loaded at 0x1739d000 > Sep 19 05:19:30.295: vcpu-0| SymBacktrace[9] eip 0x17489e3e in > function clone in object /lib/libc.so.6 loaded at 0x173b8000 > Sep 19 05:19:30.295: vcpu-0| Msg_Post: Error > Sep 19 05:19:30.295: vcpu-0| [msg.log.error.unrecoverable] VMware ESX > unrecoverable error: (vcpu-0) > Sep 19 05:19:30.295: vcpu-0| ASSERT > bora/vmcore/vmx/main/physMem.c:2148 bugNr=254266 > Sep 19 05:19:30.295: vcpu-0| [msg.panic.haveLog] A log file is > available in "/vmfs/volumes/4aaf3595-47d35fcc-a053-0030489f04bf/FreeBSD > 8.0/vmware.log". [msg.panic.haveCore] A core file is available in > "/vmfs/volumes/4aaf3595-47d35fcc-a053-0030489f04bf/FreeBSD > 8.0/vmx-zdump.003". [msg.panic.requestSupport.withLogAndCore] Please > request support and include the contents of the log file and core > file. [msg.panic.requestSupport.vmSupport.vmx86] > Sep 19 05:19:30.296: vcpu-0| To collect data to submit to VMware > support, run "vm-support". > Sep 19 05:19:30.296: vcpu-0| [msg.panic.response] We will respond on > the basis of your support entitlement. Looks like you crashed VMWare. That shouldn't happen. Send the log and core file to VMWare support and they should be able to help you. -- Dan Nelson dnel...@allantgroup.com ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: there is a way to avoid strict libraries linking?
In the last episode (Apr 14), Leinier Cruz Salfran said: > i want to know if there is a possibility to avoid current strict libraries > linking .. i will explain myself > > for example .. i have installed 'gtk' (2.18) that depends on library > 'libpng.so.5' (png) .. and i will upgrade 'png' port to a superior > version that install the library 'libpng.so.6' BUUU 'gtk' will not be > upgraded, so it will still depending on 'libpng.so.5' .. so here is my > question: there is a way to avoid this?? i means that 'gtk' load > 'libpng.so' (that is a symbolic link to 'libpng.so.6') instead of > 'libpng.so.5' at runtime The whole reason for a library version bump is because the libraries are not compatible with each other, usually due to a function being removed or its argument list changing, or a structure changing size. Symlinking one version onto another version might work, but only if the application doesn't use any of the removed or changed functions. http://article.gmane.org/gmane.comp.graphics.png.devel/3283 It's much safer to just leave the libraries alone. Just because you upgraded libpng doesn't mean that your old gtk binary will stop working (assuming you are using "portupgrade" or "portmaster -w" which preserves old libraries of course). Anyway, the FreeBSD port maintainers usually bump the revision of dependant ports when a major library like libpng gets upgraded, to force everyone to upgrade anything that depends on it. > i think this is a VERY VERY strict rule because in linux programs load > 'lib*.so' instead 'lib*.so.#' except if that program was linked to > 'lib*.so.#' at compilation time no, the rule is the same in Linux. Programs load "lib*.so.#" there also: (d...@linuxbox) ~># ldd /usr/bin/rrdtool linux-vdso.so.1 => (0x7fff4f9ff000) librrd.so.2 => /usr/lib64/librrd.so.2 (0x7fa047716000) libfreetype.so.6 => /usr/lib64/libfreetype.so.6 (0x7fa04759b000) libpng.so.3 => /usr/lib64/libpng.so.3 (0x7fa04745f000) libz.so.1 => /lib64/libz.so.1 (0x7fa04734b000) libart_lgpl_2.so.2 => /usr/lib64/libart_lgpl_2.so.2 (0x7fa047234000) libm.so.6 => /lib64/libm.so.6 (0x7fa0470df000) libc.so.6 => /lib64/libc.so.6 (0x7fa046e9f000) /lib64/ld-linux-x86-64.so.2 (0x7fa04785e000) -- Dan Nelson dnel...@allantgroup.com ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: there is a way to avoid strict libraries linking?
In the last episode (Apr 22), Steve Franks said:
> > It's much safer to just leave the libraries alone. Just because you
> > upgraded libpng doesn't mean that your old gtk binary will stop working
> > (assuming you are using "portupgrade" or "portmaster -w" which preserves
>
> Untrue. Portupgrade deletes the old
> version of the port by default. The PNG upgrade was a major PITA, because
> I installed one new port that thought it had to have it. I'm sure 98% of
> the ports I then had to upgrade would have still worked just fine even if
> rebuilt against the old libpng.
Are you sure you're talking about portupgrade? From the manpage:
-u
--uninstall-shlibs Do not preserve old shared libraries. By
default, portupgrade preserves shared libraries
on uninstallation for safety. See the
pkg_deinstall(1) manpage and check out the -P
option for details.
I've 400 MB of shared libs in /usr/local/lib/compat/pkg as proof that it
does this by default, too. I should probably clean that out someday :)
> I think the complaint here is that the port dependencies system
> frequently gives the impression/enforces the rule that new ports will
> depend on whatever the most current version of everything is in the
> ports tree at the time they were built, forcing sort of a perpetual
> upgrade cycle. IMHO this is probably due to naive port maintainers
> (such as myself) incorrectly pointing a port at libpng.5 instead of
> any libpng, or libpng >= 5. Once the ports tree is 'poisoned' in this
> fashion, there's really no going back. I'd sure vote for an audit of
> this behavior as a summer of code project.
I don't think the porter's handbook mentions the DEPENDS_* comparison
operators at all, so unless you read (and understood) the
${deptype:L}-depends target in bsd.port.mk, you might not know it existed.
--
Dan Nelson
dnel...@allantgroup.com
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: there is a way to avoid strict libraries linking?
In the last episode (Apr 22), Dan Nelson said:
> In the last episode (Apr 22), Steve Franks said:
> > (such as myself) incorrectly pointing a port at libpng.5 instead of any
> > libpng, or libpng >= 5. Once the ports tree is 'poisoned' in this
> > fashion, there's really no going back. I'd sure vote for an audit of
> > this behavior as a summer of code project.
>
> I don't think the porter's handbook mentions the DEPENDS_* comparison
> operators at all, so unless you read (and understood) the
> ${deptype:L}-depends target in bsd.port.mk, you might not know it existed.
Nevermind; it's there.
http://www.freebsd.org/doc/en/books/porters-handbook/makefile-depend.html#AEN2246
--
Dan Nelson
dnel...@allantgroup.com
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: large master.passwd
In the last episode (May 14), Roar Thron?s said: > On a site with 20k users in the master.passwd, and where NIS is not > trusted, the master.passwd is distributed to each workstation. The > pwd.db and spwd.db are sized around 10Mb. > > Sometimes, those .db files get corrupt. I suspect it has something to > do with the machines being reset etc before the sync is finished. > (The machines are dual-boot, and there are a lot of users around.) > > I did some patching, and have not seen corrupted .db-files since. > > So how usable is this patch? > Worth intregrating? > - 2048 * 1024,/* cachesize */ > + 8192 * 1024,/* cachesize */ Cachsize is already adjustable via the -s commandline switch. > + /* sync may be wise > + -roart */ > + sync(); How about an fsync() of only that file? (I don't remember whether fsync flushes metadata though) -Dan Nelson dnel...@emsphone.com To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message
Re: ifconfig: changing mac address
In the last episode (May 14), David Scheidt said: > On Sat, 15 May 1999, Greg Lehey wrote: > :It seems there's a need, and the possibility. Would somebody like > :to suggest a syntax? > > ifconfig interface ether ab:cd:ef:fe:dc:ab [options] > > makes sense to me. And the next step would be to make the kernel realize that two cards ifconfig'd with the same MAC address are meant to be bonded together as one route (lots of switches support this). I have some machines that I'd love to be able to get 20MB/sec bandwidth between transparently. -Dan Nelson dnel...@emsphone.com To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message
Re: ifconfig: changing mac address
In the last episode (May 15), Greg Lehey said: > > And the next step would be to make the kernel realize that two cards > > ifconfig'd with the same MAC address are meant to be bonded together as > > one route (lots of switches support this). I have some machines that > > I'd love to be able to get 20MB/sec bandwidth between transparently. > > I think you need to reconsider that idea. How are you going to double > the bandwidth of the wire? Two wires :) Drop two Intel EtherExpress 10/100 NICs into a Netware box, load the Intel failover/bonding .NLM, plug the NICs into adjacent ports on a compatible switch (we use BayStacks), configure the switch to bond those two ports together, and you instantly double your bandwidth. If a card fails, all traffic simply routes to the other card. I've only been able to max out both cards once (according to my mrtg graph), but it does work. It shouldn't be strictly limited to EtherExpress cards though. The general idea should work no matter what cards you have. -Dan To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message
Re: ifconfig: changing mac address
In the last episode (May 15), Greg Lehey said: > OK, now maybe I'm missing something here. But an Ethernet address is > used to identify a board. Arp binds it to an IP address. An IP > address is bound to a network. So if you're on a different network, > you get a different IP address. Why do you need the same Ethernet > address? I don't think anyone mentioned anything about having the cards on two networks. In that case, you're right, having two cards with the same MAC address doesn't help one bit. > This is very different from having two boards on the same network, > both with the same Ethernet address. As I observed earlier, that does > make sense, but it's a hot standby situation. I can't see any point > in arranging for both of them to accept or send data. Doubles the bandwidth. Especially if you are talking to multiple machines (i.e. talk to two regular boxes at 100mbit/sec each), or have another box hooked up the same way (200mbit/sec to it). Since both cards in the server have the same MAC address, the client boxes don't know anything's unusual. -Dan To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message
Re: Seti project / stats reset, new version available
In the last episode (May 18), Wes Peters said:
> Stefan Bethke wrote:
> > Would you suggest a different default nice level, then, and what
> > should it be?
>
> RTP_PRIO_IDLE of course. See rtprio(2).
>
> > One can easily modifiy ${PREFIX}/etc/rc.d/setiathome.sh to run it
> > -with nice 100, and I'm open to making a level other that 1 the
> > -default.
>
> In that case, make the start script run it at idprio:
>
> idprio setiathome
>
> Phew! That was tough, huh? ;^)
Problem is that idprio isn't safe. I used to idprio the rc5client, but
within a day or do the machine would lock up. Rc5client would get a
lock on the root of the filesystem at idprio, and if there was another
process running at 100% CPU, rc5client would never get a chance to
unlock, causing all the other processes on the system to hang.
See PR kern/5641.
-Dan Nelson
dnel...@emsphone.com
To Unsubscribe: send mail to majord...@freebsd.org
with "unsubscribe freebsd-hackers" in the body of the message
Re: timeconsuming processes on FreeBSD 3.1
In the last episode (May 19), Andre Rikkert de Koe said: > We are an ISP and we recently installed FreeBSD 3.1 on our main > logonserver. Since than almost every day we find timeconsuming > processes running while the user isn't even logged in (anymore). > These programs are mostly tin and lynx and such interactive programs. > We are sure that they were started to run in foreground. However in > the top-example "brouwert" was not logged in at that moment. Only > thing we can do is to kill the proces. > > Does anyone has a clue what's the cause of this ? It's usually due to a bug in the application. When a user exits, the stdin/stdout filedescriptors on any backgrounded processes go away. That makes any read() calls return with an error. If the application doesn't check the return value of its read(), it can go into a tight loop it'll never exit. You can check to see if this is the problem by running truss -p 39448 . Check to see if it's doing the same read() or write() over and over. Tin used to have this bug, but I thought it was fixed long ago. Lynx shoudln't have any problems either. -Dan Nelson dnel...@emsphone.com To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message
Re: a two-level port system? (fwd)
In the last episode (Jun 01), Max Khon said: > hi, there! > > On Mon, 31 May 1999, Bill Fumerola wrote: > > Not really. > > > > > E.g.: try to check out port for samba 1.9.18p10 > > > > $ cvs co -D 08/29/98 samba > > > > works for me on freefall. > > I have very (VERY!) bad link to anoncvs.freebsd.org. are there other > anoncvs servers? Be your own CVS repository. Cvsup the raw CVS tree instead of pulling a single checked-out copy; then you can check out whatever versions you want, view the commitlogs, make local changes, etc. /usr/share/examples/cvsup/cvs-supfile is a good starting point. -Dan Nelson To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message
Re: Oracle OCI code on FreeBSD
In the last episode (Jun 09), Chad David said: > I have managed to install Oracle 8.0.5 on FreeBSD 3.2-STABLE, and > everything seems to be fine, but I am unable to run an OCI program > that I am porting from Solaris. I started out with unresolved > symbols in libclntsh.so, and I "got rid" of them by relinking > libclntsh.so against /usr/compat/linux/lib/libc.so.6. > > The OCI specific code is compiled into a shared object, and is loaded > into my program via dlopen() / dlsym(), which leaves me wondering > what happens when a Linux shared object is loaded into a FreeBSD > process? Is this possible (linking against linux/lib/libc.so.6) or > am I completely out to lunch? Has anybody managed to get an OCI > program running on FreeBSD? Won't work. stdio is completely different from BSD<->Linux, so no fread/fwrite calls will work, struct direct is different (scratch opendir), ioctls are certainly different, errnos don't map the same, signals are different, etc etc etc. > When I run the program I get hit with SIGBUS as soon as the symbol in > my shared object is called. I am not really sure what other details > would be helpful, but if anyone is at all interested in the I would > be happy to supply more :). Install the linux_devel port and resign yourself to building Linux executables whenever you have to talk to Oracle. -Dan Nelson dnel...@emsphone.com To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message
Re: What is FTW?
In the last episode (Jun 10), Zhihui Zhang said: > On Wed, 9 Jun 1999, Zhihui Zhang wrote: > > Most filesystems are created from archives that were created by a > > depth first search (aka ftw). > > > > What does ftw stand for (My guess is File Tree Walk)? Can anyone > > give me examples of programs that create archives from a file tree > > in a depth first way? Do these programs rebuild the file tree from > > archive exactly as they were created? > > I have just found that ftw does stand for File Tree Walk and there is a C > library routine named ftw() (XPG4 standard) in AIX and HP-UX. However, I > can not find the same routine in FreeBSD manual pages. Maybe it is not > supported by FreeBSD. There is a set of fts* funtcions in FreeBSD (man fts); it looks like the options are very similar. -Dan Nelson dnel...@emsphone.com To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message
Re: [t...@caida.org: Re: [MRTG-DEV] CDEF's with LT and IF in .42]
In the last episode (Jun 30), Jos Backus said: > - Forwarded message from Tobi Oetiker - > > OK found your problem it is that old FreeBSD does no proper > > IEEE math ... > > > > for some comparison operations it raises an sigfpe if an NaN is > > involved ... > > > > the next release of rrdtool will come with a proper test to find > > the problem and a proper fix in the software to ignore sigfpe > > Saw this on the MRTG-DEV list. > NaN handling is perceived to be problematic, it seems. The last time this came up (and it comes up every 6 months or so), the consensus was that we would rather trap FP errors than blindly pass them on to the user application. If a program wants to ignore NaN, divide-by-zero, underflow, and overflow conditions, let it wrap the offending line of code with two fpsetmask() calls; one to mask the condition, and one to restore the previous mask. If you want to completely ignore floating point errors, call fpsetmask(0) at the top of main(). I scanned the mailinglists and the thread that covers this issue most completely is http://www.freebsd.org/cgi/mid.cgi?id=199710101907.oaa09...@millenia.srrc.usda.gov -Dan Nelson dnel...@emsphone.com To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message
Re: CPU utilization
In the last episode (Nov 10), [EMAIL PROTECTED] said: > Hi Good Morning.. > > I want to calculate the CPU utilization in my DHCP server > implementation. > I am using C language on Solaris 9. Could any one please let me know > Is there any system call can do this? This is completely the wrong mailing list for this question, but take a look at the getrusage() function. -- Dan Nelson [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Filesystem monitoring question
In the last episode (Nov 18), Cornelis Swanepoel said: > When they have completed a write operation I need to trigger some > code that will act upon the file just written. This code generates a > unique id for the file, stats the file, compresses the file(if over > set limit), generates a preview of the file and stores some info > about the file and its owner in a MySQL db and a log on a seperate > machine. It then moves the file to a different location on the same > machine (which is inaccesible to the NFS and SMB clients) and renames > it as its unique identifier. Are you sure you need to hook into the kernel? Why not just have a userland process scan the directory every 10 seconds or so for new files? Note that NFS has no concept of "file close", just reads and writes, so you will need to be able to determine whether a file has completed. This could be either timestamp based (file not modified in 60 seconds == done), or if you know the file format, you may be able to validate the contents (check for zipfile end-of-file marker, etc). -- Dan Nelson [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: [PATCH] nsswitch extensions + caching
In the last episode (Dec 06), Brooks Davis said: > On Tue, Dec 06, 2005 at 10:46:26AM -0800, Julian Elischer wrote: > > Michael Bushkov wrote: > > [...] > > > > so, I've been wonderring.. what's all the fuss about nsswitch? > > what does it get us? > > It gives us the ability use modules to provide arbitrary backends for a > variety of interfaces to system databases. For instance getpw*(), > gethost*(), etc. Michael's patch itself adds caching to our nsswitch implementation, which dramatically improves performance on slow sources (ldaps, for example). -- Dan Nelson [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: sysctl, HW_PHYSMEM, and crippled gcc
In the last episode (Dec 08), Steve Kargl said:
> Anyone have any insight into fixing gcc to make better use of system
> memory on systems with more than 4 GB. It appears that
> libiberty/physmem.c tries to use sysctl() to determine the amount of
> physical memory in a system.
>
> { /* This works on *bsd and darwin. */
> unsigned int physmem;
> size_t len = sizeof physmem;
> static int mib[2] = { CTL_HW, HW_PHYSMEM };
>
> if (sysctl (mib, ARRAY_SIZE (mib), &physmem, &len, NULL, 0) == 0
> && len == sizeof (physmem))
> return (double) physmem;
> }
>
> This works if you have less than 4GB because of the unsigned int
> physmem. I have 12 GB, which of course, when expanded to the number
> of bytes doesn't fit into a unsigned int physmem.
physmem is actually an unsigned long, not an unsigned int, so on amd64
that sysctl call should fail anyway (amd64 is LP64, so a long won't fit
into an int).
> What is the ramification? Well, gcc uses this estimate of
> memory to size internal parameters.
>
> troutmask:sgk[259] gcc -v h.c
> Using built-in specs.
> Configured with: FreeBSD/amd64 system compiler
> Thread model: posix
> gcc version 3.4.4 [FreeBSD] 20050518
> GGC heuristics: --param ggc-min-expand=30 --param ggc-min-heapsize=4096
>
> In particular, ggc-min-heapsize=4096 is ridiculously small for a
> system with 12 GB of memory.
On all my FreeBSD boxes from 128MB to 1GB of RAM, I get the exact same
heuristic values as you, so I'm not sure whether the code works at all.
I seem to remember having the opposite problem on a memory-limited
machine which insisted in allocating a relatively huge percentage of
RAM for a sort, and gnu sort uses the same physmem() call for its
dynamic sizing.
--
Dan Nelson
[EMAIL PROTECTED]
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: sysctl, HW_PHYSMEM, and crippled gcc
In the last episode (Dec 10), Giorgos Keramidas said:
> >>On Thu, Dec 08, 2005 at 05:06:16PM -0800, Steve Kargl wrote:
> >>> Anyone have any insight into fixing gcc to make better
> >>> use of system memory on systems with more than 4 GB.
> >>> It appears that libiberty/physmem.c tries to use sysctl()
> >>> to determine the amount of physical memory in a system.
> >>>
> >>> { /* This works on *bsd and darwin. */
> >>> unsigned int physmem;
> >>> size_t len = sizeof physmem;
> >>> static int mib[2] = { CTL_HW, HW_PHYSMEM };
> >>>
> >>> if (sysctl (mib, ARRAY_SIZE (mib), &physmem, &len, NULL, 0) == 0
> >>> && len == sizeof (physmem))
> >>> return (double) physmem;
> >>> }
> >>>
> >>> This works if you have less than 4GB because of the unsigned
> >>> int physmem. I have 12 GB, which of course, when expanded
> >>> to the number of bytes doesn't fit into a unsigned int physmem.
>
> Can someone with access to a system with more than 4 GB verify that the
> following works correctly?
>
> % flame:/home/keramida/tmp/physmem$ cat -n physmem.c
> % 9 int
> % 10 main(void)
> % 11 {
> % 12 uint64_t physmem;
> % 13 size_t len = sizeof physmem;
> % 14 static int mib[] = { CTL_HW, HW_PHYSMEM };
> % 15 static size_t miblen = sizeof(mib) / sizeof(mib[0]);
> % 16
> % 17 if (sysctl(mib, miblen, &physmem, &len, NULL, 0) != 0)
> % 18 err(1, "sysctl hw.physmem");
> % 19 printf("Physical memory = %ju bytes\n", (intmax_t)physmem);
> % 20 return EXIT_SUCCESS;
> % 21 }
Won't this break on x86, where physmem is 32 bits? Just use "unsigned
long", which is what the sysctl type is according to kern_mib.c .
--
Dan Nelson
[EMAIL PROTECTED]
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Does tcpdump2xplot work?
In the last episode (Dec 13), Brian Reichert said: > On Tue, Dec 13, 2005 at 11:20:49AM -0500, Brian Reichert wrote: > > Other docs say I have to do this: > > > > tcpdump -tt -S -r tcpdump.out | tcpdump2xplot > > > > But, I get the same error... > > I've traced down the issue: tcpdump now creates lines like: > > IP > ... > > And tcpdump2xplot doesn't want to see that 'IP' field. I'll try to get a > patch cobbled... You'll probably get better results using the tcptrace port, which reads capture files directly. -- Dan Nelson [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: easy question about kill command
In the last episode (Dec 16), Roman Gorohov. said: > Oliver Fromme wrote: > > > roma.a.g <[EMAIL PROTECTED]> wrote: > >> Is there anyone who can explain me, why when i say 'kill -HUP id', > >> and its failed to restart, kill say nothing? > > > There is no way for the kill command to know what the target > > process is going to do with the signal. This is entirely and only > > the business of the target process, which might chose to take the > > default action (in the case of SIGHUP it's to terminate the > > process), to ignore the signal alltogether, or to take some special > > action. Some programs use SIGHUP traditionally to rotate their > > logfiles, re-read configuration files, re-open network sockets, > > restart themselves, or other things. But that's entirely up to the > > program in question, and there is no way the kill command could > > know about it, let alone whether it was successful or not. > > Thanks for your reply. My question was about standard bsd daemons, > not about some apps with unpredictable behaviour. It still depends on what daemon you're talking about. syslogd, for example, re-reads /etc/syslog.conf and reloads its logfiles on SIGHUP. Luckily, most base daemons are started from their own /etc/rc.d/* scripts which know how that particular program works, so you can use them to start/stop/restart daemons and not have to look up pids manually. -- Dan Nelson [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Number of kevents registered in kqueue
In the last episode (Dec 16), Sen C. Farley said: > I may have missed it in the man page, but I am unable to find a way > to determine how many kevents are currently registered within a > kqueue. If there is no method for a count, how about a way to find > if a kqueue is empty or not. Besides tracking what events are still > within a kqueue, this would make for an easier way to write an event > loop. Currently, calling kevent() on an empty kqueue will still > block. I don't think there's a way currently. What I did in my local tree is modify kern_kevent so that if the magic number -1 is passed in as nchanges, it will return the entire queued event list back to the caller in *eventlist, and return the number of events as the returncode. Very useful for debugging kqueue-using programs where you want to compare what you think you're waiting for, and what the kernel thinks you're waiting for :) -- Dan Nelson [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: (no subject)
In the last episode (Dec 19), [EMAIL PROTECTED] said: > I discovered the user "operator" in UNIX , found it in the > book "Essential System Administration" by AEleen Frisch, and it has > features that I would like to use. The book says (on page 131) that > this user exists on some BSD systems and it is used for back-ups and > such. It is like superuser ( root ) in that it can access any file > regardless of the permission bits, but it operates readonly, it > cannot modify unless the permission bits allow it to do so. Actually, the "operator" user has read access to the raw device files that filesystems are mounted on. That's how it can do backups with the dump command. It has no special access to mounted filesystems themselves. -- Dan Nelson [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: copy directory structure
In the last episode (Dec 21), Ashok Shrestha said: > Do you know how to copy just a directory structure (not the files > inside it)? One way would be using mtree: "mtree -d -c -p /path | mtree -U -p /otherpath". You could also do it with find and tar: "cd /path ; find . -type d | tar Tcfn - - | ( cd /otherpath ; tar xf - )" -- Dan Nelson [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: using get_system_info() - obtaining system load averages
In the last episode (Jan 09), kamal kc said:
> dear everybody,
>
> i want to use the routine get_system_info() to get the load averages
> of the cpu. i found it that top uses it.
get_system_info is just a function /in/ top itself that gets the CPU
usage. It uses the vm.loadavg sysctl to get the load average. A
simpler way is to just call the getloadavg() function; see its manpage
for more info.
--
Dan Nelson
[EMAIL PROTECTED]
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: increasing dd disk to disk transfer rate
In the last episode (Jan 12), Christoph Kukulies said: > My notebooks' hard disk, a Hitachi Travelstar 80 GB starts to develop > read errors. I have FreeBSD and Win XP on that disk. Although FreeBSD > ist still working , the errors in the Windows partition are causing > Windows do ask for a filesystem check nearly everytime I reboot the > computer. One time the error was in the hibernate.sys file, which > impedes powering up quickly after a hibernate. > > Anyway, I decided to buy a second identical hard disk and tried to > block by block copy the old disk to the new one using > > dd if=/dev/ad2 of=/dev/ad3 conv=noerror > > The process is running now since yesterday evening and it is at 53 MB > at a transfer rate of about 1.1 MB/s. Everybody has mentioned the first obvious fix: raise your blocksize from the default 512 bytes. The second fix addresses the problem that with a single dd, you are either reading or writing. If you pipe the first dd into a second one, it'll let you run at the max speed of the slowest device. dd if=/dev/ad2 conv=noerror,sync bs=64k | dd of=/dev/ad3 bs=64k -- Dan Nelson [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: FreeBSD Real Mode interface
In the last episode (Jan 31), Dmitry Frolov said: > * Loren M. Lang <[EMAIL PROTECTED]> [28.01.2006 13:09]: > > > Is there any equivalent to the Linux Real Mode interface in FreeBSD? I > > would like to port a program called atitvout to FreeBSD, but it uses > > calls to the vesa bios in real mode on the x86 arch. I can't seem to > > find out how to do this in FreeBSD. > > LRMI <http://sourceforge.net/projects/lrmi> should work > on FreeBSD since version 0.8. And here is my FreeBSD/NetBSD patch > for atitvout (with instructions inside): > > http://kaya.nov.net/frol/patches/atitvout-0.4-bsd2.diff If this is anything like vm86 mode, check out the i386_vm86 manpage. -- Dan Nelson [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
