Re: Junior Annoying Hacker Task

[David Schultz]

Thus spake Miguel Mendez <[EMAIL PROTECTED]>:
> On Sat, 02 Feb 2002 03:29:50 -0800
> Terry Lambert <[EMAIL PROTECTED]> wrote:
> 
> Hi Terry et al,
> 
> > Let me know the form you want the hierarchy to take, so
> > you can stick it into the GTK hierarchy thingy; I'll be
> > happy to crank out some quick yacc and lex code to do
> > the parsing of the file into that format for you.  A
> > structure definition, with links, and how you want it
> > linked, would be ideal.  8-).
> 
> I've been thinking about it and I'd like to provide two frontends, a GTK one and a 
>ncurses one, what do you think?

This is getting pretty ambitious.  I'm not about to say that a
procedural interface to rc.conf wouldn't be nice, or that it couldn't
be done well, but when it comes down to it, what problem are we trying
to solve?

Certainly rc.conf supports a lot of options, and a hierarchical
arrangement would simplify things.  However, most people only need to
deal with a small subset of the options, and organization doesn't seem
to be a big issue.  Of course there's also the possibility that
someone could leave out a quotation mark and end up with a non-working
system, but that's rare and solvable with simpler tools, as has been
discussed.

Besides, both Microsoft and the linuxconf people have done far worse
than rc.conf---especially Microsoft.  Know what to do when a bad
update eats a VXD that isn't on the Windows CD?  :P

[I'm not subscribed to -current, so please CC me any replies if that's
where this thread has drifted.]

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: sparc64 tinderbox failure

[David Schultz]

Thus spake Kris Kennaway <[EMAIL PROTECTED]>:
> On Mon, Dec 30, 2002 at 03:21:22AM +, Mike Barcroft wrote:
> 
> > ===> sbin/swapon
> > cc1: warnings being treated as errors
> > /tinderbox/sparc64/src/sbin/swapon/swapon.c: In function `swaplist':
> > /tinderbox/sparc64/src/sbin/swapon/swapon.c:246: warning: field width is not type 
>int (arg 3)
> 
> Can someone please just fix this (by backing out the offending commit,
> if necessary)?

Eek, given a 64-bit size_t, the present code leaves 32 bits of it
uninitialized in the usual case.  The following patch ought to fix
the problem; I can't make sure right now because I'm out of town.

Index: swapon.c
===
RCS file: /home/ncvs/src/sbin/swapon/swapon.c,v
retrieving revision 1.14
diff -u -r1.14 swapon.c
--- swapon.c2002/12/28 23:39:47 1.14
+++ swapon.c2002/12/30 05:15:54
@@ -211,7 +211,7 @@
size_t mibsize, size;
struct xswdev xsw;
int mib[16], n, pagesize;
-   size_t hlen;
+   int hlen;
long blocksize;
long long total = 0;
long long used = 0;

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: sparc64 tinderbox failure

[David Schultz]

Thus spake Juli Mallett <[EMAIL PROTECTED]>:
> * De: Craig Rodrigues <[EMAIL PROTECTED]> [ Data: 2002-12-29 ]
>   [ Subjecte: Re: sparc64 tinderbox failure ]
> > I'm not sure if your patch will solve the problem.
> > The offending code is here:
> > 240 if (lflag) { 
> > 241 char buf[32];
> > 242 snprintf(buf, sizeof(buf), "%ld-blocks", blocksize);
> > 243 printf("%-13s %*s %*s\n",
> > 244 "Device:",
> > 245 hlen, buf,
> > 246 hlen, "Used:");
> > 247 }
> > 
> > 
> > Doesn't the printf() statement in question have the wrong number of
> > arguments?
> 
> No, it's using variable field-length specifiers.

Right.  The complaint is that hlen is 64 bits and the printf()
expects the field length specifier to be an int.  The same goes
for getbsize(&hlen, ...), so I'm not sure why the compiler didn't
complain about a type mismatch.  I guess it just coerced the
pointer to an int *.

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: sparc64 tinderbox failure

[David Schultz]

Thus spake David Schultz <[EMAIL PROTECTED]>:
> Right.  The complaint is that hlen is 64 bits and the printf()
> expects the field length specifier to be an int.  The same goes
> for getbsize(&hlen, ...), so I'm not sure why the compiler didn't
> complain about a type mismatch.  I guess it just coerced the
> pointer to an int *.

Aah, the compiler didn't complain because the getbsize() interface
was changed between -CURRENT and -STABLE, and it now takes a
'size_t *' instead of an 'int *' to fill in with the length of the
string.  (To me, this change seems absolutely ridiculous, because
the number in question is seldom greater than 10, much less 2^64.)

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: aligned_nblks calculations broken in vm_swap.c

[David Schultz]

Thus spake [EMAIL PROTECTED] <[EMAIL PROTECTED]>:
> I'm actually more than a bit of mind to rip out the entire bogus
> swap-stripe code:  If you want swap on a striped disk, you should
> use hardware, controller, vinum, ccd or raidframe to stripe.

Ccd is a nice simple solution, but by using it you lose the
ability to dynamically add and remove swap and play other tricks
like swap device prioritization.  (Actually, prioritization would
require a better abstraction for the swap subsystem anyway.)  On
the other hand, the static limit on the number of devices is
annoying.  But the point is that the swap striping code isn't
entirely redundant.

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: panic with panic: kmem_malloc(4096): kmem_map too small...

[David Schultz]

Thus spake [EMAIL PROTECTED] <[EMAIL PROTECTED]>:
> >And to that fact I have a question:
> >At the moment 8% of the disk is reserved. 
> >It being a 170Gb raid, that wastes a good 13,6Gb, which I find at lot.
> >tunefs lets me bring that down to 5% = 8,5Gb without speed penalty.
> >But is for this size of spare space such a large threshold really required??
> >And IF i would like to experiment, where do I look for the knop to turn??
> 
> Basically you don't need to reserve anything, but as you get closer to
> filling the disk the time to find a free space increases rapidly and
> your files get very fragmented.  Trouble is: they never get defragmented
> unless you copy them (or do a full dump/restore).
> 
> I'm not sure how to nail the "right" number of % down, and I'm sure
> that both Kirk and I would like to hear your numbers :-)

If someone is interested in investigating this in detail, Margo
Setzer et alii did a study a few years ago on aging filesystems in
order to measure fragmentation.  Their intent was not to measure
the effect of free space on fragmentation, but their methodology
could be applied for that purpose.  They happen to have a graph
that suggests that large file throughput drops about 20% for a
filesystem that is 75% full versus a nearly empty filesystem.

http://www.eecs.harvard.edu/~vino/fs-perf/papers/sigmetrics.ps.gz

I don't think you'll find that it's a good idea to have a
filesystem more than 90% full.  At that load factor, even a
uniform hash will take on average 2.5 probes to locate free
space, and performance can only exponentially decrease from
there.  No filesystem can avoid fragmentation and perform well
without a bit of breathing room.

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: update from 4.7 to 5.0

[David Schultz]

Thus spake Odhiambo Washington <[EMAIL PROTECTED]>:
> The box runs prettier, but I have some output of dmesg that I'd appreciate
> some explanation on. I'll mark the portions where I seek some explanation
> on the dmesg output itself. The most important one is with USB interfaces,
> because they seem to NOT start at all since updating.
[...]
> acpi_cpu: CPU throttling enabled, 2 steps from 100% to 50.0%
> 
> 
> Where is that explained? I'd really love to know about it ;)

That's just the ACPI driver telling you that your CPU has a
reduced power (half speed) mode.  My -CURRENT box doesn't support
ACPI, but glancing at the code, it looks like the relevant sysctls
are hw.acpi.cpu.performance_speed and hw.acpi.cpu.economy_speed.
Maybe someone else can point you to some real documentation, if
it's written yet.

> ad0: 19092MB  [38792/16/63] at ata0-master UDMA100
> ad2: 9771MB  [19854/16/63] at ata1-master UDMA100
> Mounting root from ufs:/dev/ad0s2a
> lock order reversal
> ^^ # this is another interesting one I'd love to know about.
> 
> 
>  1st 0xc1bfb3f8 process lock (process lock) @ /usr/src/sys/kern/kern_descrip.c:2100
>  2nd 0xc1cf3634 filedesc structure (filedesc structure) @ 
>/usr/src/sys/kern/kern_descrip.c:2107

This is a known issue; see arch@, subject ``Need help fixing lock
ordering with filedesc, proc and pipe.''  If you do a sysctl at
just the wrong time, you could deadlock the system.  The
probability of this happening is on the order of one in a million,
so it shouldn't be a concern to you.  The warning is just pointing
out that possibility so the problem can be fixed.  Note that the
fact that you got this warning means that you have WITNESS enabled
in your kernel config, which is likely to kill performance.

I don't know about the other messages, but I suspect you can
safely ignore them as long as nothing is going wrong.

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: update from 4.7 to 5.0

[David Schultz]

Thus spake Terry Lambert <[EMAIL PROTECTED]>:
> David Schultz wrote:
> > Thus spake Odhiambo Washington <[EMAIL PROTECTED]>:
> > > acpi_cpu: CPU throttling enabled, 2 steps from 100% to 50.0%
> > > 
> > >
> > > Where is that explained? I'd really love to know about it ;)
> > 
> > That's just the ACPI driver telling you that your CPU has a
> > reduced power (half speed) mode.
> 
> Can I vote for replacing "enabled" with "available", so it does
> not look like it's being turned on on you, so that you have to
> post to this mailing list asking about it, in the future?

I second that.  The message caused me a double-take the first time
I saw it, too.

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: 5.0 without swap

[David Schultz]

Thus spake Lucky Green <[EMAIL PROTECTED]>:
> I am about to set up a FreeBSD 5.0 machine without a swap partition. The
> server has 1GB of RAM. Are there any caveats that I need to consider
> during installation or configuration?

If you're using sysinstall, it might insist that you have swap.
Then again, that may be fixed by now.  Beyond that, you won't be
able to take kernel crash dumps, and you'll have to be careful
that you don't run out of RAM.

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: 5.0 without swap

[David Schultz]

Thus spake Lucky Green <[EMAIL PROTECTED]>:
> Miguel wrote:
> > Having no swap will prevent you from getting crashdumps in 
> > case of panic which, if you run 5.0, is not that unusual. 
> > Besides these days harddrives cost $1/GB, so why not setup 
> > the swap partition anyway?
> 
> I don't want cleartext cryptographic keys to ever touch magnetic media,
> thus potentially opening the door to future forensic analysis.

You can accomplish that by wiring the pages containing your
cryptographic keys, rather than effectively wiring every page in
the system by having no swap space.  Alternatively, unless you're
really paranoid, it's probably sufficient to write over your swap
partition with random data before you shut down the system.

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: background fsck did not create lost+found

[David Schultz]

Thus spake Jan Srzednicki <[EMAIL PROTECTED]>:
> This massive disk mangling occured on /usr, but still, one file in /home
> got lost - which happened to be quite important file. Background fsck
> logged:
> 
> Jan 20 16:06:30 stronghold root: /dev/ad1s1d: UNREF FILE I=1723065
> OWNER=winfried MODE=100644
> Jan 20 16:06:30 stronghold root: /dev/ad1s1d: SIZE=23397 MTIME=Jan 20
> 15:57 2003 (CLEARED)
> Jan 20 16:06:30 stronghold root: /dev/ad1s1d: Reclaimed: 0 directories, 8
> files, 16439 fragments
> Jan 20 16:06:30 stronghold root: /dev/ad1s1d: 33802 files, 13109700 used,
> 6310697 free (11577 frags, 787390 blocks, 0.1% fragmentation)
> 
> First two entries clearly correspond to the missing file, which should
> have been put in /home/lost+found. But, the poroblem is that no lost+found
> directory was created, while it should (as fsck_ffs(8) says). I guess its
> a bug, probably in the background fsck code. Still, is there any way to
> reclaim the file now, besides running strings(1) on the whole partition?

Consider what happens when you remove a large directory tree.
Thousands of directory entries may be removed, but in the
softupdates case, the inodes will stick around a bit longer.  The
same also applies to files that have been intentionally unlinked
but are still open.  To avoid a syndrome where all these thousands
of files end up in lost+found after a crash or power failure, fsck
just removes them on softupdates-enabled filesystems.

Unfortunately, this means that a newly-created file that has an
inode but no directory entry will also be cleared.  In some sense,
this race is equivalent to the situation where something went
wrong before the inode could be written.

However, when you are saving a new version of an important file,
you need to be careful that the new version (and its directory
entry) hits the disk before the old one goes away.  I know that vi
saves files in a safe way, whereas ee and emacs do not.  (Emacs
introduces only a small race, though.)  Also, mv will DTRT only if
the source and destination files live on the same filesystem.

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: background fsck did not create lost+found

[David Schultz]

Thus spake Matthew Dillon <[EMAIL PROTECTED]>:
> :However, when you are saving a new version of an important file,
> :you need to be careful that the new version (and its directory
> :entry) hits the disk before the old one goes away.  I know that vi
> :saves files in a safe way, whereas ee and emacs do not.  (Emacs
> :introduces only a small race, though.)  Also, mv will DTRT only if
> :the source and destination files live on the same filesystem.
> :
> 
> I think you have that reversed.  vi just overwrites the destination
> file (O_CREAT|O_TRUNC, try ktrace'ing a vi session and you will see). 
> I believe emacs defaults to a mode where it creates a new file and 
> renames it over the original. 
> 
> This means that there is a period of time where a crash may result in
> the loss of the file if the vi session cannot be recovered (with vi -r)
> after the fact.

vi writes and fsyncs a recovery file when it opens a file for
editing, and it fsyncs the real file before removing the recovery
file.  (I don't know how reliable vi's recovery mechanism is
because I don't use vi, but at least it's ensuring that the
recovery file is written to disk when it should be.)

In Emacs, if 'make-backup-files' is non-nil (the default), the
original file ${FILE} is renamed to ${FILE}~.  Then it writes out
and fsyncs a new file, which is perfectly safe.  If
'make-backup-files' is nil, emacs simply omits the renaming part,
unsafely overwriting the original file.  The behavior in the
latter case appears to be a bug, or at least an undocumented
feature.  Emacs even causes data loss in this case when the disk
fills up!  It needs to either do an fsync/rename or write and
fsync a backup file for the duration of the save.

Lastly, with ee, there's no backup file and no fsync.

Some ktrace snippets are below.


  3662 vi   CALL  open(0x808e260,0x2,0x180)
  3662 vi   NAMI  "/var/tmp/vi.recover/vi.HjDlgO"
  3662 vi   RET   open 4
...
  3662 vi   CALL  write(0x4,0x809a01c,0x400)
  3662 vi   GIO   fd 4 wrote 1024 bytes
   "[...]old contents[...]"
...
  3662 vi   CALL  fsync(0x4)
  3662 vi   RET   fsync 0
...
[I edit the file from "old contents" to "new contents"]
...
  3662 vi   CALL  open(0x8095140,0x601,0x1b6)
  3662 vi   NAMI  "foo"
  3662 vi   RET   open 7
...
  3662 vi   CALL  write(0x7,0x80bb000,0xd)
  3662 vi   GIO   fd 7 wrote 13 bytes
   "new contents
   "
  3662 vi   RET   write 13/0xd
...
  3662 vi   CALL  fsync(0x7)
  3662 vi   RET   fsync 0
  3662 vi   CALL  close(0x7)
  3662 vi   RET   close 0
...
  3662 vi   CALL  lseek(0x4,0,0x400,0,0)
  3662 vi   RET   lseek 1024/0x400
  3662 vi   CALL  write(0x4,0x809a01c,0x400)
  3662 vi   GIO   fd 4 wrote 1024 bytes
"[...]new contents[...]"
...
  3662 vi   CALL  fsync(0x4)
  3662 vi   RET   fsync 0

[The following bit only happens if make-backup-files is non-nil]
  3799 emacsCALL  rename(0x848c328,0x848fba8)
  3799 emacsNAMI  "/home/test/foo"
  3799 emacsNAMI  "/home/test/foo~"
  3799 emacsRET   rename 0
...
[This part happens unconditionally]
  3799 emacsCALL  open(0x848c328,0x601,0x1b6)
  3799 emacsNAMI  "/home/test/foo"
  3799 emacsRET   open 3
  3799 emacsCALL  write(0x3,0xbfbfae24,0x3)
  3799 emacsGIO   fd 3 wrote 3 bytes
   "new"
  3799 emacsRET   write 3
  3799 emacsCALL  write(0x3,0xbfbfae24,0x9)
  3799 emacsGIO   fd 3 wrote 9 bytes
   " contents"
  3799 emacsRET   write 9
  3799 emacsCALL  fsync(0x3)
  3799 emacsRET   fsync 0
  3799 emacsCALL  close(0x3)
  3799 emacsRET   close 0

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: current- and BSD related stupid question

[David Schultz]

Thus spake Harald Schmalzbauer <[EMAIL PROTECTED]>:
> With 4.x I had a /etc/make.conf where I could force gcc to optimize for my
> CPU with -march.
> 
> This file (/etc/defaults/make.conf) vanished, but I can see something
> similar now without any rule set.
> 
> Does gcc (or any compiler stage like preprocessor) analyze the current
> building system??
> Just for interest; I don't have any problems but there are quiet a few
> things under 5.0 which are completely new to me and I'd like to keep in
> touch with this great OS.

What used to be /etc/defaults/make.conf is now
/usr/share/examples/etc/make.conf.  It works
just as it always has.

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: background fsck did not create lost+found

[David Schultz]

Thus spake Garrett Wollman <[EMAIL PROTECTED]>:
> <<[EMAIL PROTECTED]> said:
> 
> > Would that be a big problem to allow some fsck option not to erase all
> > these softupdates-pending inodes, but to put them in lost+found as usual?
> 
> It certainly couldn't be done with the background fsck, because
> background fsck works on a snapshot and not the running filesystem;
> thus, it cannot make any allocations -- it can only deallocate things.

Actually, that should work just fine.  When background fsck
notices an unreferenced inode in the snapshot, it could create a
file in the underlying filesystem.  The easy way to do this is to
copy the data with the standard open(2)/write(2)/close(2)
interfaces.  After the copy, the original data blocks are
deallocated as usual.  A more efficient implementation might
require a special kernel interface that creates a directory entry,
given an inode number and path.  Unfortunately, I think it is
possible that the unreferenced inode has not been initialized,
even though it is allocated in the inode bitmap, so you could
potentially get random junk.

Such a feature sounds reasonable, although I'm not sure how useful
it would really be.  If you have software that introduces a race
window where you can lose data because it does updates
incorrectly, hacking the operating system to make the race window
slightly smaller is not the best solution.

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: I've just had a massive file system crash

[David Schultz]

Thus spake Greg Lehey <[EMAIL PROTECTED]>:
> I've been thinking about what happened, and I have a possibility: the
> session before shutdown included a lot of writing to that file system,
> and I did a shutdown -p.  It's possible that the shutdown powered off
> the system before the disk had flushed its cache.  For the moment I'm
> avoiding shutdown -p, but when I get home I'll try to provoke it
> again.

FreeBSD's ``fix'' for this problem is the same as Windows 98's.
Specifically, there is a 5-second delay (tuneable:
kern.shutdown.poweroff_delay) after all buffers are flushed but
before the power is cut.  Maybe we ought to be sending FLUSH
CACHE commands to all drives and waiting for them to finish.

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: I've just had a massive file system crash

[David Schultz]

Thus spake Nate Lawson <[EMAIL PROTECTED]>:
> On Fri, 24 Jan 2003, David Schultz wrote:
> > Thus spake Greg Lehey <[EMAIL PROTECTED]>:
> > > I've been thinking about what happened, and I have a possibility: the
> > > session before shutdown included a lot of writing to that file system,
> > > and I did a shutdown -p.  It's possible that the shutdown powered off
> > > the system before the disk had flushed its cache.  For the moment I'm
> > > avoiding shutdown -p, but when I get home I'll try to provoke it
> > > again.
> > 
> > FreeBSD's ``fix'' for this problem is the same as Windows 98's.
> > Specifically, there is a 5-second delay (tuneable:
> > kern.shutdown.poweroff_delay) after all buffers are flushed but
> > before the power is cut.  Maybe we ought to be sending FLUSH
> > CACHE commands to all drives and waiting for them to finish.
> 
> da(4) does a SYNC CACHE (see daclose() and dashutdown()).

Good.  I was referring to IDE in this case, because I assume
that's what Greg's laptop uses.  The ATA driver flushes the cache
when the device is closed, but I don't think that happens during
shutdown.  It probably needs to register a shutdown hook like the
SCSI driver.  Also, the driver is a bit optimistic about how long
the flush will take; it times out after 5 seconds, whereas the ATA
spec says a flush can take up to 30 seconds.

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: [PATCH]: newfs(8) FS_OPTSPACE vs FS_OPTTIME bug

[David Schultz]

Thus spake Maxim Konovalov <[EMAIL PROTECTED]>:
> newfs(8) incorrectly claims that FS_OPTTIME is unavailable when
> minfree is less than MINFREE. MINFREE is defined in ufs/ffs/fs.h:
> 
> #define MINFREE  8
> 
> But relevant code in ufs/ffs/ffs_alloc.c uses hardcoded value:
> 
> 288 if (fs->fs_minfree <= 5 ||
> 289 fs->fs_cstotal.cs_nffree >
> 290 (off_t)fs->fs_dsize * fs->fs_minfree / (2 * 100))
> 291 break;
> 292 log(LOG_NOTICE, "%s: optimization changed from SPACE to TIME\n",
> 
> tunefs(8) metions 5% too.

The code you quote in ffs_alloc.c doesn't do what you think it
does.  The minfree comparison there is an exception to the rule
``if there is too much fragmentation, switch optimization from
space to time''.  The kernel must do this because the calculation
on lines 289 and 290 doesn't work very well if minfree is small.

The code in newfs works in the opposite direction: it forces
optimization from time to space if minfree is too small (<8%).
The rationale here is different: if you set minfree very low, you
must expect space to be very tight, so the filesystem should
optimize from space from the start.

So these numbers are different on purpose, because they address
separate issues.  The tunefs(8) comment you refer to seems to have
been introduced in rev. 1.2 of tunefs.8.  I revised the comment in
rev. 1.18, but even then I didn't catch the error.  I think we
need to s/of 5% and less/below the default value/.  The manpage
should also point out that the time/space optimization
restrictions based on minfree are enforced by newfs/tunefs when
minfree is changed, and not by the kernel.  They can be overridden
by subsequently asking for space or time optimization again.

Also, newfs does not match tunefs exactly in its treatment of
minfree.  Tunefs will additionally force time optimization if
minfree is >= the default.  Early comments in CVS logs seem to
indicate that tunefs's behavior was the intended one.

**OR**

Here is another idea: rip out of tunefs and newfs the magic that
forces space/time optimization.  Instead, when someone tries to
specify a small value for minfree, just warn them why they
shouldn't do that.  The one bit of minfree magic in ffs_alloc.c
stays because it's important.

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: I've just had a massive file system crash

[David Schultz]

Thus spake Greg Lehey <[EMAIL PROTECTED]>:
> I've been thinking about what happened, and I have a possibility: the
> session before shutdown included a lot of writing to that file system,
> and I did a shutdown -p.  It's possible that the shutdown powered off
> the system before the disk had flushed its cache.  For the moment I'm
> avoiding shutdown -p, but when I get home I'll try to provoke it
> again.

Just a heads up: Soeren tells me he will commit a fix for this in
his next ATA meta-commit.  I have patches if wanted.

I still can't figure out why the problem would trash your entire
home directory, though.  Even if the disk reordered writes and
failed to write some sectors, directory entries that were not
being actively modified shouldn't have become corrupted, as far as
I know.  (Maybe your disk does track-at-once writes and just
happened to be flushing the last few sectors from its cache when
the power was cut.)  Perhaps someone could ask Kirk, although it
may take an actual hosed filesystem to diagnose what happened.

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: boot delay when testing for ata devices

[David Schultz]

Thus spake Enache Adrian <[EMAIL PROTECTED]>:
> FreeBSD used to have an irritating boot delay in 4.4. It was gone in
> ~4.6, never reappeared in -STABLE, but is there in -CURRENT.
> (I've seen it also mentioned in some vmware documentation).
> 
> Applying this patch fixes it:
> 
> --- /arc/freebsd/src/sys/dev/ata/ata-all.cSun Jan 19 23:54:13 2003
> +++ sys/dev/ata/ata-all.c Mon Jan 27 21:02:34 2003
> @@ -514,7 +514,7 @@
>  
>  /* apparently some devices needs this repeated */
>  do {
> - if (ata_command(atadev, command, 0, 0, 0, ATA_WAIT_INTR)) {
> + if (ata_command(atadev, command, 0, 0, 0, ATA_IMMEDIATE)) {
>   ata_prtdev(atadev, "%s identify failed\n",
>  command == ATA_C_ATAPI_IDENTIFY ? "ATAPI" : "ATA");
>   free(ata_parm, M_ATA);
> 
> I'm not familiar with the ata code, but I can't see from ata-all.c
> which harm this could cause.

There is already at least one person for whom this code doesn't
work, so this patch doesn't seem like a good idea.  Someone needs
to figure out exactly what delays are needed here to make
everyone's hardware actually work.

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: add ext2fs to the module list in modules/Makefile

[David Schultz]

Thus spake Juli Mallett <[EMAIL PROTECTED]>:
> * De: Max Khon <[EMAIL PROTECTED]> [ Data: 2003-01-27 ]
>   [ Subjecte: Re: add ext2fs to the module list in modules/Makefile ]
> > hi, there!
> > 
> > On Tue, Jan 28, 2003 at 12:40:12AM +0200, Enache Adrian wrote:
> > 
> > > On Mon, Jan 27, 2003 at 01:36:35PM -0800, Steve Kargl wrote:
> > > > Portions of the ext2fs source are covered by the GPL.  You
> > > > need to rebuild the kernel with "option EXT2FS".  The
> > > > FreeBSD cannot create a ext2fs.ko and comply with the GPL.
> > > 
> > > This is weird.
> > > Builting it as part of kernel is ok, but separate, as a module
> > > isn't.
> > 
> > IIRC NetBSD has BSD-copyrighted ext2fs implementation
> 
> Closely tied to their VFS implementation, which is different, of course,
> last I heard it was a fairly heavy task to port it, but something a lot
> of people would like to see.

Does it work any better/worse than FreeBSD's implementation?

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: Seat-belt for source upgrades from stable to current

[David Schultz]

Thus spake Christopher Vance <[EMAIL PROTECTED]>:
> On Wed, Jan 29, 2003 at 10:35:40PM -0500, Garance A Drosihn wrote:
> : How about requiring the user to touch some file in / or /boot which
> : indicates the branch-tag that's acceptable for installworlds?  Then
> : you just need to propagate the tag from the 'cvs co' stage to some
> : file under /usr/src (such as /usr/src/CVS/Tag ).
> 
> Some of use cvsup and won't have CVS/Tag.

OT: Is there a good way to get the CVS metadata in /usr/src and
/usr/ports without transferring the entire source tree over the
network?  On some machines, I'd like to be able to do a CVS
{diff,log,update} now and then, but I don't have the disk space
for the entire repository.  I usually end up blowing away /usr/src
and fetching a new copy from a CVS server, but I'm sure this is
far from ideal for the people who pay for that server's bandwidth.

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: Seat-belt for source upgrades from stable to current

[David Schultz]

Thus spake Steve Kargl <[EMAIL PROTECTED]>:
> On Thu, Jan 30, 2003 at 07:09:16PM -0800, David Schultz wrote:
> > OT: Is there a good way to get the CVS metadata in /usr/src and
> > /usr/ports without transferring the entire source tree over the
> > network?  On some machines, I'd like to be able to do a CVS
> > {diff,log,update} now and then, but I don't have the disk space
> > for the entire repository.  I usually end up blowing away /usr/src
> > and fetching a new copy from a CVS server, but I'm sure this is
> > far from ideal for the people who pay for that server's bandwidth.
> > 
> 
> anoncvs
> 
> See the handbook for info.

That's a great answer...to a different question.  ;-)

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: Seat-belt for source upgrades from stable to current

[David Schultz]

Thus spake Mike Makonnen <[EMAIL PROTECTED]>:
> Use the r version of the cvs commands (like cvs rlog and rdiff). They operate on
> the repository remotely, so you don't need to have the files checked out localy.

That's a pretty good solution, and I use those occasionally.  It
would be a perfect solution if there were an 'rupdate', so I don't
have to (cd /tmp; cvs co src/file/i/want.c)
&& cp /tmp/src/file/i/want.c /where/i/want/it
&& rm -rf /tmp/src
all the time.

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: Seat-belt for source upgrades from stable to current

[David Schultz]

Thus spake Christopher Vance <[EMAIL PROTECTED]>:
> On Thu, Jan 30, 2003 at 09:07:11PM -0800, Steve Kargl wrote:
> : On Thu, Jan 30, 2003 at 08:05:06PM -0800, David Schultz wrote:
> : > Thus spake Steve Kargl <[EMAIL PROTECTED]>:
> : > > On Thu, Jan 30, 2003 at 07:09:16PM -0800, David Schultz wrote:
> : > > > OT: Is there a good way to get the CVS metadata in /usr/src and
> : > > > /usr/ports without transferring the entire source tree over the
> : > > > network?  On some machines, I'd like to be able to do a CVS
> : > > > {diff,log,update} now and then, but I don't have the disk space
> : > > > for the entire repository.  I usually end up blowing away /usr/src
> : > > > and fetching a new copy from a CVS server, but I'm sure this is
> : > > > far from ideal for the people who pay for that server's bandwidth.
> : > > > 
> : > > 
> : > > anoncvs
> : > > 
> : > > See the handbook for info.
> : > 
> : > That's a great answer...to a different question.  ;-)
> : 
> : It's the correct answer.  I assumed that you knew
> : how to use cvs.
> 
> cvsup gets me everything I need to track and compile both current and
> stable.
> 
> I don't want to be forced into using cvs when there's a better tool
> available (for some definition of better).  I get paid to use cvs at
> work, and that's how I know to choose something else...
> 
> For a while, I used to grab the whole repo (with cvsup), and used cvs
> to get current and stable out of it, but now I consider that a waste
> of space/time, and have reverted to just using cvsup to get the tags I
> want.
> 
> I'm not a FreeBSD developer, and very rarely (just a handful of times)
> have had to modify existing stuff to do what I want, so I don't need
> my own repo to commit to.  With that, disappers any need to use cvs.
> 
> Perhaps you can explain why cvsup is the wrong answer...

I don't know about Steve, but cvsup is the wrong answer for me
because it's a mirroring tool and not a version control tool.
Among the things I would like to do are:

- Update to a specific version of a specific file from the
  repository.

- Generate a diff between two revisions in the repository,
  or between a version in the repository and some local
  patches of my own.

- View logs for particular files.

I asked the question in hopes that there would be some neat
feature of cvsup that mocked up some CVS metadata for me, but
since nobody has mentioned any such thing, I guess I'm out of
luck.  Mirroring the entire repository is not an option on
machines with less than 6 GB of spare disk.[1]  Transferring the
entire source tree over the network via anoncvs is suboptimal when
all I really want is a few kilobytes of 'CVS' subdirectories.
But I guess it will have to do for now.


[1] When the system is an aging dual PPro or 200MHz Alpha using
SCSI, buying new drives is not practical.

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: Seat-belt for source upgrades from stable to current

[David Schultz]

Thus spake Ruslan Ermilov <[EMAIL PROTECTED]>:
> On Thu, Jan 30, 2003 at 09:58:15PM -0800, David Schultz wrote:
> > I don't know about Steve, but cvsup is the wrong answer for me
> > because it's a mirroring tool and not a version control tool.
> > Among the things I would like to do are:
> > 
> > - Update to a specific version of a specific file from the
> >   repository.
> > 
> > - Generate a diff between two revisions in the repository,
> >   or between a version in the repository and some local
> >   patches of my own.
> > 
> > - View logs for particular files.
> > 
> > I asked the question in hopes that there would be some neat
> > feature of cvsup that mocked up some CVS metadata for me, but
> > since nobody has mentioned any such thing, I guess I'm out of
> > luck.  Mirroring the entire repository is not an option on
> > machines with less than 6 GB of spare disk.[1]  Transferring the
> > entire source tree over the network via anoncvs is suboptimal when
> > all I really want is a few kilobytes of 'CVS' subdirectories.
> > But I guess it will have to do for now.
> > 
> > 
> > [1] When the system is an aging dual PPro or 200MHz Alpha using
> > SCSI, buying new drives is not practical.
> > 
> Well then learn how to use anoncvs?

I use anoncvs.  My question was, given a source or ports tree
without any CVS metadata, is there a way to obtain the tiny
corresponding CVS/Entries and CVS/Repository files without
checking out an entirely new copy of the tree via a busy, often
overloaded, anoncvs server.

Keep in mind that I'm not asking a ``can it be done?'' question.
I've been doing it for years.  Rather, I'm asking, ``can it be
done more efficiently?''  So far, the best answer I've received is
``mirror the repository locally'' (thank you, David), which I
already do on one machine to maintain a local branch and will
consider doing on others.

I have also considered writing a script to parse embedded
$FreeBSD$ tags out of source files and mock up a CVS/Entries and
CVS/Repository based on that information.  Perhaps I will try that
and see if it works passably well.

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: Seat-belt for source upgrades from stable to current

[David Schultz]

Thus spake Giorgos Keramidas <[EMAIL PROTECTED]>:
> On 2003-01-30 21:38, David Schultz <[EMAIL PROTECTED]> wrote:
> > Thus spake Mike Makonnen <[EMAIL PROTECTED]>:
> > > Use the r version of the cvs commands (like cvs rlog and rdiff). They operate on
> > > the repository remotely, so you don't need to have the files checked out localy.
> >
> > That's a pretty good solution, and I use those occasionally.  It
> > would be a perfect solution if there were an 'rupdate', so I don't
> > have to (cd /tmp; cvs co src/file/i/want.c)
> > && cp /tmp/src/file/i/want.c /where/i/want/it
> > && rm -rf /tmp/src
> > all the time.
> 
>   # cd /tmp
>   # cvs -d 'repo magic' export -rHEAD src/file/i/want.c
> 
> Does `cvs export' do the trick for you?

Actually, 'cvs export' does the opposite of what I want to do.
But thank you for mentioning it, because it's a good way to
explain what I'm trying to do, so people don't keep assuming that
my problem is not knowing how to use CVS.

'cvs checkout': create a copy of the sources AND CVS's
administrative directories

'cvs export': just create a copy of the sources

What I want to do: just create the administrative directories

I'm aware that making CVS magically discover what version I have
is a bit much to ask for, so I'm willing to tell it what tag to
use.  Unfortunately, I don't think the facility I want exists, but
I'm going to try to implement a workaround when I have a moment.

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: State of the Union Report (backout request department)

[David Schultz]

Thus spake Wesley Morgan <[EMAIL PROTECTED]>:
> On Fri, 31 Jan 2003, Mike Barcroft wrote:
> 
> > The archives might not be telling the whole story.  A lot of times
> > these things get handled behind closed doors, whether private e-mail
> > or developer-only lists.  Thankfully though, most conflicts *do* get
> > resolved. :)
> 
> I have always LOVED watching the commits and backouts. I find it much more
> exciting to watch the actual development commit by commit, watch the
> brainiacs audit each other, and resolve to the best course. It seems much
> better than the way Linux traditionally did it (although they seem to have
> moved to bitkeeper) and much more like a professional development team.

The problem is that once something is in the tree and someone asks
for a backout, there is a stigma attached to it.  It's as though
the original committer would be admitting that he's wrong by
backing it out, even if his commit is justified.  So he gets
defensive about it (and sometimes everyone else gets annoyed while
their builds fail), and thus a DSW breaks out and people spend
hours of their time being generally nasty and unproductive.
People need to think of backouts as a form of bookkeeping and not
as an admission of error.  They should also try to minimize their
occurrence, i.e. I shouldn't be able to read through the CVS logs
and predict ``uh oh, there's gonna be hell about this...''

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: Seat-belt for source upgrades from stable to current

[David Schultz]

Thus spake Juli Mallett <[EMAIL PROTECTED]>:
> * De: Giorgos Keramidas <[EMAIL PROTECTED]> [ Data: 2003-01-31 ]
>   [ Subjecte: Re: Seat-belt for source upgrades from stable to current ]
> > On 2003-01-30 21:38, David Schultz <[EMAIL PROTECTED]> wrote:
> > > Thus spake Mike Makonnen <[EMAIL PROTECTED]>:
> > > > Use the r version of the cvs commands (like cvs rlog and rdiff). They operate 
>on
> > > > the repository remotely, so you don't need to have the files checked out 
>localy.
> > >
> > > That's a pretty good solution, and I use those occasionally.  It
> > > would be a perfect solution if there were an 'rupdate', so I don't
> > > have to (cd /tmp; cvs co src/file/i/want.c)
> > >   && cp /tmp/src/file/i/want.c /where/i/want/it
> > >   && rm -rf /tmp/src
> > > all the time.
> > 
> > # cd /tmp
> > # cvs -d 'repo magic' export -rHEAD src/file/i/want.c
> > 
> > Does `cvs export' do the trick for you?
> 
> Further, export -d somedir might be useful in this situation to get the
> files where you want them, though sometimes -d does not DWIM, so I'm
> not sure :)

That works when the revision has a symbolic tag associated with
it, but export seems to be picky and won't do numeric revision
numbers of particular files.  I guess I could always go by date.
And yes, -d is picky, too; you can't export into your working
directory.  Still, not bad...

It doesn't address the entire problem I mentioned before, because
I still want to do diffs between local revisions and the
repository.  This thread is just a subthread of the original
thread in which I guess we're talking about how to do a 'cvs
update' without a copy of the repository.  (Sorry Giorgos, your
idea actually works for updates, just not for the original
question I was asking.  I was confused.  ENOSLEEP.)

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: rand() is broken

[David Schultz]

Thus spake Andrey A. Chernov <[EMAIL PROTECTED]>:
> Yes, first value correlation is there, but old formulae have even worse
> effect "The random sequences do not vary much with the seed", as source
> file comments and whole discussion about old RNG bad effects shown. I.e.  
> for different time+PID sequence, especially increased monotonically, like
> in common practice, you'l got the same random sequence with old formulae
> (which can't be called "works fine" because this fine work was the main
> reason for change). So, returning to old formulae is not an option.
> 
> The real problem is not in formulae, but in srand() funclion. This simple
> patch can fix first value correlation, and I plan to commit it, if we all
> agree. I not find better value for NSHUFF right now, but think
> that something like 10 will be enough to fight corellation completely.
> Some generating picture tests needed.

Throwing away the first 10 numbers is probably not good enough to
eliminate randomness with respect to the initial seed.  Knuth
suggests throwing away the first 2000, but he is conservative.[1]
But no amount of throwing away the initial sequence will help as
long as the generator itself doesn't look very random.
Specifically, rand() isn't very interesting in the lower-order
bits, and it spectacularly fails nearly all of Marsaglia's
randomness tests.[2]  If rand() is a concern to someone, they
should either find LCG parameters that produce better behavior,
or research a replacement algorithm.


[1] His PRNG is available from
http://www-cs-faculty.stanford.edu/~knuth/programs/rng.c

[2] http://stat.fsu.edu/~geo/diehard.html (you need ports/lang/f2c)

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: rand() is broken

[David Schultz]

Thus spake Bakul Shah <[EMAIL PROTECTED]>:
> > As I said, I don't know how big a concern this is.  But last time
> > it was enough of a concern to make us keep rand() as it was.
> 
> [I know you are talking about rand() but Mark Murray's
> earlier email about wanting to re-implement random() really
> concerned me so I want to make sure my point gets across]
> 
> Not changing random() was of real concern to me when I was
> doing chip simulations.  ASIC design verification folks won't
> be happy if the rug is pulled out from under them.  In
> general crypto and simulation needs are different and I don't
> trust the crypto guys to look out for the simulation guys!
> 
> I don't care any more if rand() is changed but _please_ leave
> random() alone!  And it would be nice to indicate *why* in
> the source code for the next time this discussion comes up.

If you need guaranteed reproducible random numbers that won't
change from system to system or across libc versions, you need to
roll your own PRNG.  A simple linear congruential generator such
as the original BSD algorithm might look random enough for your
purposes.  But I must admit that I was surprised when Andrey
Chernov pointed out that rand() had been replaced between -CURRENT
and -STABLE; I thought it was simply common knowledge that rand()
was broken, and nobody was interested in fixing it.

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: rand() is broken

[David Schultz]

Thus spake Andrey A. Chernov <[EMAIL PROTECTED]>:
> On Sat, Feb 01, 2003 at 23:06:50 -0800, Kris Kennaway wrote:
> > FreeBSD's rand() implementation has been broken for the past 23
> > months, since the following commit:
> 
> > i.e. the first value returned from rand() is correlated with the seed
> > given to srand().  This is a big problem unless your seed is randomly
> > chosen over its entire integer range.  I noticed this because awk
> > exhibits the same problem, and the script seeds the generator with a
> > PID.  The script works fine under 4.x since the rand() implementation
> > does not have this "feature".
> 
> Yes, first value correlation is there, but old formulae have even worse
> effect "The random sequences do not vary much with the seed", as source
> file comments and whole discussion about old RNG bad effects shown. I.e.  
> for different time+PID sequence, especially increased monotonically, like
> in common practice, you'l got the same random sequence with old formulae
> (which can't be called "works fine" because this fine work was the main
> reason for change). So, returning to old formulae is not an option.
> 
> The real problem is not in formulae, but in srand() funclion. This simple
> patch can fix first value correlation, and I plan to commit it, if we all
> agree. I not find better value for NSHUFF right now, but think
> that something like 10 will be enough to fight corellation completely.
> Some generating picture tests needed.

The correlation is still present with your patch and NSHUFF set to
10.  For instance, try seeding rand() with contiguous monotonically
increasing integers, and observe the four lowest-order bits.

Just for the heck of it, I ran Marsaglia's tests on the rand()
implementation in -CURRENT.  The arc4random() implementation
passed with flying colors as expected, whereas rand() seems to
have some slight defects, particularly in the lowest and highest
order bits.  When I looked at rand()'s behavior with respect to
different seeds, it failed miserably, both with and without your
patch.  The results are available at
http://www.csua.berkeley.edu/~das/marsaglia/

I'm not necessarily advocating changing the algorithm at all,
given that it's well known that many rand() implementations are
not very random.  But I also don't buy the argument that ``rand()
should never ever change.''  If someone wants to do the work to
improve the algorithm, that's fine with me.  David Wagner has
collected some links on randomness that might be helpful:
http://www.cs.berkeley.edu/~daw/rnd/index.html

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: Final fix for correlation problem (was Re: rand() is broken)

[David Schultz]

Thus spake Thomas David Rivers <[EMAIL PROTECTED]>:
> I'm afraid I don't understand the fix... and how it
> seems to affect the historical behaviour of srand()/rand().
> 
> How does it address the understanding that if I use
> srand(28), I will get exactly the same sequence of
> numbers srand(28) produced yesterday, last week, 
> last year?
> 
> I have worked with programs that depend on exactly
> that behavior.
> 
> That is,  given the same input seed - I expect
> to see the same "random" sequence again.  
> 
> This requirement would seem to indicate that changing
> srand()/rand() isn't really possible...  And, also,
> I believe, why random() was introduced...
> 
> Please, oh please, don't change that behavior in 
> srand()/rand().

There are two sides to this argument.  One side says that rand()
should always produce the same sequence for a given seed no matter
what.  This argument is bogus, because if you need exact
reproducability across all platforms for all time, you should be
using your own PRNG.

The other side says that rand() should be a reasonably good RNG
for most applications.  It would be nice if things worked out that
way, but there are enough broken rand() implementations out there
that most people don't rely upon them.  So I'd be inclined to call
this argument bogus as well, except that I know someone who has
been burned by a bad rand() while trying to test a randomized
graph algorithm in C.

My final thoughts are that having a better rand() implementation
is a Good Thing, but it isn't that big a deal, and it certainly
isn't worth all of this bickering.

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: rand() is broken

[David Schultz]

Thus spake Andrey A. Chernov <[EMAIL PROTECTED]>:
> On Sun, Feb 02, 2003 at 16:26:39 -0800, David Schultz wrote:
> > 
> > The correlation is still present with your patch and NSHUFF set to
> > 10.  For instance, try seeding rand() with contiguous monotonically
> > increasing integers, and observe the four lowest-order bits.
> 
> Since you already have running framework for that, could you please 
> test it with NSHUFF picked from 100-2000 range?

Rather than me showing you more semi-meaningful numbers from
Marsaglia's tests, why don't you look at the following sequence,
which I get by taking the lowest four bits of the 201st number in
the rand() sequence for seeds of (0, 1, 2, ...).

f c 9 6 2 f c 8 5 2 e b 8 4 1 e b 7 4 1 d a 7 3 0 d 9 6 3 f c 9 
6 2 f c 8 5 2 e b 8 4 1 e b 7 4 1 d a 7 3 0 d 9 6 3 f c 9 6 2 f 
c 8 5 2 e b 8 4 1 e a 7 4 1 d a 7 3 0 d 9 6 3 f c 9 6 2 f c 8 5 
2 e b 8 4 1 e a 7 4 1 d a 7 3 0 d 9 6 3 f c 9 5 2 f c 8 5 2 e b 
8 4 1 e a 7 4 1 d a 7 3 0 d 9 6 3 f c 9 5 2 f c 8 5 2 e b 8 4 1 
e a 7 4 0 d a 7 3 0 d 9 6 3 f c 9 5 2 f c 8 5 2 e b 8 4 1 e a 7 
4 0 d a 7 3 0 d 9 6 3 f c 9 5 2 f b 8 5 2 e b 8 4 1 e a 7 4 0 d 
a 7 3 0 d 9 6 3 f c 9 5 2 f b 8 5 2 e b 8 4 1 e a 7 4 0 d a 7 3 
0 d 9 6 3 f c 9 5 2 f b 8 5 2 e b 8 4 1 e a 7 4 0 d a 6 3 0 d 9 
6 3 f c 9 5 2 f b 8 5 2 e b 8 4 1 e a 7 4 0 d a 6 3 0 d 9 6 3 f 

Notice that 'f c 9' repeats in regular intervals and is always
followed by a 5 or 6.  There is a similar pattern for 'e a 7'.  I
think this pretty much demonstrates that the algorithm isn't good
enough to generate high-quality randomness with respect to
different seed values.  I'm not suggesting that it absolutely must
be replaced, since most rand() implementations aren't very good in
the first place, but I'm pointing out that to do a good job of
fixing it once and for all is harder than you might think.

The program to generate this sequence follows.  I ran this on a
several-month-old -CURRENT system with the new generator, but
without your latest patches.

#include 
#include 

int main() {
int i, j;

for (i = 0; ; i++) {
srand(i);
for (j = 0; j < 200; j++)
rand();
printf("%x ", rand() & 0x0f);
if (i % 32 == 31)
putchar('\n');
}
}

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: rand() is broken

[David Schultz]

Thus spake Eric Hodel <[EMAIL PROTECTED]>:
> David Schultz ([EMAIL PROTECTED]) wrote:
> 
> > Rather than me showing you more semi-meaningful numbers from
> > Marsaglia's tests, why don't you look at the following sequence,
> > which I get by taking the lowest four bits of the 201st number in
> > the rand() sequence for seeds of (0, 1, 2, ...).
> > 
> 
> f c 9 6 2 f c 8 5 2 e b 8 4 1 e b 7 4 1 d a 7 3 0 d 9 6 3
> f c 9 6 2 f c 8 5 2 e b 8 4 1 e b 7 4 1 d a 7 3 0 d 9 6 3
> f c 9 6 2 f c 8 5 2 e b 8 4 1 e a 7 4 1 d a 7 3 0 d 9 6 3
> f c 9 6 2 f c 8 5 2 e b 8 4 1 e a 7 4 1 d a 7 3 0 d 9 6 3
> f c 9 5 2 f c 8 5 2 e b 8 4 1 e a 7 4 1 d a 7 3 0 d 9 6 3
> f c 9 5 2 f c 8 5 2 e b 8 4 1 e a 7 4 0 d a 7 3 0 d 9 6 3
> f c 9 5 2 f c 8 5 2 e b 8 4 1 e a 7 4 0 d a 7 3 0 d 9 6 3
> f c 9 5 2 f b 8 5 2 e b 8 4 1 e a 7 4 0 d a 7 3 0 d 9 6 3
> f c 9 5 2 f b 8 5 2 e b 8 4 1 e a 7 4 0 d a 7 3 0 d 9 6 3
> f c 9 5 2 f b 8 5 2 e b 8 4 1 e a 7 4 0 d a 6 3 0 d 9 6 3
> f c 9 5 2 f b 8 5 2 e b 8 4 1 e a 7 4 0 d a 6 3 0 d 9 6 3
> f 
> 
> > Notice that 'f c 9' repeats in regular intervals and is always
> > followed by a 5 or 6.  There is a similar pattern for 'e a 7'.  I
> > think this pretty much demonstrates that the algorithm isn't good
> > enough to generate high-quality randomness with respect to
> > different seed values.  I'm not suggesting that it absolutely must
> > be replaced, since most rand() implementations aren't very good in
> > the first place, but I'm pointing out that to do a good job of
> > fixing it once and for all is harder than you might think.
> 
> A littele modification shows just how similar these sequences are :)

Yeah, I saw the periodicity when I asked less(1) to select
particular subsequences.  I guess it's a bit more impressive when
you select the right modulus.  ;-)

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: rand() is broken

[David Schultz]

Thus spake Andrey A. Chernov <[EMAIL PROTECTED]>:
> On Mon, Feb 03, 2003 at 21:40:20 -0800, David Schultz wrote:
> I don't try to make rand() good for high-quality pseudo-randomness,
> because it can be done by price of speed and, more important, big state
> size. Due to rand_r() restriction state size can be one word only, so we
> can choose rand() algorithm only from those which pass this
> restrictions.

You can do better than the present generator with 32 bits of state.
See the following page by Neal Wagner (not to be confused with David Wagner):
http://www.cs.utsa.edu/~wagner/laws/rng.html
The section on LCGs suggests that the multiplier FreeBSD uses (7^5)
is not particularly good, and points out some better values suggested
by Knuth.  I can't find the original discussion in TAOCP vol. 2, but
I take N. Wagner's word that the numbers have been blessed by the holy
hand of Knuth.  I'm sure you can find more information if you search
the literature.  I apologize, but I don't have time to help you right
now, and rand() isn't really a concern to me.

> Returning to current algorithm, I am interested in good NSHUFF value in 
> the range 100-2000. Do you have any findings there?

Well, if 0 doesn't work, and 10 doesn't work, and 100 doesn't
work, then I'm not too hopeful about 2000.  I appeal to Asimov's
zero, one, infinity law.

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: Compiling with high optimization?

[David Schultz]

Thus spake Ray Kohler <[EMAIL PROTECTED]>:
> Has anyone tried building world/kernel with high optimizations (-O2,
> -O3) recently? What breaks? (Booby prize to whoever says "common sense"
> ;) I last tried it quite a few months ago and the resolver died on me,
> don't know what else. I'm not really thinking of running like that, but
> I am curious about others' experiences.

First, let me answer the question that you really meant to ask but
forgot to, namely, ``How much of a performance difference does -O3
make over -O for the kernel/world?''  The answer is ``very little,
for most purposes.''  So if you do use higher optimization levels,
at least do a little benchmarking to make sure it was worth it.
To answer your second question, higher optimization levels usually
work, but there *will* be new bugs.  I know of several libc
problems due to -fstrict-aliasing, and I'm told that the inline
assembly for TCP checksumming can still break.

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: Compiling with high optimization?

[David Schultz]

Thus spake Terry Lambert <[EMAIL PROTECTED]>:
> David Schultz wrote:
> > Thus spake Ray Kohler <[EMAIL PROTECTED]>:
> > > Has anyone tried building world/kernel with high optimizations (-O2,
> > > -O3) recently? What breaks? (Booby prize to whoever says "common sense"
> > > ;) I last tried it quite a few months ago and the resolver died on me,
> > > don't know what else. I'm not really thinking of running like that, but
> > > I am curious about others' experiences.
> > 
> > First, let me answer the question that you really meant to ask but
> > forgot to, namely, ``How much of a performance difference does -O3
> > make over -O for the kernel/world?''  The answer is ``very little,
> > for most purposes.''  So if you do use higher optimization levels,
> > at least do a little benchmarking to make sure it was worth it.
> 
> Actually, failure to use optimization suppresses some compilation
> warnings, particularly those which normally print from using some
> variables without initializing them.

I think you're thinking of dataflow analysis, which I believe gcc
does with -O and higher optimization levels.  So unless you're
using -O0, I would expect that you'd get all the warnings you
want.

> There are a number of places, particularly on non-i386 platforms,
> where optimization actually doesn't work.  I think that's why it
> was turned off for the libc compilation, and why the bug crept in.
> 
> It's probably useful to compile world with optimization occasionally,
> to make compilation-time detectable bugs like that to show up, but, as
> you point out, it'd probably be a *bad* idea to actually use the
> resulting code, at least until after the next GCC import, which will
> supposedly fix the Alpha optimizer.

Yes, the possibility of being bitten by compiler bugs is certainly
higher with higher optimization levels.  Alpha with -O2 seems to
have been broken for years, and I have seen strange things happen
on IA64 as well.  But the i386 code generators have received much
wider testing and debugging, so there is somewhat less danger there.

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: Compiling with high optimization?

[David Schultz]

Thus spake Terry Lambert <[EMAIL PROTECTED]>:
> David Schultz wrote:
> > Thus spake Terry Lambert <[EMAIL PROTECTED]>:
> > > Actually, failure to use optimization suppresses some compilation
> > > warnings, particularly those which normally print from using some
> > > variables without initializing them.
> > 
> > I think you're thinking of dataflow analysis, which I believe gcc
> > does with -O and higher optimization levels.  So unless you're
> > using -O0, I would expect that you'd get all the warnings you
> > want.
> 
> See the thread "Re: tmpfile breakage on setuid executables".
> 
> Kris accidently introduced a bug that had to do with whether or
> not a variable was used before it was initialized.  The compiler
> didn't complain when he checked it before committing it because
> optimization was off by default; it should have complained, e.g.:
> 
> "x.c:9:warning: `foo' might be used uninitialized in this function"

Thanks, I saw that.  I'm just pointing out that all you need is -O
to get that warning:

das@bloody-mary:~> cat foo.c
int main() {
int foo;

return foo;
}
das@bloody-mary:~> gcc -o foo foo.c -Wall -O
foo.c: In function `main':
foo.c:2: warning: `foo' might be used uninitialized in this function


This is one of those things that gcc3 does pretty well, but Sun's
javac does very badly.  First of all, in Java, the above warning
is considered an error (except *maybe* in JDK 1.4, which is broken
and slow on IA64 so I can't use it).  Second, the dataflow
analysis in javac is braindead, so the compiler is almost always
wrong.  When I get the same warning from gcc, it's usually right
unless interprocedural analysis would be required.


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: Compiling with high optimization?

[David Schultz]

Thus spake Adrian Chadd <[EMAIL PROTECTED]>:
> On Sat, Feb 08, 2003, David Schultz wrote:
> 
> > Yes, the possibility of being bitten by compiler bugs is certainly
> > higher with higher optimization levels.  Alpha with -O2 seems to
> > have been broken for years, and I have seen strange things happen
> > on IA64 as well.  But the i386 code generators have received much
> > wider testing and debugging, so there is somewhat less danger there.
> 
> Yet squid under i386 freebsd is .. well, finds -O bugs in gcc.
> We gave up trying -O under FreeBSD a long time ago. :-)

The last time someone told me, ``gcc -O is broken'', it turned out
that they were doing some stack fiddling, and gcc's optimizations
broke their faulty assumptions.  On the other hand, I'm sure gcc -O
does have bugs.  Do you have an example snippet that gets miscompiled?

> (note: I've seen better performance gains by telling gcc exactly what
> CPU you have over -O65536 ..)

Strangely, gcc in FreeBSD 5.0 actually generates *slower* code
when compiling for more recent architectures than when compiling
for a 386.  I don't know whether that is a bug in gcc or whether
gcc is using some fancy feature like SSE that the kernel handles
poorly on context switches.  I think there was some discussion on
the lists about it earlier.

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: Compiling with high optimization?

[David Schultz]

Thus spake Marcin Dalecki <[EMAIL PROTECTED]>:
> David Schultz wrote:
> 
> >Strangely, gcc in FreeBSD 5.0 actually generates *slower* code
> >when compiling for more recent architectures than when compiling
> >for a 386.  I don't know whether that is a bug in gcc or whether
> >gcc is using some fancy feature like SSE that the kernel handles
> >poorly on context switches.  I think there was some discussion on
> >the lists about it earlier.
> The reason is that the optimization done by GCC are ill balanced.
> All the scheduling of instractions and what a not - which would be
> fine on a micro scope level is causing so much higher pressure
> on the CPUs caches that the code is actually loosing.

Interesting.  So they redid the code generation for gcc 3 and
their new tricks backfired.  But at least they fixed the
completely braindead things gcc 2.9x was doing with alignment,
floating point, and combinations thereof, and they got the
compiler to do more reasonable things on ia64.  Any idea when they
will have fixed the i386 anti-optimizations?

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: MSDOSFS wastes 256k when nothing is mounted!

[David Schultz]

Thus spake Poul-Henning Kamp <[EMAIL PROTECTED]>:
> Somebody: please fix so this doesn't suck.

Does msdosfs even have an active maintainer?  There seem to be
about half a dozen PRs open against it, one of which is a
semi-obvious 4-line patch I submitted last April.

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: Best method to produce patches?

[David Schultz]

Thus spake David Leimbach <[EMAIL PROTECTED]>:
> I am about to try to make some changes to FreeBSD current...
> 
> Should I begin to use read-only CVS instead of CVSup for this work or 
> is it possible to generate diffs based on CVSup'd sources?
> 
> What is the recommend method to use for playing with the source?
> 
> I already found a small change in libc that should probably get 
> committed but I want to generate the patch properly for everyone's 
> approval.

The best thing to do is to have a local copy of the entire
repository, synced via cvsup.  If you have multiple machines, you
can even run a cvsup server on one of them, and sync them all from
that.

On older hardware that lacks sufficient disk space for the entire
repo, I use anoncvs, but that's much more annoying. You need to
hack up CVS/Entries manually to add and delete files, for
instance.

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: PLEASE TEST - ATA driver patch (ATANG)...

[David Schultz]

Thus spake Soeren Schmidt <[EMAIL PROTECTED]>:
> I've prepared a patch that brings the ATA driver to the next level.
> 
> This is mainly to prepare for other platforms which demanded
> some changes to the driver infrastructure.
> 
> It also fixes lots of outstanding problems and adds support several
> new chipsets including SiS (thanks to Christoph Kukulies for 
> sponsoring a brand new SiS648 based board making that possible!)
> 
> Please test it out and remember that this is WIP, so use protective 
> glasses and rubber gloves, you have been warned.

Cool.  I'll try these patches out this weekend.  Is there a chance
you could integrate the other half of the patch I sent you in
January---the part that sends a FLUSH CACHE command to all disks
at shutdown time?  This measure should ensure that disks with
large caches write out all data before the power to them is cut.
Or are you working on a better way of solving this problem?

There's also the problem jjramsey reported where some (perhaps
broken) hardware required a small delay following the completion
of an IDENTIFY command before the correct data are available to
read, but that problem doesn't concern me directly.

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: cvs commit: src/lib/libc/stdlib rand.c

[David Schultz]

Thus spake Kris Kennaway <[EMAIL PROTECTED]>:
> I think we should commit this patch (to -current) and fix all the
> problems that pop up.  For example, it's used in awk (which started
> this set of changes), and in some of the XFree86 libraries.
...
> +__warn_references(rand_r,
> + "warning: rand_r() does not produce high-quality random numbers and should not 
>generally be used");

Many programmers who use rand() are aware that it isn't very good,
but don't care for their particular application.  For instance,
for games or for randomized backoff in network protocols, you
might just want a sequence of numbers that looks kinda random, and
you don't care that there happens to be a pattern in the
lowest-order bits that you see only if you look carefully.  rand()
isn't like gets() because it's nearly impossible to write a robust
program using gets().

It might make sense to put in the warning just to check whether
someone used rand() when they really wanted cryptographic-quality
randomness, but people would probably get annoyed if the next
release of FreeBSD nagged them about every use of rand().

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: cvs commit: src/lib/libc/stdlib rand.c

[David Schultz]

Thus spake Kris Kennaway <[EMAIL PROTECTED]>:
> On Mon, Feb 17, 2003 at 04:40:48PM +1100, Tim Robbins wrote:
> 
> > I disagree. It's safe to use rand() in games and in certain kinds of
> > simulations when you don't care that the distribution isn't quite
> > uniform, or when you prefer speed over quality. I don't think rand()
> > needs a warning message like gets() &c. because it's not as dangerous.
> 
> The problem is that there are a number of applications that use it
> when they should not.  I've given examples of two of them, and there
> are probably lots of others I haven't noticed.  For example, I just
> checked, and libICE appears to use rand() for cookie generation.  This
> is completely bogus, and insecure.
> 
> Note that I was only suggesting this patch be committed to -current
> for purposes of finding out what these applications are, and fixing
> them as appropriate.

Then how about wrapping the warning in an #ifdef, so people who
want to find inappropriate uses of rand() can do so for as long as
they want, and everyone else who uses -CURRENT is not affected?

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: devastating 5.0R crash

[David Schultz]

Thus spake Paul A. Mayer <[EMAIL PROTECTED]>:
> I'm presently running fsck_ffs from a live cd.  It fsck'ed / with some 
> complaints about an unreadable sector.  It's now on /var and reporting 
> vast, vast numbers of sectors as unreadable, with "UNEXPECTED SOFT 
> UPDATE INCONSISTENCY".  I've not visited /usr yet. I hope my disk (with 
> less than 800 hours of service) hasn't been somehow physically borked. 
> (The windows partition boots and runs seemingly fine.)

If many (say, more than a track's worth) of sectors are unreadable
(e.g. you get a ``medium error''), your hard disk is probably
failing.

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: cvs commit: src/lib/libc/stdlib rand.c

[David Schultz]

Thus spake Jacques A. Vidrine <[EMAIL PROTECTED]>:
> On Mon, Feb 17, 2003 at 04:40:48PM +1100, Tim Robbins wrote:
> > I disagree. It's safe to use rand() in games and in certain kinds of
> > simulations when you don't care that the distribution isn't quite
> > uniform,
> 
> Safe, maybe.  But I think it still shouldn't be used.
> See my posting of two years ago:
> 
> http://groups.google.com/groups?&selm=97b83t%2414q3%241%40FreeBSD.csie.NCTU.edu.tw&rnum=1
> >
> 
> BTW, I don't care if linking a program with rand() gives an obnoxious
> warning or not.  Just pointing out that rand() is less useful than it
> might seem.

Yes, as with most interfaces, there are ways to use rand()
incorrectly.  But there also exist ways to use it correctly.
That's why people would get annoyed if their programs that
have compiled for years start coughing up spurious warnings.
rand() is not like gets(); it is almost impossible to use
the latter in a robust program.  On the other hand, I like
the idea of people going out and fixing all the programs that
use rand() incorrectly.

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: Posix testsuites?

[David Schultz]

Thus spake David Leimbach <[EMAIL PROTECTED]>:
> I have been looking into helping with the C99 conformance stuff and I wondered if 
>the 
> following would be helpful?
> 
> http://posixtest.sourceforge.net/
> 
> I am sure some of you knew about this... I guess I wonder if a link on the C99 web 
>page
> is appropriate under resources and links.

The list you want is standards@.

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: -fno-builtin world breaks in gperf

[David Schultz]

Thus spake Dag-Erling Smorgrav <[EMAIL PROTECTED]>:
> /usr/obj/usr/src/i386/usr/lib/libstdc++.so: undefined reference to `fabsl'
> *** Error code 1
> 
> Our libm doesn't seem to support long double at all, yet our libstdc++
> requires long double support.  It seems to correctly detect the
> absence of a "real" fabsl() and the presence of the gcc builtin, but
> then goes on to use the fabsl() instead of __builtin_fabsl() in at
> least one instance (src/contrib/libstdc++/libmath/stubs.c).  There's a
> similar problem with sqrtl().

Various people are working on C99 conformance issues right now.
Implementing long double versions of all library functions is
likely to be one of the more time-consuming problems.  Details are
available at http://www.FreeBSD.ORG/projects/c99/ .

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: background fsck deadlocks with ufs2 and big disk

[David Schultz]

Thus spake Martin Blapp <[EMAIL PROTECTED]>:
> I just wanted to tell that I can deadlock one of my current boxes
> with a ufs2 filesystem on a 120GB ATA disk. I can reproduce
> the problem. The background fsck process hangs some time at the
> same place always at the same place, sometimes the box freezes
> after some time.
> 
> The same box works fine with ufs1.

IIRC, Kirk was trying to reproduce this a little while ago in
response to similar reports.  He would probably be interested
in any new information.

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: Any ideas why we can't even boot a i386 ?

[David Schultz]

Just out of curiosity, is your agenda to convince everyone to nix
386 support altogether or to fix 386 support?  I'm not against
either, although I consider the latter goal to be a bit silly.

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: sparc64 tinderbox failure

[David Schultz]

Thus spake Doug Barton <[EMAIL PROTECTED]>:
> I made the suggestion in another forum that we create subscription lists
> for these tinderbox messages so that those interested could see them, and
> those not interested would not need to filter them out.
> 
> Is anyone else interested in doing it that way? Or is this a solution
> looking for a problem?

I think most people who track -CURRENT are subscribed to current@
precisely because they want to know when things break.  Similarly
for -STABLE, but that's a moot point because -STABLE doesn't break
very often.  For people who don't agree with this, there's always
procmail; the reports aren't all that frequent that filtering is
some sort of burden.  I do, however, like the idea of
consolidating them and providing more concise summaries.

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: Any ideas why we can't even boot a i386 ?

[David Schultz]

Thus spake Terry Lambert <[EMAIL PROTECTED]>:
> John Baldwin wrote:
> > I doubt the usefulness of this.  i386 kernels were just accidentally
> > broken for almost a month and a half without anyone noticing.
> 
> People who build embedded devices that need to be supported in
> the field, and want to worry about their software, and not the
> platform it runs on, don't use -current, FWIW.

Moot point.  People who build embedded devices have separate,
usually modern, machines for building their kernels.

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: Plea for base system trim

[David Schultz]

Thus spake Subscriber <[EMAIL PROTECTED]>:
> Would the powers that be please consider removing sendmail,
> bind and openssl from the base system, as was done for perl
> with 5.0?

Please don't restart this flamewar.  When we have a better
installer, then it may be possible in the future to select between
multiple mailers (for example), but nobody is going to rip out
widely-used functionality because someone has a pet peeve.  (And
yes, this *is* just a pet peeve; if it were not, you'd see people
complaining about ntpd, ipfilter, pnpinfo, etc. instead.)  Check
the archives.

As far as security updates are concerned, keep in mind that base
system components are actually *better* supported by the FreeBSD
security officer than are ports.  If you really want the ports
version of something, put the appropriate NO_* option in your
make.conf and install the ports version instead.

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: 4.8-PRE upgrade to FreeBSD 5 errors

[David Schultz]

Thus spake Axel Gruner <[EMAIL PROTECTED]>:
> Checking to see if your booted Kernel is fresh enough..
> /usr/obj/usr/src/bin/sh/sh -c 'echo "Testing installed kernel fpr new
> sigaction(2) syscall"' pid 60424/sh),uid0: exited on signal 12 (core
> dumped) Bad system call (core dumped)
> ***Error code 140
> Stop in /usr/src
> ***Error code 1
> Stop in /usr/src

You need to do things in the order specified in UPDATING.  This
message is the result of a sanity check that verifies that you're
running the new kernel before you try to install a new world.

> Well, ok, i read the first line and booted my machine, so that the new
> Kernel can boot. Also if the machine will not boot correctly i will fall
> to single user mode. This happend. Back in single user mode i did a
> second "installworld", and, surprise surprise, it worked without errors.
> Ok, i rebooted the machine.
> 
> So, now on bootup i get these messages:
> 
> link_elf: symbol lminor undefined
> link_elf: symbol splhigh undefined

You seem to be trying to load outdated modules.

> And if i want to login, i get these:
> 
> login: $MYNAME
> login:in openpam_load_module(): no pam:skey.so found
> login:pam_start(): failed to load module
> 
> I cant login.
> So, does this mean, some modules are missing? And how do i get the
> system back running perfectly? 

You should still be able to log in as root, in single user mode if
nothing else.  Doing a proper installworld/mergemaster should
rectify this problem.

> And, a third error message appears on boottime:
> 
> Warning: userland calling deprecated sysctl, please rebuild world
> 
> Well, i booted the machine back in single user mode, mount all the
> devices, and did a buildworld, buildkernel, installkernel, mergemaster
> and installworld again. The result? The same errors, plus something
> other i did not recognize the first time (but was there):

Did the installworld actually work this time, or did you get the
error message you mentioned above?  Check the modification dates
on your /bin and /sbin binaries to see what old crud you might be
running.

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: 4.8-PRE upgrade to FreeBSD 5 errors

[David Schultz]

Thus spake Axel Gruner <[EMAIL PROTECTED]>:
> > > link_elf: symbol lminor undefined
> > > link_elf: symbol splhigh undefined
> > You seem to be trying to load outdated modules.
> 
> Hmm, well, so what to do here? I switched of any module in loader.conf.

THe location of modules moved between 4.x and 5.0, so you may have
to clear out the old ones (/modules).  However, I haven't had any
problems with the kernel finding the wrong ones myself, so maybe
this has something to do with the other failures you experienced.

> Yes, in single user mode, no problem. 
> I did a second run there. Buildworld, buildkernel, installkernel,
> mergemaster, installworld... Same result. Same errors. 
[...]
> Yes, it worked that time. 
> The problem is, if i try to run /usr/sbin/mergemaster after the
> installworld, i get an error: cant cd to /usr/src/etc

Well, then mergemaster didn't work, and that explains why you're
still getting errors. ;-)  Make sure you actually have a
/usr/src/etc.

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: possible unionfs bug

[David Schultz]

On Sun, Jul 20, 2003, Divacky Roman wrote:
> Hi,
> 
> I might be wrong but this:
> 
> free(mp->mnt_data, M_UNIONFSMNT);   /* XXX */
>   mp->mnt_data = 0;
>   
> seems to me wrong and might cause crashes etc.
> am I correct or wrong?
> 
> its from union_vfsops.c:384

What's wrong with it?  It's just freeing the memory it allocated
earlier.  FFS does the same thing.
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: maildir with softupdates

[David Schultz]

On Wed, Jul 23, 2003, Attila Nagy wrote:
> Hello,
> 
> Is this statement still valid?
> 
> "ext3 is unsafe for maildir, and with softupdates, so is ffs."
> http://www.irbs.net/internet/postfix/0202/0358.html

The statement is FUD; this is a topic that mailer people love to
complain about.  It's only true if your MTA doesn't call fsync()
when it wants to guarantee that the file it just wrote is on
stable storage.  Most filesystems don't guaranteed 100%
synchronous semantics for regular data unless you ask for them
explicitly, due to the performance implications.  The statement
you quote used to be true for ext3 due to an inadequacy in its
fsync() implementation, and I'm not sure if that was ever fixed.
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: vfprintf() has a 4096-byte memory leak?

[David Schultz]

On Sat, Aug 02, 2003, Ryan T. Dean wrote:
> "Poul-Henning Kamp" wrote:
> >In message <3F2B9C59.3060209 at cytherianage.net 
> >>, "Ryan T. 
> >Dean" writes:
> >>/Hey all-
> /> >/I was doing some app debugging tonight, and noticed what appears 
> to /> >/be a memory leak in vfprintf().
> /> 
> >This is probably the buffer which stdio uses for all I/O.
> >
> >Try calling 
> > setbuf(stdout, NULL);
> > setbuf(stderr, NULL);
> >
> >as the very first thing in your program, and you will probably see
> >that it does not allocate the 4k buffer, you may also be able to
> >measure the performance change due to this.
> >
> >In other words, what you see is perfectly normal, and to be expected,
> >but if it is a problem for you, the above is the workaround.
> 
> Aha.  setbuf(stdout, NULL); does prevent the buffer from being allocated.
> However, in the case of stdout and stderr, if you don't setbuf() it to 
> null, a buffer is malloc'd.  The corresponding free() is in fclose.  So, if 
> you [f]printf() to stdout/stderr, and fclose(stdout/stderr), you are fine.  
> If, however, you don't know that buffers are being allocated for 
> stdin/stdout and don't fclose() or setbuf() to NULL, a 4096-byte chunk of 
> memory is allocated and never freed.  For shits and giggles, I checked a 
> DeadRat 8 box - no buffering by default.  I guess the only reason I'm 
> worried is the
> potential number of programs in the ports tree originally written on a 
> system
> where stdout/stderr behave differently, or people (like myself) who didn't
> have a clue any sort of output buffering was enabled on stdout/stderr, and 
> as a result have memory leaks.  If the porter did their job, it shouldn't 
> be an issue (was caught, patched, and the patch submitted upstream), but, 
> then, we never know, right?

stdio's buffering routines make a one-time allocation for their
buffers, and this memory does not get freed because it can
potentially get used during the entire time the program is
running.  This isn't a bug.  See the archives for details.
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: warnpassword and warnexpire in 5.1 login.conf

[David Schultz]

On Sat, Aug 02, 2003, Mats Larsson wrote:
> 
> Hello!
> 
> Tried this question to the questions list with no response, perhaps
> current is the correct list for questions related to 5.1-RELEASE??
> 
> I am trying to use warnexpire and warnpassword in login.conf but with no
> result, are the warnexpire and warnpassword still used in 5.1 or have they
> been superseded with a PAM module in the same way as minpasswordlen and
> minpasswordcase??

They're part of the pam_unix PAM module now, but they should still
work.  I tried them a few months ago, and I don't remember any
special steps being necessary.  You ran cap_mkdb(1), right?
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: warnpassword and warnexpire in 5.1 login.conf

[David Schultz]

On Tue, Aug 05, 2003, Mats Larsson wrote:
> Sure, run cap_mkdb on every edit on login.conf
> 
> The values im trying to use there are the following:
> :warnexpire=28d:\
> :warnpassword=14d:\
> 
> And with pw i use the following to test with: (also with -e option)
> pw usermod user -p +10d
> 
> The only thing im getting now is i warning in messages when i try to login
> into a locked account.
> 
>   Aug  5 12:14:39 marvin sshd[55256]: error: PAM: user accound has expired

This looks reasonable.

> And the following varning when password is old:
>   Aug  5 12:27:38 marvin sshd[55386]: error: PAM: OK
>   Aug  5 12:27:40 marvin sshd[55390]: fatal: PAM: chauthtok not supprted with 
> privsep
> 
> Is there perhaps a better PAM way of doing this things now??

Hmm... Apparently you can't change an expired password with a
privilege-separated OpenSSH.  I don't know whether that can be
fixed, but perhaps des@ has some insight.
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: 64 bit quantities in statfs ?

[David Schultz]

On Mon, Aug 18, 2003, Matthew Dillon wrote:
> As part of the DragonFly effort we are going to increase the 
> mount path limit from 80 chars to 1024.
> 
> This will change the statfs structure.  I thought I would adopt the
> 64 bit changes that 5.x has made to keep things synchronized.
> 
> Except... there don't appear to be any 64 bit changes to struct
> statfs in 5.x.  Am I missing something here?  Is there an 'nstatfs'
> structure that I have not seen?  The following probably need to be 64
> bit entries:
> 
>   f_blocks
>   f_bfree
>   f_bavail
>   f_files
>   f_ffree
>   f_syncwrites
>   f_asyncwrites
>   f_syncreads
>   f_asyncreads

Yep, looks broken.  In the POSIX standard, the functionality of
statfs() is provided by statvfs(), so implementing the latter may
be a way out that doesn't involve breaking any ABIs.
Unfortunately, the implementation of statvfs() in FreeBSD 5.X
relies on statfs(), so it's broken despite having the correct
field sizes.  Moreover, statvfs() is underspecified such that it
isn't actually required to do anything useful...
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: 64 bit quantities in statfs ?

[David Schultz]

On Mon, Aug 25, 2003, Garrett Wollman wrote:
> < said:
> 
> > Yep, looks broken.  In the POSIX standard, the functionality of
> > statfs() is provided by statvfs(), so implementing the latter may
> > be a way out that doesn't involve breaking any ABIs.
> 
> statfs() is a lot more useful interface than statvfs().  I'd like to
> keep statvfs() as the ``standard'' interface, rather than extending it
> to cover all of the information that statfs() has.
> 
> In order to grow statfs() we need to rev libc.  It might be
> appropriate to do that in the 5.2 time frame, if we are still
> anticipating that 5.2 will be the -stable crossover point.  RE team?

We can't fix statfs() until 6.0.  statvfs() is potentially just as
useful, and it doesn't suffer from the same problems.  Despite
being underspecified by the standard, many systems, e.g. Solaris,
make it convey at least as much information as statfs().
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: __fpclassifyd

[David Schultz]

On Fri, Aug 29, 2003, Christoph Kukulies wrote:
> 
> I did a cvsup and rebuild of world and ports, portupgrade,
> reinstalled mod_php4, apache and still get this
> sh apache.sh start
> Syntax error on line 237 of /usr/local/etc/apache/httpd.conf:
> Cannot load /usr/local/libexec/apache/libphp4.so into server: 
> /usr/local/libexec/apache/libphp4.so: Undefined symbol "__fpclassifyd"
> 
> I'm pretty sure that my binaries are in sync. A dangling libraries somewhere?

Something is probably out of date, but I don't know whether it's
Apache or some library that Apache depends on.  It seems like
you're loading a 4.X libc on a 5.X system.  Try running 'ldd' on
the Apache binary (not the wrapper script), and do the same for
all the libraries that ldd mentions.
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

ATAng panic: ata_dmasetup: transfer active on this device!

[David Schultz]

I have a machine that panics reliably within ten minutes of
operation with ATAng.  It subsequently locks up, so I can't obtain
a dump, and I get a small amount of random filesystem corruption
upon rebooting.  Everything was fine with ATAog.

I don't have a serial cable handy at the moment, but the DDB
traceback looks like this:

panic: ata_dmasetup: transfer active on this device!
cpuid = 0
[...]
panic [...]
ata_dmastart(d3075000,4000,0,101) [...]
ata_pci_dmastart(d3075000,4000,0,20) [...]
ata_transaction [...]
ata_start [...]
ata_completed [...]
taskq_run [...]
[...]

I can attach a serial cable and get some more information if
necessary.  The dual-processor machine in question has one 200GB
UATA 100 drive attached to ad0 and two 36GB SATA drives on ad4 and
ad6, mirrored through ccd(4).
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: vm_map.c LOR

[David Schultz]

On Sat, Sep 27, 2003, Florian C. Smeets wrote:
> Hey guys,
> 
> got this one with today's current:
> 
> lock order reversal
>  1st 0xc28008ac vm object (vm object) @ /space/src/sys/vm/vm_map.c:2195
>  2nd 0xc082f110 system map (system map) @ /space/src/sys/vm/vm_kern.c:328
> Stack backtrace:
> backtrace(c037b3b0,c082f110,c0390c85,c0390c85,c0390b1a) at backtrace+0x17
> witness_lock(c082f110,8,c0390b1a,148,0) at witness_lock+0x672
> _mtx_lock_flags(c082f110,0,c0390b1a,148,3) at _mtx_lock_flags+0xba
> _vm_map_lock(c082f0b0,c0390b1a,148,c03dcfe0,2b4) at _vm_map_lock+0x36
> kmem_malloc(c082f0b0,1000,101,cf083b18,c030c587) at kmem_malloc+0x3a
[...]

This is a false positive.
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: ata_dmasetup: transfer active on this device!

[David Schultz]

On Mon, Oct 27, 2003, Robert Watson wrote:
> 
> Bumped into this panic on a kernel from Oct 4 running on my notebook. 
> Unfortunately, I don't have a good trace because I didn't have access to a
> serial console, or a debugging kernel :-(.  It might well already be
> fixed, but I figured I'd post about it just in case.  Basically, I was
> shutting down my notebook, and did a "shutdown -p NOW".  During the
> shutdown, the system panicked. 
> 
> panic: ata_dmasetup: transfer active on this device!
> ...
> ata_dmastart+0x26
> ata_pci_dmastart+0x29
> ata_transaction+0x999
> ata_start+0x1c9
> ata_completed+0x230
> taskqueue_run+...

I have had this problem since the day ATAng was committed.  I last
verified that it was still an issue about two weeks ago, but I'm
reluctant to continue testing on my main machine (the only one
that triggers the bug) because it leads to random data corruption.

The configuration that *consistently* produces this bug within
five minutes of operation involves two 36GB SATA disks mirrored
with ccd(4).  My best guess is that there's a race in the ATAng
code that is exacerbated by ccd(4).  By brief inspection, it
appears that there are a number of problems with the locking.  For
instance, ata_interrupt() does an ATA_UNLOCK_CH() with no matching
ATA_LOCK_CH().  (This is not detected by witness because ATAng
uses its own locking primitives, which have sleep/wakeup races
among other things...)  Unfortunately, I don't understand the code
well enough to fix it and I don't have the time to understand it.
If you ever find that this problem has gone away, please let me
know so I can sync the ATA driver in my tree.
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: __fpclassifyd problem

[David Schultz]

I'm just catching up on -CURRENT, but I wanted to point out that
this was fixed last night in:

src/lib/msun/src/e_scalbf.c,v1.8
src/lib/msun/src/e_scalb.c,v1.10

The fix was to use the old versions of isnan() and isinf()
specifically in the two places in libm where they are needed.
This problem illustrates that we need to be careful as we roll
forward support for C99 features that cannot be implemented as
regular functions, but that are regular functions in C89.  I
expect that once all that support is in the tree, we can do a
single libm version number bump and remove the compatibility hacks.
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: __fpclassifyd problem

[David Schultz]

On Fri, Oct 31, 2003, John Angelmo wrote:
> But still after importing e_scalb.c or e_scalbf.c and rebuilding gives 
> me this:
> 
> 
> cc -fpic -DPIC -O -pipe -march=pentium3 -D_IEEE_LIBM
> -D_ARCH_INDIRECT=i387_   -c i387_s_tan.S  -o i387_s_tan.So
> building shared library libm.so.2
> e_scalb.So: In function `__ieee754_scalbf':
> e_scalb.So(.text+0x0): multiple definition of `__ieee754_scalbf'
> e_scalbf.So(.text+0x0): first defined here
> *** Error code 1

Sounds like you might have accidentally imported e_scalbf.c
twice---once as e_scalbf.c and once as e_scalb.c.
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: Unfortunate dynamic linking for everything

[David Schultz]

On Tue, Nov 18, 2003, Robert Watson wrote:
> (2) Shells again, because they will be fork()d and exec()d frequently
> during heavily scripted activities, such as system boot, periodic
> events, large make jobs, etc.  And presumably the only shell of
> interest is sh, although some of the supporting non-builtin binaries
> may also be of interest. 

This is the only performance argument that truly seems compelling,
particularly since fork()/exec() performance for dynamic binaries
in FreeBSD really sucks as compared to Solaris and AIX (and
presumably other systems that have gone dynamic long ago).
I think this is mostly an aspect of the dynamic linker and the
lack of prebinding rather than performance problems in the VM system.

Most of the other arguments both for and against seem less
important.  The disk space argument that people have been giving
is particularly bogus given that disk capacity is increasing 60%
per year and the size of /bin isn't.  The NSS argument is only
valid because NSS was implemented specifically to require dlopen()
rather than a daemon, so that disadvantage is in no way
fundamental.

The best reason in my mind for going dynamic hasn't even been
mentioned much.  In a fully dynamic system, it is possible to
change kernel ABIs without breaking anything as long as the kernel
and libc are in sync.  Thus, we wouldn't have three versions of
statfs() and two versions of accept(), along with other cruft in
the kernel to support binaries going back to 4.2BSD.  The
compatX.Y libraries would suffice.  Also, developers would have
greater flexibility to extend or otherwise improve system
interfaces between minor releases.  Unfortunately, this would mean
giving up on ABI compatibility for static binaries entirely, so I
doubt it would fly anytime soon.  Moreover, the performance issues
we have with dynamic binaries now really are significant, at least
as of a few months ago.  (Maybe mdodd's prebinding work addresses
some of that.)

Dragonfly has addressed the same issue in a different, perhaps
more elegant way, wherein even static binaries have forward and
reverse ABI compatibility.  But it does still require the kernel
to mock up legacy fields in shared structures forever, and it
doesn't permit me to, for instance, take advantage of the fact
that mmap(), msync(), mprotect(), minherit(), madvise(), and
mincore() have essentially the same access checks and rewrite them
to use a common monitor.
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: Unfortunate dynamic linking for everything

[David Schultz]

On Tue, Nov 18, 2003, Scott Long wrote:
> > The additional hole of exploiting the system through the shared libs
> > is a negative tradeoff.
> 
> Exploits in libraries happen though.  The LD_LIBRARY_PATH attack is an old
> one that most Unixes are hopefully hardened against.

FreeBSD had a lingering LD_LIBRARY_PATH-related vulnerability
until Sunday, actually[1].  ;-)  But I don't mean to dispute your
point.  Like most of the other arguments in this bikeshed, there
is nothing fundamental about the LD_LIBRARY_PATH problem---nothing
that can't be fixed easily.


[1] The bug is either that nologin(8) respected LD_LIBRARY_PATH or
that sshd(8) and login(1) allow environment poisoning, depending
on your point of view.
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: Unfortunate dynamic linking for everything

[David Schultz]

On Tue, Nov 18, 2003, Robert Watson wrote:
> On systems like Mac OS X, use of a "shared
> region" permits not only use of prebinding, but assumptions of common load
> addresses for common libraries.  In addition, the "shared region" approach
> allows the reuse of page table and VM meta-data across many processes. 
> Leaving aside special-purpose optimizations like that, using some simple
> heuristics, we could probably easily reduce the number of load addressed
> dramatically, making prebinding a far more feasible approach, and
> something we should explore thoroughly.

IIRC, Dyson proposed something like that a while ago.  It's a neat
idea, particularly if it gets us away from PIC on register-starved
architectures such as the i386.

> John Dyson's suggestion of "mixed mode" linking, in which we statically
> link known required parts of binaries, but permit dynamic linking of
> "plug-in"  functionality is another quite reasonable solution, as long as
> we take care of the potential issue of "plug-ins" relying on symbols from
> a library that is statically linked to the application, but not in the set
> of symbols depended on by the application.

Wouldn't the first shared object you loaded instantly pull in
libc.so as a dependency, giving you pieces of both a static and a
dynamic libc?  It seems that that would break something...

> For big applications like KDE, Open Office, et al, prebinding probably is
> the right approach, and hopefully we have plenty of time before 5.3 to
> refine Matthew's work and see if we can't get things up to speed.

For these applications, the fork/exec overhead is so far into the
noise and there are so many shared libraries that dynamic linking
wins outright, prebinding or not.  Maybe for servers that fork a
lot this would be more contentious.
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: Port of Niels Provos's file descriptor allocation code

[David Schultz]

On Thu, Nov 27, 2003, Tim Robbins wrote:
> I've ported Niels Provos's file descriptor allocation code to FreeBSD
> in case anyone wants to try it out & run some benchmarks. If the performance
> boost turns out to be worth the added complexity, I might clean it up a
> bit and commit it.

I've used a similar data structure for a special-purpose allocator
before, and it had extremely low allocation time overhead---
basically a few memory references for every level of the tree
in the common case.  Unless for some strange reason it pessimizes
processes with a small number of file descriptors significantly,
it would be really great to have this in the tree!
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: KSE test code?

[David Schultz]

Thus spake Ivan Voras <[EMAIL PROTECTED]>:
> Is there any example code available that just tests (a "proof of
> concept"-like) the KSE system on FreeBSD 5-current?

See src/tools/KSE/ksetest.

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: cvs commit: src/sys/vm vm_map.c vm_map.h vm_pageout.c

[David Schultz]

Thus spake Garrett Wollman <[EMAIL PROTECTED]>:
> < said:
> 
> > A real problem is that a swapped out process' uarea has to be
> > paged back in, even when no memory is available.  I don't think
> > there's an easy way around that, given that you need the uarea and
> > kernel stack to handle the signal.
> 
> But you don't, actually -- at least not to ``handle'' a SIGKILL.  In
> that case, you should be able to simply destroy the process and free
> whatever swap it has been allocated without ever giving it control.
> So is the issue that we don't want to send SIGKILL too aggressively,
> and send some other signal to give the process a chance to exit
> gracefully?

Perhaps you don't technically need it since swapping doesn't swap
out very much these days (and should probably go away).  But you'd
need to make some minor changes to signal delivery, or write a
separate mechanism, in order to kill a process without swapping it in.

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: OSS SBLive driver causes kernel panic with 5.0 current

[David Schultz]

Thus spake Andre Guibert de Bruet <[EMAIL PROTECTED]>:
> 
> On Fri, 7 Mar 2003, Jody Franklin wrote:
> 
> > I'd been keeping up with current (world/kernel) every other week or so,
> > and until this week I had no real problems. But after the build I did on
> > March 3rd my soundcard driver (4Front's SBLive/Audigy driver) causes a
> > kernel panic on load. If I don't load the driver the system boots fine,
> > and runs with no other problems.
> >
> > This is the message I get from the debugger when I load the driver:
> >
> > panic: Invalid major (-1030904368) in make_dev
> >
> > I've posted this info to their support forums also, their last responce
> > was to "see what they broke".
> 
> Please don't cross-post -current and -questions.
> 
> Major numbers are now being allocated dynamically. Sounds like the emu10k1
> driver doesn't like this. My guess is, it's probably in the process of
> being converted over...

His post refers to the commercial driver, which still needs to be
converted.

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: .2 isn't a valid double and other problems

[David Schultz]

Thus spake Franz Klammer <[EMAIL PROTECTED]>:
> cvsup and build (kernel + userland, empty /usr/obj):
> FreeBSD ds9.webonaut.com 5.0-CURRENT FreeBSD 5.0-CURRENT #0: Sun Mar 16
> 17:53:22 CET 2003  [EMAIL PROTECTED]:/usr/obj/usr/src/sys/DS9  i386
> 
> 
> since my update from yesterday (above), fontconfig can't read his
> configuration file. it claims the .2 isn't a valid double. 
> (line 228 of fonts.conf)
> 
> metacity seems can't also not correct read his config if numbers 
> like 0.7 used.
> 
> there are also problems with script-fu from gimp-devel wich cause 
> a segfault:
> 
> ... many similar line like the blow deletet ...
> gimp-1.3: Corrupt segment 1 in gradient file
> '/usr/X11R6/share/gimp/gradients/Yellow_Orange.ggr'.
> 
> (gimp-1.3:58219): Gimp-Core-WARNING **: (): no matching segment for
> position 0,067
> gimp-1.3: fatal error: Segmentation fault

You're using a locale in which the decimal point character is a
comma.  The breakage is my fault.  I'm waiting to hear back from
the vendor about a (trivial) patch, but I may just check the fix
in sooner anyway.  For the moment, please use the following:

Index: gdtoaimp.h
===
RCS file: /cvs/src/contrib/gdtoa/gdtoaimp.h,v
retrieving revision 1.2
diff -u -r1.2 contrib/gdtoa/gdtoaimp.h
--- gdtoaimp.h  12 Mar 2003 20:20:22 -  1.2
+++ gdtoaimp.h  15 Mar 2003 23:14:12 -
@@ -203,6 +203,7 @@
 #endif
 
 #define INFNAN_CHECK
+#define USE_LOCALE
 
 #undef IEEE_Arith
 #undef Avoid_Underflow

Index: contrib/gdtoa/strtodg.c
===
RCS file: /cvs/src/contrib/gdtoa/strtodg.c,v
retrieving revision 1.1.1.1
diff -u -r1.1.1.1 strtodg.c
--- contrib/gdtoa/strtodg.c 12 Mar 2003 20:18:18 -  1.1.1.1
+++ contrib/gdtoa/strtodg.c 16 Mar 2003 00:27:41 -
@@ -337,6 +337,9 @@
int j, k, nbits, nd, nd0, nf, nz, nz0, rd, rvbits, rve, rve1, sign;
int sudden_underflow;
CONST char *s, *s0, *s1;
+#ifdef USE_LOCALE
+   CONST char *s2;
+#endif
double adj, adj0, rv, tol;
Long L;
ULong y, z;

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: HEADS UP: Don't upgrade your Alphas!

[David Schultz]

Thus spake Ruslan Ermilov <[EMAIL PROTECTED]>:
> Hold off upgrading your Alphas for a moment.
> Something broke libc recently that results in
> (at least) floating point exceptions from
> awk(1) (this is not related to today's awk
> upgrade).
> 
> I've been able to reproduce this on beast.freebsd.org
> by building the fresh libc.a and linking awk with
> it, and running a test case.
> 
> I haven't been able to reproduce this with 8th
> March libc, so the time window for the breakage
> is low.
> 
> I suspect the recent gtdoa commit to libc; we
> will know that is less than an hour.

Whups.  You're probably using a locale in which the decimal point
is not a period.  In that case, please use the patches I just
posted to the following thread, which I just CC'd you.

If it really is an Alpha issue and not a locale issue, awk is
probably dying on one of the scripts used by the kernel build.
If you could send me the command line that causes awk to die,
that would be helpful.  I'm running a kernel build on beast
right now to see if I can reproduce a problem.  I have a meeting
in a few minutes, but I'll be back in five hours or so to follow
up on this.

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: HEADS UP: Don't upgrade your Alphas!

[David Schultz]

Thus spake Ruslan Ermilov <[EMAIL PROTECTED]>:
> Yes, as I have suspected, the gdtoa change is responsible
> for a breakage.  libc corresponding to this lib/libc works:
> 
>   cvs -q up -P -d -D'2003/03/12 20:20:00'
> 
> : Using /home/ru/w/f/usr.bin/awk/nawk nawk...
> 
> This version, together with contrib/gdtoa, doesn't:
> 
>   cvs -q up -P -d -D'2003/03/12 20:29:59'
> 
> : Using /home/ru/w/f/usr.bin/awk/nawk nawk...
> : nawk: floating point exception 8
> :  input record number 325, file
> :  source line number 84
> 
> To see the breakage, one needs to install new libc, and
> run (assuming that /usr/bin/nawk is dynamically linked)
> make in usr.bin/truss; this will run the awk(1) script
> that exhibits one of these FPEs.
> 
> P.S.  Hmm, I didn't test this on i386, as I found this
> bug when attempting to produce a cross-release of i386
> on Alpha, so i386's may be affected too.  Will see.

The bug is Alpha specific; makenewsyscalls.sh will die if you are
using an awk compiled with the new sources, regardless of what
architecture you are building for.  This is because floating point
support on Alpha is broken unless you specifically tell gcc to
unbreak it by specifying -mieee.  This week is really bad for me,
so unless there's a quick fix, I won't get a chance to look into
it further until Thursday night.  Here's a way (other than using
-mieee when compiling awk) to hack around the problem.  It may not
work with higher optimization levels.

Index: lib.c
===
RCS file: /home/ncvs/src/contrib/one-true-awk/lib.c,v
retrieving revision 1.1.1.3
diff -u -r1.1.1.3 lib.c
--- lib.c   17 Mar 2003 07:59:58 -  1.1.1.3
+++ lib.c   18 Mar 2003 07:09:45 -
@@ -678,7 +678,7 @@
char *ep;
errno = 0;
r = strtod(s, &ep);
-   if (ep == s || r == HUGE_VAL || errno == ERANGE)
+   if (ep == s || isinf(r) || errno == ERANGE)
return 0;
while (*ep == ' ' || *ep == '\t' || *ep == '\n')
ep++;

That said, can someone out there with Alpha FP clue let me know
why silent NaN's are broken?  The architecture guide says that
software support is needed to support quiet vs. signalling NaNs,
but the default gcc settings seem to do this incorrectly.  For
instance, the following is wrong:

[EMAIL PROTECTED]:~> cat foo.c
#include 
int main () { return (NAN == HUGE_VAL); }
[EMAIL PROTECTED]:~> gcc foo.c
[EMAIL PROTECTED]:~> ./a.out
Floating exception
[EMAIL PROTECTED]:~>

FWIW, the bug is caused by the fact that our strtod() didn't used
to understand the string "nan" as specified in ANSI C99, and now
it does.  awk detects whether a given token is a number using
strtod(), and since quiet NaNs seem to incorrectly cause
exceptions on Alpha, it chokes on the "nanosleep" symbol in
syscalls.master.  I'm hoping there's a better solution than
disabling support for NaNs in strtod().

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: HEADS UP: Don't upgrade your Alphas!

[David Schultz]

Thus spake Terry Lambert <[EMAIL PROTECTED]>:
> David Schultz wrote:
> > This is because floating point
> > support on Alpha is broken unless you specifically tell gcc to
> > unbreak it by specifying -mieee.
> 
> Sounds like the ability to turn "-mieee" off at all, let alone
> making it the default, is bad?  If so, why is that the way it
> is configured?
> 
> > I'm hoping there's a better solution than
> > disabling support for NaNs in strtod().
> 
> Make "-mieee" on by default?

Well, I was secretly hoping I'd get that answer from the Powers
That Be, but I think it's off by default on purpose for efficiency
reasons.  That would imply that I need to come up with a workaround
that involves not generating NaN values, *period*.  Sigh.

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: libm problem

[David Schultz]

Thus spake Till Riedel <[EMAIL PROTECTED]>:
> [EMAIL PROTECTED] $ cat test.c
> #include "math.h"
> 
> int main()
> {
>   int base=8;
>   int dim=2;
>   float res;
>   res=pow((float)base,(float)dim);
>   printf("%f\n",res);
>   return 0;
> }
> [EMAIL PROTECTED] $ gcc -lm test.c
> [EMAIL PROTECTED] $ ./a.out
> 1.00
> 
> what happened to my libm???
> on my 4.8 box the result is 64 by the way :-)
> CPU: Pentium 4 (2411.60-MHz 686-class CPU)
> Origin = "GenuineIntel"  Id = 0xf27  Stepping = 7
> any suggestions?

I can't reproduce your problem on dual PPro or Celeron systems.
Did you have any optimizations other than -O set when you made
world?  It would be helpful if you could drop into gdb and give me
the output of 'print/x {int}&res' right after the call to pow().

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: Port breakage (isnan undeclared)

[David Schultz]

Thus spake Tim Robbins <[EMAIL PROTECTED]>:
> On Thu, Mar 20, 2003 at 12:55:22AM -0800, Kris Kennaway wrote:
> 
> > Several ports have become broken recently with the following error:
> > 
> > ../../../include/osg/Math:149: `isnan' undeclared (first use this function)
> > 
> > http://bento.freebsd.org/errorlogs/i386-5-latest/osg-0.9.3.log
> > http://bento.freebsd.org/errorlogs/i386-5-latest/gnucap-0.31.log
> > http://bento.freebsd.org/errorlogs/i386-5-latest/fractorama-1.6.4.log
> > 
> > Can someone please investigate?
> 
> The prototypes for isnan() &c. need to be put back into math.h, and their
> source files need to be un-deprecated.

C99 requires that isnan() be a macro, since it can take arguments
of multiple types and C doesn't support templates or overloading.
Technically, redundant function and macro implementations can
coexist, but that's gross.

A better solution may be to define _GLIBCPP_USE_C99 to 1 in
libstdc++.  Among other things, this tells the C++ library to
capture standard C99 macros such as isnan() in a wrapper in the
std namespace before undefing them as it does now.  The
appropriate configure option is --enable-c99, BTW.  If a real C++
guru could make sure this doesn't break anything else I would be
grateful.  What I don't understand is why the libstdc++
all-your-macros-are-belong-to-us magic gets pulled in when you say
'#include ' instead of .  That's going to break
programs (such as fractorama) that expect isnan() and friends to
be in the global namespace.  Again, comments from someone with C++
fu would be appreciated.

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: libm problem

[David Schultz]

Thus spake Till Riedel <[EMAIL PROTECTED]>:
> I now know the thing that makes it break.
> 
> cc -O2 -pipe -mcpu=pentiumpro -c /usr/src/lib/msun/src/e_pow.c
> works fine!
> cc -O0 -pipe -march=pentium4 -c /usr/src/lib/msun/src/e_pow.c
> ... works
> but...
> cc -O -pipe -march=pentium4 -c /usr/src/lib/msun/src/e_pow.c
> breaks it. Hey its only gcc :-), nothing to worry about.
> I think that 0 is nice number: so why don't optimize everithing to down it.
> -O seems to minimize numbers not calculation time. Does anyone know the
> flag to turn that off.

If you have the time and inclination, I suggest reporting this
problem to the gcc folks.  For now, don't use Pentium 4
optimizations.  Not only are they apparently broken, they also
generate slower code than you get by optimizing for a PPro, or
even a 386.

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: libm problem

[David Schultz]

Thus spake Anti <[EMAIL PROTECTED]>:
> On Sat, 22 Mar 2003 10:28:46 -0800
> Steve Kargl <[EMAIL PROTECTED]> wrote:
> 
> > Pentium 4 is definitely broken on 5.x.  Perhaps, we should remove
> > the footshooting.
> > 
> > --- bsd.cpu.mk.orig Sat Mar 22 10:23:42 2003
> > +++ bsd.cpu.mk  Sat Mar 22 10:27:11 2003
> > @@ -62,7 +62,9 @@
> >  .  elif ${CPUTYPE} == "k5"
> >  _CPUCFLAGS = -march=pentium
> >  .  elif ${CPUTYPE} == "p4"
> > -_CPUCFLAGS = -march=pentium4
> > +# XXX gcc 3.2.2 appears to generate bad code on FreeBSD 5.x
> > +#_CPUCFLAGS = -march=pentium4
> > +_CPUCFLAGS = -march=pentiumpro
> >  .  elif ${CPUTYPE} == "p3"
> >  _CPUCFLAGS = -march=pentium3
> >  .  elif ${CPUTYPE} == "p2"
> 
> 
> pentium3 would be better than pentiumpro on a p4 i think...

You would think so, but in my (limited) testing on a P3, this does
not appear to be the case with gcc3 in -CURRENT.  Optimizing for a
Ppro worked the best, IIRC.

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: "Just" building the lib part of world

[David Schultz]

Thus spake David Leimbach <[EMAIL PROTECTED]>:
> Or even better would be just building libc.  I have been working on my 
> getpwnam_r assignment...
> examining implementations in both Darwin and NetBSD and started trying 
> to implement some of
> this code for FreeBSD... Its not anywhere even near the goal in sight 
> as I am still learning the
> build system.
> 
> Do I always have to build world or can I get away with just making some 
> subdirectories?  If so
> what is the best way to do this?
> 
> Rebuilding gcc each time I just want to test out my code is a real drag 
> :)

Try:

cd src/lib/libc && make && make install

That will remake any modified files in libc and install libc.  You
can remake all files by doing a 'make clean' first.  Note that if
you have not previously populated an object directory (e.g. with
'make world', the object files will be placed in src/lib/libc.

You may also want to try

make -DNOCLEAN buildworld

which recompiles only the sources that have changed and things
that depend on them.

Keep in mind that the well-documented approach is conservative to
prevent people from shooting themselves in the foot, so you need
to be careful.  For instance, if your libc changes break ABI
compatability, you could screw yourself over with the approach above.

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: Unclean sync in current

[David Schultz]

Thus spake Kevin Oberman <[EMAIL PROTECTED]>:
> I've been seeing this for a couple of weeks since I updated my laptop to
> CURRENT. I do a normal shutdown (-p or -r) and reboot. The shutdown
> looked normal, with no problems reported with the sync, but, when the
> system is rebooted, the partitions are all shown as possibly
> unclean. From my dmesg:
> Mounting root from ufs:/dev/ad0s3a
> WARNING: / was not properly dismounted
> WARNING: / was not properly dismounted
> WARNING: /tmp was not properly dismounted
> WARNING: /usr was not properly dismounted
> WARNING: /var was not properly dismounted
> 
> All disks are mounted with soft-updates enabled. 
> 
> I don't see any other reports of this. Is this unique to my system?

Unlike the SCSI driver, the ATA driver does not send a flush cache
command to your disks before powering off the system.  The kernel
waits for five seconds in either case, but for some disks that may
not be sufficient.

The following patch should fix that, although it may have rotted a
bit in the last two months given Soeren's sweeping ATA changes.  I
also edited an unrelated change out of the diff, which might
confuse patch(1).  If you run into problems getting it to apply,
let me know and I'll fix it when I'm back from vacation.

Index: sys/dev/ata/ata-all.c
===
RCS file: /home/ncvs/src/sys/dev/ata/ata-all.c,v
retrieving revision 1.163
diff -u -r1.163 ata-all.c
--- sys/dev/ata/ata-all.c   2003/01/19 20:18:07 1.163
+++ sys/dev/ata/ata-all.c   2003/01/27 09:17:02
@@ -77,6 +77,7 @@
 static void ata_intr(void *);
 static int ata_getparam(struct ata_device *, u_int8_t);
 static int ata_service(struct ata_channel *);
+static void ata_shutdown(void *arg, int howto);
 static void bswap(int8_t *, int);
 static void btrim(int8_t *, int);
 static void bpack(int8_t *, int8_t *, int);
@@ -1160,7 +1161,32 @@
 return error;
 }
 
+/*
+ * This procedure is called during shutdown,
+ * after all dirty buffers have been flushed.
+ */
 static void
+ata_shutdown(void *arg, int howto)
+{
+struct ata_channel *ch;
+int ctlr;
+
+#ifdef DEV_ATADISK
+/* Flush the caches of each open ATA disk device. */
+for (ctlr = 0; ctlr < devclass_get_maxunit(ata_devclass); ctlr++) {
+   if (!(ch = devclass_get_softc(ata_devclass, ctlr)))
+   continue;
+   ch->lock_func(ch, ATA_LF_LOCK);
+   if (ch->devices & ATA_ATA_MASTER && ch->device[MASTER].driver)
+   ad_sync(&ch->device[MASTER]);
+   if (ch->devices & ATA_ATA_SLAVE && ch->device[SLAVE].driver)
+   ad_sync(&ch->device[SLAVE]);
+   ch->lock_func(ch, ATA_LF_UNLOCK);
+}
+#endif /* DEV_ATADISK */
+}
+
+static void
 ata_drawer_start(struct ata_device *atadev)
 {
 ATA_INB(atadev->channel->r_io, ATA_DRIVE);   
@@ -1516,5 +1542,10 @@
printf("ata: config_intrhook_establish failed\n");
free(ata_delayed_attach, M_TEMP);
 }
+
+/* Register a handler to flush write caches on shutdown */
+if ((EVENTHANDLER_REGISTER(shutdown_post_sync, ata_shutdown,
+  NULL, SHUTDOWN_PRI_DEFAULT)) == NULL)
+   printf("ata: shutdown event registration failed!\n");
 }
 SYSINIT(atadev, SI_SUB_DRIVERS, SI_ORDER_SECOND, ata_init, NULL)
Index: sys/dev/ata/ata-disk.c
===
RCS file: /home/ncvs/src/sys/dev/ata/ata-disk.c,v
retrieving revision 1.139
diff -u -r1.139 ata-disk.c
--- sys/dev/ata/ata-disk.c  2002/12/17 16:26:22 1.139
+++ sys/dev/ata/ata-disk.c  2003/01/27 09:17:04
@@ -255,10 +255,8 @@
 disk_invalidate(&adp->disk);
 disk_destroy(adp->dev);
 devstat_remove_entry(&adp->stats);
-if (flush) {
-   if (ata_command(atadev, ATA_C_FLUSHCACHE, 0, 0, 0, ATA_WAIT_READY))
-   ata_prtdev(atadev, "flushing cache on detach failed\n");
-}
+if (flush)
+   ad_sync(atadev);
 if (adp->flags & AD_F_RAID_SUBDISK)
ata_raiddisk_detach(adp);
 ata_free_name(atadev);
@@ -289,8 +287,7 @@
 
 adp->device->channel->lock_func(adp->device->channel, ATA_LF_LOCK);
 ATA_SLEEPLOCK_CH(adp->device->channel, ATA_CONTROL);
-if (ata_command(adp->device, ATA_C_FLUSHCACHE, 0, 0, 0, ATA_WAIT_READY))
-   ata_prtdev(adp->device, "flushing cache on close failed\n");
+ad_sync(adp->device);
 ATA_UNLOCK_CH(adp->device->channel);
 adp->device->channel->lock_func(adp->device->channel, ATA_LF_UNLOCK);
 return 0;
@@ -765,6 +762,20 @@
return ATA_OP_CONTINUES;
 }
 return ATA_OP_FINISHED;
+}
+
+/*
+ * Flush the write cache of the given device.
+ */
+int
+ad_sync(struct ata_device *atadev)
+{
+int error;
+
+/* XXX The ATA standard says this command can take up to 30 seconds. */
+if ((error = ata_command(atadev,ATA_C_FLUSHCACHE,0,0,0,ATA_WAIT_READY)))
+   ata_prtdev(atadev, "flushing cache failed\n");
+return error;
 }
 
 static void
Index: sys/dev/ata/ata-disk.h

Re: several background fsck panics

[David Schultz]

Thus spake Alexander Langer <[EMAIL PROTECTED]>:
> I had several panics related to background fsck now.  Once I disabled
> background fsck, all went ok.
> 
> It began when I pressed the reset buttons on several boots while the
> system was still doing fscks.
[...]
> Mar 24 21:48:59 fump kernel: panic: ufs_dirbad: bad dir

You would have gotten this one without bgfsck as well the next
time you tried to look the offending directory.  Background fsck
only expedited the panic by reading all the directories on the
system in order to perform its checks.  Basically, the panic is
the kernel's way of telling you that something is unexpectedly
wrong with the filesystem (due in this case to ATA write caching),
and that it is going to give up rather than risk causing further
damage.  UFS, as well as most other filesystems, are not designed
to tolerate failures on the part of the hardware to honor its
guarantees, so it's hard to do better without inventing a new
filesystem.
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: [Re: several background fsck panics

[David Schultz]

Thus spake Terry Lambert <[EMAIL PROTECTED]>:
> o Put a counter in the first superblock; it would be
>   incremented when the BG fsck is started, and reset
>   to zero when it completes.  If the counter reaches
>   3 (or some command line specified number), then the
>   BG flagging is ignored, and a full FG fsck is then
>   performed instead.  I like this idea because it will
>   always work, and it's not actually a hack, it's a
>   correct solution.

I'm glad you like it because AFAIK, it is already implemented.  ;-)

> o Implement "soft read-only".  The place that most of
>   the complaints are coming from is desktop users, with
>   relatively quiescent machines.  Though swap is used,
>   it does not occur in an FS partition.  As a result,
>   the FS could be marked "read-only" for long period of
>   time.  This marking would be in memory.  The clean bit
>   would be set on the superblock.  When a write occurs,
>   the clean bit would be reset to "dirty", and committed
>   to disk prior to the write operation being permitted
>   to proceed (a stall barrier).  I like this idea because,
>   for the most part, it eliminates fsck, both BG and FG,
>   on systems that crash while it's in effect.  The net
>   result is a system that is statistically much more
>   tolerant of failures, but which still requires another
>   safety net, such as the previous solution.

I was thinking of doing something like this myself as part of an
``idle timeout'' for disks.  (Marking the filesystem clean after a
period of quiescence would actually interfere with ATA disks'
built-in mechanism for spinning down after a timeout, which is
important for laptops, so the OS would have to track the true
amount of idle time.)  Annoyingly, I can never get the disk
containing /var to remain quiescent for long while cron is running
(even without any crontabs), and I hope this can be solved without
disabling cron or adding a nontrivial hack to bio.
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: MIDI

[David Schultz]

Thus spake Thanjee Neefam <[EMAIL PROTECTED]>:
> I was very happy when compiling my 5.0 kernel. For the first time "device
> midi" compiled without giving any errors. This abnormal excitement only
> led to misery when I discovered after rebooting that there still was no
> MIDI. 
> Is MIDI going to be implemented soon? Who is working on it? Can I help
> them? (I am not a very good programmer, but I can hack pre existing code,
> and I am good at testing). MIDI is the ONLY thing stopping me from
> running FreeBSD exclusively.

FYI, the non-free OSS driver supports MIDI:

http://www.opensound.com/bsd.html
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: CSTD=c99 breaks package creation

[David Schultz]

On Thu, Jun 12, 2003, Tim Robbins wrote:
> On Wed, Jun 11, 2003 at 07:37:01PM -0700, Kris Kennaway wrote:
> 
> > It's possible that there's either a bug in gcc or there is C code in
> > the system that has a different meaning when interpreted to C99
> > standards.
> 
> I think I may have found the problem, and I think it's in GNU tar.
> 
> GNU tar does this:
> 
> #ifndef __attribute__
> /* This feature is available in gcc versions 2.5 and later.  */
> # if __GNUC__ < 2 || (__GNUC__ == 2 && __GNUC_MINOR__ < 5) || __STRICT_ANSI__
> #  define __attribute__(Spec) /* empty */
> # endif
> #endif
> 
> machine/_types.h does this:
> 
> typedef int __attribute__((__mode__(__DI__)))   __int64_t;
> typedef unsigned int __attribute__((__mode__(__DI__)))  __uint64_t;
> 
> If __attribute__ is empty, __int64_t becomes a synonym for int. Bad.

Wow!  Nice catch...
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: Problems tuning kmem_map on 5.1-REL box.

[David Schultz]

On Tue, Jun 17, 2003, Peter Losher wrote:
> Hi -
> 
> If this sort of question is better asked on a more specialized list
> @freebsd.org, please let me know. (Since 5.1 is still considered a
> developmental release, one doesn't know if either -current or -stable
> would be more appropriate.  So I am sending it to -current.
> 
> So, I recently put a Quad-Xeon server (w/ 4GB of RAM) into production as
> primarily as a master FTP server for ISC as well as for FreeBSD (US/IPv6
> side of ftp.freebsd.org).  All has been going well for the past couple of
> weeks until 5.1 was released. Traffic spiked, and the server started
> panicking every 12 hours:
> 
> > panic: kmem_malloc(4096): kmem_map too small: 377487360 total allocated
> 
> So after looking at the archives, I read that FreeBSD had issues with
> installed memory above 2GB, so I set the sysctl kern.maxvnodes=13, but
> to no effect:
> 
> > panic: kmem_malloc(16384): kmem_map too small: 293519360 total allocated
> 
> I have since removed my NMBCLUSTER setting in the kernel, so it would be
> set automatically, I have even set VM_KMEM_SIZE_SCALE="2" (I haven't messed
> with the other KMEM kernel options, as I was hoping to avoid it)  And a
> couple of hours ago I updated the kernel with the latest kern_malloc.c file
> in the RELENG_5_1 branch.

To allow the kmem_map to exceed 200 MB, you'll also need to tweak
VM_KMEM_SIZE_MAX to (for example) '(1024 * 1024 * 1024)'.  BTW,
the formula, which I stole from vmparam.h, is:

min(max(VM_KMEM_SIZE, Physical memory/VM_KMEM_SIZE_SCALE), VM_KMEM_SIZE_MAX)
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: Problems tuning kmem_map on 5.1-REL box.

[David Schultz]

On Thu, Jun 19, 2003, Peter Losher wrote:
> On Wed, 18 Jun 2003, David Schultz wrote:
> 
> > To allow the kmem_map to exceed 200 MB, you'll also need to tweak
> > VM_KMEM_SIZE_MAX to (for example) '(1024 * 1024 * 1024)'.  BTW,
> > the formula, which I stole from vmparam.h, is:
> >
> > min(max(VM_KMEM_SIZE, Physical memory/VM_KMEM_SIZE_SCALE), VM_KMEM_SIZE_MAX)
> 
> Looks like I may have found some kind of error, adding this to my kernel
> config:
> 
> # KVM
> options VM_KMEM_SIZE_SCALE="4"
> options VM_KMEM_SIZE_MAX="(1024 * 1024 * 1024)"
> 
> (1024MB of KVM, 4096MB/4, 1024MB MAX KVM size)
> 
> Cause the following warning and compile bomb:
> 
> % config KVM_FIX
> WARNING: unknown option `*' removed from ../compile/KVM_FIX/opt_vm.h

This looks like a limitation of config(8).

> Changing the options to:
> 
> # KVM
> options VM_KMEM_SIZE_SCALE="4"
> options VM_KMEM_SIZE_MAX="(1024*1024*1024)"
> 
> Configs and compiles cleanly, but panics when rebooting with the new
> kernel:
> 
> kmem_suballoc: bad status return of 3.
> panic: kmem_suballoc

That means there wasn't a large enough contiguous region for the
kmem_map.  I guess I forgot to tell you that you also need to
adjust KVA_PAGES upwards so that the kernel has enough virtual
address space to fit the map.  The following should do:

options KVA_PAGES=512
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: write access to dos partition hangs system completely

[David Schultz]

On Sun, Jun 15, 2003, Andreas Klemm wrote:
> FreeBSD titan.klemm.apsfilter.org 5.1-RC FreeBSD 5.1-RC #0: Sun Jun  1 14:21:32 CEST 
> 2003 [EMAIL PROTECTED]:/usr/src/sys/i386/compile/TITAN5  i386
> 
> When I mount my dos partition read write and copy
> some data to it it immediately freezes the system.
[...]
> da0: Attempt to query device size failed: NOT READY, Medium not present
> (da0:umass-sim0:0:0:0): READ CAPACITY. CDB: 25 0 0 0 0 0 0 0 0 0 
> (da0:umass-sim0:0:0:0): CAM Status: SCSI Status Error
> (da0:umass-sim0:0:0:0): SCSI Status: Check Condition
> (da0:umass-sim0:0:0:0): NOT READY asc:3a,0
> (da0:umass-sim0:0:0:0): Medium not present
> (da0:umass-sim0:0:0:0): Unretryable error
> Opened disk da0 -> 6
> (da0:umass-sim0:0:0:0): READ CAPACITY. CDB: 25 0 0 0 0 0 0 0 0 0 
> (da0:umass-sim0:0:0:0): CAM Status: SCSI Status Error
> (da0:umass-sim0:0:0:0): SCSI Status: Check Condition
> (da0:umass-sim0:0:0:0): NOT READY asc:3a,0
> (da0:umass-sim0:0:0:0): Medium not present
> (da0:umass-sim0:0:0:0): Unretryable error
> Opened disk da0 -> 6
> Mounting root from ufs:/dev/ad2s2a
> ad0: hard error cmd=read fsbn 115471868 of 115471868-115471871 status=51 error=40
> pid 640 (squid), uid 65534: exited on signal 6
> pid 674 (squid), uid 65534: exited on signal 6
> pid 676 (squid), uid 65534: exited on signal 6
> pid 678 (squid), uid 65534: exited on signal 6
> pid 680 (squid), uid 65534: exited on signal 6
> ad0: hard error cmd=read fsbn 115471868 of 115471868-115471871 status=51 error=40
> ad0: hard error cmd=read fsbn 115471871 status=51 error=40
> ad0: hard error cmd=read fsbn 115471868 of 115471868-115471871 status=51 error=40
> ad0: hard error cmd=read fsbn 115471871 status=51 error=40

I don't know which of these devices your DOS partition is on, but
the root problem seems to be the hardware.  That said, msdosfs
does hang when a write error occurs, so that may be your problem.
(It gets into a loop in which it retries forever.)  I submitted
kern/37035 regarding this over a year ago, although I don't know
to what extent the original problem or the trivial patch I posted
still apply to 5.X.
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: questions about VM_KMEM_SIZE_SCALE

[David Schultz]

On Tue, Jun 24, 2003, Jay Kuri wrote:
> 
> Hi there,
> 
> Can anyone shed some light on the implications of adjusting
> VM_KMEM_SIZE_SCALE?  In particular I'm wondering if I increase this to,
> say, 2, what happens?  I must admit I don't know how KVA is different from
> KVM or total RAM... so the note in kern_malloc ("on an x86 with 256M KVA,
> try to keep VM_KMEM_SIZE_MAX at 80M or below") doesn't shed enough light
> on the matter.  What are the implications of VM_KMEM_SIZE getting large?
> 
> Does changing this affect memory available to user programs if it's unused
> by the kernel?  

No, KVA_PAGES affects the memory available to user programs.  (You
have a 4 GB address space on i386 to split between user programs
and the kernel.)  Within the kernel's share of this address space,
memory is split into submaps, such as the mb_map (for the
network), buffer_map for the filesystem buffer cache, and the
kmem_map for just about everything else.  These submaps are
size-limited to prevent any one of them from getting out of hand.

The vm_kmem_map is sized automatically based on the amount of
memory you have.  Specifically,

kmem_map_size = min(max(VM_KMEM_SIZE, Physical memory/VM_KMEM_SIZE_SCALE),
VM_KMEM_SIZE_MAX)

The default value for VM_KMEM_SIZE_SCALE is 3, and the default
VM_KMEM_SIZE_MAX is 200MB.
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: Hyperthreading

[David Schultz]

On Fri, Jun 27, 2003, Brooks Davis wrote:
> On Fri, Jun 27, 2003 at 06:39:12PM -0500, Glenn Johnson wrote:
> > Thanks.  I had read the smp manual page.  I know _how_ to enable HTT; I
> > was wondering whether I _should_ enable it.  It seems the answer is that
> > it is not beneficial in its current state because the scheduler does not
> > yet differentiate between physical and logical processors.
> 
> It's more complicated then that.  For many users, it's true that HTT is
> not useful due to the scheduling issues, but for some applications where
> you keep all the CPUs busy, it does help.  Somewhat suprisingly,
> [EMAIL PROTECTED] performs better with HTT enabled then without.  The individual
> workunits take longer to process, but the overall throughput is better
> (4 workunits every 6hrs instead of 2 workunits every 4hrs).

Hyperthreading will generally give you better thoughput because
you get better utilization of the hardware; when one functional
unit would normally be idle due to a pipeline bubble, the other
logical CPU may be able to provide work for it.  On the other
hand, as you observe, latency is worse.  In particular, if you're
running a web browser on one processor, it's competing for
resources with your [EMAIL PROTECTED] client on the other processor, even
though the [EMAIL PROTECTED] client is niced.
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: HEADS UP: new KSE signal code

[David Schultz]

On Sat, Jun 28, 2003, David Xu wrote:
> I begin to commit KSE signal code, libkse will
> be broken for a while.

Umm...if it's known to be broken, then why did you commit it?
If you want people to test KSE and report bugs, the version in
the tree needs to be of consistently good quality.
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: HEADS UP: new KSE signal code

[David Schultz]

On Sat, Jun 28, 2003, Julian Elischer wrote:
> he means that between the time the commits start and finish there may be
> an inconsistant period.. Why is everyone so eager to jump down everyone
> else's throat these days?

The statement that ``libkse will be broken for a while'' gave me
the impression that it would be broken for a couple of days, not a
couple of minutes, and I was just in the process of upgrading
before trying to diagnose a signal problem involving Java.  I
apologize for the misinterpretation.
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: HEADS UP: new KSE signal code

[David Schultz]

On Sat, Jun 28, 2003, David Leimbach wrote:
> Because we aren't working on anything and need something to do... so we 
> find
> ways to think about how we can enforce quality without understanding 
> how stuff
> works first maybe?

Umm...no, but thanks for the insult.  How about: Because we are
working at 3 AM to figure out why signals aren't getting delivered
to java properly and we see an email saying that things will be
more broken ``for a while'' and misinterpret what ``for a while''
means in this context.  See previous post.
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: SMP page update

[David Schultz]

On Tue, Jul 15, 2003, Alp ATICI wrote:
> I was wondering whether the SMPng page at http://www.freebsd.org/smp is 
> updated as those features are added. Because it seems like no feature 
> update for long.
> 
> For instance is the preemptible kernel going to be a part of 5.x series or 
> going
> to be left to 6.x?

The kernel has been preemptible for a long time.  I don't know
anything about the webpage.
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: [-CURRENT tinderbox] failure on sparc64/sparc64

[David Schultz]

On Thu, Jul 17, 2003, Dag-Erling Smorgrav wrote:
> On Thu, Jul 17, 2003 at 09:58:10AM +0200, Harti Brandt wrote:
> > I have no idea how a program can core in vfork(). Probably a vm problem?
> 
> Most likely a KSE-related problem in vfork().  Try replacing vfork() with
> fork() in make(1) and see if the problem goes away.  Warning: build times
> may increase significantly...

I would guess that the problem doesn't occur in the vfork() call
itself, but in the child process (gzip?), and there's a problem
that causes the child to be incompletely divorced from the parent.
Is there any trick to reproducing this problem?  I just did a make
universe on i386 and didn't see it, but maybe my sources are too
old.

It would be interesting to see if fork() fixes the problem.  With
the VM optimizations, vfork() is only about 20% faster than
fork(), so build times shouldn't be significantly impacted.
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "[EMAIL PROTECTED]"