7-BETA4/7-RC1 crashes randomly at shutdown
Hello. I have been following RELENG_7 and then RELENG_7_0. I update /usr/src daily and install them. I think the crash started around 12/18. Since then, I started seen pancing at umount during shutdown. I am using GENERIC kernel with SCHED_ULE. This crash tends to happen after some amount of I/O performed. I cannot really tell what kind of I/O will trigger this umount failure, yet. I got a core and 'where' command. Does anyone have any ideas or suggestions? Thanks, Hiro # kgdb /usr/obj/usr/src/sys/GENERIC/kernel.debug /var/crash/vmcore.3 [GDB will not be able to debug user-mode threads: /usr/lib/libthread_db.so: Undefined symbol "ps_pglobal_lookup"] GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-marcel-freebsd". Unread portion of the kernel message buffer: <118>/etc/rc.shutdown: WARNING: $samba_enable is not set properly - see rc.conf(5). <118>Writing entropy file: <118>. <118>. <118>Dec 26 01:11:56 XX syslogd: exiting on signal 15 Waiting (max 60 seconds) for system process `vnlru' to stop...done Waiting (max 60 seconds) for system process `bufdaemon' to stop...done Wait iSnygn c(imnagx dis60k s,s evcnoonddess) rfeomra isynsitneg.m. .p3r ocess `syncer' to stop...3 3 1 1 1 1 0 0 0 0 done All buffers synced. GEOM_ELI: Detached da0.eli on last close. acpi_ec0: warning: EC done before starting event wait GEOM_ELI: Detached ad4s2f.eli on last close. Fatal trap 12: page fault while in kernel mode cpuid = 0; apic id = 00 fault virtual address = 0x188 fault code = supervisor read, page not present instruction pointer = 0x20:0xc07479d4 stack pointer = 0x28:0xe3c71b1c frame pointer = 0x28:0xe3c71b34 code segment= base 0x0, limit 0xf, type 0x1b = DPL 0, pres 1, def32 1, gran 1 processor eflags= interrupt enabled, resume, IOPL = 0 current process = 1 (init) trap number = 12 panic: page fault cpuid = 0 Uptime: 1h24m11s Physical memory: 2025 MB Dumping 155 MB: 140 124 108 92 76 60 44 28 12 #0 doadump () at pcpu.h:195 195 pcpu.h: No such file or directory. in pcpu.h (kgdb) where #0 doadump () at pcpu.h:195 #1 0xc0753e87 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:409 #2 0xc0754149 in panic (fmt=Variable "fmt" is not available. ) at /usr/src/sys/kern/kern_shutdown.c:563 #3 0xc0a689ec in trap_fatal (frame=0xe3c71adc, eva=392) at /usr/src/sys/i386/i386/trap.c:899 #4 0xc0a68c70 in trap_pfault (frame=0xe3c71adc, usermode=0, eva=392) at /usr/src/sys/i386/i386/trap.c:812 #5 0xc0a6961c in trap (frame=0xe3c71adc) at /usr/src/sys/i386/i386/trap.c:490 #6 0xc0a4f5ab in calltrap () at /usr/src/sys/i386/i386/exception.s:139 #7 0xc07479d4 in _mtx_lock_sleep (m=0xc8abfa18, tid=3306133088, opts=0, file=0x0, line=0) at /usr/src/sys/kern/kern_mutex.c:335 #8 0xc07cc94f in vgone (vp=0xc8abf990) at /usr/src/sys/kern/vfs_subr.c:2471 #9 0xc5742a79 in ?? () #10 0xc8abf990 in ?? () #11 0xc55a9a70 in ?? () #12 0x in ?? () #13 0xc5743c20 in ?? () #14 0x016c in ?? () #15 0xc5746074 in ?? () #16 0x3002 in ?? () #17 0xc8abf990 in ?? () #18 0xe3c71bf0 in ?? () #19 0x0004 in ?? () ---Type to continue, or q to quit--- #20 0xc50f9660 in ?? () #21 0xe3c71ba8 in ?? () #22 0xc57420b4 in ?? () #23 0xc55a9a70 in ?? () #24 0xc5746000 in ?? () #25 0x1002 in ?? () #26 0xe3c71bf0 in ?? () #27 0xc50f9660 in ?? () #28 0xc569d550 in ?? () #29 0xe3c71c00 in ?? () #30 0xc07c6ac1 in dounmount (mp=0xc55a9a70, flags=-982228992, td=0x10) at /usr/src/sys/kern/vfs_mount.c:1270 Previous frame inner to this frame (corrupt stack?) (kgdb) ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "[EMAIL PROTECTED]"
(no subject)
___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Packet loss every 30.999 seconds
On Dec 25, 2007, at 12:27 AM, Kostik Belousov wrote: What fs do you use ? If FFS, are softupdates turned on ? Please, show the total time spent in the softdepflush process. Also, try to add the FULL_PREEMPTION kernel config option and report whether it helps. FFS with soft updates on all filesystems. With your latest uio_yield() in MNT_VNODE_FOREACH patch it's a little harder to provoke packet loss. Standard nightly crontabs and a tar -cf - / > /dev/null no longer cause drops. A make buildkernel will though. root 38 0.0 0.0 0 8 ?? DL Mon08PM 0:04.62 [softdepflush] Building a new kernel with KTR and FULL_PREEMPTION now. -- mark ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: SMP on FreeBSD 6.x and 7.0: Worth doing?
Adrian Chadd wrote: On 25/12/2007, Brett Glass <[EMAIL PROTECTED]> wrote: It sounds like you're pretty convinced you know what the problem is. Again, I'd have to instrument either the FreeBSD kernel or Squid to be 100% sure. But it APPEARS that it's a problem with large numbers of threads trying to do file I/O simultaneously and blocking. I'd still check the namei cache; Squid can and does chew through stupid amounts of pathnames. Its why I hacked up that "ifs" thing a few years ago but was suddenly (contractually) required to not hack on caching for a while.. Yes, Squid is the ideal application for IFS. Do you still have any of your work on this, and would you be able to share it? Scott ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: SMP on FreeBSD 6.x and 7.0: Worth doing?
Mike Tancsa wrote: At 12:12 PM 12/24/2007, Scott Long wrote: For others who might want help with this, tweaking vfs.ufs.dirhash_maxmem is what is needed. A bit of a balancing act is needed if you're on i386 since you'll risk exhausting KVA unless you also tweak KVA_PAGES. Hi Scott, How does one know if the vfs.ufs.dirhash_maxmem is set too high and are exhausting KVA ? Panics, freezes, failure to exec new programs, failure to create pipes, etc. Scott ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: SMP on FreeBSD 6.x and 7.0: Worth doing?
On 26/12/2007, Scott Long <[EMAIL PROTECTED]> wrote: > Yes, Squid is the ideal application for IFS. Do you still have any of > your work on this, and would you be able to share it? It'd be easy to rewrite it from scratch if IFS were recovered. In fact, the whole point behind IFS, way back when, is I could layer a user-space directory hierarchy on top of a kernel provided space and then do "stuff" (I had a POP3 Maildir-like server written using IFS back then.) The squid code wasn't difficult at all. The biggest problem back then was rebuilding the disk index - didn't I have some code to export the inode allocation bitmap via a special file in the filesystem so I didn't have to stat() each individual inode, or didn't I end up comitting that? I'm happy to work on that later on next year. I've got enough non-disk Squid code to rewrite and optimise over the next few months; the storage side is going to have to wait a while. Adrian -- Adrian Chadd - [EMAIL PROTECTED] ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Packet loss every 30.999 seconds
Mark Fullmer wrote: On Dec 25, 2007, at 12:27 AM, Kostik Belousov wrote: What fs do you use ? If FFS, are softupdates turned on ? Please, show the total time spent in the softdepflush process. Also, try to add the FULL_PREEMPTION kernel config option and report whether it helps. FFS with soft updates on all filesystems. With your latest uio_yield() in MNT_VNODE_FOREACH patch it's a little harder to provoke packet loss. Standard nightly crontabs and a tar -cf - / > /dev/null no longer cause drops. A make buildkernel will though. root 38 0.0 0.0 0 8 ?? DL Mon08PM 0:04.62 [softdepflush] Building a new kernel with KTR and FULL_PREEMPTION now. FYI FULL_PREEMPTION causes performance loss in other situations. Kris ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: SMP on FreeBSD 6.x and 7.0: Worth doing?
Scott, Adrian: Even more interesting would be a storage schema for caches that rests on top of FreeBSD's GEOM facility. One could bypass all filesystems but still take advantage of the driver architecture. --Brett Glass At 06:09 AM 12/26/2007, Scott Long wrote: >Yes, Squid is the ideal application for IFS. Do you still have any of your >work on this, and would you be able to share it? > >Scott ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: SMP on FreeBSD 6.x and 7.0: Worth doing?
On 27/12/2007, Brett Glass <[EMAIL PROTECTED]> wrote: > Scott, Adrian: > > Even more interesting would be a storage schema for caches that rests > on top of FreeBSD's GEOM facility. One could bypass all filesystems > but still take advantage of the driver architecture. The biggest bonuses to gain high throughput with web caches, at least with small objects, is to apply temporal locality to them and do IO in $LARGE chunks. You then want to pull tricks with your memory cache so you throw away RAM in cluster-sized chunks - the objects grouped by temporal locality above - because really, if you throw away a 4k page, your costs of performing disk IO to read that 4k versus reading say, 32k or 64k, are pretty freaking similar (the same if you happen to be on the same track, slightly more if you're straddling tracks.) So you also want to pull those tricks. If you have two small objects (<64k) which are 50% likely to be fetched together, then by grouping them into one IO operation you're effectively slicing the seeks needed in half with very little impact. Well, there's an impact - you suddenly start pulling lots more data off disk. Could -that- be done without too much trouble? I've looked at madvise() to pull these tricks with mmap()'ed backing files but, again, I've not hit the point where I'm looking to optimise Squid's disk storage. There's just too much catching up to do to varnish's memory-only workload performance. Damn you phk. :) -- Adrian Chadd - [EMAIL PROTECTED] ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: SMP on FreeBSD 6.x and 7.0: Worth doing?
At 08:32 AM 12/26/2007, Adrian Chadd wrote: >The biggest bonuses to gain high throughput with web caches, at least >with small objects, is to apply temporal locality to them and do IO in >$LARGE chunks. By "temporal locality" I assume you mean that you expect items that are fetched at the same time to both be fetched the next time also. Sort of a "working set" concept for Web pages. Correct? >You then want to pull tricks with your memory cache so you throw away >RAM in cluster-sized chunks - the objects grouped by temporal locality >above - because really, if you throw away a 4k page, your costs of >performing disk IO to read that 4k versus reading say, 32k or 64k, are >pretty freaking similar (the same if you happen to be on the same >track, slightly more if you're straddling tracks.) So you also want to >pull those tricks. If you have two small objects (<64k) which are 50% >likely to be fetched together, then by grouping them into one IO >operation you're effectively slicing the seeks needed in half with >very little impact. Well, there's an impact - you suddenly start >pulling lots more data off disk. And you need more buffer space. The key, I think, is to avoid needing that buffer space on multiple levels. The file system may prefetch large chunks and then the Web cache might do so also, doubling the overhead. >Could -that- be done without too much trouble? I've looked at >madvise() to pull these tricks with mmap()'ed backing files but, >again, I've not hit the point where I'm looking to optimise Squid's >disk storage. There's just too much catching up to do to varnish's >memory-only workload performance. Damn you phk. :) I don't know much about Varnish, but I'd been told that it is not a replacement for Squid. In any event, I certainly WOULD like to see a cache that had a true first-level cache in memory and a second-level cache on disk. The way Squid works now, it never keeps copies of objects in RAM once they've been evicted -- a major flaw, IMHO. This may account for the performance disadvantage relative to Varnish. After all, some objects are accessed by many people at certain times of day, and it pays to "promote" them to RAM during those periods. --Brett ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "[EMAIL PROTECTED]"
USB umass kernel panic
Hello, In the last few days I've been experiencing a reproducible kernel panic whenever plugging in my new digital camera (Pentax Optio Z10). Tried with a freshly cvsupped system too. I'm using 7.0. Here is a manual transcript of the panic message, I have some core dumps too, if needed. They're a little big as usual. umass0: uhub3 (da0:umass-sim0:0:0:0): unsupported block size 0 (da0:umass-sim0:0:0:0): lost device panic: kmem_malloc(-1407700992): kmem_map too small: 14807040 total allocated uptime: etc. (I thought the rest is not really needed) I;m available to give any information required to get into the problem. Perhaps the camera is doing something strange(block size 0) but the panic is surely out of place... Thank you in advance for any help. -- Guido Falsi <[EMAIL PROTECTED]> ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: usb/umass, devfs: this sucks
On Wed, 2007-12-26 at 00:35 -0700, M. Warner Losh wrote: > In message: <[EMAIL PROTECTED]> > Mikhail Teterin <[EMAIL PROTECTED]> writes: > : If we want people to give FreeBSD a try in good faith, it is both > profoundly > : stupid and dishonest on our part to claim, we have a working USB-system... > It > : does not matter, how great our buffer-sharing VM is, if a home user can't > : process their photos with a FreeBSD-powered computer. > > In an ideal world, it would be perfect. > > However, all the USB hardware I've recently purchased actually does > work without a hitch on my FreeBSD system. Older card readers are > more problematic. > > Rather than complain about the system, how about merging RELENG_7's > usb stack to RELENG_6? Or fixing a few bugs from the PRs that are > filed? I did my time in a big push for 7.0, maybe some other people > can help out a little too? Fixing and merging are good, but it seems to me (as an occasional patch contributor without commit privileges) that the bottleneck for USB is still in the handling of incoming patches. At the moment, there are 162 USB PRs to close. Out of these, 103 are marked with "[patch]", but only 42 of those are actually in "patched" state. Almost all of the latter were apparently handled as part of Warner's summer push. The USB stack has to deal with many third-party devices, most of which will not be immediately available for testing by FreeBSD developers. This means that we are more or less forced to rely on external patch contributors (such as myself) to provide workarounds for the problems caused by various hardware peculiarities. Usually, it shouldn't take more than a basic code review to accept these patches, so this would be a good place to start if you want to improve USB handling in FreeBSD. Look at it from my perspective: I would be happy to complete my fix for the infamous five-year-old usb/46176, so people can finally detach umass devices without having to manually unmount them first. However, it will undoubtedly take a non-trivial amount of time to reproduce and eliminate the remaining issues. I'm more likely to put in that effort if I believe that my patches may actually end up in CURRENT, but if a one-line fix such as that in usb/78984 has not been applied after more than a year, how long will it take for patches that involve multiple subsystems? /Henrik ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: SMP on FreeBSD 6.x and 7.0: Worth doing?
At 08:11 AM 12/26/2007, Scott Long wrote: How does one know if the vfs.ufs.dirhash_maxmem is set too high and are exhausting KVA ? Panics, freezes, failure to exec new programs, failure to create pipes, etc. Is there anyway to know ahead of time one is getting close to the stage where all those "bad things" start to happen ? ---Mike ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "[EMAIL PROTECTED]"
PR backlog [was: Re: usb/umass, devfs: this sucks]
On Wed, Dec 26, 2007 at 06:15:16PM +0100, Henrik Gulbrandsen wrote:
> Fixing and merging are good, but it seems to me (as an occasional patch
> contributor without commit privileges) that the bottleneck for USB is
> still in the handling of incoming patches [...] if a one-line fix
> such as that in usb/78984 has not been applied after more than a year,
> how long will it take for patches that involve multiple subsystems?
I'll put on my bugmeister hat for a moment.
First, I share your frustration.
Second, unfortunately, it's not just USB. We suffer from this problem in
several other areas, notably, patches for the userland utilities ("bin").
There are two interrelated problems that create a chicken-and-egg
situation:
- the PR tool is insufficient for our needs;
- there's not a "culture" of going and fixing bugs outside of the code
one usually works on.
As for 1), once the releases are out, I intend to start working on defining
what "our needs" are. As I've stated before in other places, until we
understand those, and get community buy-in to define an actual "process"
rather than just a particular PR system, it's unwise just to change the
PR system. (IMHO, there's no reason to further automate a process that
doesn't work correctly.) I hope to have something concrete to present at
BSDCan in May.
As for 2), I've scratched my head for what to do about that for a while,
and not been able to make much progress. Here's what we've tried:
- The creation of a weekly posting "bugs the bugmeister team thinks are
ready for commit". This doesn't seem to have attracted the desired
attention. Perhaps this is a problem of "push" not being the right
solution here; perhaps it just hasn't been publicized enough.
- The creation of a hack for classifying PRs, the "[tag]" convention.
This is simply working around the weakness of the tool. However,
it is sufficient to be able to generate weekly email sorting the
PRs by tag, and another email showing only PRs with patches, also
sorted by tag. If you are a committer, it's also possible to run
queries via:
~gnats/tools/showwithtag
to get a summary.
- Trying to get more traffic on the freebsd-bugbusters@ mailing list.
- Trying to create some interest in #freebsd-bugbusters on EFNet on IRC.
This has not attracted enough committers to be viable yet.
- Holding some "bugathons" (idea stolen from NetBSD) where we try to get
committers to come onto the IRC channel at particular weekends to try
to interactively work through PRs. This had some success, but we have
not done one in a while.
The odd thing is that for ports, the existing PR system -- plus, most
importantly, the hacks we've added on top of it -- works reasonably well.
- Each port has an explicit "maintainer" field (even though many of
the entries are null). Most of the src codebase does not, therefore
no one in particular "owns" it.
- We've taken advantage of that to layer a PR auto-assigner on top, that
also sets things to 'feedback'.
- portsmon is also able to track PRs by the port they affect, and semi-
weekly reminder emails are sent out.
But the first of these items is really particular to ports. Also, more
ports PRs than src PRs are upgrades/bugfixes, rather than true bug reports
that need substantial investigation (in fact, the ratio is probably
exactly reversed). This means we can clear a proportionally larger number
of ports PRs. All of this has helped get the port PR count down over the
last several years, to the point where it no longer seems as overwhelming
as the backlog in the other areas. The size of the backlog creates a
substantial disincentive to try to fix a handful -- thus perpeturating
the cycle.
What we all need to understand is that the PR count will never be at zero;
if we can instead settle for a steady-state, where the most concrete PRs
can be worked on as they arrive, then I'll feel we've have made great
progress.
Unfortunately I don't have any brilliant insights as to how to make the
work more interesting; most of my ideas have the focus of simply making
it less frustrating.
I'll throw the floor open for brainstorming at this point.
mcl
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: PR backlog
Mark and Henrik make a number of good points here. Rather than reply to the details, I'm going to make a couple of quick observations. As a project we're not leveraging the community sufficiently when it comes to contributions. The current system of patch review and submission is very hap-hazard. If you happen to get the attention of the right person at the right time, then it goes in. If not, patches can languish a long time in the PR system. The PR system is also the wrong tool for the job. While Mark touches on the cultural issues in play, they are exacerbated by the misapplication of a problem system to be a patch submission and tracking system. Maybe we need to adopt a practice from the Linux community. At least for arm kernel patches, there is a two step process: submit it to a mailing list for review and refinement, with the second step being submitting it into a queue. I'm not sure the details we need to be successful in the FreeBSD project. Many of the USB patches in the PR system I left alone because I didn't have the time and/or knowledge to evaluate them for inclusion, or I saw something obviously wrong in the patch. When I was trying to just get through the obviously trivial patches. Warner ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: usb/umass, devfs: this sucks
In message: <[EMAIL PROTECTED]> Henrik Gulbrandsen <[EMAIL PROTECTED]> writes: : Look at it from my perspective: I would be happy to complete my fix for : the infamous five-year-old usb/46176, so people can finally detach umass : devices without having to manually unmount them first. However, it will : undoubtedly take a non-trivial amount of time to reproduce and eliminate : the remaining issues. There's no patches in this bug. : I'm more likely to put in that effort if I believe : that my patches may actually end up in CURRENT, but if a one-line fix : such as that in usb/78984 has not been applied after more than a year, : how long will it take for patches that involve multiple subsystems? The patch in this bug is wrong. There are devices that are an odd number of sectors. This is one of the places where the obvious patch isn't necessarily right. Warner ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: ifconfig options?
Hi,
On 24/12/2007, Krassimir Slavchev <[EMAIL PROTECTED]> wrote:
> 'ifconfig -l [address_family]' does not work correct on RELENG_7
>
>
> FreeBSD 6.3-BETA2:
> # ifconfig -l
> em0 em1 plip0 lo0 pflog0
>
> #ifconfig -l ether
> em0 em1
>
> But:
> FreeBSD 7.0-BETA4:
> # ifconfig -l
> em0 em1 plip0 lo0 pflog0
>
> #ifconfig -l ether
> em0 em1 plip0 lo0 pflog0
>
> I need this functionality to get all ethernet interfaces. Is there other
> way to do this?
>
To revert this functionality try this patch please.
--- /usr/src/sbin/ifconfig/ifconfig.c 2007-12-26 23:25:17.0 +0300
+++ /tmp/ifconfig.c 2007-12-26 23:18:53.0 +0300
@@ -298,9 +298,12 @@
* Are we just listing the interfaces?
*/
if (namesonly) {
- if (ifindex > 1)
- printf(" ");
- fputs(name, stdout);
+ if (afp == NULL || afp->af_af != AF_LINK ||
+ sdl->sdl_type == IFT_ETHER) {
+ if (ifindex > 1)
+ printf(" ");
+ fputs(name, stdout);
+ }
continue;
}
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: SMP on FreeBSD 6.x and 7.0: Worth doing?
Brett Glass wrote on Fri, Dec 21, 2007 at 10:31:24PM -0700: > > As has been reported in some other messages on this list, Linux is > currently blowing FreeBSD away. It's taking as much as 20% less > time to get through the benchmark, depending on exactly how the > random shuffle came out. This is with 4 GB RAM, the GENERIC FreeBSD > SMP kernel (using SCHED_ULE), and aufs as the storage schema for Squid. How much CPU load is there on the server during this benchmark? As I have measured and reported since 1997, if you have CPU load and do throughput timing at the same time, then FreeBSD always defaults to satisfy the CPU eating processes and starves network (at least TCP) and Linux is the other way round, keeps the throughput up but CPU needing processes lose out. Kind of opposite of what you'd expect from the ninja macho networker OS (FreeBSD) and the desktop friendly OS (Linux). Anyway, the real question is whether you get anything in return, aka is there anything running on the server that uses CPU but is not directly bound to the network throughput and hence gets work done quicker. > It appears, though I'd need to instrument the code more to be sure, > that the slowdown is coming from file I/O. I could measure the above effects with things coming out of nowhere, aka data made up by a process. If you are observing the same phaenomenon then it has nothing to do with file I/O. > Could it be that there > less concurrency or more overhead in FreeBSD file operations than > there is in Linux? Even with SoftUpdates turned on, the cache > volume mounted with -noatime, and aufs (which uses kqueues -- a > FreeBSD invention -- to optimize multithreaded disk access), the > benchmark shows FreeBSD losing out. Why? All that async and softupdates stuff only matters if you use a lot of small files, which you shouldn't do in the first place if you are building a cache of some sorts. In all likelyness the cache that you are building is satisfying requests out of the RAM, even if the cache is file-backed. You need to verify that before you can blame file I/O. So the real questions right now are: - how much CPU load (post top output) - how much disk activity (post iostat or vmstat output) - can't hurt to have a netstat throughput output, too Martin -- %%% Martin Cracauer <[EMAIL PROTECTED]> http://www.cons.org/cracauer/ FreeBSD - where you want to go, today. http://www.freebsd.org/ ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: SMP on FreeBSD 6.x and 7.0: Worth doing?
On Wed, 26 Dec 2007 21:13:32 +0100 Uwe Doering <[EMAIL PROTECTED]> wrote: > Perhaps this works on later versions of FreeBSD, too. Confirmed, it does. [EMAIL PROTECTED] uname -a FreeBSD kg-work.kg4.no 6.3-PRERELEASE FreeBSD 6.3-PRERELEASE #5: Sun Nov 4 16:19:56 CET 2007 [EMAIL PROTECTED]:/usr/obj/usr/src/sys/SS51G i386 [EMAIL PROTECTED] sysctl -a| grep kvm vm.kvm_size: 1073737728 vm.kvm_free: 247459840 [EMAIL PROTECTED] [EMAIL PROTECTED] uname -a FreeBSD kg-vm.kg4.no 7.0-BETA4 FreeBSD 7.0-BETA4 #0: Sun Dec 2 16:34:41 UTC 2007 [EMAIL PROTECTED]:/usr/obj/usr/src/sys/GENERIC amd64 [EMAIL PROTECTED] sysctl -a | grep kvm vm.kvm_free: 1327493120 vm.kvm_size: 2147479552 HTH -- Regards, Torfinn Ingolfsen ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: SMP on FreeBSD 6.x and 7.0: Worth doing?
Mike Tancsa wrote: At 08:11 AM 12/26/2007, Scott Long wrote: How does one know if the vfs.ufs.dirhash_maxmem is set too high and are exhausting KVA ? Panics, freezes, failure to exec new programs, failure to create pipes, etc. Is there anyway to know ahead of time one is getting close to the stage where all those "bad things" start to happen ? At least on FreeBSD 4.11 you can do sysctl -a|grep kvm and get something like this: vm.kvm_size: 1065353216 vm.kvm_free: 348127232 Perhaps this works on later versions of FreeBSD, too. Now, if vm.kvm_free drops to 10% or so of vm.kvm_size and continues to fall, and vfs.ufs.dirhash_mem still hasn't hit the vfs.ufs.dirhash_maxmem limit, it's time to get concerned. Of course, you can also use the vm.kvm_* values to dimension vfs.ufs.dirhash_maxmem properly in the first place. Regards, Uwe -- Uwe Doering | EscapeBox - Managed On-Demand UNIX Servers [EMAIL PROTECTED] | http://www.escapebox.net ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: PR backlog
I agree with this and there is also the 2nd issue of commiters not responding to those working on updating ports. Point and Fax I have emaile Sobomax about fixing and updating the asterisk ports. and have sent patches but never got a reply in 9 months. If your going to be a maintainer you need to respond to those working on projects already in ports. On December 26, 2007 10:42:24 am M. Warner Losh wrote: > Mark and Henrik make a number of good points here. Rather than reply > to the details, I'm going to make a couple of quick observations. > > As a project we're not leveraging the community sufficiently when it > comes to contributions. The current system of patch review and > submission is very hap-hazard. If you happen to get the attention of > the right person at the right time, then it goes in. If not, patches > can languish a long time in the PR system. > > The PR system is also the wrong tool for the job. While Mark touches > on the cultural issues in play, they are exacerbated by the > misapplication of a problem system to be a patch submission and > tracking system. Maybe we need to adopt a practice from the Linux > community. At least for arm kernel patches, there is a two step > process: submit it to a mailing list for review and refinement, with > the second step being submitting it into a queue. I'm not sure the > details we need to be successful in the FreeBSD project. > > Many of the USB patches in the PR system I left alone because I didn't > have the time and/or knowledge to evaluate them for inclusion, or I > saw something obviously wrong in the patch. When I was trying to just > get through the obviously trivial patches. > > Warner > ___ > [EMAIL PROTECTED] mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-usb > To unsubscribe, send any mail to "[EMAIL PROTECTED]" -- Welcome to the World. An the World gets smaller. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: usb/umass, devfs: this sucks
On Wed, 2007-12-26 at 11:48 -0700, M. Warner Losh wrote: > In message: <[EMAIL PROTECTED]> > Henrik Gulbrandsen <[EMAIL PROTECTED]> writes: > : Look at it from my perspective: I would be happy to complete my fix for > : the infamous five-year-old usb/46176, so people can finally detach umass > : devices without having to manually unmount them first. However, it will > : undoubtedly take a non-trivial amount of time to reproduce and eliminate > : the remaining issues. > > There's no patches in this bug. Correct. I was hoping to have a complete fix before making any official announcements, especially since the approach may be a bit controversial. Fixing underlying issues one by one is not necessarily as clean as doing a complete redesign, but it's pretty efficient and good for merging too. The patches to CURRENT were described in this post to the USB list: http://lists.freebsd.org/pipermail/freebsd-usb/2007-November/004127.html > : I'm more likely to put in that effort if I believe > : that my patches may actually end up in CURRENT, but if a one-line fix > : such as that in usb/78984 has not been applied after more than a year, > : how long will it take for patches that involve multiple subsystems? > > The patch in this bug is wrong. There are devices that are an odd > number of sectors. This is one of the places where the obvious patch > isn't necessarily right. You're right. I realized that myself right after I sent the first email. The patch was already two years old when I submitted it, and I guess I must have focused a little too much on solid-state devices with "large" megabytes, where the sector count is very likely to be a power of two. The patch can be written as a quirk fix for this specific umass device, but (as I've mentioned elsewhere) I'd prefer something that handles the entire category of problems at once. Perhaps an extra line containing "if (is_power_of_two(maxsector))" could be a working compromise, but this can be discussed in a smaller setting than the current one. /Henrik ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: usb/umass, devfs: this sucks
In message: <[EMAIL PROTECTED]> Henrik Gulbrandsen <[EMAIL PROTECTED]> writes: : On Wed, 2007-12-26 at 11:48 -0700, M. Warner Losh wrote: : > In message: <[EMAIL PROTECTED]> : > Henrik Gulbrandsen <[EMAIL PROTECTED]> writes: : > : Look at it from my perspective: I would be happy to complete my fix for : > : the infamous five-year-old usb/46176, so people can finally detach umass : > : devices without having to manually unmount them first. However, it will : > : undoubtedly take a non-trivial amount of time to reproduce and eliminate : > : the remaining issues. : > : > There's no patches in this bug. : : Correct. I was hoping to have a complete fix before making any official : announcements, especially since the approach may be a bit controversial. : Fixing underlying issues one by one is not necessarily as clean as doing : a complete redesign, but it's pretty efficient and good for merging too. : : The patches to CURRENT were described in this post to the USB list: : http://lists.freebsd.org/pipermail/freebsd-usb/2007-November/004127.html These patches look good on their surface, but I want to analyse them in more detail. I had recalled seeing something like this, but I couldn't turn it up in my searching of PR database recently. I'd forgotten it was in the usb list... There's problems even when you just pop the flash card out of the adapter.. I'll see if these fix those or not... : > : I'm more likely to put in that effort if I believe : > : that my patches may actually end up in CURRENT, but if a one-line fix : > : such as that in usb/78984 has not been applied after more than a year, : > : how long will it take for patches that involve multiple subsystems? : > : > The patch in this bug is wrong. There are devices that are an odd : > number of sectors. This is one of the places where the obvious patch : > isn't necessarily right. : : You're right. I realized that myself right after I sent the first email. : The patch was already two years old when I submitted it, and I guess I : must have focused a little too much on solid-state devices with "large" : megabytes, where the sector count is very likely to be a power of two. : : The patch can be written as a quirk fix for this specific umass device, : but (as I've mentioned elsewhere) I'd prefer something that handles the : entire category of problems at once. Perhaps an extra line containing : "if (is_power_of_two(maxsector))" could be a working compromise, but : this can be discussed in a smaller setting than the current one. There's already a quirk for this problem that has been applied to the few devices that it affects. Maybe we need to add one more to the list? There was also talk of forcing a read to the last sector if the sector count was odd, but it never got past talk. Interestingly enough, I found the fact that FreeBSD's dd worked while Linux's didn't at a NIST analysis of forensic tools presented at a conference years ago. Warner ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: PR backlog
On Wed, Dec 26, 2007 at 12:43:06PM -0800, Richard Neese wrote: > Point and Fax I have emaile Sobomax about fixing and updating the asterisk > ports. and have sent patches but never got a reply in 9 months. For situations like this you need to email [EMAIL PROTECTED] We already have defined parameters for resetting inactive maintainers. mcl ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: -net or kernel: where does this belong?
- "Per olof Ljungmark" <[EMAIL PROTECTED]> wrote:
> Posted on -questions but got no response, trying -current and -stable
> as
> well. Sorry for the cross-post but I'm getting kind of desperate...
...
> Since quite a while I have had problems with {CURRENT|RELENG_7} SMP
> machines that lock up when accessed from a remote location over a vpn
> (ipsec) link and sees a ICMP_REDIRECT.
...
Unfortunately, the "How To Repeat" description is rather ... thin.
How do you have IPSec configured? Are you accessing the server remotely via
ssh within the VPN tunnel, or any remote access?
Tom
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "[EMAIL PROTECTED]"
