Re: loadable aio
Mike Smith <[EMAIL PROTECTED]> writes:
> > Terry Lambert <[EMAIL PROTECTED]> writes:
> > > There is so much "goo" around the module loading these days; there
> > > are incursions into "mount" and all sorts of other programs that
> > > should not know about module loading.
> >
> > The kldload(2) interface alone is enough to make me cringe. The way
> > in which it locates a module to load appears to be black magic.
>
> What part of searching a path for a matching file is "black magic"?
>
> Shells have been doing this for decades...
%%%
/*
* Load /boot/kernel/procfs.ko
* XXX: why does this work?
*/
chdir("/");
kldload("procfs");
/*
* Load /boot/kernel/procfs.ko
* XXX: why does this work?
*/
chdir("/");
kldload("procfs.ko");
/*
* Load /boot/kernel/procfs.ko
*/
kldload("/boot/kernel/procfs.ko"); /* Proper interface */
/*
* Move procfs.ko from /boot/kernel to /tmp, then load the copy in /tmp.
*/
system("/bin/mv /boot/kernel/procfs.ko /tmp");
chdir("/tmp");
kldload("procfs.ko"); /* XXX: this doesn't work. */
%%%
If that's not black magic, I'd like to know what is. I'd like to
refer you to the kldload(2) manual, but unfortunately it doesn't
document how kldload(2) works. :(
Best regards,
Mike Barcroft
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message
Re: loadable aio
On Mon, Dec 31, 2001 at 03:48:07AM -0500, Mike Barcroft wrote:
> Mike Smith <[EMAIL PROTECTED]> writes:
> > > Terry Lambert <[EMAIL PROTECTED]> writes:
> > > > There is so much "goo" around the module loading these days; there
> > > > are incursions into "mount" and all sorts of other programs that
> > > > should not know about module loading.
> > >
> > > The kldload(2) interface alone is enough to make me cringe. The way
> > > in which it locates a module to load appears to be black magic.
> >
> > What part of searching a path for a matching file is "black magic"?
> >
> > Shells have been doing this for decades...
>
> %%%
> /*
> * Load /boot/kernel/procfs.ko
> * XXX: why does this work?
> */
> chdir("/");
> kldload("procfs");
>
> /*
> * Load /boot/kernel/procfs.ko
> * XXX: why does this work?
> */
> chdir("/");
> kldload("procfs.ko");
>
> /*
> * Load /boot/kernel/procfs.ko
> */
> kldload("/boot/kernel/procfs.ko");/* Proper interface */
>
> /*
> * Move procfs.ko from /boot/kernel to /tmp, then load the copy in /tmp.
> */
> system("/bin/mv /boot/kernel/procfs.ko /tmp");
> chdir("/tmp");
> kldload("procfs.ko"); /* XXX: this doesn't work. */
> %%%
>
> If that's not black magic, I'd like to know what is. I'd like to
> refer you to the kldload(2) manual, but unfortunately it doesn't
> document how kldload(2) works. :(
Okay, so it's not documented in the manual, but one look at the source
should suffice :)
As Mike said, there is a search path. However, the current directory
is tried first. If a file by that name is not found in the current
directory, the search path is, well, searched ;) The search path
is available in the kern.module_path sysctl or in the output of
'kldconfig -r'.
This is similar to what shells have been doing for decades, with
the added feature of an implicit '.' at the start of the search path.
G'luck,
Peter
--
Do you think anybody has ever had *precisely this thought* before?
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message
Re: loadable aio
On Mon, Dec 31, 2001 at 10:59:40AM +0200, Peter Pentchev wrote:
>
> Okay, so it's not documented in the manual, but one look at the source
> should suffice :)
>
> As Mike said, there is a search path. However, the current directory
> is tried first. If a file by that name is not found in the current
> directory, the search path is, well, searched ;) The search path
> is available in the kern.module_path sysctl or in the output of
> 'kldconfig -r'.
>
> This is similar to what shells have been doing for decades, with
> the added feature of an implicit '.' at the start of the search path.
*oof*
Don't know what I was thinking.
There is NO implicit '.'. There is NO searching the current
directory first. Only the kern.modules_path search path is used.
This explains why your last case - cd("/tmp"); kldload("procfs"); -
does not work.
If you specify the full path, though, that is tried first before
the search path.
G'luck,
Peter
--
The rest of this sentence is written in Thailand, on
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message
kldload(2) family (was Re: loadable aio)
[Moved to -arch, BCC'd to -hackers.] Peter Pentchev <[EMAIL PROTECTED]> writes: > Okay, so it's not documented in the manual, but one look at the source > should suffice :) :( > As Mike said, there is a search path. However, the current directory > is tried first. If a file by that name is not found in the current > directory, the search path is, well, searched ;) The search path > is available in the kern.module_path sysctl or in the output of > 'kldconfig -r'. Oh no, it's worse than I feared. > This is similar to what shells have been doing for decades, with > the added feature of an implicit '.' at the start of the search path. Yes, the usual approach shells take is much better designed. Here's how I would design this interface: o _kldload(2) accepts a file path (similar to open(2)) o kldunload(2) accepts a filename (no path) o kldload(3) accepts a module name (procfs), file name (procfs.ko), or file path (/boot/kernel/procfs.ko). o Search paths for kldload(3) are controlled by the environment variable `KLDPATH' (similar to MANPATH and PATH). o When kldload(3) locates a module file, it calls _kldload(2). o kldload(8) uses kldload(3) o kldunload(8) uses kldunload(2) The main advantage of this design is that it allows a Unix programmer to utilize it. :) Best regards, Mike Barcroft To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Several fields on the Dialog
) and the numeric field next to it.The dialog symbol is the name your COBOL program uses to refer to the dialog box. The numeric field shows the number that will be assigned to that name via a 78-level XE "78-level" XE "level 78" , which the Resource Manager automatically generates in mfres. available.com Description: Binary data
Re: 4.4-STABLE crashes - suspects new ata-driver over wd-drivers
It seems Nils Holland wrote: > root@poison> pciconf -r -b pci0:0:0 0x0:0xff Thanks! this was a kernel without the corruption fix, and it shows that you need it, the MWQ bug has been fixed in your BIOS... I have a new improved patch in the works that covers more chipset comboes, it'll go into -current shortly, and I hope to get permission to get it in 4.5, but so far the RE@ doesn't respond... -Søren To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: userland program panics freebsd 4.3
On 2001-12-18 15:29 -0500, Michael Scheidell <[EMAIL PROTECTED]> wrote: > I have a userland program that canpanic/reboot a freebsd 4.3 system. > Hardware is Intel isp1100 (mbx440 motherboard) 850MHZ pIII, 256mb ram, 640mb > swapfile > software is 'nessusd' (network security scanner) hits the ethernet port > pretty hard when running. > If I read the dumpdev right, it is crashing in the vm section of the kernel, > refrencing a structure that is not within kernel space? This appears to a duplicate of kern/32681, currently assigned to Dag-Erling Smorgrav <[EMAIL PROTECTED]>. You can insert the code shown below as a work-around: > list > 677 > 678 if ((error = getvnode(p->p_fd, SCARG(uap, fd), &fp)) != 0) > 679 return (error); > 680 mp = ((struct vnode *)fp->f_data)->v_mount; if (mp == NULL) return (ENOENT); > 681 sp = &mp->mnt_stat; > 682 error = VFS_STATFS(mp, sp, p); > 683 if (error) > 684 return (error); Regards, STefan To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: userland program panics freebsd 4.3
> > I've had similar problems running nessus against a range of 32 class B > networks (i.e. 2mio addresses) with some 1 hosts expected to be > found. Since I need to complete the scan during the next two weeks > (it's running in batch mode right now; but I had trouble with hanging > nessusd processes and I guess I'll have to manually rescan many hosts.) > > If you are interested, we could share our experience running nessus > under FreeBSD ... No German, I am stuck with english. as for hanging processes, this is probaly due to the bug in the FBSD distributer libpcap, hanging on pcap_next(). I got those also in nessus 1.09 this was fixed in nessusd 1.1x, which uses its one libpcap. I would be interested in seeing if nessus 1.10 or 1.11x panics freebsd at your site. -- Michael Scheidell Secnap Network Security, LLC [EMAIL PROTECTED] 1+(561) 368-9561 See updated IT Security News at http://www.fdma.com/ To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Re: userland program panics freebsd 4.3
On 2001-12-31 10:29 -0500, Michael Scheidell <[EMAIL PROTECTED]> wrote: > as for hanging processes, this is probaly due to the bug in the FBSD > distributer libpcap, hanging on pcap_next(). IIRC, most processes were sleeping in select(). But truss revealed, that some processes were running in a loop executing signal mask functions and not responding to a kill -TERM ... > I got those also in nessus 1.09 > this was fixed in nessusd 1.1x, which uses its one libpcap. Then I must have a different problem, since I'm currently using 1.1.10 (will try 1.1.11 when I'm back in the office on January 8th). > I would be interested in seeing if nessus 1.10 or 1.11x panics freebsd at > your site. Sure it does ;-) http://www.FreeBSD.org/cgi/query-pr.cgi?pr=32681 And it is the same problem you observed (and even the stack trace looks quite similar). The problem occurs on -stable and -current and with SMP and non-SMP kernels. The cause is a NULL pointer dereference in that fstatfs system call, where some pointer hanging off a vnode is cleared. Nessusd tries to read from /proc/PID (for PID = process IDs of plugins spawned) in order to see whether some plugin is still running. (The second method, used only if there is no PROCFS, is to call kill(PID, 0), which will check if a signal could be delivered. That method should probably be prefered to the reading of procfs anyway, since the latter takes 5 system calls instead of a single one in the case of kill() ...) Regards, STefan To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Re: userland program panics freebsd 4.3
> > http://www.FreeBSD.org/cgi/query-pr.cgi?pr=32681 > > The cause is a NULL pointer dereference in that fstatfs system > call, where some pointer hanging off a vnode is cleared. Nessusd > tries to read from /proc/PID (for PID = process IDs of plugins > spawned) in order to see whether some plugin is still running. > (The second method, used only if there is no PROCFS, is to call > kill(PID, 0), which will check if a signal could be delivered. > That method should probably be prefered to the reading of procfs > anyway, since the latter takes 5 system calls instead of a single > one in the case of kill() ...) would this patch fix it on the nessus side?, and it SEEMS to be faster. also, several 'core dumps' listed on nessusd.messages werein fact programs that also were listed as finished. cd ../nessus-core/nessusd --- utils.c.origMon Dec 17 12:02:23 2001 +++ utils.c Mon Dec 31 11:20:12 2001 @@ -898,6 +898,9 @@ if(!pid) return 0; +#ifndef FREEBSD +## panics FREEBSD 4.3 and 4.4, might be fixed in FREEBSD 4.5 + /* * First method : attempt to open /proc/ * (we first check that we can open /proc/ourpid because @@ -917,6 +920,7 @@ else return 0; } +#endif /* * Second method, we attempt to use kill. But first, we * wait() for the process, just in case it's a zombie. -- Michael Scheidell Secnap Network Security, LLC [EMAIL PROTECTED] 1+(561) 368-9561 See updated IT Security News at http://www.fdma.com/ To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Kernel Memory Limit
Anjali Kulkarni wrote: > I have tried this too, it makes absoutely no difference at all. My mallocs > fail after a certain no. of runs of my code(and there is no memory leak), > and there was no difference by increasing MAXDSIZ/DFLDSIZ. You were asking how to increase your VM space. What you should have been asking is how to keep your malloc's from failing. Most kernel memory allocations are type-stable. That means that once something is allocated for one purpose, it can't be reallocated for another purpose, even if that original purpose represented a burst usage condition. This is good and bad. It's good, in that it makes it very easy to debug certain classes of problems. It's bad, in that most of those classes of problems have already been debugged (I think I found one of the last of these when I found the credential reference count overflow, more than six months ago). Most of the allocations are also intrinsically type-stable, in that when you allocate a page for an object type, it gets apportioned out, and unless all objects in that memory region are given back to the system, it can't be coelesced back to a whole page, and then given back to the system. It gets very tempting sometimes, to go to a "handle" based system, like the Macintosh, so that the kernel can move memory allocations around out from under the programs doing the allocation, without then leaving invalid pointer references all over the place because of hidden pointer cloning. In any case, what this means is that: 1) The steady state of kernel memory is page fragmented This means that you cannot expect large allocations to work, except at boot time, for any allocations that require contiguous physical pages. It's possible to work around this, but we would have to add ELF attribution of pages in addition to the tiny number of bits we support (ELF supports 32), to identify code in the paging (and then also relocation) path, since that would let us move around physical memory (in page chhunks) transparent to the other kernel subsystem's usage of memory. 2) The steady state of pages is fragmented This means that you really need to get your pages at the earliest possible opportunity, or you won't be able to reclaim them. The exception to this rule is clean pages acting as buffer for FS data pages (clean backing objects), and dirty swap pages, which could be forced to swap. Within these limits, it's possible to do larger allocations of pages, discontiguous in the kernel virtual address space. One common appraoch, if the only problem you need to solve is a contiguous region of kernel virtual memory space, is to allocate the page maps up front, but not to fill them in with backing pages until later (this is how the mbuf, socket, intcb, pctcb, and similar allocations based on the maximum number of files allowed are typically handled currently). To do this, allocate a zone using ZALLOCI() (a zone in which the allocations are permitted to occur at interrupt time, by way of page fault handling alone, rather than by way of allocation of KVA space, which might involve forced swapping, which can not be done at interrupt time). This must be done at startup. Another approach is to use boot time allocation, where a region that is physically contiguous is allocated in the startup code in /sys/i386/i386/machdep.c; this is difficult to do, unless you understand the code in great detail (it is one of the tutorials in my "FreeBSD memory mamangement at startup" article -- not yet published). I recommend *against* this approach, for your usage, since it requires static allocation of a chunk of memory which can then not be reused for any other purpose. This is most useful in a dedicated purpose embedded system product (for example). Most likely, what you want is a defragger of code and data not in the paging path. Mike Smith and Matt Dillon probably could point you in the right direction on this, but you will need some compiler work (#pragma's for setting section attribution) to make it work, and to identify code in FreeBSD's paging path and your defragger code itself). -- Terry To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: FreeBSD performing worse than Linux?
Matthew Dillon wrote:
> new-reno was artifically limiting the max number of in-transit
> packets to 4. This is probably why the USB ethernet worked
> with 4.3, but it destroyed TCP performance for everything else
> and was removed.
>
> I don't think there is kernel solution to the USB problems, short
> of fixing the driver (if that's even possible). The packets are
> being sent from the server and the only thing the client has
> control over is the receive window advertisement. You could artifically
> reduce the window advertisement on the client to deal with lots of small
> packets but it would destroy performance for larger packets / data
> streams.
Cap the window size via the driver. This will probably require
another attribute on all drivers, indicating "max allowable window
size" ("0" could mean "any", and be used as the default so that we
don't end up flaking out someone talking to Mars).
-- Terry
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message
Re: Kernel Memory Limit
Matthew Dillon wrote: > KVM is only 1G, and a lot of is used-up. You cannot allocate > (directly map) hundreds of megabytes of kernel memory. You can crank up the KVA space, though the handbook is wrong for -release, and woefully out of date for -current. You can also do big allocations, if you do them at startup, but they require heroic measures and have their own limitations. See my other (longer) posting for a clearer explanation. -- Terry To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: FreeBSD performing worse than Linux?
:Cap the window size via the driver. This will probably require
:another attribute on all drivers, indicating "max allowable window
:size" ("0" could mean "any", and be used as the default so that we
:don't end up flaking out someone talking to Mars).
:
:-- Terry
It won't help. The problem is that the server is sending more
data then the client USB adapter can handle. If the client limits
its window size with the expectation that the server will send it
small packets, then any *other* transfer that uses larger packets
will have terrible performance.
-Matt
Matthew Dillon
<[EMAIL PROTECTED]>
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message
Re: FreeBSD performing worse than Linux?
Matthew Dillon wrote:
> :Cap the window size via the driver. This will probably require
> :another attribute on all drivers, indicating "max allowable window
> :size" ("0" could mean "any", and be used as the default so that we
> :don't end up flaking out someone talking to Mars).
>
> It won't help. The problem is that the server is sending more
> data then the client USB adapter can handle. If the client limits
> its window size with the expectation that the server will send it
> small packets, then any *other* transfer that uses larger packets
> will have terrible performance.
I'm talking about capping the maximum negotiable window size
over the USB adapter...
How can this cause problems, since (1) the only thing we are talking
about is window size, not packet size (MTU), and (2) the only thing
it will effect is the TCP stream post negotiation, so it should not
result in a slowdown for any virtual circuit other than those that
*NEED* a slowdown?
-- Terry
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message
Re: FreeBSD performing worse than Linux?
:I'm talking about capping the maximum negotiable window size :over the USB adapter... : :How can this cause problems, since (1) the only thing we are talking :about is window size, not packet size (MTU), and (2) the only thing :it will effect is the TCP stream post negotiation, so it should not :result in a slowdown for any virtual circuit other than those that :*NEED* a slowdown? : :-- Terry If you look at the tcp dump Josef sent, the server was sending 32-byte (data) packets over TCP. It only took 8 packets or so to blow up the USB adapter. 8x32 = 256 bytes. So unless you intend to cap the window size at, say, 128 bytes, this will solve the ssh issue but destroy performance for everything else going over the USB ethernet. -Matt Matthew Dillon <[EMAIL PROTECTED]> To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: FreeBSD performing worse than Linux?
the driver should return ENOBUFS and TCP should repect that.. I think it already does in fact.. On Mon, 31 Dec 2001, Matthew Dillon wrote: > :I'm talking about capping the maximum negotiable window size > :over the USB adapter... > : > :How can this cause problems, since (1) the only thing we are talking > :about is window size, not packet size (MTU), and (2) the only thing > :it will effect is the TCP stream post negotiation, so it should not > :result in a slowdown for any virtual circuit other than those that > :*NEED* a slowdown? > : > :-- Terry > > If you look at the tcp dump Josef sent, the server was sending 32-byte > (data) packets over TCP. It only took 8 packets or so to blow up the USB > adapter. 8x32 = 256 bytes. So unless you intend to cap the window size > at, say, 128 bytes, this will solve the ssh issue but destroy performance > for everything else going over the USB ethernet. > > -Matt > Matthew Dillon > <[EMAIL PROTECTED]> > > > To Unsubscribe: send mail to [EMAIL PROTECTED] > with "unsubscribe freebsd-hackers" in the body of the message > To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: FreeBSD performing worse than Linux?
: :the driver should return ENOBUFS :and TCP should repect that.. :I think it already does in fact.. How does the client's driver returning ENOBUFS prevent the server from sending too many packets? -Matt To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: FreeBSD performing worse than Linux?
:Matt: make up your mind. : :Either: : :1) The server is sending too many packets : :OR : :2) The server is NOT sending too many packets : :The correct way to deal with #1 is to negotiate a smaller window size :so that "too many packets are not sent". : :Actually, the PSC rate-halving algorithm would probably fix this :right up... : :-- Terry Terry, I don't think you quite understand the problem. From the server's point of view NOTHING IS WRONG. Any non-USB-ethernet client would have no problem whatsoever with the large number of small packets that the server is sending, nor can the server just arbitrarily decide that 8 packets is too much, because doing so screws up bandwidth over long-haul links. The server can not arbitrarily limit its window size because the problem only occurs with small packets. So how is the server supposed to figure out the difference between between normal packet loss and packet loss due to the client having a USB adapter and not being able to handle lots of tiny packets? New-Reno might be able to help but it would have to be hacked up a bit to send a double ack to the server to force the server to reduce its congestion window. New-Reno is a mess and I am not volunteering to hack it up more. -Matt Matthew Dillon <[EMAIL PROTECTED]> To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: FreeBSD performing worse than Linux?
my stupid :-) On Mon, 31 Dec 2001, Matthew Dillon wrote: > > : > :the driver should return ENOBUFS > :and TCP should repect that.. > :I think it already does in fact.. > > How does the client's driver returning ENOBUFS prevent the server from > sending too many packets? > > -Matt > > To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: FreeBSD performing worse than Linux?
On Mon, 31 Dec 2001, Terry Lambert wrote: > Matthew Dillon wrote: > > :the driver should return ENOBUFS > > :and TCP should repect that.. > > :I think it already does in fact.. > > > > How does the client's driver returning ENOBUFS prevent the server from > > sending too many packets? > > Matt: make up your mind. > > Either: > > 1)The server is sending too many packets > > OR > > 2)The server is NOT sending too many packets > > The correct way to deal with #1 is to negotiate a smaller window size > so that "too many packets are not sent". Windows are set in BYTES not packets.. what IS set in packets is the backoff winsow size (I forget it's real name) in the slow-start code.. there may be something you could do there... > > Actually, the PSC rate-halving algorithm would probably fix this > right up... > > -- Terry > To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
You can be more successful now..!!
Title: $10K in 30 Days!! I'll Show YOU How To Make At Least $10,000 In 30 Days or Less Every Month... NO RISK!!! Would you like to make at least $10,000 (in 30 days or less) every month? Here's how you can make that money (and more... ) every month, and make your first $10,000 in 30 days or less - with absolutely no risk! Click Here For Complete Details! If you have received this message in error and wish to be removed from future mailings CLICK HERE To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
PRO/1000 vs PRO/100 Performance
Does anyone have a feel for how the PRO/1000 XT card/driver compares to the eepro100/fxp driver as a 100Mb/s solution? Thanks, Dennis To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: FreeBSD performing worse than Linux?
Matthew Dillon wrote: > Terry, I don't think you quite understand the problem. From the server's > point of view NOTHING IS WRONG. Any non-USB-ethernet client would have > no problem whatsoever with the large number of small packets that the > server is sending, nor can the server just arbitrarily decide that 8 > packets is too much, because doing so screws up bandwidth over long-haul > links. The server can not arbitrarily limit its window size because > the problem only occurs with small packets. So how is the server > supposed to figure out the difference between between normal packet > loss and packet loss due to the client having a USB adapter and not > being able to handle lots of tiny packets? Uh, Matt, the window size is a negotiated value between the server and the client. What the server wants is irrelevant, if the client wants a smaller window, since the value is MIN(server_wants,client_wants). So if the problem is that the FreeBSD USB<->ethernet connected client can't handle that many outstanding packets, then the place to fix it is _obviously_ the FreeBSD USB<->ethernet driver. Julian did similar work already in the InterJet in order to control the advertised window size on the client, and therefore do what the AltQ code is supposed to do, but without congesting the intermediate router. > New-Reno might be able to help but it would have to be hacked up a bit > to send a double ack to the server to force the server to reduce its > congestion window. New-Reno is a mess and I am not volunteering to > hack it up more. The Rate-Halving algorithm is used with FACK in order to halve the window size in the presence of failures (like the one being seen here). Again, the place that this should happen is on the client, not the server. I don't understand why you think this is a server problem, when it is not the server hardware where the failure is occuring? -- Terry To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: FreeBSD performing worse than Linux?
Terry, I give up. Maybe if you actually tried to go in and fix it you would see what I'm talking about. For your information: * Julian's work has nothing to do with this particular problem. It has to do with constricted bandwidth. This issue has nothing to do with USB's bandwidth limitations. * The window size is not negotiated between client and server (other then whether window scaling is used or not, which is non-applicable). Each side's TCP protocol receiver controls the window size it advertises. * The window size has no bearing on the number of in-transit packets, unless you make assumptions in regards to the packet size. That's the crux of the problem faced by the receiver in this case. * If you think reducing the receive window on the fly is easy, try it and you will find out that it isn't as easy as you might have thought. * And if you think you can handle that, now try figuring out, in the receiver, how to dynamically increase and reduce the window based on the size of the packets being received and try to discern the difference between packet loss and a driver problem, to then limit the number of in-transit packets (through a massive window reduction). Good luck! * I never said the server was the problem. To the contrary, I've been saying that the client is the problem.. USB ethernet is broken, period. As I have said, it may be possible to make new-reno on the client (receiver) side to cause the transmit side to close its congestion window (and to keep it closed, or fairly closed). But I aint gonna be the one to try to do it. -Matt Matthew Dillon <[EMAIL PROTECTED]> To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: loadable aio
> > What part of searching a path for a matching file is "black magic"?
> >
> > Shells have been doing this for decades...
>
> %%%
> /*
> * Load /boot/kernel/procfs.ko
> * XXX: why does this work?
> */
> chdir("/");
> kldload("procfs");
You should only need the last kldload call. Any other magic is probably
left over from times when kldload didn't work properly.
> If that's not black magic, I'd like to know what is. I'd like to
> refer you to the kldload(2) manual, but unfortunately it doesn't
> document how kldload(2) works. :(
The synopsis for kldload(2) should be "find and load the named kernel
module".
In essence, it takes either a module name and searches the "right"
places for it, or it takes a fully-qualified pathname and loads
exactly that module. That's all; anything else is a bug in the
implementation and should be fixed.
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message
Re: FreeBSD performing worse than Linux?
Matthew Dillon wrote: > > Terry, I give up. Maybe if you actually tried to go in and fix it > you would see what I'm talking about. For your information: I don't have a USB/Ethernet adapter. > * Julian's work has nothing to do with this particular problem. It > has to do with constricted bandwidth. This issue has nothing to do > with USB's bandwidth limitations. The specific work in question is rate limiting the *other end* of a TCP connection. This would let a client with a problem like this *slow the server down*, without any code changes on the server, and without buffer congestion on the server. You have not seen this work, I think. > * The window size is not negotiated between client and server (other > then whether window scaling is used or not, which is non-applicable). > Each side's TCP protocol receiver controls the window size it > advertises. So you are saying that if the client advertised a smaller TCP window, then the server would have less simultaneously oustanding packets in flight before an ACK was required. I think you and I are just using different definitions of "negotiated": the client controls the transmit window which is available to the server. So we are agreed: If the client with the USB bogosity were to advertise a smaller window, then the server would have less packets in flight; and since the problem appears (to me) to be that there are more packets in flight from the server than the USB<->ethernet dongle is capable of buffering to send down the slow (USB) side of the link, then this would prevent the server from buffer-overflowing the dongle into losing packets. It seems to me that this is a clear case of impedence mismatch, with a fast link trying to jam more packets down a slow link than is possible, with the ACKs being sent by the dongle, rather than delayed until it has sufficient buffer space available. > * The window size has no bearing on the number of in-transit packets, > unless you make assumptions in regards to the packet size. That's > the crux of the problem faced by the receiver in this case. This is why I suggested that the PSC rate-halving algorithm would probably be useful. The real problem here is that more data is being sent to the dongle than it is able to forward. The packet trace bears this conclusion out. > * If you think reducing the receive window on the fly is easy, > try it and you will find out that it isn't as easy as you > might have thought. Both Julian's code and the PSC code do this. I wasn't suggesting writing the code from scratch. However, I have to say that I've implicitly done this sort of limitation in a commercial product, before, anolg with Jeffrey Hsu, by controlling transmit and receive window sizes in a proxy server. Clearly, a web server on a 1G link talking to a client on the other end of a 28.8k link would quickly overrun a proxy server's transmit buffers, otherwise, as the received data is shoved out at a much slower rate. This failure mode is consistant with the use of a much slower USB link on the other side of a USB<->ethernet dongle. The problem here is that the don'e itself is not going to do the necessary flow rate limiting (it doesn't, or this discussion would have never happened in the first place), so you have to trigger it as a secondary effect, like the PSC rate halving, or the code Julian did to force the advertised window sizes smaller over time. Julian's code was exactly analogous to the "dongle" situation, except the "dongle" was a CISCO router on the other end of a slow link from an InterJet, where we wanted a single flow to not monopolize all the transmit buffers on the router, so that we would stil be able to get data through. This basically meant controlling the amount of the pool taken up, times the pool retention time, on the router, for any given set of flows. As a side effect, you could intentionally reduce the limit on the advertised window size, until the flow rate dropped. This is basically the balance the TCP Rate Halving code seeks to achieve (the balance between buffer congestion losses and the overall throughput). Please see: http://www.psc.edu/networking/rate_halving.html > * And if you think you can handle that, now try figuring out, in the > receiver, how to dynamically increase and reduce the window based on > the size of the packets being received and try to discern the difference > between packet loss and a driver problem, to then limit the number > of in-transit packets (through a massive window reduction). Good luck! Julian already did this, as I stated. > * I never said the server was the problem. To the contrary, I've > been saying that the client is the problem.. USB ethernet is > broken, period. > > As I have said, it may be possible to make new-reno on the client > (receiver) side to cause the transmit side to
Re: FreeBSD performing worse than Linux?
An underlying issue here is why applications decide to set TCP_NODELAY options on sockets, rather than just letting Nagle's algorithm do the right thing. I recall some handwaving about this in the X server some years ago to make mouse movements "smoother". For the problem at hand, if both the client and server machines didn't do TCP_NODLEAY, then there'd only be one packet smaller than the TCP MSS in flight between the transmitter and receiver at any one time. I think that poking OpenSSH to not set the TCP_NODELAY option "fixed" this problem. I was just pondering the TCP implementation in 4.5-PRERELEASE, and it doesn't look like there's any explicit delay after a write going on, other than Nagle's algorithm, in the TCP packetization code. So setting TCP_NODELAY is almost certain the Wrong Thing for most applications to do. Perhaps there ought to be a warning in the man page about being a poor network citizen, flooding the Internet with tinygrams and otherwise making the performance of your application generally suck. louie To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: FreeBSD performing worse than Linux?
In a message written on Mon, Dec 31, 2001 at 02:44:42PM -0800, Matthew Dillon wrote: > * I never said the server was the problem. To the contrary, I've > been saying that the client is the problem.. USB ethernet is > broken, period. I think several people are trying to say the same thing off this point but not saying it the same way, so: * Hacking TCP (or pretty much anything else) is the _wrong_ solution to this problem. The solution is to "fix" USB ethernet and/or not use it. * Since this is a fairly well defined, easy to reproduce packet loss situation that TCP seems to not handle particularly well, it might be worth using it to investigate if there is a generic TCP improvment that could be made. Fair enough? -- Leo Bicknell - [EMAIL PROTECTED] - CCIE 3440 PGP keys at http://www.ufp.org/~bicknell/ Read TMBG List - [EMAIL PROTECTED], www.tmbg.org To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: FreeBSD performing worse than Linux?
"Louis A. Mamakos" wrote: > > An underlying issue here is why applications decide to set TCP_NODELAY > options on sockets, rather than just letting Nagle's algorithm do > the right thing. I recall some handwaving about this in the X server > some years ago to make mouse movements "smoother". OK, I've been making an assumption here that is perhaps incorrect, but is consistant with the traces posted so far: I've assumed that the data is lost between the ethernet side of the dongle and the USB side of the dongle. If the FreeBSD USB driver is really a dog (I have a hard time buying that) or the dongle USB hardware is superior to that in the PC being used (I have a hard time buying that one, either), then it's just possible that the data losses are FreeBSD's, and not a result of dongle buffer overrun. You'd need a breakout box for USB, and a USB system without the problem (Linux?) to prove it via dumps, though. That said: There is no problem with turning on TCP_NODELAY. If anything, the resulting data sent through the dongle would be *less* bursty (being application paced), and tend to be *less* likely to fill up any pool in the middle to the point of overflow. This really has all the earmarks of a pool congestion (not a link congestion) problem in the dongle, to my mind. > For the problem at hand, if both the client and server machines didn't > do TCP_NODLEAY, then there'd only be one packet smaller than the > TCP MSS in flight between the transmitter and receiver at any one > time. I think that poking OpenSSH to not set the TCP_NODELAY option > "fixed" this problem. I think it masked it. If you had a less CPU-intensive test application, I think you would see that you can still overrun the buffer. The masking is a natural effect of decreasing the overall payload encapsulation overhead. But increase the amount of payload, and you should still see the problem rear its ugly head, since the link rate differential is so large on either side of the dongle. A fugly way of handling the problem would be to initiate a flow control mechanism between the dongle and the FreeBSD ethernet driver. This is tantamount to the hardware flow control required for the MNP modem knock-offs by Microcomm competitors who used flow control as a means of keeping the pool size manageable, so that they could really cheap out on putting RAM in the modem, and undercut Microcomm's prices. For this to work, you'd have to have cooperative firmware in the dongles; I think that taking that approach is a lost cause, unless we are downloading our own firmware. I think we will have to tackle it another way. Teaching the driver about TCP windows, and delaying transmissions until 50% of the receive window is satisfied would probably work, but is a very unsatisfactory approach (even though we know that transmit over the link is not going to have the same problem as receive, due to the speed differential across the dongle). -- Terry To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: FreeBSD performing worse than Linux?
:An underlying issue here is why applications decide to set TCP_NODELAY :options on sockets, rather than just letting Nagle's algorithm do :the right thing. I recall some handwaving about this in the X server :some years ago to make mouse movements "smoother". : :For the problem at hand, if both the client and server machines didn't :do TCP_NODLEAY, then there'd only be one packet smaller than the :TCP MSS in flight between the transmitter and receiver at any one :time. I think that poking OpenSSH to not set the TCP_NODELAY option :"fixed" this problem. : :I was just pondering the TCP implementation in 4.5-PRERELEASE, and :it doesn't look like there's any explicit delay after a write going :on, other than Nagle's algorithm, in the TCP packetization code. So :setting TCP_NODELAY is almost certain the Wrong Thing for most :applications to do. Perhaps there ought to be a warning in the :man page about being a poor network citizen, flooding the Internet :with tinygrams and otherwise making the performance of your application :generally suck. : :louie Yes, you are correct. There is no real reason for ssh to set TCP_NODELAY on FreeBSD and, in fact, I believe it didn't used to. We should just turn it off. -Matt Matthew Dillon <[EMAIL PROTECTED]> To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: FreeBSD performing worse than Linux?
Matthew Dillon wrote: > Yes, you are correct. There is no real reason for ssh to set > TCP_NODELAY on FreeBSD and, in fact, I believe it didn't used to. > We should just turn it off. FWIW, I agree that it should not be set. Setting socket options has the unfortunate side effect of grabbing more per connection memory than would otherwise be necessary. Still, it's legal to set the option, even if it's not really a sane thing to do in many cases, so disallowing it is not a "fix" for anything. -- Terry To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: FreeBSD performing worse than Linux?
Disabling Nagle's algorithm for no good reason has very poor scaling behavior. This is what happens when TCP_NODELAY is enabled on a socket. If you look at the work function for most network elements, the part that runs out of gas first is per-packet forwarding performance. Sure, you need to have adequate bus bandwidth to move stuff through a box, but it's performing per-packet forwarding operations and policy which is the resource that's most difficult to make more of. I think this is true for toy routers based on PC platform as well as high-end boxes like the Cisco 12000 series. Juniper managed adequate forwarding performance using specialized ASIC implementions in the forwarding path. Of this statement, I'm sure; in my day job at UUNET, I talk to all the major backbone router vendors, and forwarding performance (and also reasonable routing protocol implementions) is a show-stopper requirement they labor mightily over. So here was have a mechanism with wonderful properties - it's a trivial yet clever implementation of a self tuning mechanism to prevent tinygrams from being generated by a TCP without all manner of complicated timers. It give great performance on LAN and other high-speed interconnects where remote echo type applications are demanding, yet over long delay paths where remote echo is gonna suck no matter what you do, it automatically aggregates packets. Nagle's algorithm and Van Jacobson's slow-start algorithm allowed the Internet to survive over congested paths. And they did so with a bunch of self-tuning behavior independent of the bandwidth*delay product of the path the connection was running over. It was and is amazing stuff. Likewise, the original problem in this thread is likely caused by some part of the USB Ethernet implementation having inadequate per-packet resources. It's probably not about the number of bytes, but the number of transactions. You see here a modern reimplementation of essentially the same problem that the 3COM 3C501 ISA ethernet card had 15 years ago - back to back packets were consistantly dropped because of the poor per-packet buffering implementation. It was absolutely repeatable. Sure, it's "legal" to generate streams of tinygrams and not use Nagle's algorithm to aggregate the sender's traffic, but it's just plain rude and on low bandwidth links, it sucks because of all the extra 40 byte headers you're carrying around. I'm sure TCP_NODELAY got added because it sounds REALLY C00L to make the interactive thing go better. But clearly people don't understand the impact of turning on the cleverly named option and how it probably doesn't really improve things. louie To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Running out of bufferspace
Hi,
First of all happy new year, etc. =)
My problem: I'm writing a device driver kernel module that uses kernel
level sosend() from the d_write write() function. But it runs out of
bufferspace (error 55) when I really stress it (sending 15 megs in 3-4 secs
over UDP with 32 or 60K packets).
Thing is, my mbufs are fine, peak is 1/3rd of max. I also use sowriteable()
to check if it's safe to send, and sbspace() tells me I have the full 128K
sndbuf that I configured available.
So I have 2 questions...
1) which buffers are there that I can run out of here and where can I check
their status?
2) Should I free() the uio I get from userspace after I have passed it to
the sosend?
Side note on 1): after I run out of bufferspace with my little driver the
box continues to function fine as far as I can tell, except that
mount_smbfs has trouble:
[ root@hera:~ ] # truss mount -t smbfs //drwilco@ceres/D$ /ceres/d_drive/
readlink("/etc/malloc.conf",0xbfbffa94,63) ERR#2 'No such file or
directory'
mmap(0x0,4096,0x3,0x1002,-1,0x0) = 671551488 (0x28071000)
break(0x807c000) = 0 (0x0)
break(0x807d000) = 0 (0x0)
open(".",0,00) = 3 (0x3)
chdir(0xbfbff090)= 0 (0x0)
sigaction(SIGSYS,0xbfbfeaac,0xbfbfea94) = 0 (0x0)
__getcwd(0xbfbff090,0x400) = 0 (0x0)
sigaction(SIGSYS,0xbfbfea94,0x0) = 0 (0x0)
fchdir(0x3) = 0 (0x0)
close(3) = 0 (0x0)
stat("/ceres/d_drive",0xbfbfeff4)= 0 (0x0)
fork() = 3331 (0xd03)
smbfs: can't get server address: syserr = No buffer space available
SIGNAL 20
wait4(0xd03,0xbfbff088,0x0,0x0) = 3331 (0xd03)
exit(0x1) process exit, rval = 256
A different tactic could be that I pull a m_devget() or some other mbuf
creation stunt and copy the data from the uio into the mbuf. Do I have to
clear the mbuf and/or uio after passing that mbuf to sosend?
Any pointers (even to some docs, though I've done heavy googling) or hints
would be greatly appreciated.
DocWilco
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message
Re: FreeBSD performing worse than Linux?
>Yes, you are correct. There is no real reason for ssh to set >TCP_NODELAY on FreeBSD and, in fact, I believe it didn't used to. >We should just turn it off. Nagle + delayed acks don't really get along all that well, and often result in character echoes lagging .25-.5 seconds behind the keystrokes (due to round trip + 200ms). This really makes editing files and other interactive jobs rather painful. -DG David Greenman Co-founder, The FreeBSD Project - http://www.freebsd.org President, TeraSolutions, Inc. - http://www.terasolutions.com President, Download Technologies, Inc. - http://www.downloadtech.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: FreeBSD performing worse than Linux?
:>TCP_NODELAY on FreeBSD and, in fact, I believe it didn't used to. :>We should just turn it off. : : Nagle + delayed acks don't really get along all that well, and often :result in character echoes lagging .25-.5 seconds behind the keystrokes (due :to round trip + 200ms). This really makes editing files and other interactive :jobs rather painful. : :-DG : :David Greenman I think that has been fixed. Try it. It doesn't lag for me. The turn-around echo of the keystroke should be pushed out instantly. -Matt To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
