Re: has anyone installed 5.1 from a SCSI CD?
Peter Jeremy wrote:
> On Sun, Sep 28, 2003 at 06:14:25PM -0400, Sergey Babkin wrote:
> >BTW, I have another related issue too: since at least 4.7
> >all the disk device nodes have charcater device entries in /dev.
>
> 'block' vs 'character' has nothing to do with random or sequential
> access and any application that thinks it does is broken. Any
> application that directly accesses devices must understand all the
> various quirks - ability to seek, block size(s) supported, side-
> effects of operations etc.
As opposed to the kernel understanding them, and representing the
classes of devices uniformly to the user level software.
> Yes, block devices must be random access,
> but character devices can be either random or sequential-only
> depending on the physical device.
But character devices can't be "random-only". Therefore, you
can assume the ability to perform random access on block devices,
and you can assume character devices require sequential access,
and your software will "just work", without a lot of buffer
copying around in user space.
> The only purpose for block devices was to provide a cache for disk
> devices. It makes far more sense for this caching to be tightly
> coupled into the filesystem code where the cache characteristics
> can be better controlled.
Actually, there are a number of other uses for this. The number
one use is for devices whose physical block size is not an even
power of two less than or equal to the page size. The primary
place you see this is in reading audio tracks directly off CD's.
Another place this is useful is in the UDF file system that Apple
was prepared to donate to the FreeBSD project several years ago.
DVD's are recorded in two discrete areas, one of which is an
index section, recorded as ISO9660, and one of which is recorded
as UDF. By providing two distinct devices to access the drive,
it was possible to mount the character device as ISO9660, and
then access the UDF data via the block device. Again, we are
talking about physical block sizes of which the page size is not
an even power of 2 multiple.
Another use for these devices is to avoid the need for some form
of intermediary blocking program (e.g. "dd", etc.) for accessing
historical archives on tape devices. Traditional blocking on
tape devices is 20K, and by enforcing this at the block device
layer, it's possible to deal with streaming of tape devices without
adding an unacceptable CPU overhead.
Another issue is Linux emulation; Linux itself has only block
devices, not character, and when things are the right size
and alignment, the block devices "pass through" and act like
character devices. However... this means that Linux software
which depends on this behaviour will not run on FreeBSD under
emulation.
Finally, block devices serve the function of double-buffering a
device for unaligned and/or non-physical block sized accesses.
The benefit to this is that you do not need to replicate code in
*every single user space program* in order deal with buffering
issues. There has historically been a lot of pain involved in
maintaining disk utilities, and in porting new FS's to FreeBSD,
as a result of the lack of block devices to deal with issues like
this.
I'll agree that the change has been "mostly harmless" -- additional
pain, rather than actually obstructing code from being written
(except that Apple didn't donate the UDF code and it took years to
reproduce it, of course, FreeBSD doesn't appear to have suffered
anything other than a migration of FS developers to other platforms).
On the other hand, a lot of the promised benefits of this change
never really materialized; for example, even though it's "more
efficient" in theory, Linux performance still surpasses FreeBSD
performance when it comes to raw device I/O (and Linux has only
*block* devices). We still have to use a hack ("foot shooting")
to allow us to edit disklabels, rather than using an ioctl() to
retrive thm or rewrite them as necessary, etc., and thus use
user space utilities to do the work that belongs below an abstract
layer in the kernel.
I'm not saying that FreeBSD should switch to the Linux model -- though
doing so would benefit Linux emulation, and, as Linux demonstrates,
it does not have to mean a loss of performance -- but to paint it as
something "everyone agreed upon" or even something "everyone has
grown to accept" is also not a fair characterization.
-- Terry
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: user malloc from kernel
earthman wrote:
> how to allocate some memory chunk
> in user space memory from kernel code?
> how to do it correctly?
If your intent is to allocate a chunk of memory which is shared
between your kernel and a single process in user space, the
normal way of doing this is to allocate the memory to a device
driver in the kernel, and then support mmap() on it to establish
a user space mapping for the kernel allocated memory. In general,
you must do this so that the memory is wired down in the kernel
address space, so that if you attempt to access it in the kernel
while the process you are interested in sharing with is swapped
out, you do not segfault and trap-12 ("page not present") panic
your kernel.
If your intent is to share memory with every process in user
space (e.g. similar to what some OS's use to implement zero
system call gettimeofday() functions, etc.), then you want to
allocate the memory in kernel space (still), make sure it's on
a page boundary, and set the PG_G and PG_U bits on the page(s)
in question.
-- Terry
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: user malloc from kernel
Pawel Jakub Dawidek wrote: > On Mon, Sep 29, 2003 at 06:56:13PM +0300, Peter Pentchev wrote: > +> I mean, won't the application's memory manager attempt to allocate the > +> next chunk of memory right over the region that you have stolen with > +> this brk(2) invocation? Thus, when the application tries to write into > +> its newly-allocated memory, it will overwrite the data that the kernel > +> has placed there, and any attempt to access the kernel's data later will > +> fail in wonderfully unpredictable ways :) > > I'm not sure if newly allocated memory will overwrite memory allocated > in kernel, but for sure process is able to write to this memory. > > Sometime ago I proposed model which will allow to remove all copyin(9) > calls and many copyout(9), but I'm not so skilled in VM to implement it. You probably need two pages; one R/O in user space and R/W in kernel space, and one R/W in both user and kernel space. The copyin() elimination would use the R/W page. Frankly, I have to say that you aren't saving much by eliminating copyin() this way, and most of your overhead is going to be data copies with pointers, and it doesn't really matter where you get the pointers into the kernel, the bummer is going to be copying around the data pointed to by the pointers. For the copyout, you'd probably get a rather larger benefit if you could implement getpid(), getuid(), getgid(), getppid(), and so on, in user space entirely, just by referencing the common read-only page. You could probably also benefit significantly by deobfuscating the timer code and using a flip-flop timer and externalizing the calibration information in a single globally read-only page (PG_G, PG_U, R/O mapping one place, kernel-only R/W mapping another), and then using it to implement a zero system call gettimeofday() operation (there's really no need to have a huge list of timers, if updates are effectively atomic at the clock interrupt, and you use a flip-flop pointer to only two contexts instead of a huge number of them). Specifically, you could find yourself with a huge performance improvement in anything that has to log in the Apache/SQUID styles, which require a *lot* of logging, which would mean a *lot* of system calls. You could also use a knote for this, which is only returned when other knote's are returned, and not otherwise, but that would be a lot less friendly to third party source code that was not specifically adulterated for FreeBSD. -- Terry ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: user malloc from kernel
On Mon, Sep 29, 2003 at 05:04:05PM -0700, Brian O'Shea wrote: > --- Peter Pentchev <[EMAIL PROTECTED]> wrote: [actually, Pawel wrote this:] > > > Here you got sample kernel module which do this: > > > > > > http://garage.freebsd.pl/usmalloc.tgz > > > http://garage.freebsd.pl/usmalloc.README [and then I replied] > > > > E... but won't this interfere *badly* with userland programs > > which attempt to allocate memory after making the syscall in question? > > Couldn't the user library interface to this new system call just > malloc() the memory first in the process, and then pass the pointer > and size to the kernel via the system call interface? This would > ensure that malloc() doesn't touch the desired range of memory until > it is freed by the user. You'd just have to be careful not to free > it until the kernel is done with it. This would be my preferred solution, too, although it might mean that when you are not exactly sure of how much memory the kernel will need, you either overallocate and pass a max-size argument (see most of the socket calls like accept(2), getsockname(2), getpeername(2), etc), or you do two syscalls, one for determining the size needed, and another for actually copying the data (see sysctl(3)). The second approach is more robust, but it does have the overhead of an additional syscall and might also possibly be vulnerable to a race (if the data should change between the two invocations). This, of course, could be worked around by having another couple of syscalls for locking and unlocking the data - lock, query size, fetch, unlock - but that would open a whole new can of worms :) G'luck, Peter -- Peter Pentchev [EMAIL PROTECTED][EMAIL PROTECTED][EMAIL PROTECTED] PGP key:http://people.FreeBSD.org/~roam/roam.key.asc Key fingerprint FDBA FD79 C26F 3C51 C95E DF9E ED18 B68D 1619 4553 If wishes were fishes, the antecedent of this conditional would be true. pgp0.pgp Description: PGP signature
Re: USB2.0 external hub and ehci question
On Tue, Sep 30, 2003 at 05:26:33AM +0200, Barry Bouwsma wrote: > [Drop hostname part of IPv6-only address above to obtain IPv4-capable e-mail, > or just drop me from the recipients and I'll catch up from the archives] > > > Hallo Hackers, I suppose I should post this to -current as the code in > question is derived from there, but I'm running it on RELENG_4, so... > > I've ported the USB controller codes (uhci, ohci, and ehci) from -current > to 4.9-PRERELEASE in order to try and add USB2.0 support to 4.x, and I see > something that I also saw with the NetBSD ehci codes back last December; > namely, that I can't attach an external hub, supposedly with USB2.0 > capability, and have it be recognized. > > First, I seem to have no problems building just the uhci and ohci codes > into the usb.ko kernel module, and using them, though I haven't thoroughly > crash-tested them. > > I've mixed all three controller codes, with the result that the hub is > not seen. Nor is the external drive. Which I attribute to my own > incompetence more than anything. So to make things easier, I ditched > all but the ehci code and ignored the check for companion controllers, > to limit testing to just that. > > With an external USB2.0 drive connected, I am able to see and mount it. > When I connect the external hub in its place, I get the error that the > port was disabled, STALLED -- just as I saw under old NetBSD. > > I haven't built -current, or a more recent NetBSD, to see if their > behaviour is any different when faced with this hub. Is it possible > I need some sort of quirks entry for this device, which I can use as > a USB1.x device fine? Or do I not even get that far? cypress hubs stall the controll endpoint without a reason when running high speed - even if it had one the specs say that the control endpoint shouldn't stall at all. I have a workaround for the probing problem, but USB2 hubs won't work anyway, because at ehci is missing support for interrupt endpoints. Maybe there are other show stopppers too. -- B.Walter BWCThttp://www.bwct.de [EMAIL PROTECTED] [EMAIL PROTECTED] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
IPFW
hello there, I was thinking you have to add some thing about the limiting rules some more control about the way it makes the dynamic rules like in allow tcp from any to any 21,22,80 limit dst-port 50 this would make a dynamic rules to limit each port to 50 but what if I want to control this to limit the total of these ports into 50 ? same with hosts there should be some control over how IPFW creates the dynamic rules. Thanks Regards, ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Custom installworld
It's sometimes necessary to have a set of custom scripts run as part of the installation routine (possibly security changes, possibly local customizations). Is there a hook in the makefiles which would allow local functions to be run? What about generalizing this to work for most common (buildworld, installworld, etc) or all targets? Thank you, James -- James Howard [EMAIL PROTECTED] 202-390-4933 ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: kern/50216: kernel panic on 5.0-current when use ipfw2 with dynamic rules
I reproduced it on the latest 5.1current. Here is the backtrace: # GNU gdb 5.2.1 (FreeBSD) Copyright 2002 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-undermydesk-freebsd"... panic: Most recently used by cred panic messages: --- panic: Most recently used by cred Stack backtrace: syncing disks, buffers remaining... 628 628 628 628 628 628 628 628 628 628 628 628 628 628 628 628 628 628 628 628 giving up on 520 buffers Uptime: 16m0s Dumping 255 MB 16 32 48 64 80 96 112 128 144 160 176 192 208 224 240 --- Reading symbols from /usr/obj/usr/src/sys/IPFW/modules/usr/src/sys/modules/acpi/acpi.ko.debug...done. Loaded symbols for /usr/obj/usr/src/sys/IPFW/modules/usr/src/sys/modules/acpi/acpi.ko.debug Reading symbols from /boot/kernel/daemon_saver.ko...done. Loaded symbols for /boot/kernel/daemon_saver.ko #0 doadump () at /usr/src/sys/kern/kern_shutdown.c:240 240 dumping++; (kgdb) bt #0 doadump () at /usr/src/sys/kern/kern_shutdown.c:240 #1 0xc01a0221 in boot (howto=256) at /usr/src/sys/kern/kern_shutdown.c:372 #2 0xc01a05b7 in panic () at /usr/src/sys/kern/kern_shutdown.c:550 #3 0xc02817f7 in mtrash_ctor (mem=0xc29f8a80, size=0, arg=0x0) at /usr/src/sys/vm/uma_dbg.c:137 #4 0xc028002e in uma_zalloc_arg (zone=0xc083ab40, udata=0x0, flags=257) at /usr/src/sys/vm/uma_core.c:1413 #5 0xc0194a23 in malloc (size=3229854528, type=0xc03020c0, flags=257) at /usr/src/sys/vm/uma.h:234 #6 0xc021e03f in add_dyn_rule (id=0xcd7bfc90, dyn_type=39 '\'', rule=0xc2815e00) at /usr/src/sys/netinet/ip_fw2.c:976 #7 0xc021e43e in install_state (rule=0xc28f3a80, cmd=0xc28f3ac0, args=0xcd7bfc70) at /usr/src/sys/netinet/ip_fw2.c:1140 #8 0xc021f4dc in ipfw_chk (args=0xcd7bfc70) at /usr/src/sys/netinet/ip_fw2.c:1942 #9 0xc0221dd7 in ip_input (m=0xc0ed9800) at /usr/src/sys/netinet/ip_input.c:489 #10 0xc0211a82 in swi_net (dummy=0x0) at /usr/src/sys/net/netisr.c:236 #11 0xc018c762 in ithread_loop (arg=0xc0ebec80) at /usr/src/sys/kern/kern_intr.c:534 #12 0xc018b76f in fork_exit (callout=0xc018c5e0 , arg=0x0, frame=0x0) at /usr/src/sys/kern/kern_fork.c:796 (kgdb) # Here is my full ipfw rule set script: # cat ./ipfwpanic.sh dumpon -v /dev/ad0s1b /sbin/ipfw add allow tcp from any to any established /sbin/ipfw add allow ip from a.b.c.0 to a.b.c.d /sbin/ipfw add allow tcp from any to a.b.c.d 80 limit src-addr 20 setup /sbin/ipfw add allow ip from a.b.c.d to any And I added "IPFIREWALL_DEFAULT_TO_ACCEPT" into kernel configure file. # Here is my test script. I installed an apache on that machine, and use ab to connect 80 port. cat panicstart.sh #!/bin/sh number=0 while (test $number -lt 1) do echo "$number" ab -c 100 http://a.b.c.d/ number=`expr $number + 1` done # This problem can be reproduced on both MP and UP machine. I've tested it on a dell poweredge2650(with 2 P4xeon, HTT enabled/disabled) and a DIY PC(1 PIII CPU). The backtrace I post above is produced on PC(1CPU). ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: nForce MCP network driver - working
Q wrote: > Hi, > > I am in the final stages of porting the NVidia Linux nForce MCP network > driver to FreeBSD-5.1 and am after some experienced users/developers > with access to this hardware to do some testing to find out what breaks, > and what doesn't work. My driver makes use of the Linux nvnetlib.o API > library, and should therefore be compliant with the NVidia Linux > distribution license. > > The driver currently appears to be stable on my hardware (an MSI K7N420 Pro), > although I haven't done much stress testing, nor do I have access to an > nForce2 based motherboard to test. I have a set of nForce2 and nForce3 based boards, but they all run 5.x and the nForce3 is an athlon64 system. > This is still very much a work in progress, but it has been stable > enough for me to actually use productively so I thought I would share > the wealth, so to speak, with the rest of the community. > > If you are interested in testing this, email me offline. I am also > interested in how many people would like to see a FreeBSD-4.x version. > > PS: I am still waiting for NVidia to reply to any of my emails. :( Fun fun. :-( Cheers, -Peter -- Peter Wemm - [EMAIL PROTECTED]; [EMAIL PROTECTED]; [EMAIL PROTECTED] "All of this is for nothing if we don't go to the stars" - JMS/B5 ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
