Re: 20TB Storage System (fsck????)
Max Clark wrote: > Ohh, that's an interesting snag. I was under the impression that 5.x w/ PAE > could address more than 4GB of Ram. The kernel being able to address the RAM does not meant that the KVA+UVA space is larger than 4G. At best, you could take the uiomove/copyin/copyout performance hit, and move both of thse to 4G, each, rather than 4G total. That still limits you to 4G. > If fsck requires 700K for each 1GB of Disk, we are talking about 7GB of Ram > for 10TB of disk. Is this correct? Will PAE not function correctly to give > me 8GB of Ram? To check 10TB of disk? No, it will not. > Is there anyway to bypass this requirement and split fsck into smaller > chunks? Being able to fsck my disk is kinda important. Yes. Limit the number of CG bitmaps you examine simultaneously, and make the operation multiple pass over the disk. This is not that hard a modification to fsck, and it can be done fairly quickly by anyone who understands the code. The code in time to fsck the disk will go up inversely proportionally to the amount of RAM it's allowed to use, which is limited to the UVA size minus the fsck program size itself, and the fsck buffers used for things like FS metadata for a given file/directory. > I have zero experience with either itanium or opteron. What is the current > status of support for these processors in FreeBSD? What would the preferred > CPU be? Will there be PCI cards that I would not be able to use in either of > these systems? I have no idea whether these systems support a larger UVA size, or how much memory you could jam into them... -- Terry ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: possible change for 4.9 rc file?
I oppose this going in prior to 4.9 just on general principles. The concept looks sound on a very light review though. If you'd like more in depth review, please forward your patch to [EMAIL PROTECTED] Doug -- This .signature sanitized for your protection ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: 20TB Storage System (fsck????)
On Thu, Sep 04, 2003 at 01:12:45AM -0700, Terry Lambert wrote: > > Yes. Limit the number of CG bitmaps you examine simultaneously, > and make the operation multiple pass over the disk. This is not > that hard a modification to fsck, and it can be done fairly > quickly by anyone who understands the code. The code in time to > fsck the disk will go up inversely proportionally to the amount > of RAM it's allowed to use, which is limited to the UVA size > minus the fsck program size itself, and the fsck buffers used for > things like FS metadata for a given file/directory. > > Pardon my ignorance but does the number of inodes in the filesystem have a significant impact on the memory requirement of fsck? I ask as it was previously stated the smallest file on the 10TB filessytem would be 500MB which would enable a vastley reduced number of inodes and possibly very large block fragment and cluster sizes? ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
RE: non reliable nmi
> From: Mike Silbersack [mailto:[EMAIL PROTECTED] > On Thu, 28 Aug 2003, Don Bowman wrote: > > > I have machdep.ddb_on_nmi=1. > > I can drop into the debugger with the magic > > key sequence. However, when i hit the NMI > > jumper, i don't always go there. Sometimes > > I do. > > The system is 4-way SMP [2xHTT xeon processors] > > with 4.7. > > > > Any suggestion on where my NMI might be going? > > Is your NMI about 106K in size, and does it have the subjects > "Thank you", > "Wicked screensaver" and others? If so, I think I know where it's > going... ? it was actually a serious question. The nmi header on my board goes into the ICH-3 of my chipset, but shorting the jumper out doesn't always enter the debugger. It does sometimes. If I set the NMI_NOW bit in the ICH-3 I always enter the debugger. I was curious if anyone else had seen this behaviour. --don ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: non reliable nmi
On Thu, Sep 04, 2003 at 09:16:23AM -0400, Don Bowman wrote: > > From: Mike Silbersack [mailto:[EMAIL PROTECTED] > > On Thu, 28 Aug 2003, Don Bowman wrote: > > > > > I have machdep.ddb_on_nmi=1. > > > I can drop into the debugger with the magic > > > key sequence. However, when i hit the NMI > > > jumper, i don't always go there. Sometimes > > > I do. > > > The system is 4-way SMP [2xHTT xeon processors] > > > with 4.7. > > > > > > Any suggestion on where my NMI might be going? > > > > Is your NMI about 106K in size, and does it have the subjects > > "Thank you", > > "Wicked screensaver" and others? If so, I think I know where it's > > going... > > ? > it was actually a serious question. The nmi header on > my board goes into the ICH-3 of my chipset, but shorting the > jumper out doesn't always enter the debugger. It does > sometimes. If I set the NMI_NOW bit in the ICH-3 I always > enter the debugger. I was curious if anyone else had seen > this behaviour. I haven't kept quite up to date on my x86 hardware lately (read: in the past five to ten years), but I distinctly remember a time when everyone referred to x86's NMI as a joke: a non-maskable interrupt that anyone could mask using a simple CLI instruction. Not sure if this is still the case, others would have to say if today's processors still may get so wedged that a NMI request would simply be ignored. The other possibility is some kind of kernel mess-up, bad enough that an NMI does indeed reach the processor, the processor does invoke the NMI handler, yet the handler (DDB) is quite unable to actually do any useful work - messed up data structures and such. G'luck, Peter -- Peter Pentchev [EMAIL PROTECTED][EMAIL PROTECTED][EMAIL PROTECTED] PGP key:http://people.FreeBSD.org/~roam/roam.key.asc Key fingerprint FDBA FD79 C26F 3C51 C95E DF9E ED18 B68D 1619 4553 This sentence no verb. pgp0.pgp Description: PGP signature
Re: FYI - Just got a kernel panic - RELENG_4
On Fri, 29 Aug 2003, Kris Kennaway wrote: > On Thu, Aug 28, 2003 at 05:34:29PM -0400, Marc Ramirez wrote: > > > > supped as of ~ 1:40pm EST today > > > > The panic: > > > > Fatal trap 12: page fault while in kernel mode > > This is probably hardware-related, but it's possible it may be due to > a kernel problem. As a first step, you need to try and get a gdb > traceback with debugging symbols as explained in the developer's > handbook on the website. I forgot to turn off the cvsup cron job; I went to rebuild the kernel with DDB, and found out that my code base had changed... Everything works now. I have been unable to reproduce the crash... Thanks for taking the time, though. Marc. -- Marc Ramirez Blue Circle Software Corporation 513-688-1070 (main) 513-382-1270 (direct) http://www.bluecirclesoft.com http://www.mrami.com (personal) ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Reminder: BSDCon next week in San Mateo!
This is just a friendly reminder e-mail that the BSD Conference is taking place in San Mateo next week, and that if you're planning to attend and haven't yet registered, you might want to. Or, just turn up and register at the door. There's a really strong lineup of FreeBSD-related papers, especially relating to new features in the 5-CURRENT development line. I've attached a list of just some of the interesting things that will be going on there: they include a number of tutorials relating to development and administration, technical session presentations relating to the development of FreeBSD, development of products using FreeBSD, and the deployment of FreeBSD-based systems. And, as always, there will be a variety of invited talks, BoFs and work-in-progress sessions. USENIX has extended their early registration pricing, and also (I believe) has an online registration discount. Multi-employee discounts are also available for companies sending more than one employee. You can find out more about the location, schedule of events, etc, at: http://www.usenix.org/events/bsdcon03/ I look forward to seeing you there! Robert N M Watson FreeBSD Core Team, TrustedBSD Projects [EMAIL PROTECTED] Network Associates Laboratories Several excellent tutorials including one on developing storage extensions using GEOM Keynote: Computing Fallacies (or, What Is the World Coming To?) Reasoning about SMP in FreeBSD devd-A Device Configuration Daemon ULE: A Modern Scheduler for FreeBSD An Automated Binary Security Update System for FreeBSD Building a High-performance Computing Cluster Using FreeBSD build.sh: Cross-building NetBSD Invited Talk: Long Range 802.11 WANs BSD Status Reports GBDE-GEOM Based Disk Encryption Cryptographic Device Support for FreeBSD Enhancements to the Fast Filesystem to Support Multi-Terabyte Storage Systems Invited Talk: Social and Technical Implications of Nonproprietary Software Running BSD Kernels as User Processes by Partial Emulation and Rewriting of Machine Instructions A Digital Preservation Network Appliance Based on OpenBSD Using FreeBSD to Render Realtime Localized Audio and Video Work in Progess Reports (WiPs) Tagging Data in the Network Stack: mbuf_tags Fast IPSec: A High-Performance IPsec Implementation The WHBA Project: Experiences "deeply embedding" NetBSD Invited Talk: Post-Digital Possibilities ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: non reliable nmi
In <[EMAIL PROTECTED]>, on 09/04/03 at 09:12 AM, Peter Pentchev <[EMAIL PROTECTED]> said: >I haven't kept quite up to date on my x86 hardware lately (read: in the >past five to ten years), but I distinctly remember a time when everyone >referred to x86's NMI as a joke: a non-maskable interrupt that anyone >could mask using a simple CLI instruction. Not sure if this is still the >case, others would have to say if today's processors still may get so >wedged that a NMI request would simply be ignored. CLI doesn't stop an NMI, but you can mask off NMI in the CMOS RAM. On standard PC-AT platforms anyway. -- Clark ** This email, and any files transmitted with it, are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please advise [EMAIL PROTECTED] ** ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
getblk() may return NULL, but most caller's don't check for it
Hi All, While perusing the FreeBSD kernel source code today I noticed that there are several reasons that the getblk() function in vfs_bio.c might decide to return a NULL return code. However, checking both 4-stable and 5-current I can find only place (in 5-curren) where the return value from getblk() is actually checked for the possibility of being NULL (in function cluster_rbuild() in vfs_cluster.c near line 389). In all other instances that I can find the returned value is quickly dereferenced w/o any check. What am I missing here? Is this really a bug, or are there extenuating circumstances that prevent getblk() from returning NULL in actual operation? Thanks for any insight that you may provide. Bob -- Bob Willcox Maslow's Maxim: [EMAIL PROTECTED] If the only tool you have is a hammer, Austin, TX you treat everything like a nail. ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: 20TB Storage System (fsck????)
On Wed, 3 Sep 2003, Tim Kientzle wrote: > Max Clark wrote: > > Ohh, that's an interesting snag. I was under the impression that 5.x w/ PAE > > could address more than 4GB of Ram. > > That's >4G of memory in the system. 32-bit processors > are still limited to 4G processor address space, which means > <3G per process (allowing some memory for kernel operations). > You can't get around that unless you either go for a 64-bit > processor or do some complex coding to break your application > storage across multiple processes. It's worse than that, becasue I think that to handle >4GB of ram you need to limit your processes to about 2G of virtual space. ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: non reliable nmi
On 04-Sep-2003 [EMAIL PROTECTED] wrote: > In <[EMAIL PROTECTED]>, on 09/04/03 >at 09:12 AM, Peter Pentchev <[EMAIL PROTECTED]> said: > >>I haven't kept quite up to date on my x86 hardware lately (read: in the >>past five to ten years), but I distinctly remember a time when everyone >>referred to x86's NMI as a joke: a non-maskable interrupt that anyone >>could mask using a simple CLI instruction. Not sure if this is still the >>case, others would have to say if today's processors still may get so >>wedged that a NMI request would simply be ignored. > > CLI doesn't stop an NMI, but you can mask off NMI in the CMOS RAM. On > standard PC-AT platforms anyway. The bug is probably in FreeBSD in that we mask the NMI LVT entry in the local APIC on all the APs and only leave it enabled in the BSP. Thus, if your NMI gets sent to one of the APs, it is effectively ignored. You can try fixing this in mpapic.c like so: Index: mpapic.c === RCS file: /usr/cvs/src/sys/i386/i386/mpapic.c,v retrieving revision 1.64 diff -u -r1.64 mpapic.c --- mpapic.c25 Aug 2003 09:48:47 - 1.64 +++ mpapic.c4 Sep 2003 18:40:47 - @@ -74,7 +74,9 @@ /* setup LVT2 as NMI, masked till later... */ temp = lapic.lvt_lint1; temp &= ~(APIC_LVT_M | APIC_LVT_TM | APIC_LVT_IIPP | APIC_LVT_DM); - temp |= 0x00010400; /* masked, edge trigger, active hi */ + temp |= 0x0400; /* edge trigger, active hi */ + if (PCPU_GET(cpuid) == 0) + temp |= APIC_LVT_MSET; /* masked till later */ lapic.lvt_lint1 = temp; -- John Baldwin <[EMAIL PROTECTED]> <>< http://www.FreeBSD.org/~jhb/ "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: 20TB Storage System (fsck????)
On 4 Sep 2003, at 11:53, Julian Elischer wrote: > On Wed, 3 Sep 2003, Tim Kientzle wrote: > > > Max Clark wrote: > > > Ohh, that's an interesting snag. I was under the impression that > > > 5.x w/ PAE could address more than 4GB of Ram. > > > > That's >4G of memory in the system. 32-bit processors > > are still limited to 4G processor address space, which means > > <3G per process (allowing some memory for kernel operations). > > You can't get around that unless you either go for a 64-bit > > processor or do some complex coding to break your application > > storage across multiple processes. > > > It's worse than that, becasue I think that to handle >4GB of ram you > need to limit your processes to about 2G of virtual space. > Our experience has been that with 4GB of RAM (or more) you really must increase your KVA to 2GB, leaving only 2GB of UVA. So, I would concur with what Julian said. ;-) With the lack of third party filesystem support in FreeBSD, might you be better served by looking at a Linux system running ReiserFS or one of the other file systems designed for such behemoth disk systems? These days, I think Sun even gives away Solaris licenses with their low end x86 servers, so that might even be an option. UFS is great, but there are other filesystems out there that have already addressed such problems from their use in academic, government, and scientific computing where gigantic filesystems tend to be more prevalent. >From my brief research on the subject, the FreeBSD community has been highly resistant to supporting third party filesystems precisely because nobody with such needs as yours has ever contributed the code necessary to make third party filesystem support a reality. The response is usually something like "if you want it, go program it and submit it to a committer for review," which is really the only reasonable response that can be made given FreeBSD's goals. C'est la vie. Sincerely, Andrew Kinney President and Chief Technology Officer Advantagecom Networks, Inc. http://www.advantagecom.net ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: 20TB Storage System (fsck????)
On Thu, 4 Sep 2003, Andrew Kinney wrote: > > > > Our experience has been that with 4GB of RAM (or more) you > really must increase your KVA to 2GB, leaving only 2GB of UVA. > So, I would concur with what Julian said. > > thrown> ;-) > > With the lack of third party filesystem support in FreeBSD, might > you be better served by looking at a Linux system running > ReiserFS or one of the other file systems designed for such > behemoth disk systems? > > These days, I think Sun even gives away Solaris licenses with their > low end x86 servers, so that might even be an option. > > UFS is great, but there are other filesystems out there that have > already addressed such problems from their use in academic, > government, and scientific computing where gigantic filesystems > tend to be more prevalent. > UFS2 will make the filesystem.. All we need is a way to FIX such a filesystem. My brief analysis of this indicates that a 'serial' fsck should be possible. What this would do is read through the filesystem metadata, creating several 'list' files on another filesystem. These would then be duplicated and sorted on several different fields, and then recombined in a 'merge' manner, to produce lists of unallocated files, bad directory entries, duplicate allocated blocks etc. etc. This would probably be workable in a similar order of magnitute of time as a normal fsck, except 'offline' and able to handle a much larger filesystem. julian ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: 20TB Storage System (fsck????)
On Thu, Sep 04, 2003, Andrew Kinney wrote: > Our experience has been that with 4GB of RAM (or more) you > really must increase your KVA to 2GB, leaving only 2GB of UVA. > So, I would concur with what Julian said. > > thrown> ;-) > > With the lack of third party filesystem support in FreeBSD, might > you be better served by looking at a Linux system running > ReiserFS or one of the other file systems designed for such > behemoth disk systems? I understand that ReiserFS fares rather poorly on large filesystems. People have complained about it overflowing the kernel heap when they try to create massive volumes. Perhaps more recent versions have addressed this, and I haven't verified this myself. > These days, I think Sun even gives away Solaris licenses with their > low end x86 servers, so that might even be an option. Umm...the Sun UFS implementation doesn't support >2TB volumes, since it uses 32-bit block numbers. > UFS is great, but there are other filesystems out there that have > already addressed such problems from their use in academic, > government, and scientific computing where gigantic filesystems > tend to be more prevalent. Yep. The most significant problem I see with UFS is that recovering from a crash or power failure is an O(data) operation, and that just doesn't scale well. Other designs (the most prevalent of which involve logging) support O(1) recovery. > >From my brief research on the subject, the FreeBSD community > has been highly resistant to supporting third party filesystems > precisely because nobody with such needs as yours has ever > contributed the code necessary to make third party filesystem > support a reality. The response is usually something like "if you > want it, go program it and submit it to a committer for review," > which is really the only reasonable response that can be made > given FreeBSD's goals. There's probably only a handful of committers who have access to the quantities of storage that would make this kind of filesystem development interesting. Meanwhile, UFS2 is perfectly adequate for most systems. I would imagine that an interested company would have to step forward and fund this sort of work. Moreover, there are licensing issues that would have to be considered to do a port of an existing filesystem... ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
