Re: using FreeBSD to create a completely new OS
Forgot to mention this project vary well turn into my PhD thesis ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: using FreeBSD to create a completely new OS
On Mon, Dec 10, 2012 at 2:56 AM, Aryeh Friedman wrote: > On Mon, Dec 10, 2012 at 2:38 AM, Zaphod Beeblebrox wrote: > I know (about the list not being google) and have seen the formats for > MBR's (even wrote a few by hand) the question was how to extract it > from the virtual HDD (see above) namely is I have a prototype MBR > that should be made I want to check if it was actually written to the > virtual HDD correctly The simplest thing I can think of is to attach said virtual disk to a FreeBSD guest and use dd. Sure ... I'm pretty sure the virtual disk you're using is documented, but follow the path of least resistance. > From my initial research there is almost nothing that depends on > actual ASM except for a very small number of things that create the > environment needed to run the equiv of the JVM (no garbage collection > at this level) natively on an x86. My guess the total amount of ASM > needed is less then a thousand instructions (99% are macro's mostly > for performance reasons). The other 1% is just to get the protected > mode and the native JVM loaded and started. If you mean swtch.s here, I suppose you're interpreting what I said rightly. > Note when I say JVM I actually mean a collection of VM's that model > real HW (not some abstract HW like the sun JVM does) and if the VM > just happens to be identical to the native HW then it skips the > emulation part of the VM (i.e. direct execution on the actual HW). > The first such VM I am planning is x86 (64-bit). Namely if your > running on a PC then it runs natively but on anything else it runs on > a VM identical to a PC. Note that this pretty much makes java pretty much worthless. I say that carefully as the seemingly "good" thing that makes the extra effort in java "worth it" is the guarantee against the halting problem that makes the security madness possible. If your "vm" is i386, you can't offer that guarantee. The JVM is very specially crafted for this purpose. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: using FreeBSD to create a completely new OS
On Sun, 09 Dec 2012 23:48:12 EST Aryeh Friedman wrote: > For personal hobby reasons I want to write an OS completely from > scratch (due to some aspects of the design no existing OS is a > suitable starting place)... what I mean is I want to start with the > MBR (boot0) and go on from there... I only have one *REAL* machine to > work with which means I need to work with something like > emulators/virtualbox-ose... I also want to do as many automated tests > as possible (for example seeing if the installer copied the MBR [and > later other stuff] correctly to the virtual HDD) for this reason I > have a few questions on vb (or perhaps QEMU if not possible in vb): > > 1. Can it be scripted? > 2. Is there any documentation on the various virtual HDD formats and > such (that way I can check the "physical" drive and not by indirect > query)? > > Also can people give me some idea of a good general > development/testing framework the one I have in mind so far is: You may wish to check out http://wiki.osdev.org/Expanded_Main_Page http://wiki.osdev.org/Projects ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: using FreeBSD to create a completely new OS
The reason for Java has nothing to do with VM's per se but with some of the networking features I have in mind. Also from a theoretical point of view any turning complete vm would do (the actual HW differences are fairly easy to abstract in the model I have in mind and polymorphism is a must for various other reasons that would take far to long to explain so that means either C++ or Java... from a pure CS point of view C++ is the wrong choice (in my mind the only arg for it is it has ptr 's which in reality often are too easy to misuse anyways and that's the reason for wrapping them in Java like I described in the OP). ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: using FreeBSD to create a completely new OS
> Note that this pretty much makes java pretty much worthless. I say > that carefully as the seemingly "good" thing that makes the extra > effort in java "worth it" is the guarantee against the halting problem > that makes the security madness possible. If your "vm" is i386, you > can't offer that guarantee. The JVM is very specially crafted for > this purpose. On the halting problem unless my basic theory of computation is very out of date it is not possible for any turing complete machine to guerntee against the halting problem btw I think the AI crowd would love to hear that Java is a oracle (pun intended) which is likely more then any human can do (as far I know the current thinking is humans are somewhere between UTM's and oracles) ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: using FreeBSD to create a completely new OS
Hi, On Mon, 10 Dec 2012 03:08:11 -0500 Aryeh Friedman wrote: > Forgot to mention this project vary well turn into my PhD thesis I would then extremely careful with Java. There are many reasons why assembler and C are still the languages used for OS programming. Erich > ___ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to > "freebsd-hackers-unsubscr...@freebsd.org" ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: using FreeBSD to create a completely new OS
On Mon, Dec 10, 2012 at 3:44 AM, Erich Dollansky wrote: > Hi, > > On Mon, 10 Dec 2012 03:08:11 -0500 > Aryeh Friedman wrote: > >> Forgot to mention this project vary well turn into my PhD thesis > > I would then extremely careful with Java. > > There are many reasons why assembler and C are still the languages used > for OS programming. Thats why I have spent almost 5 years doing pencil and paper designs to make sure it fits into the Java computational model but still has full HW access ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: using FreeBSD to create a completely new OS
> Forgot to mention this project vary well turn into my PhD thesis
That's a rather different context than "personal hobby reasons".
I was going to suggest this anyway, but for a dissertation project
I would _very strongly_ suggest that you thoroughly research Linus
Torvalds' experiences in implementing Linux, as well as any other
low-level OS implementation projects that you can find in the
literature (Multics, Unix, and OS/360 at least; Symbol 2R also
comes to mind). Much of what those earlier implementors learned
about "building an OS from scratch" will be directly applicable.
(Granted Linus started with the SVID and you'll be writing your own
specification, but the process of getting from {specification} to
{code that actually works} is almost guaranteed to involve many of
the same issues. PhD work is supposed to advance the state of the
art, not rediscover the hard way what could have been learned in
the library.)
Perhaps equally important, I hope you're planning to figure out
early on what "significant contribution to knowledge[1]" you will
be making, and consider with your advisor whether it's necessary
to go to all the work of building an entirely new OS to make that
contribution. Again, you want to be standing on the shoulders of
those who have gone before, not retracing their footsteps.
[1] For those not familiar, that's the phrase typically used to
describe the level of work that a PhD dissertation is supposed
to report -- or at least it was when I was in graduate school.
P.S. Just for general interest, what university?
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: using FreeBSD to create a completely new OS
> P.S. Just for general interest, what university? CUNY grad center (under a special arrangement that allows me to skip the course work part of grad school) [I have not formally started yet] As to your complexity issues I do not plan to do the whole OS just the stuff up to mid-level I/O (for FreeBSD this is also known as user land I/O) ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: using FreeBSD to create a completely new OS
On 12/10/12 10:02 AM, Perry Hutchison wrote: Hi, >> Forgot to mention this project vary well turn into my PhD thesis > > That's a rather different context than "personal hobby reasons". > > I was going to suggest this anyway, but for a dissertation project > I would _very strongly_ suggest that you thoroughly research Linus > Torvalds' experiences in implementing Linux, as well as any other > low-level OS implementation projects that you can find in the > literature (Multics, Unix, and OS/360 at least; Symbol 2R also > comes to mind). Much of what those earlier implementors learned Also Plan9 (http://plan9.bell-labs.com/plan9/) and Amoeba (http://en.wikipedia.org/wiki/Amoeba_distributed_operating_system) Grtnx, -- Bert Kiers XS4ALL UNIX systeembeheerder, suspected terrorist 1984 was not meant as a manual ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: using FreeBSD to create a completely new OS
> ... a special arrangement that allows me to skip > the course work part of grad school ... [shudder] I hope that "special arrangement" includes passing the final exams, or otherwise demonstrating that you already know the content, of at least the minimum course work that would ordinarily be required :) > [I have not formally started yet] So you don't yet have a formally-assigned advisor :( I hope you at least have had some substantial (although necessarily informal) discussions with the department head, and/or the professor whom you anticipate will be your advisor, before doing a lot of work that might turn out not to be useful in your pursuit. > I do not plan to do the whole OS just the stuff up to mid-level I/O > (for FreeBSD this is also known as user land I/O) While the userland is by far the _largest_ part, the low-level kernel is likely to be the _trickiest_ part to get right ;) Another OS for your research list: Mach (which had been around for a while before being adopted as the basis of Darwin, the MacOS X kernel). Darwin is highly modular; depending on what you're doing, you may be able to write less code from scratch by reusing or adapting some parts of it. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: kernel module parallel build?
On Dec 6, 2012, at 1:28 AM, Garrett Cooper wrote: > On Wed, Dec 5, 2012 at 3:51 PM, Damien Fleuriot wrote: > > ... > >> All trolling aside, I believe an awesome fix to be setting module override >> in /etc/make.conf to only build the 4-5 specific modules one needs. >> >> To be honest I think this configuration tweak should be advertised a bit >> more as it definitely speeds up kernel builds. >> >> I would be happy to check if this is advertised in the handbook in the >> "rebuilding kernel" section and enhance its visibility if required. >> >> I can provide en_US and fr_FR. > > +1. Please write it up if you can; it's much quicker/better than the > kitchen sink approach if you know what you're doing and don't have to > build for a large set of platforms. > Thanks! > -Garrett I've had a look at the handbook and the MODULES_OVERRIDE bit is already well written and definitely visible: http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/kernelconfig-building.html I believe no rewrite is necessary. However, I fail to see any mention of nextboot. It might be a good idea to let people know they can boot their new kernel just once to see if it works, and revert to their old kernel if boot should fail. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: calendar(1) regressions
Hi! On 2012-12-10, Greg 'groggy' Lehey wrote: > > Unfortunately r205821 [1] has caused several regressions to calendar(1). > > Relevant PRs: > > > > bin/157718 > > bin/162211 > > bin/168785 > > bin/170930 > > I think we fix bugs rather than revert the commits. Of course it's preferred but I didn't see any progress happening. As a result some of the basic functionality is broken (especially in stable/8). Maybe I should have debugged and fixed those bugs but I wasn't really keen to debug a large commit adding mostly features which I don't care much. > I've had a chat with edwin@, and I'll look at the issues Real Soon > Now. This is great news! Thanks. -- Jaakko ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: using FreeBSD to create a completely new OS
If you consider VirtualBox for automated tests, you should make some testing life cycle. I'd recommend to you: 1. create sparse file (truncate -s 1G for example) 2. mdconfig -af file (produced md0) 3. work with md0 as if it was real hw disk (write boot sectors, fs, etc), then destroy md0 (mdconfig -du 0) 4. create vmdk for file ( https://github.com/yerenkow/freebsd-vm-image/blob/master/freebsd-firmware/create-scsi-vmdk.sh -- this helps me create vmdk for raw file and feed them to VmWare/Virtualbox just fine) 5. boot VirtualBox using cli (and run some tests). I had some experience in running in-virtualbox-os tests a while ago, but there was real OS there, and not test one. Hope this helps. -- Regards, Alexander Yerenkow ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: using FreeBSD to create a completely new OS
suitable starting place)... what I mean is I want to start with the MBR (boot0) and go on from there... I only have one *REAL* machine to work with which means I need to work with something like emulators/virtualbox-ose... I also want to do as many automated tests as possible (for example seeing if the installer copied the MBR [and later other stuff] correctly to the virtual HDD) for this reason I have a few questions on vb (or perhaps QEMU if not possible in vb): 1. Can it be scripted? for such usage i would recommend qemu. Virtualbox is for production use with common OSes, and is very ineffective in batch usage and inflexible, but fast. qemu is slow but you can have anything. including precise definition of your virtual hardware such (that way I can check the "physical" drive and not by indirect query)? just use "raw" format. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Building v8 on FreeBSD with clang
I'm having a problem building google v8 on FreeBSD with clang. I get this error: /usr/bin/ld: final link failed: Nonrepresentable section on output clang++: error: linker command failed with exit code 1 (use -v to see invocation) gmake[1]: *** [/root/v8/out/x64.release/cctest] Error 1 gmake[1]: Leaving directory `/root/v8/out' gmake: *** [x64.release] Error 2 I needed to use gmake rather than make, and I added the following to use clang: setenv CC /usr/bin/clang setenv CXX /usr/bin/clang++ setenv GYP_DEFINES "clang=1" Some version info: # clang -v FreeBSD clang version 3.1 (branches/release_31 156863) 20120523 Target: x86_64-unknown-freebsd9.0 Thread model: posix # gmake -v GNU Make 3.82 Built for amd64-portbld-freebsd9.1 command used to checkout v8 code: # svn checkout http://v8.googlecode.com/svn/trunk/ v8 # uname -a FreeBSD thug 9.1-RC3 FreeBSD 9.1-RC3 #0 r242324: Tue Oct 30 00:58:57 UTC 2012 r...@farrell.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC amd64 ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: calendar(1) regressions
On Monday, 10 December 2012 at 16:27:39 +0200, Jaakko Heinonen wrote: > > Hi! > > On 2012-12-10, Greg 'groggy' Lehey wrote: >>> Unfortunately r205821 [1] has caused several regressions to calendar(1). >>> Relevant PRs: >>> >>> bin/157718 >>> bin/162211 >>> bin/168785 >>> bin/170930 >> >> I think we fix bugs rather than revert the commits. > > Of course it's preferred but I didn't see any progress happening. As a > result some of the basic functionality is broken (especially in > stable/8). Maybe I should have debugged and fixed those bugs but I > wasn't really keen to debug a large commit adding mostly features which > I don't care much. IIRC the main issue in 8 was that debug output was appearing as a matter of course. It would be relatively trivial to MF9 (i.e. copy the 9-STABLE version), but of course that would only solve part of the problem. Do you want an MFC to 8 when I'm done? Greg -- Sent from my desktop computer. Finger g...@freebsd.org for PGP public key. See complete headers for address and phone numbers. This message is digitally signed. If your Microsoft MUA reports problems, please read http://tinyurl.com/broken-mua pgpUfyx3iwSIY.pgp Description: PGP signature
Re: 9.x -- New Install -- serious partition misalignment
Apparently, I owe everyone an apology. I found out (a bit too late) that the WD 1 TB "black" drive that had come to me in sealed anti-static bag with a warning on it about "Advanced Format" drives (and possible OS incompatability) was not in fact itself an AF drive. Rather it is a traditional drive with 512 byte sectors. I tried doing a test install of FreeBSD 9.1-RC3 onto a different drive... one that I am quite sure had the new AF 4KB physical blocks and sure enough, all partitions were properly and automagically aligned on 4KB boundaries. I have put in a request to cancel the PR that I had filed regarding this issue. My thanks for Nathan Whitehorn for his input and forbearance. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: calendar(1) regressions
"Greg 'groggy' Lehey" wrote: > > --cHMo6Wbp1wrKhbfi > Content-Type: text/plain; charset=us-ascii > Content-Disposition: inline > > On Monday, 10 December 2012 at 16:27:39 +0200, Jaakko Heinonen wrote: > > > > Hi! > > > > On 2012-12-10, Greg 'groggy' Lehey wrote: > >>> Unfortunately r205821 [1] has caused several regressions to calendar(1). > >>> Relevant PRs: > >>> > >>> bin/157718 > >>> bin/162211 > >>> bin/168785 > >>> bin/170930 > >> > >> I think we fix bugs rather than revert the commits. > > > > Of course it's preferred but I didn't see any progress happening. As a > > result some of the basic functionality is broken (especially in > > stable/8). Maybe I should have debugged and fixed those bugs but I > > wasn't really keen to debug a large commit adding mostly features which > > I don't care much. > > IIRC the main issue in 8 was that debug output was appearing as a > matter of course. It would be relatively trivial to MF9 (i.e. copy > the 9-STABLE version), but of course that would only solve part of the > problem. Do you want an MFC to 8 when I'm done? > > Greg > -- > Sent from my desktop computer. > Finger g...@freebsd.org for PGP public key. > See complete headers for address and phone numbers. > This message is digitally signed. If your Microsoft MUA reports > problems, please read http://tinyurl.com/broken-mua > Hi Greg I too filed send-prs re. rgeression bugs in recent broken calendar, hopefully you can look/ fix please. http://www.berklix.com/~jhs/src/bsd/fixes/FreeBSD/src/gen/usr.bin/calendar/REAME.JHS bin/15182 http://www.berklix.com/~jhs/src/bsd/fixes/FreeBSD/src/gen/usr.bin/calendar/send-pr.impossible_date.no_customise bin/157718 http://www.berklix.com/~jhs/src/bsd/fixes/FreeBSD/src/gen/usr.bin/calendar/core_dump_shell.no_customise bin/162211 http://www.berklix.com/~jhs/src/bsd/fixes/FreeBSD/src/gen/usr.bin/calendar/send-pr.first_last_in_month.no_customise -- PS for Anyone, doesnt need to be Greg misc/157748 This PR is NOT a regression coding error, but corrections & additions for a lot of badly sorted & incorrect src/ data in wrong calendar files etc. It doesn't need a programmer to fix it, any commiter could make corrections & also someone with a sense of orthogonality could shuffle data around, as some current calendar data in src/ is in wrong files, & also in some cases date is wrong. http://www.berklix.com/~jhs/src/bsd/fixes/FreeBSD/src/gen/usr.bin/calendar/calendars/calendar.holiday.send-pr http://www.berklix.com/~jhs/src/bsd/fixes/FreeBSD/src/gen/usr.bin/calendar/calendars/ Cheers, Julian -- Julian Stacey, BSD Unix Linux C Sys Eng Consultant, Munich http://berklix.com Reply below not above, like a play script. Indent old text with "> ". Send plain text. Not: HTML, multipart/alternative, base64, quoted-printable. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: calendar(1) regressions
On Tuesday, 11 December 2012 at 3:52:32 +0100, Julian H. Stacey wrote: > "Greg 'groggy' Lehey" wrote: >> On Monday, 10 December 2012 at 16:27:39 +0200, Jaakko Heinonen wrote: >>> >>> Hi! >>> >>> On 2012-12-10, Greg 'groggy' Lehey wrote: > Unfortunately r205821 [1] has caused several regressions to calendar(1). > Relevant PRs: > > bin/157718 > bin/162211 > bin/168785 > bin/170930 > > I too filed send-prs re. rgeression bugs in recent broken calendar, > hopefully you can look/ fix please. > > bin/157718 > bin/162211 These two are mentioned above. > bin/15182 > http://www.berklix.com/~jhs/src/bsd/fixes/FreeBSD/src/gen/usr.bin/calendar/send-pr.impossible_date.no_customise This one was closed a little over 11 years ago. Since then, the code base has changed completely, but it seems that another bug (one that I'm currently scratching my head about) has overtaken it. I'll include your example in my test cases. > PS for Anyone, doesnt need to be Greg > > misc/157748 OK, I'll take a look when I'm done here. Greg -- Sent from my desktop computer. Finger g...@freebsd.org for PGP public key. See complete headers for address and phone numbers. This message is digitally signed. If your Microsoft MUA reports problems, please read http://tinyurl.com/broken-mua pgpo85JBjkQbE.pgp Description: PGP signature
Re: calendar(1) regressions
"Greg 'groggy' Lehey" wrote: > > --ryJZkp9/svQ58syV > Content-Type: text/plain; charset=us-ascii > Content-Disposition: inline > > On Tuesday, 11 December 2012 at 3:52:32 +0100, Julian H. Stacey wrote: > > "Greg 'groggy' Lehey" wrote: > >> On Monday, 10 December 2012 at 16:27:39 +0200, Jaakko Heinonen wrote: > >>> > >>> Hi! > >>> > >>> On 2012-12-10, Greg 'groggy' Lehey wrote: > > Unfortunately r205821 [1] has caused several regressions to calendar(1). > > Relevant PRs: > > > > bin/157718 > > bin/162211 > > bin/168785 > > bin/170930 > > > > I too filed send-prs re. rgeression bugs in recent broken calendar, > > hopefully you can look/ fix please. > > > > bin/157718 > > bin/162211 > > These two are mentioned above. Ah ! > > bin/15182 > > http://www.berklix.com/~jhs/src/bsd/fixes/FreeBSD/src/gen/usr.bin/calendar/send-pr.impossible_date.no_customise > > This one was closed a little over 11 years ago. Since then, the code Whoops :-) I should sleep ! > base has changed completely, but it seems that another bug (one that > I'm currently scratching my head about) has overtaken it. I'll > include your example in my test cases. Cheers :-) > > PS for Anyone, doesnt need to be Greg > > > > misc/157748 > > OK, I'll take a look when I'm done here. > > Greg Thanks Greg :-) Cheers, Julian -- Julian Stacey, BSD Unix Linux C Sys Eng Consultant, Munich http://berklix.com Reply below not above, like a play script. Indent old text with "> ". Send plain text. Not: HTML, multipart/alternative, base64, quoted-printable. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
