Re: What if the console device is only accessible from one CPU in a multiprocessor system?
On Tue, Mar 04, 2025 at 01:26:42PM -0800, Jason Thorpe wrote: > The firmware selects which CPU module gets to boot the system (the > criteria used is unimportant here), and the UART on that module is > connected to the console port and the RTC on that module is the > authoritative RTC for the system. > > Obviously, this is a lot more problematic for the console device. We > currently have no restrictions in the kernel overall around which CPU > gets to access the console device for printing messages, and don’t > even get me started on how weird this all gets with DDB. My first thought over morning coffe was to add a thin layer on the cnputc/cngetc/cnpollc interface level. Basically have a stack of cn_tabs (only the necessary parts perhaps). On the Laser/TurboLaser machines have "virtual" cnputc/cngetc/cnpollc that, on the boot CPU, calls the "real" cnputc/cngetc that frobs the hardware and, on the other CPUs, xcalls to the boot CPU. I presume you can easily identify the boot CPU. I don't see a way around the xcall. The "thin layer" may come in handy when we want to deal with multiple devices acting as console like e.g. Linux has it. There you can have console on e.g. frame buffer and serial simultaneously. I'm surprised you expect weirdness with DDB. I thought DDB runs on the boot CPU and pauses all other CPUs. Doing xcalls to the other CPUs for stack traces etc. I.e. it's a non-issue for DDB. I did a bit of nxr'ing on cnputc and on subr_prf.c but that quickly turned into a maze of twisty little passages to large for my short term memory. The main thing I remember is that kprintf ends up cnputc'ing every single character. Sending a bunch of characters in an xcall might be less overhead. You are looking for an excuse to pull out the rototiller, aren't you? :-) --chris
Tickless NetBSD with high-resolution timers GSOC project
Hi, Hope everyone is doing well. I am Dorjoy. I am from Bangladesh. I am interested in the Tickless NetBSD with high-resolution timers[1] project. I wanted to know if this project is still up for grabs and if a mentor is available for this project. I would like to start to discuss this one and see if I can contribute to netbsd. I have been looking into the related code (kern_timeout.c etc) and the call wheel based data structure (the paper referenced in kern_timeout.c as well) for the last few days. 1. The call wheel based data structure works really well because there is a periodic timer interrupt and the sleep durations are based on those ticks. Insertion, deletion into the hierarchical call wheel is basically O(1). When there is a periodic timer interrupt, we basically move ahead one tick on the wheel and call any callbacks (or move timers to the next level) which is O(number of callbacks). For high resolution timer, we won't have any periodic timer at that resolution. So I believe we will need to just schedule an interrupt at the "next" event. As we cannot use the call wheel based data structure here (because we can't move one tick to the next in this case) we will need to be able to efficiently insert, delete and find the next timer. I looked at how linux does it a little bit (include/linux/hrtimer.h, include/linux/hrtimer_types.h, include/linux/hrtimer_defs.h, include/linux/timerqueue_types.h) and it seems like the ktime_t is a signed 64 bit value (nanosecond resolution) and red-black tree is being used as the data structure. I believe we can do the same for netbsd. Maybe 64 bit unsigned value (I am not sure if signed is needed) for the resolution at nanosecond (as opposed to the "tick" used now in kern_timeout.c) and a red-black tree for storing the timers. That would mean insertion, deletion, finding "next" would be O(log n). I don't yet have a complete picture of what the code would look like and how these interrupts are scheduled/work in general but it makes sense to have the data structure discussion first. 2. I don't know yet what the API should look like but I believe they should be similar to what we already have now like the functions in kern_timeout.c. So that we can replace the users of the kern_timeout.c functions, right? 3 and 4 depend on the above so let's discuss how 1 and 2 should be like first. Does the above description of 1 and 2 look like a reasonable proposal for the upcoming GSOC and is there a mentor available for this project? Please let me know if you have any suggestion on the data structure / API so that we can discuss and be certain what high resolution timers should look like in netbsd. Thanks! Regards, Dorjoy
Re: Introductory email in reference to GSoC 2025 (https://wiki.netbsd.org/projects/project/swap-auto/)
Hello, I wasn't able to get internet working on NetBSD under QEMU and since I'm completely unfamiliar with NetBSD and I'll definitely need more time than I have currently, I will not apply for the moment. But, I was thinking about it and realized that changes to the scheduler would also be required to avoid thrashing and to implement time-outs. I'm interested in this though and in case it is still available later on (outside of GSoC or perhaps next year's GSoC) I'll reach out to you again. Thanks and regards, Anup K Parikh. On Sat, Mar 8, 2025 at 7:05 PM anup parikh wrote: > > Hello, > > My name is Anup. I'd like to participate in the project at > https://wiki.netbsd.org/projects/project/swap-auto/. I'm not a student but I > think I'm eligible as I've never participated in GSoC or other project based > FOSS internships. I'm interested in the FOSS industry and I'd like to > contribute and build experience around the same to transition into being an > active member of the FOSS industry. I'm currently a mentee in ACM's Sigplan-M > program and am learning Agda out of interest in its domains. I also practice > almost daily on Leetcode - you can check my profile at > https://leetcode.com/u/parikhanupk/. Please let me know if you want to know > anything else or offer some pointers to me in the context of this project. > > I've answered some questions from > https://wiki.netbsd.org/projects/application/ as follows: > > Goal: > Auto create swap on memory pressure > > Deliverables: > Code and documentation > > Overview: > 1. Study/Research the swapping behavior of NetBSD kernel > 2. Identify files/functions where modifications need to be made > 3. Write code along with documentation > > Similar software availability: > Not that I know of > > Is the project a port or a rewrite: > No. This will be new code added into NetBSD code and it will be licensed > under a NetBSD compatible license. > > NetBSD experience: > Not yet. I am an absolute newcomer to NetBSD. I'm sending this first email as > an expression of interest in this project. I'll do this by the next weekend > (March 16 2025) - all of it like installing, configuration, building kernel > and userland. I haven't worked on pkgsrc and this project, I think, wouldn't > require working on pkgsrc, however, I typically use a custom Linux based > system built natively from sources (it's heavily based on LFS and BLFS > projects, except that it's a naive but near complete automation). > > Prior experience: > I've done this in userspace when some of my builds failed due to memory > pressure (for example LLVM, Libreoffice, Firefox, etc.) where I added swap > file creation in specific build scripts. Please note that this was just a few > additional lines in shell scripts. Also around 2010/2011 when I was working > on a chess related program, I did this sort of thing in userspace - this was > a custom allocator that programs could use to allocate as much memory as one > wanted by specifying physical RAM limits. For example, one could allocate a > billion 32-byte objects (32 GiB) with a RAM limit of 256 MiB and my library > handled everything else (the user accessed their objects by index). It worked > well for its purpose (sequential access). I wrote this in C# and I do not > have the code but I'm mentioning this as it's something that relates to this > project. Perhaps usage specific behavior could be added to the swapping > mechanism. > > Thanks and regards, > Anup K Parikh.
Re: ataraid(4) missing disk handling
On Mon, Mar 10, 2025 at 04:43:30PM +0200, Andrius V wrote:
> Taylor has a good analysis and explanation of the issue
> https://mail-index.netbsd.org/netbsd-bugs/2024/03/26/msg082202.html,
> which I also noticed by testing ATA RAID setup on VIA controllers.
> [...]
Good analysis.
> Given that three statuses are defined for adi_status
> (https://nxr.netbsd.org/xref/src/sys/dev/ata/ata_raidvar.h#75), I
> probably need to check if any of the flags are defined ((adi_status &
> (ADI_S_ONLINE | ADI_S_ASSIGNED | ADI_S_SPARE)) instead
> (https://netbsd.org/~andvar/ata_raid_fix2.diff).
The first part seems fine. Although I didn't look at what the difference
between vn_bdev_open() vs. the deleted code is.
In the second part this bit is weird for me to read:
+ else
+ vp = NULL;
if (vp == NULL) {
I'd write it as:
- vp = ata_raid_disk_vnode_find(adi);
+ vp = NULL;
+ if (adi->adi_status &
+ (ADI_S_ONLINE | ADI_S_ASSIGNED | ADI_S_SPARE))
if (vp == NULL) {
/*
* XXX This is bogus. We should just mark the
But Taylor seemed to be fine with it if I remember the discussion on ICB
correctly. A matter of style I guess.
> Another alternative is to check that adi->adi_dev IS NULL as Taylor
> proposed in his analysis thread.
But that proposal never explained how an adv with adi->adi_dev==NULL ended
up on the ataraid_disk_vnode_list which isn't supposed to happen!
--chris
Re: ataraid(4) missing disk handling
On Tue, Mar 11, 2025 at 04:33:51PM +0200, Andrius V wrote: > The major difference is that vn_bdev_open() increments > (*vpp)->v_writecount++; > That prevents from the v_writecount > 0 assert failure on vn_close() > (https://nxr.netbsd.org/xref/src/sys/dev/ata/ld_ataraid.c#249) > > My only concern was that on successful path incremented value will > stay equal 1 and if it may cause unexpected later on. But vn_close() decrements v_writecount. So that is fine. --chris
