Robert, You are mostly on the right track, unfortunately I had not implemented the upper half driver for either of these devices since they were really just testing the PCI bus. The edu test device actually makes some sense to have a character driver implemented for so I could probably wire up an example of that tomorrow evening. I'm guessing for your use case you can get away with mostly using the ioctl to configure your device. If you have tight timing requirements based on an IRQ I would probably define those as well in the driver and not userspace and then use the char or block device interface to access the information you need.
Be aware that with QEMU you are going to pay the price of both the interrupt handler logic from Linux and NuttX. So your performance might be impacted. https://www.linux-kvm.org/page/Virtio/Block/Latency Where I left off there was an issue with some interrupts when booting via UEFI so I need to look into that before I could recommend booting directly for development. Hope that helps, Brennan On Tue, Aug 18, 2020, 1:39 PM Robert Faron <robe...@cmslaser.com> wrote: > Brennan, > > I have started writing a proper driver, however I am still a bit > confused. There are 2 different PCI device examples and I'm not exactly > sure which I should be modeling my driver after. There is qemu_pci_test.c > and there is qemu_edu.c. I have taken the qemu_edu.c and renamed it > cms_pci.c and then converted all the functions to suit my hardware. I have > also written many other functions that my FPGA card uses. However I can't > seem to find where either of the qemu examples are ever actually used. I > can see were they are enabling in drivers/pci.c in that struct and I have > added mine. I would like to get this setup and working the proper way so I > might test the timing of accessing the card through the kernel and outside > it to see which is the method that we will have to use. > > It has taken me this long due to finding errors in the counter/timer > sections of the FPGA. Soon the manufacture of the card should have an > updated vhdl for the FGPA so that the timers work properly. > > > > -----Original Message----- > From: Brennan Ashton <bash...@brennanashton.com> > Sent: Monday, August 3, 2020 8:49 PM > To: dev@nuttx.apache.org > Subject: Re: AMD64 arch > > Robert, > I ended up with a little less time this weekend than I had hoped for. > Getting the "console" working is a little more tricky because of > restrictions put in place on modern machines that are booting in UEFI > mode. I was able to add some logic for booting via UEFI and extracting > framebuffer information to be able to draw on the screen, just need to glue > it together and then at least the output bit will work with most video > hardware. The input is a little more tricky since that usually requires > PCI + USB + HID. > > You had mentioned that you hard coded the memory addresses found in the > scan. This probably means that you did not create a driver interface. > Neither of the two PCI examples that I created expose a driver interface, > but I could tweek the pci_edu driver to show how this would work. You > would register your driver in drivers/pci.c in this struct: > > struct pci_dev_type_s *pci_device_types[] = { > #ifdef CONFIG_VIRT_QEMU_PCI_TEST &pci_type_qemu_pci_test, > > #endif /* CONFIG_VIRT_QEMU_PCI_TEST */ > > #ifdef CONFIG_VIRT_QEMU_EDU &pci_type_qemu_edu, > > #endif /* CONFIG_VIRT_QEMU_EDU */ > > NULL, > > }; > > From there you would then create and register a character or block device > and register the interfaces with it. The qemu_edu_probe shows how your > driver can collect this information which could then be stored in a struct > for your instance of your device. Then your application would make a > read/write/ioctrl call to /dev/mypcidev which would then be handled by the > kernel. Your application should not be writing directly to physical memory > addresses or doing things like registering interrupts, that should be left > to the device driver. > > --Brennan > > On Mon, Aug 3, 2020 at 11:47 AM Robert Faron <robe...@cmslaser.com> wrote: > > > > Brennen, > > > > Again I wanted to thanks you for the help you have provided thus far. > > > > After digging in to accessing pci devices in qemu I found that to > virtualize PCi hardware you first need a board that supports VT-d. The Asus > 885M-E motherboard only supports VT-x which allows for virtualization of > 64bit OSes but doesn't support direct access to I/O. So this morning I > started using Gigabyte H110N motherboard that does support VT-d. It took a > bit of time to get the device to start using the vfio-pci kernel driver > which allows qemu to access the card. Once that was complete I can now > start qemu with "-device vfio-pci,host=01:00:0" and the qemu session now > can see the card. The qemu_pci_init now can see the card and the Bar0 & > Bar1 memory addresses. Using that I have written a very simple test > program that can be enabled using "make menuconfig" called cms. Running > cms from the nsh prompt access the card and turns on the isolated digital > output then reads the state and reverses it. > > > > To continue my testing I would like to get the program cms to autorun > > that way I can at least test the I/o functionality. I will modify my > > program which is using hardcoded memory addresses to use the address > > found during a scan of the PCI configuration data so when it boots I > > can find the correct address as I assume it will be different when not > > in qemu > > > > I have tried using com1 (0x3f8) connected to coolterm on another > computer to see if any terminal could be accessed but haven't had any > success. I know you said you were going to work on getting a console up but > having a serial terminal would work for now to keep my progress moving. > > > > -Robert > > > > > > > > -----Original Message----- > > From: Brennan Ashton <bash...@brennanashton.com> > > Sent: Friday, July 31, 2020 5:21 AM > > To: dev@nuttx.apache.org > > Subject: Re: AMD64 arch > > > > Robert, > > I dug into this a little more, and I am fairly confident that the PCI > subsystem will find your device since it properly traverses the bridges and > can identify multiple buses. > > > > A couple things that are probably important to check, you are not using > the COM port on your motherboard for this testing right? There is no > support for a VGA console right now and the OS is expecting to be able to > write to the physical serial port. > > > > The warning you are getting in Grub is because the OS is asking for the > bootloader to put the VGA controller in Text Mode, unfortunately that > warning you are getting about the console is letting us know that the Grub > is using UEFI which does not allow switching to text mode. I had > implemented a VGA text mode console, but we cannot use this because of this > limitation. I started adding support for the graphics mode so we can get a > console drawn on the screen to help with debugging, but that is going to > have to be a weekend project. I did get the graphics mode configured so I > suspect it will not be too bad, but there are a lot of moving bits. > > > > --Brennan > > > > On Thu, Jul 30, 2020 at 2:24 PM Brennan Ashton < > bash...@brennanashton.com> wrote: > > > > > > Robert, > > > I'll take a look this evening and get back to you. > > > > > > This has run on bare hardware before, as well as in a jailhouse > > > cell, so it should be fairly close to running on your hardware. I > > > do most of my testing inside of QEMU since it is much easier for me > > > to attach the debugger, but I'll see if I can still get this running > > > directly on my NUC. > > > > > > As for the multiple PCI busses I will need to look at that as well. > > > From what I remember the enumeration code structures handle the bus > > > id, but I would need to look at how we identify the buses that we > > > scan. > > > > > > This is actually all timely because I just got some PCIe hardware > > > that I have been waiting for for a few months, so let's see what we > can do. > > > > > > --Brennan > > NOTICE: This e-mail message and all attachments transmitted with it > > may contain trade secrets and confidential information intended solely > > for the use of the addressee. If the reader of this message is not the > > intended recipient, you are hereby notified that any reading, > > dissemination, distribution, copying, or other use of this message or > > its attachments is strictly prohibited. If you have received this > > message in error, please notify the sender immediately by telephone at > > 407-679-9716, and delete this message and all copies and backups > > thereof. Thank you > NOTICE: This e-mail message and all attachments transmitted with it may > contain trade secrets and confidential information intended solely for the > use of the addressee. If the reader of this message is not the intended > recipient, you are hereby notified that any reading, dissemination, > distribution, copying, or other use of this message or its attachments is > strictly prohibited. If you have received this message in error, please > notify the sender immediately by telephone at 407-679-9716, and delete this > message and all copies and backups thereof. Thank you >
