Thanks Simon, The NVMe driver does work, at least for me. There are some issues with cache and Device Tree mapping, but I think those are handled in 2021.04.
I attempted to add a remove function to the nvme code. I patterned it after a couple of sata drivers remove commands. It looks like this: static int nvme_remove(struct udevice *udev) { struct nvme_dev *dev = dev_get_priv(udev); free(dev->prp_pool); nvme_free_queues(dev, 0); return 0; } However, I don't understand how it is called. I looked into the cmd interface but I got lost in the linker_lists.h code. I'm a Hardware Engineer and my lack of C experience is showing. It seems to me that since the nvme is a block device, there should be a function associated with the block commands for stopping or unbinding the blk device and that should call nvme.remove automatically. I dug around in the code looking for the linkage but was not able to find it. Can you or one of the other experts give me a pointer to a bit of C code that shows the linkage and an example of how to unbind the block device? I also thought about putting a back door command into the probe function, but I have to find the existing udevice in order to call the probe function. Also I hesitate because that seems like the wrong way to go. Any assistance would be greatly appreciated. On another note: I have created a driver for PCIe on Cyclone V. It should work for all V series Altera (intel) FPGAs. I kept the driver separate from the 10 series PCIe driver published by intel because I feel that they may want to modify that code in the future for a new series of FPGAs and I don't want to interfere with that process. Can I submit it as a patch to this list, or is there a different way to submit patches for review? Thanks much for your support, Brian On Fri, May 7, 2021 at 12:39 PM Simon Glass <s...@chromium.org> wrote: > Hi Brian, > > On Fri, 7 May 2021 at 11:03, Brian McKee <rayd...@gmail.com> wrote: > > > > Hi Gents. > > > > Background: working with the socfpga fork of u-boot-2020.10, I've added > a driver for the Altera V series FPGA PCIe controller to u-boot (borrowed > completely from intel's 10 series FPGA driver and the linux kernel driver). > I got it working through a series of hacks to the pcie and nvme drivers. I > haven't been able to get 2021.04 Denx version to work, I think because the > PCIe -> memory interface is not enabled by 'bridge enable'. I could > probably hack that, but for now I'm sticking with 2020.10 socfpga. Based on > my experiments though, it looks like the hacks I deployed to 2020.10 are > not necessary on 2021.04. I'm waiting for socfpga to port to 2021.04. > > > > I have a problem though. To keep cost down, I have a 2MB QSPI flash > device on the board. And to keep software developers happy, the almost 8 > MiB FPGA image should be stored on the SSD so field upgrades are less > likely to brick the hardware. > > > > I have created a stripped down FPGA that only contains the PCIe > controller and the HPS module. lzma compressed it is less than 300KB. It > fits quite nicely into the free space of the QSPI NOR Flash memory. > > > > I have u-boot installed on the QSPI, booting and I've setup commands to > grab the stripped down FPGA image, decompress it, and load it into the > FPGA. Then initialize PCI and NVME scan and it all works! > > > > However, when I reload the FPGA from the NVME, pcie is okay, but the > nvme crashes. I'm not sure why. It might be getting reset by the FPGA > re-load. All of it's programming disappears and it drops off the bus. > > > > I have proven that if I perform a u-boot 'reset' after the full FPGA is > loaded, do a 'bridge enable', 'pci' and 'nvme scan' the nvme comes back to > life using the full fpga image. > > > > I have been trying to figure out how to trick u-boot into > re-initializing the nvme controller, but I haven't figured out a hack to do > so yet. > > > > Ideally, I'd like to be able to unload the nvme driver (and possibly the > pcie driver) and then reload them but there doesn't appear to be a way to > do that in u-boot at this time. > > > > Do any of you guys have any suggestions for what I can try to > reinitialize the nvme driver? > > You could check if the remove() method of the nvme is implemented. If > not you could implement it. Does the NVME depend on the FPGA? Are > there any reset lines controlled by the FPGA? > > I tried NVMe on an Intel device (a Chromebook) a while back and it > just hung. I haven't fiddled with it since. > > > > > Thanks much for your support. > > > > Brian > > Regards, > Simon > -- -- Consciousness moves everything.