How to tell linux retrain and scan PCIe bus?

Asked 2 years ago Modified 2 months ago Viewed 4k times

We have an embedded board that has an iMX8M-Plus Processor and Linux v5.4.161. This board has one PCIe bus and that one is connected to an
FPGA. When we power up the board, the FPGA is not yet configured, so it acts as if it was not on the PCIe bus.
3
Once the Linux is fully booted, we configure the FPGA and only after that it starts acting as a PCIe endpoint (device).

At this point, when I run lspci -> it returns nothing. When I first execute echo "1" > /sys/bus/pci/rescan as suggested here and here and then
lspci , I still get nothing.

But if I reboot the linux without reseting the FPGA, it starts being visible in the lspci list. Rebooting the linux is not an option for us. Somehow I need
to tell the linux that whatever it's doing at the boot time, please do it again at runtime. But I couldn't find a solution for this so far.

According to the Texas Instrument support forum, they said if the PCIe link is not trained at the boot time, rescan command never works.

At the boot time, while linux loads a pci driver, it tries to establish a PCIe link, I can see that with an oscilloscope, PERST pin is asserted and
PCIE_CLK generated for a while and then stops if it can not detect any device. But the rescan command never does that.

Also in the system there is no pcie device to execute echo 1 > $pcidevice/remove in order to make rescan functional. Or there is no device or bus to
set power off and on back like echo 0 > /sys/bus/pci/slots/.../power

I also learned that there was a method in old linux times (v2.6) called adding a Fake PCIe Device which physically doesn't exist to solve this problem.
For that I took the fakephp.c driver from an old linux repo and ported it to ours. After solving a couple of deprecated function problems, it is compiled
for Linux Kernel v5.4. modprobe fakephp worked and driver loaded but somehow I didn't get this fake device in my device list. Here it is mentioned that
the fakephp driver was removed from mainstream linux since PCI core has similar functionality, but he never mentioned how.

Short of the story is that, I am stuck here, I need my FPGA to be visible in the lspci list without restarting the linux.

linux-kernel embedded-linux pci-e

Share Improve this question Follow asked Mar 23, 2022 at 16:43
Mehmet Fide
1,695 1 22 36

For hot-plug NVMe that I worked on the firmware must leave room in its device table. Devices can't be moved once they are active. – stark Mar 23, 2022 at 18:19

Just a guess: Try your idea of echo 1 > $pcidevice/remove with pcidevice = the root port. Then set rescan. – prl Mar 24, 2022 at 5:47

@prl when the pcie link is not trained at the boot time, later on there is no device or root port defined in the sys folder to remove. – Mehmet Fide Mar 24, 2022 at 12:53

2 Answers Sorted by: Highest score (default)

I recommend configuring the FPGA in u-boot to get away from these kinds of problems. Connect up SPI pins to FPGA's config pins & run it in Slave
configuration mode.
0
Share Improve this answer Follow answered Jun 1, 2022 at 19:58
Otto
1 1

As it’s currently written, your answer is unclear. Please edit to add additional details that will help others understand how this addresses the question asked. You can find
more information on how to write good answers in the help center. – Community Bot Jun 2, 2022 at 7:05

Try at the Linux prompt:

0 sudo echo 1 | sudo tee /sys/bus/pci/rescan

Share Improve this answer Follow edited Jan 18 at 1:45 answered Jan 18 at 1:38
Dharman ♦ Mark McDougald
32k 25 91 139 1 1

This doesn't seem to have to do anything with programming. – Dharman ♦ Jan 18 at 1:46