linux - 为什么没有分配给 pci 设备的 BAR 地址？

Question

问题描述如下:
linux启动后，运行命令“lspci -v”，我们可以看到可以找到pci设备，但是没有分配地址给设备。进一步查看linux启动信息，发现有很多类似“bridge window mem ...”的信息。不确定这些消息是正常的还是实际上是地址分配失败的迹象。

我运行的处理器是Freescale p2010，是powerpc架构。 bootloader是我们自己设计的，没有接触pci相关的东西。所以我们完全依赖 Linux 内核自己完成完整的 pci 枚举。

我们在 Linux 内核 pci 枚举开始之前做的唯一一件事就是为 pci 空间使用配置一个 LAW 窗口(特别是 LAW4)。此外，当然还要根据我们的硬件配置对dts文件进行适当的修改。我们认为在完成所有这些准备工作后，Linux 内核应该能够使所有 pci 东西正常工作，然而，它没有。

那么，有没有人知道如何解决这个问题，或者曾经遇到过类似的问题，并且可以对此有所了解？非常感谢!

下面附有更多信息:
========内核引导参数===============

root=/dev/ram rw ip=off console=ttyS0,9600

========dts中的PCI段设置=======

pci0: pcie@fbf0a000 {
    interrupt-map-mask = <0xf800 0x0 0x0 0x7>;
    interrupt-map = <
        00000 0x0 0x0 0x1 &mpic 0x0 0x1
        00000 0x0 0x0 0x2 &mpic 0x1 0x1
        00000 0x0 0x0 0x3 &mpic 0x2 0x1
        00000 0x0 0x0 0x4 &mpic 0x3 0x1>;

    interrupt-parent = <&mpic>;
    interrupts = <26 2>;
    bus-range = <0 255>;
    ranges = <0x2000000 0x0 0x80000000 0x80000000 0x0 0x40000000
          0x1000000 0x0 0x0 0xd3000000 0x0 0x100000>;
    clock-frequency = <33333333>;
    #interrupt-cells = <1>;
    #size-cells = <2>;
    #address-cells = <3>;
    reg = <0xfbf0a000 0x1000>;
    compatible = "fsl,mpc8548-pcie";
    device_type = "pci";
        fsl,pcie-access-big-endian;

    pcie@0 {
        reg = <0x0 0x0 0x0 0x0 0x0>;
        #size-cells = <2>;
        #address-cells = <3>;
        device_type = "pci";
        ranges = <0x2000000 0x0 0x80000000
              0x2000000 0x0 0x80000000
              0x0 0x40000000

              0x1000000 0x0 0x0
              0x1000000 0x0 0x0
              0x0 0x100000>;
    };
};

==============“lspci -v”的结果===============

# lspci -v
0000:00:00.0 Class 0b20: 1957:007b (rev 21)
        !!! Invalid class 0b20 for header type 01
        Flags: bus master, fast devsel, latency 0
        Memory at <ignored> (32-bit, non-prefetchable)
        Bus: primary=00, secondary=01, subordinate=07, sec-latency=0
        I/O behind bridge: 00000000-00000fff
        Memory behind bridge: 00000000-000fffff
        Prefetchable memory behind bridge: 0000000000000000-0000000000000000
        Capabilities: [44] Power Management version 2
        Capabilities: [4c] #10 [0041]

0000:01:00.0 Class 0604: 10b5:8606 (rev ba)
        Flags: bus master, fast devsel, latency 0
        Bus: primary=01, secondary=02, subordinate=07, sec-latency=0
        Capabilities: [40] Power Management version 3
        Capabilities: [48] Message Signalled Interrupts: 64bit+ Queue=0/2 Enable+
        Capabilities: [68] #10 [0052]
        Capabilities: [a4] #0d [0000]

0000:02:01.0 Class 0604: 10b5:8606 (rev ba)
        Flags: bus master, fast devsel, latency 0
        Bus: primary=02, secondary=03, subordinate=03, sec-latency=0
        Capabilities: [40] Power Management version 3
        Capabilities: [48] Message Signalled Interrupts: 64bit+ Queue=0/2 Enable+
        Capabilities: [68] #10 [0162]
        Capabilities: [a4] #0d [0000]

0000:02:04.0 Class 0604: 10b5:8606 (rev ba)
        Flags: bus master, fast devsel, latency 0
        Bus: primary=02, secondary=04, subordinate=04, sec-latency=0
        Capabilities: [40] Power Management version 3
        Capabilities: [48] Message Signalled Interrupts: 64bit+ Queue=0/2 Enable+
        Capabilities: [68] #10 [0162]
        Capabilities: [a4] #0d [0000]

0000:02:05.0 Class 0604: 10b5:8606 (rev ba)
        Flags: bus master, fast devsel, latency 0
        Bus: primary=02, secondary=05, subordinate=05, sec-latency=0
        Capabilities: [40] Power Management version 3
        Capabilities: [48] Message Signalled Interrupts: 64bit+ Queue=0/2 Enable+
        Capabilities: [68] #10 [0162]
        Capabilities: [a4] #0d [0000]

0000:02:07.0 Class 0604: 10b5:8606 (rev ba)
        Flags: bus master, fast devsel, latency 0
        Bus: primary=02, secondary=06, subordinate=06, sec-latency=0
        Capabilities: [40] Power Management version 3
        Capabilities: [48] Message Signalled Interrupts: 64bit+ Queue=0/2 Enable+
        Capabilities: [68] #10 [0162]
        Capabilities: [a4] #0d [0000]

0000:02:09.0 Class 0604: 10b5:8606 (rev ba)
        Flags: bus master, fast devsel, latency 0
        Bus: primary=02, secondary=07, subordinate=07, sec-latency=0
        Capabilities: [40] Power Management version 3
        Capabilities: [48] Message Signalled Interrupts: 64bit+ Queue=0/2 Enable+
        Capabilities: [68] #10 [0162]
        Capabilities: [a4] #0d [0000]

0000:03:00.0 Class 0580: 10ee:7011
        Subsystem: 10ee:0007
        Flags: fast devsel, IRQ 17
        Capabilities: [40] Power Management version 3
        Capabilities: [48] Message Signalled Interrupts: 64bit+ Queue=0/0 Enable-
        Capabilities: [60] #10 [0002]

0000:04:00.0 Class 0580: 10ee:7011
        Subsystem: 10ee:0007
        Flags: fast devsel, IRQ 16
        Capabilities: [40] Power Management version 3
        Capabilities: [48] Message Signalled Interrupts: 64bit+ Queue=0/0 Enable-
        Capabilities: [60] #10 [0002]

0000:05:00.0 Class 0200: 14e4:8372 (rev 13)
        Subsystem: 14e4:8372
        Flags: fast devsel, IRQ 17
        Memory at <ignored> (64-bit, non-prefetchable)
        Capabilities: [48] Power Management version 3
        Capabilities: [50] Vital Product Data
        Capabilities: [58] Message Signalled Interrupts: 64bit+ Queue=0/3 Enable-
        Capabilities: [ac] #10 [0002]

0000:06:00.0 Class 0200: 14e4:8372 (rev 13)
        Subsystem: 14e4:8372
        Flags: fast devsel, IRQ 18
        Memory at <ignored> (64-bit, non-prefetchable)
        Capabilities: [48] Power Management version 3
        Capabilities: [50] Vital Product Data
        Capabilities: [58] Message Signalled Interrupts: 64bit+ Queue=0/3 Enable-
        Capabilities: [ac] #10 [0002]

#                                                           

=======================Linux内核启动信息====================

irq: irq 47 on host /soc@fbf00000/pic@40000 mapped to virtual irq 47
PCI: Probing PCI hardware
pci 0000:00:00.0: ignoring class b20 (doesn't match header type 01)
pci 0000:00:00.0: supports D1 D2
pci 0000:00:00.0: PME# supported from D0 D1 D2 D3hot D3cold
pci 0000:00:00.0: PME# disabled
pci 0000:01:00.0: reg 10: [mem 0x00000000-0x0001ffff]
pci 0000:01:00.0: PME# supported from D0 D3hot D3cold
pci 0000:01:00.0: PME# disabled
pci 0000:00:00.0: PCI bridge to [bus 01-ff]
pci 0000:00:00.0:   bridge window [io  0x0000-0x0000] (disabled)
pci 0000:00:00.0:   bridge window [mem 0x00000000-0x000fffff] (disabled)
pci 0000:00:00.0:   bridge window [mem 0x00000000-0x000fffff pref] (disabled)
irq: irq 0 on host /soc@fbf00000/pic@40000 mapped to virtual irq 16
pci 0000:02:01.0: PME# supported from D0 D3hot D3cold
pci 0000:02:01.0: PME# disabled
pci 0000:02:04.0: PME# supported from D0 D3hot D3cold
pci 0000:02:04.0: PME# disabled
pci 0000:02:05.0: PME# supported from D0 D3hot D3cold
pci 0000:02:05.0: PME# disabled
pci 0000:02:07.0: PME# supported from D0 D3hot D3cold
pci 0000:02:07.0: PME# disabled
pci 0000:02:09.0: PME# supported from D0 D3hot D3cold
pci 0000:02:09.0: PME# disabled
pci 0000:01:00.0: PCI bridge to [bus 02-ff]
pci 0000:01:00.0:   bridge window [io  0xf000-0x0000] (disabled)
pci 0000:01:00.0:   bridge window [mem 0xfff00000-0x000fffff] (disabled)
pci 0000:01:00.0:   bridge window [mem 0xfff00000-0x000fffff pref] (disabled)
irq: irq 1 on host /soc@fbf00000/pic@40000 mapped to virtual irq 17
irq: irq 3 on host /soc@fbf00000/pic@40000 mapped to virtual irq 18
pci 0000:03:00.0: reg 10: [mem 0x00000000-0x00003fff]
pci 0000:03:00.0: PME# supported from D0 D1 D2 D3hot
pci 0000:03:00.0: PME# disabled
pci 0000:02:01.0: PCI bridge to [bus 03-ff]
pci 0000:02:01.0:   bridge window [io  0xf000-0x0000] (disabled)
pci 0000:02:01.0:   bridge window [mem 0xfff00000-0x000fffff] (disabled)
pci 0000:02:01.0:   bridge window [mem 0xfff00000-0x000fffff pref] (disabled)
pci 0000:04:00.0: reg 10: [mem 0x00000000-0x00003fff]
pci 0000:04:00.0: PME# supported from D0 D1 D2 D3hot
pci 0000:04:00.0: PME# disabled
pci 0000:02:04.0: PCI bridge to [bus 04-ff]
pci 0000:02:04.0:   bridge window [io  0xf000-0x0000] (disabled)
pci 0000:02:04.0:   bridge window [mem 0xfff00000-0x000fffff] (disabled)
pci 0000:02:04.0:   bridge window [mem 0xfff00000-0x000fffff pref] (disabled)
pci 0000:05:00.0: reg 10: [mem 0x80400000-0x8043ffff 64bit]
pci 0000:05:00.0: PME# supported from D0 D3hot D3cold
pci 0000:05:00.0: PME# disabled
pci 0000:02:05.0: PCI bridge to [bus 05-ff]
pci 0000:02:05.0:   bridge window [io  0xf000-0x0000] (disabled)
pci 0000:02:05.0:   bridge window [mem 0xfff00000-0x000fffff] (disabled)
pci 0000:02:05.0:   bridge window [mem 0xfff00000-0x000fffff pref] (disabled)
pci 0000:06:00.0: reg 10: [mem 0x80500000-0x8053ffff 64bit]
pci 0000:06:00.0: PME# supported from D0 D3hot D3cold
pci 0000:06:00.0: PME# disabled
pci 0000:02:07.0: PCI bridge to [bus 06-ff]
pci 0000:02:07.0:   bridge window [io  0xf000-0x0000] (disabled)
pci 0000:02:07.0:   bridge window [mem 0xfff00000-0x000fffff] (disabled)
pci 0000:02:07.0:   bridge window [mem 0xfff00000-0x000fffff pref] (disabled)
pci 0000:02:09.0: PCI bridge to [bus 07-ff]
pci 0000:02:09.0:   bridge window [io  0xf000-0x0000] (disabled)
pci 0000:02:09.0:   bridge window [mem 0xfff00000-0x000fffff] (disabled)
pci 0000:02:09.0:   bridge window [mem 0xfff00000-0x000fffff pref] (disabled)
PCI: Cannot allocate resource region 0 of device 0000:05:00.0, will remap
PCI: Cannot allocate resource region 0 of device 0000:06:00.0, will remap
pci 0000:01:00.0: BAR 8: can't assign mem (size 0x400000)
pci 0000:01:00.0: BAR 0: can't assign mem (size 0x20000)
pci 0000:02:01.0: BAR 8: can't assign mem (size 0x100000)
pci 0000:02:04.0: BAR 8: can't assign mem (size 0x100000)
pci 0000:02:05.0: BAR 8: can't assign mem (size 0x100000)
pci 0000:02:07.0: BAR 8: can't assign mem (size 0x100000)
pci 0000:03:00.0: BAR 0: can't assign mem (size 0x4000)
pci 0000:02:01.0: PCI bridge to [bus 03-03]
pci 0000:02:01.0:   bridge window [io  disabled]
pci 0000:02:01.0:   bridge window [mem disabled]
pci 0000:02:01.0:   bridge window [mem pref disabled]
pci 0000:04:00.0: BAR 0: can't assign mem (size 0x4000)
pci 0000:02:04.0: PCI bridge to [bus 04-04]
pci 0000:02:04.0:   bridge window [io  disabled]
pci 0000:02:04.0:   bridge window [mem disabled]
pci 0000:02:04.0:   bridge window [mem pref disabled]
pci 0000:05:00.0: BAR 0: can't assign mem (size 0x40000)
pci 0000:02:05.0: PCI bridge to [bus 05-05]
pci 0000:02:05.0:   bridge window [io  disabled]
pci 0000:02:05.0:   bridge window [mem disabled]
pci 0000:02:05.0:   bridge window [mem pref disabled]
pci 0000:06:00.0: BAR 0: can't assign mem (size 0x40000)
pci 0000:02:07.0: PCI bridge to [bus 06-06]
pci 0000:02:07.0:   bridge window [io  disabled]
pci 0000:02:07.0:   bridge window [mem disabled]
pci 0000:02:07.0:   bridge window [mem pref disabled]
pci 0000:02:09.0: PCI bridge to [bus 07-07]
pci 0000:02:09.0:   bridge window [io  disabled]
pci 0000:02:09.0:   bridge window [mem disabled]
pci 0000:02:09.0:   bridge window [mem pref disabled]
pci 0000:01:00.0: PCI bridge to [bus 02-07]
pci 0000:01:00.0:   bridge window [io  disabled]
pci 0000:01:00.0:   bridge window [mem disabled]
pci 0000:01:00.0:   bridge window [mem pref disabled]
pci 0000:00:00.0: not setting up bridge for bus 0000:01
pci_bus 0000:00: resource 0 [io  0xffafd000-0xffbfcfff]
pci_bus 0000:00: resource 1 [mem 0x80000000-0xbfffffff]
Registering qe_ic with sysfs...
bio: create slab <bio-0> at 0
vgaarb: loaded
Generic PHY: Registered new driver

Answer 1

在我们的一个 SuperMicro 上:启用以下 BIOS 选项:

高级 -> PCIe/PCI/PnP 配置:

PCI PERR/SERR Support   - Enabled    - event log pci errors.

Above 4G Decoding       - Enabled    - 64bit O/S and many pcie devices.

SR-IOV Support          - Enabled    - Linux gets dmesg "no space for" errs.

首先是一个建议...