• NEW_PCIB

NEW_PCIB is a kernel option to enable a "new" PCI-PCI bridge driver. The "new" driver actively manages the resource windows in the PCI-PCI bridge from which resources for child devices are allocated. The "old" driver used whatever ranges the firmware provided. This includes the resources managed include SYS_RES_MEMORY and SYS_RES_IOPORT for memory and I/O BARs as well as the recently added PCI_RES_BUS to manage the PCI bus numbers assigned to secondary buses behind bridges.

Currently, the NEW_PCIB driver is not enabled on all platforms. The goal of this page is to coordinate making that happen.

NEW_PCIB Platforms

Steps to support NEW_PCIB

To support, the base NEW_PCIB driver (which supports managing memory and I/O port windows), the platform specific code needs the following changes:

1. The Host-PCI bridge drivers need to properly handle bus_adjust_resource() calls for SYS_RES_MEMORY and SYS_RES_IOPORT resources. Typically, this means forwarding the requests up the tree to the device that allocates the resource from a resource manager (struct rman). That driver's bus_adjust_resource method should use rman_adjust_resource(). For an example, see nexus_adjust_resource() in sys/x86/x86/nexus.c.

2. Any bus drivers that provide resources to child PCI devices (e.g. Host-PCI bridge drivers and any parent devices that the bridge drivers pass bus_alloc_resource up to) need to only configure bus space handles and tags in the bus_activate_resource and bus_deactivate_resource methods. Currently, the NEW_PCIB driver relies on the ability to initialize its own resource managers to manage the resource ranges assigned to its I/O windows. When a child device allocates a resource from a PCI-PCI bridge, a resource is allocated from that rman, but that resource is then passed up the tree via bus_activate_resource to obtain a bus space tag and handle. This only works if the bus space tag and handle logic is self-contained in bus_activate_resource and bus_deactivate_resource.

3. Test and then enable in the platform's DEFAULTS config file.

Steps to support PCI_RES_BUS

Once a platform supports the base NEW_PCIB driver, the next step is to add support for PCI bus number management. This requires the following changes:

1. Define a value for PCI_RES_BUS in <machine/resource.h. The PCI-PCI bridge driver and PCI bus driver enable support for PCI bus number management automatically if this is defined.

2. The Host-PCI bridge drivers need to handle requests to allocate (bus_alloc_resource), adjust (bus_adjust_resource), and release (bus_release_resource) PCI_RES_BUS bus numbers. The PCI layer provides a library of routines to manage these requests with a separate resource pool of bus numbers for each PCI domain (or segment). All the Host-PCI bridge driver must do is provide the appropriate domain number when calling the pci_domain_*_bus methods to implement these requests. An example of this can be found in the ACPI Host-PCI bridge driver in sys/dev/acpica/acpi_pcib_acpi.c

Future Work

Support for hotplug PCI-e should build on this work (and will likely require both NEW_PCIB and PCI_RES_BUS).

One issue with the current approach is that passing a request allocated from the PCI-PCI bridge's rman up to a parent device for bus_activate_resource is a gross hack and a layering violation. Also, bus_activate_resource is currently overloaded to do two tasks: 1) mark a resource as actively in use (e.g. a PCI BAR that is active is decoding the assigned address range) and 2) allocate a bus space tag and handle so that the device driver can use the bus_space API to access a resource. Sometimes a resource needs the first property, but having the second property thrown in has unfortunate side effects (e.g. KVA wasteage on 32-bit platforms).

One idea to fix both of these problems would be to split bus_activate_resource up. However, care must be taken to avoid a huge API incompatibility. One approach that may be able to do both would be:

1. Add a new pair of bus methods that allow arbitrary sub-ranges of a given resource to be "mapped" (meaning allocate a bus space tag and handle): bus_map_resource(bus, child, resource, start, length, *tag, *handle) and bus_unmap_resource(bus, child, resource, start, length, tag, handle).

2. Change struct resource to support having a linked list of all mappings so that when a resource is free'd, all mappings for that resource can be released. It would probably be best if this was an optional library call in the rman API that bus drivers can call to add a known mapping to a given resource rather than having them automatically added somehow.

3. When the PCI-PCI bridge driver sub-allocates a resource from a window, it locates the corresponding range of the resource it has allocated from its parent backing the window and uses bus_map_resource to obtain a mapping for the sub-range of that resource that corresponds to the child resource to obtain a mapping for the child resource.

4. To maintain API compatibility, the RF_ACTIVE flag passed to bus_alloc_resource will call bus_map_resource after bus_activate_resource succeeds to allocate the "full" mapping of the resource that sets the public tag and handle members of struct resource. However, a new RF_UNMAPPED flag will inhibit this step allowing a driver to mark a resource as active but chose to only map a subset of a resource for driver access. This last requirement is the trickiest as it means changing just about every bus driver in the tree that has a non-trivial bus_alloc_resource method to support RF_UNMAPPED.

CategoryProject