-->
This article clarifies some confusion that vendors have experienced about how hardware that complies with PCI Power Management (PCI-PM) interacts with device drivers in the operating system and about how PCI-PM integrates with ACPI. For more information, see https://www.uefi.org/specifications
Device drivers and PCI power management
This discussion assumes that you are familiar with how Windows Driver Model (WDM) drivers handle power management events, as described in the current Windows DDK. In general, the responsibilities for device drivers are as follows:
Bus drivers: Bus drivers are responsible for enumerating, configuring, and controlling devices. For PCI-PM, the PCI driver is responsible for reading the PCI-PM registers to determine the capabilities of the hardware. When POWER IRPs request power state changes, the PCI driver writes to the PCI power management registers to set the hardware to different Dx states.
When a device is enabled for wake-up, the PCI driver writes to PCI-PM registers to enable the device to fire PME (ACPI will also take an action, see the next section). Finally, when ACPI determines that the PCI bus is waking the system, the PCI driver scans PCI configuration space looking for which device is asserting PME, disables PME in that device, and notifies the driver for that device.
Device driver: The specific driver for the device is responsible for saving and restoring device context, and requesting power state changes as the policy owner for the device. When the device driver receives a POWER IRP requesting a lower device power state change, the device driver is responsible for saving any proprietary device context needed to later turn on the device. In some cases, there may be nothing to save.
- Tech support scams are an industry-wide issue where scammers trick you into paying for unnecessary technical support services. You can help protect yourself from scammers by verifying that the contact is a Microsoft Agent or Microsoft Employee and that the phone number is an official Microsoft global customer service number.
- PATCH v2 07/14 MIPS: jz4780: DTS: Add nodes for ingenic pinctrl and gpio drivers, Paul Cercueil, 2017/01/22 PATCH v2 06/14 MIPS: jz4740: DTS: Add nodes for ingenic pinctrl and gpio drivers, Paul Cercueil, 2017/01/22 PATCH v2 09/14 MIPS: JZ4780: CI20: Add pinctrl configuration for several drivers, Paul Cercueil, 2017/01/22.
- Diff -git a/Documentation/cpu-freq/pcc-cpufreq.txt b/Documentation/cpu-freq/pcc-cpufreq.txt index 0a94224ad296.9e3c3b33514c 100644- a/Documentation/cpu-freq/pcc.
- PATCH v2 09/14 MIPS: JZ4780: CI20: Add pinctrl configuration for several drivers, Paul Cercueil, 15:55 PATCH v2 06/14 MIPS: jz4740: DTS: Add nodes for ingenic pinctrl and gpio drivers, Paul Cercueil, 15:54.
CD4511 7-Segment Driver IC CD4511 is a BCD to 7-segment latch decoder driver IC formed with CMOS logic and NPN bipolar transistor output devices on an immovable structure. This IC is used where we need to driving common-cathode displays like 7-segment display, low voltage fluorescent display, and incandescent display.
PCI-PM registers are strictly the domain of the PCI driver--the IHV's device driver does not need to access any of these registers. Doing so would cause the system to not work reliably. The device driver's responsibility is to perform only proprietary actions.
Integrating ACPI and PCI PM
Some devices, particularly motherboard video devices in portables, may require both PCI Power Management as well as ACPI Source Language Assembler (ASL) to completely power manage the device. The PCI Power Management registers would control the internal state of a device, such as internal clocks and power planes. ASL would control the external state, such as external clocks and power planes, or in the case of video controllers, ASL would control the video backlights. Note that ASL and PCI-PM can only be combined on motherboard devices.
The OnNow architecture is a layered architecture, handling the integration of the device driver, PCI driver, and ACPI driver (and ASL) naturally. The following scenarios show the order in which drivers are called to handle these devices.
Note
For the above scenarios to work as described, a WDM driver must forward POWER IRPs correctly as described in the current version of the Microsoft Windows DDK.
Scenario 1: Turning off a device
- Device driver: Saves proprietary device state.
- PCI driver: Saves Plug and Play configuration, disables the device (interrupts and BARs), and puts the device in D3 using PCI-PM registers.
- ACPI driver: Runs ASL code (_PS3 and _OFF for power resources no longer in use) to control the state external to the chip.
Scenario 2: PCI power management and device drivers
- ACPI driver: Runs ASL code (_PS0 and _ON for any OnNow required power resources) to control the state external to the chip.
- PCI driver: Puts the device in D0 using PCI-PM registers and restores Plug and Play configuration (interrupts and BARs--these might be different from what the device was previously on).
- Device driver: Restores proprietary context in the device.
Scenario 3: Enabling wake-up
- Device driver: Sets proprietary registers in the chip to enable wake-up. For example, in pattern matching network wake-up, this is when the patterns would be programmed into the adapter.
- PCI driver: Sets the wake-up enable bits in the PCI PM registers to allow the device to assert PME.
- ACPI driver: Enables the GPE in the chip set associated with PME (as described by the _PRW object listed under the root PCI bus).
Scenario 4: Wake-up
- ACPI driver: Wakes and scans the GPE status bits for wake-up events, disabling GPEs for set GPE status bits, and running any _Lxx or _Exx methods associated with set GPE bits. In response to a wake-up notification on the PCI bus, the ACPI driver will complete the PCI driver's WAIT_WAKE IRP to notify the PCI driver that it is waking the system.
- PCI driver: Scans configuration space looking for any devices with a set PME status bit. For each device, it disables PME and completes the WAIT_WAKE IRP for that device to inform the driver that it is asserting wake-up. The PCI driver stops scanning for wake devices when it has made a complete pass through all PCI devices having not found any asserting PME and when PME stops being asserted.
- Device driver: Requests the device be put in D0 (see scenario 2) and sets any proprietary registers in the chip required to handle the wake-up event.
Call to action on PCI power management and device drivers
- Integrate ACPI and PCI-PM capabilities into your devices as described in this article.
- The PCI Power Management specification is available at https://www.pcisig.com. This link leaves the Microsoft.com site.
- ACPI Specification available at https://www.uefi.org/specifications. This link leaves the Microsoft.com site.
- The ACPI Component Architecture (ACPICA) compiler can be found at https://acpica.org/downloads/binary-tools.
Drivers Mips Tci Test
As SmartNICs become more popular, more decision makers at companies are being asked to look at the way SmartNICs work – specifically the different offload models related to the OVS datapath, and the pros and cons of each model. In this blog, I will go through and explain these various models. As we go through each model, you will realize that not all of them require offload of the OVS datapath to NIC hardware. In some cases the datapath is actually moved up to the user space as well (some call it on-loading). I will cover all of them using a consistent tabular format for easy readability and comparison. Here we go…
Model 1: OVS datapath in the kernel
Summary of features, pros (in green) and cons (in red):
Location of OVS datapath (kernel, user, NIC) | Kernel | Most mature and proven model Broadest vendor support Easier to integrate and enhance the datapath with other kernel-based networking features such as Conntrack, BPF |
---|---|---|
Policy enforcement using OVS datapath (yes, no) | Yes | Suitable for SDN and Network Virtualization |
OpenStack Orchestration support (yes, no) | Yes | Suitable for cloud-based deployments Supported with most OpenStack distributions |
VM hardware independence (yes, no) | Yes | Uses Virtio driver in the VM making the VMs hardware independent and enabling support of broad array of guest operating systems and live VM migration |
Performance, CPU core use for datapath processing (Mpps, No. of cores) | 1-2 Mpps Using 4 CPU Cores | Poor datapath performance Performance degrades rapidly with more flows and more or complex policy rules Consumes many CPU cores for datapath performance resulting in lowest server utilization and TCO |
Model 2: SR-IOV is used where the OVS datapath is bypassed
Summary of features, pros (in green) and cons (in red):
Location of OVS datapath (kernel, user, NIC) | Kernel but not used (bypassed using SR-IOV) | Mature and proven model Broadest vendor support |
---|---|---|
Policy enforcement using OVS datapath (yes, no) | No Relies on policy enforcement at TOR switch | Not suitable for SDN Not suitable Network Virtualization at scale |
OpenStack Orchestration support (yes, no) | Yes | Suitable for cloud-based deployments where SDN-based policy enforcement is not required Supported with most OpenStack distributions. |
VM hardware independence (yes, no) | No | Uses vendor-specific driver in the VM making the VMs hardware dependent Live VM migration is not supported |
Performance, CPU core use for datapath processing (Mpps, No. of cores) | Close to 30 Mpps No CPU cores as the OVS datapath is not used | Excellent performance delivering packets from network ports to VMs |
Model 3: OVS datapath is moved to the user space using DPDK
Summary of features, pros (in green) and cons (in red):
Location of OVS datapath (kernel, user, NIC) | User space | Broad vendor support Difficult to enhance features leveraging other kernel datapath implementations such as in Conntrack and BPF |
---|---|---|
Policy enforcement using OVS datapath (yes, no) | Yes | Suitable for SDN and Network Virtualization |
OpenStack Orchestration support (yes, no) | Yes | Suitable for cloud-based deployments Supported with many OpenStack distributions |
VM hardware independence (yes, no) | Yes | Uses Virtio driver in the VM making the VMs hardware independent and enabling support of broad array of guest operating systems and live VM migration |
Performance, CPU core use for datapath processing (Mpps, No. of cores) | 6-8 Mpps Using 4 CPU Cores | Good datapath performance Performance degrades rapidly with more flows and more or complex policy rules Consumes many CPU cores for datapath performance resulting in low server utilization and TCO |
Model 4: OVS datapath in the kernel is fully offloaded to the SmartNIC, hardware-dependent VMs
Summary of features, pros (in green) and cons (in red):
Location of OVS datapath (kernel, user, NIC) | Kernel and SmartNIC Fallback to kernel OVS for control traffic and new/first flow | Uses kernel-compliant and upstreamed TC flower based offload of the OVS datapath Offload mechanism is included in the RHEL 7.5 distribution Kernel-based offload makes it easier to enhance features leveraging other kernel datapath implementations such as in Conntrack and BPF Available in the latest kernel releases only until backported to older versions |
---|---|---|
Policy enforcement using OVS datapath (yes, no) | Yes | Suitable for SDN and Network Virtualization. |
OpenStack Orchestration support (yes, no) | Yes | Suitable for Cloud-based deployments Supported with newer OpenStack distributions such as Queens and RHOSP 13 Available in the latest OpenStack releases only until backported to older versions |
VM hardware independence (yes, no) | Yes | Uses vendor-specific driver in the VM making the VMs hardware dependent Live VM migration is not supported |
Performance, CPU core use for datapath processing (Mpps, No. of cores) | 25-28 Mpps No CPU cores used for datapath processing | Excellent datapath performance Performance is maintained well with more flows and more or complex policy rules Frees all CPU cores from datapath processing resulting in high server utilization and TCO |
Model 5: OVS datapath is fully offloaded to the SmartNIC, VMs are hardware-independent
Summary of features, pros (in green) and cons (in red):
Location of OVS datapath (kernel, user, NIC) | Kernel and SmartNIC Fallback to kernel OVS for control traffic and new/first flow | Uses kernel-compliant and upstreamed TC flower-based offload of the OVS datapath Offload mechanism is included in the RHEL 7.5 distribution Kernel-based offload makes it easier to enhance features leveraging other kernel datapath implementations such as in Conntrack and BPF Available in the latest kernel releases only until backported to older versions |
---|---|---|
Policy enforcement using OVS datapath (yes, no) | Yes | Suitable for SDN and Network Virtualization |
OpenStack Orchestration support (yes, no) | Yes | Suitable for Cloud-based deployments Supported with newer OpenStack distributions such as Queens and RHOSP 13 Available in the latest OpenStack releases only until backported to older versions |
VM hardware independence (yes, no) | Yes | Uses Virtio driver in the VM making the VMs hardware independent and enabling support of broad array of guest operating systems and live VM migration Uses one of the two modes: Relay Agent with vhost/DPDK and Virtio 1.0, or vDPA (vhost datapath acceleration) with Virtio 1.1 Consumes 1-3 CPU cores for processing the Relay Agent in user space |
Performance, CPU core use for datapath processing (Mpps, No. of cores) | 25-28 Mpps No CPU Cores used for datapath processing | Excellent datapath performance Performance is maintained well with more flows and more or complex policy rules Frees all CPU cores from datapath processing resulting in high server utilization and TCO |
Model 6: OVS datapath is offloaded to the SmartNIC, VMs are hardware-independent
Summary of features, pros (in green) and cons (in red):
Location of OVS datapath (kernel, user, NIC) | Kernel and SmartNIC Fallback to kernel OVS for control traffic and new/first flow | Uses kernel-compliant and upstreamed TC flower-based offload of the OVS datapath Offload mechanism is included in the RHEL 7.5 distribution Kernel-based offload makes it easier to enhance features leveraging other kernel datapath implementations such as in Conntrack and BPF Available in the latest kernel releases only until backported to older versions |
---|---|---|
Policy enforcement using OVS datapath (yes, no) | Yes | Suitable for SDN and Network Virtualization |
OpenStack Orchestration support (yes, no) | Yes | Suitable for cloud-based deployments Supported with newer OpenStack distributions such as Queens and RHOSP 13 Available in the latest OpenStack releases only until backported to older versions |
VM hardware independence (yes, no) | Yes | Uses Virtio driver in the VM making the VMs hardware independent and enabling support of broad array of guest operating systems and live VM migration Uses hardware implementation of Direct Virtio on the NIC |
Performance, CPU core use for datapath processing (Mpps, No. of cores) | 25-28 Mpps No CPU cores used for datapath processing | Excellent datapath performance Performance is maintained well with more flows and more or complex policy rules Frees all CPU cores from datapath processing resulting in high server utilization and TCO |
Model 7: OVS datapath is in the user space and some functions are offloaded to the SmartNIC
Summary of features, pros (in green) and cons (in red):
Location of OVS datapath (kernel, user, NIC) | User space Partial offload of datapath functions to NIC or SmartNIC hardware using DPDK Flow API | Difficult to enhance features leveraging kernel datapath implementations and innovations such as in Conntrack and BPF |
---|---|---|
Policy enforcement using OVS datapath (yes, no) | Yes | Suitable for SDN and Network Virtualization |
OpenStack Orchestration support (yes, no) | Yes | Suitable for cloud-based deployments |
VM hardware independence (yes, no) | Yes | Uses Virtio driver in the VM making the VMs hardware independent and enabling support of broad array of guest operating systems and live VM migration |
Performance, CPU core use for datapath processing (Mpps, No. of cores) | 10-15 Mpps 4-8 CPU cores used for datapath processing | Good datapath performance, full line rate for 10GbE networks Performance degrades rapidly with more flows and more or complex policy rules Consumes many CPU cores for datapath performance resulting in low server utilization and TCO |
Model 8: Both the OVS datapath and control plane are offloaded to the SmartNIC
Drivers Mips Tci Training
Summary of features, pros (in green) and cons (in red):
Location of OVS datapath (kernel, user, NIC) | SmartNIC Control plane runs in the user space on an ARM or MIPS CPU on the SmartNIC Datapath slow path runs in the kernel or user space on the ARM and fast path runs on accelerator chip available on some SmartNICs | Useful for bare metal cloud applications where the service provider has no control over what host operating system is used on the server Best when SDN and cloud orchestration is implemented via the SmartNIC (not through the host) |
---|---|---|
Policy enforcement using OVS datapath (yes, no) | Yes | Suitable for SDN and Network Virtualization Non-Bare Metal Use Case: When the host runs OVS control plane and datapath, the version that runs on the SmartNIC can easily go out of sync with the version that runs on the host, causing anomalies and feature inconsistencies related to SDN deployments |
OpenStack Orchestration support (yes, no) | Yes | Suitable for cloud-based deployments Non-Bare Metal Use Case: When the host runs OVS control plane and datapath, the version that runs on the SmartNIC can easily go out of sync with the version that runs on the host, causing anomalies and feature inconsistencies related to cloud orchestration. |
VM hardware independence (yes, no) | No | Uses vendor-specific driver in the VM making the VMs hardware dependent Live VM migration is not supported |
Performance, CPU core use for datapath processing (Mpps, No. of cores) | 15-20 Mpps No CPU cores used for datapath processing OVS control plane processing on the host consumes less than a CPU core. This is eliminated by this approach | Good datapath performance, full line rate for 10GbE networks for small packets and 25GbE networks for mid-sized packets |