Multi-Root I/O Virtualization and Sharing Specification

PCIe

需积分: 44 45 浏览量更新于2023-03-16 2 收藏 1.36MB PDF 举报

身份认证购VIP最低享 7 折!

领优惠券(最高得80元）

资源详情

资源推荐

Multi-Root I/O Virtualization and

Sharing Specification

Revision 1.0

May 12, 2008

Multi-Root I/O Virtualization and Sharing Specification, Rev. 1.0

Contents

1. ARCHITECTURAL........................................................................................................... 15

1.1. ARCHITECTURAL OVERVIEW ......................................................................................... 15

1.2. COMPARING PCIE, SR-IOV, AND MR-IOV................................................................... 18

1.2.1. PCIe Topology ...................................................................................................... 18

1.2.2. SR-IOV Topology.................................................................................................. 20

1.2.3. MR-IOV Topology................................................................................................. 21

1.2.4. MRA Components ................................................................................................. 23

1.2.5. MR-IOV and ARI (Alternative Routing Identifier)................................................ 30

1.2.6. MR-IOV Relationship to SR-IOV and ATS ........................................................... 30

1.3. MR TRANSACTION LAYER OVERVIEW........................................................................... 31

2. MR PROTOCOL CHANGES ........................................................................................... 35

2.1. MR LINK AND FLOW CONTROL NEGOTIATION .............................................................. 35

2.1.1. MR Data Link Control and Management State Machine (MR-DLCMSM).......... 35

2.1.2. MR Link Protocol Negotiation (DL_NegotiateMR).............................................. 38

2.1.3. MR Flow Control Initialization Protocol (DL_InitMR) ....................................... 42

2.2. TLP PREFIX TAGGING.................................................................................................... 51

2.2.1. MR Switch Transaction Layer Processing............................................................ 53

2.2.2. MR Device Transaction Layer Processing ........................................................... 55

2.2.3. Global Key Processing ......................................................................................... 56

2.2.4. MR TLP Dataflow Examples ................................................................................ 57

2.3. PER-VH RESET.............................................................................................................. 59

2.3.1. Per-VH Reset Example ......................................................................................... 59

2.3.2. Reset DLLP Format.............................................................................................. 64

2.3.3. Reset DLLP Processing ........................................................................................ 65

2.4. MR FLOW CONTROL...................................................................................................... 71

2.4.1. FC Information Tracked by Transmitter .............................................................. 71

2.4.2. FC Information Tracked by Receiver ................................................................... 74

2.4.3. Electrical Idle Inference ....................................................................................... 76

2.5. MR MESSAGE PROCESSING ........................................................................................... 76

2.5.1. Interrupts............................................................................................................... 76

2.5.2. PME Turn Off Processing..................................................................................... 77

2.5.3. PM_PME Processing............................................................................................ 77

2.6. ORDERING RULES IN MR............................................................................................... 78

2.7. MISCELLANEOUS CHANGES ........................................................................................... 78

2.8. MISCELLANEOUS NON-CHANGES .................................................................................. 79

3. INITIALIZATION AND RESOURCE ALLOCATION ................................................ 81

3.1. MR TOPOLOGY INITIALIZATION .................................................................................... 81

3.1.1. Initial State After Fundamental Reset................................................................... 82

3.1.2. Initial MR-PCIM Location Policy ........................................................................ 86

3.1.3. Topology Discovery .............................................................................................. 86

3.1.4. Component Discovery........................................................................................... 89

Multi-Root I/O Virtualization and Sharing Specification, Rev. 1.0

3.1.5. VH and VF Mapping Policy.................................................................................. 91

3.1.6. VH and VF Mapping Implementation................................................................... 92

3.1.7. MR-PCIM Failover............................................................................................... 98

3.2. MR DEVICE INITIALIZATION.......................................................................................... 99

3.2.1. Enabling MR Operation...................................................................................... 100

3.2.2. Managing Flow Control ..................................................................................... 101

3.2.3. Managing VF Mapping....................................................................................... 101

3.2.4. Managing VF Migration..................................................................................... 104

3.3. MR ROOT PORT INITIALIZATION ................................................................................. 108

4. CONFIGURATION.......................................................................................................... 109

4.1. CONFIGURATION FIELD SUMMARY .............................................................................. 110

4.2. DEVICE CONFIGURATION SPACE.................................................................................. 119

4.2.1. Device MR-IOV Extended Capability................................................................. 120

4.2.2. Device VL Arbitration Table............................................................................... 136

4.2.3. LVF Table ........................................................................................................... 136

4.2.4. Function Table.................................................................................................... 137

4.2.5. Misc. Device Configuration Space Requirements .............................................. 151

4.3. SWITCH CONFIGURATION SPACE ................................................................................. 152

4.3.1. Switch MR-IOV Extended Capability ................................................................. 155

4.3.2. Switch VS Authorization Bitmap......................................................................... 163

4.3.3. Switch Port Table................................................................................................ 164

4.3.4. Switch VL Arbitration Table............................................................................... 184

4.3.5. Switch VS Table .................................................................................................. 184

4.3.6. Switch VS Bridge Table ...................................................................................... 188

4.3.7. Miscellaneous Switch Configuration Space Requirements................................. 202

4.4. VL ARBITRATION TABLE............................................................................................. 208

4.5. PERFORMANCE MONITORING AND STATISTICS COLLECTION....................................... 209

4.5.1. Configuration Space Fields ................................................................................ 211

4.5.2. Statistics Descriptor Table.................................................................................. 214

4.5.3. Statistics Block Table.......................................................................................... 221

4.5.4. Statistics Counter Table...................................................................................... 222

5. ERROR HANDLING....................................................................................................... 225

5.1. PCIE ERROR MAPPING TO MR..................................................................................... 225

5.2. MR ERRORS................................................................................................................. 228

6. HOT PLUG........................................................................................................................ 231

6.1. MRA SWITCH.............................................................................................................. 231

6.1.1. PCI Express Capability: Slot Capability Register.............................................. 232

6.1.2. PCI Express Capability: Slot Control Register .................................................. 233

6.1.3. PCI Express Capability: Slot Status Register..................................................... 235

6.1.4. PCI Express Capability: PCI Express Capabilities Register............................. 236

6.1.5. Hot-Plug Virtual Signals Interface Registers ..................................................... 237

6.1.6. Physical Slot Registers........................................................................................ 238

6.1.7. Physical Hot-Plug Signals Interface................................................................... 238

6.2. VIRTUAL DEVICE MIGRATION ..................................................................................... 238

Multi-Root I/O Virtualization and Sharing Specification, Rev. 1.0 
 
5
6.3. BASE PCI EXPRESS DEVICE MIGRATION ..................................................................... 239 
7.  POWER MANAGEMENT .............................................................................................. 241 
7.1. OVERVIEW ................................................................................................................... 241 
7.2. VIRTUAL D-STATE....................................................................................................... 241 
7.3. LINK POWER STATES ................................................................................................... 242 
7.4. MULTI-ROOT ASPM.................................................................................................... 242 
7.5. SLOT CLOCK AND COMMON CLOCK CONFIGURATION ................................................. 243 
7.6. MULTI-ROOT WAKE-UP .............................................................................................. 243 
7.6.1.  PME Triggers Beacon/Wake#............................................................................. 244 
7.6.2.  Beacon/Wake# Triggers MSI.............................................................................. 244 
7.6.3.  Beacon/WAKE# Triggers Beacon/WAKE#......................................................... 244 
7.7. MULTI-ROOT PME TURN OFF ..................................................................................... 245 
7.8. MULTI-ROOT POWER CONTROLLER............................................................................. 245 
7.9. MULTI-ROOT POWER BUDGETING ............................................................................... 246 
8.  CONGESTION MANAGEMENT .................................................................................. 247 
8.1. OVERVIEW ................................................................................................................... 247 
8.2. CONGESTION ISOLATION.............................................................................................. 248 
8.2.1.  Virtual Links........................................................................................................ 248 
8.2.2.  Bypass Queues .................................................................................................... 255 
8.2.3.  Flow Control Rules............................................................................................. 256 
8.3. PERFORMANCE MONITORING AND STATISTICS COLLECTION....................................... 257 
ACKNOWLEDGEMENTS ..................................................................................................... 259 
 
Figures 
FIGURE 1-1:  GENERIC SERVER BLADE CONFIGURATION............................................................... 16 
FIGURE 1-2:  EXAMPLE SERVER BLADE CONFIGURATION USING MR-IOV TECHNOLOGY ............ 16 
FIGURE 1-3:  EXAMPLE PLATFORM CONFIGURATION WITHOUT SR-IOV OR MR-IOV TECHNOLOGY
............................................................................................................................................... 18 
FIGURE 1-4:  EXAMPLE PLATFORM CONFIGURATION WITH SR-IOV TECHNOLOGY....................... 20 
FIGURE 1-5:  TWO VIRTUAL HIERARCHIES (VH) IMPLEMENTED OVER SHARED PHYSICAL 
COMPONENTS ........................................................................................................................ 22 
FIGURE 1-6:  PHYSICAL COMPONENTS THAT CAN BE SUPPORTED IN AN MR-IOV TOPOLOGY ...... 23 
FIGURE 1-7:  PCIE RP AND MRA RP FUNCTIONAL BLOCK COMPARISON..................................... 24 
FIGURE 1-8:  NON-IOV, SR-IOV, AND MRA DEVICE FUNCTIONAL BLOCK COMPARISON............ 25 
FIGURE 1-9:  MRA PCIM IN AN MR-IOV TOPOLOGY................................................................... 28 
FIGURE 1-10:  PCIE SWITCH AND MRA SWITCH FUNCTIONAL BLOCK COMPARISON.................... 29 
FIGURE 1-11:  EXAMPLE MULTI-ROOT TOPOLOGY........................................................................ 31 
FIGURE 1-12:  EXAMPLE MULTI-ROOT TOPOLOGY AS VIEWED FROM HOST A.............................. 33 
FIGURE 1-13:  EXAMPLE MULTI-ROOT TOPOLOGY AS VIEWED FROM HOST C.............................. 34 
FIGURE 2-1:  MR DATA LINK CONTROL AND MANAGEMENT STATE MACHINE (MR-DLCMSM) 36 
FIGURE 2-2:  EXAMPLE MR TO MR INITIALIZATION SEQUENCE.................................................... 38 