PCI Express 4.

0
Electrical compliance test overview
Agenda
 PCI Express 4.0 electrical
compliance test overview
 Required test equipment
 Test procedures:
 Transmitter Electrical testing
 Transmitter Link Equalization
testing
 Receiver Link Equalization
testing
 PLL Bandwidth testing
 Q&A

2
PCI Express: terminology and history
PCIE Terminology History
 PCI Express Standards are  A new version of each of these
maintained by the PCI-SIG specifications is developed for each
 Peripheral Components Interface generation of PCIE
Special Interest Group

 2005: PCIe 1.0, 2.5 Gb/s

 PCI Express specifications:
 BASE specification: defines  2007: PCIe 2.0, 5 Gb/s
device behavior at the chip level  2010: PCIe 3.0, 8 Gb/s
 CEM (Card ElectroMechanical)  2018: PCIe 4.0, 16 Gb/s
specification: defines device
behavior at the card connector
 Test specification: how to test a
device for CEM spec compliance

3
The PCI Express 4.0 Timetable
WS101 WS102 WS103 WS104 WS105 WS106

April 2017 August 2017 October 2017 December 2017 April 2018 August? 2018

First Gen4 Likely Gen4

Not a Gen4 workshop FYI workshop
preliminary workshop
Possible Gen4 Possible first Gen4
Gen4 preliminary workshop
FYI workshop compliance workshop

 Preliminary workshop: Primary purpose is test and specification development. Test results are not
required to be shared with device vendors.

 FYI workshop: Vendors receive pass/fail results but no official integrator’s list. At least 2 FYI
workshops are run before official compliance testing begins.

 Compliance workshop: Test specification is complete and approved. Devices are officially tested,
and passing devices are added to the integrator’s list.

4
Compliance Test Specification overview

Defined PCI Express 4.0 compliance tests:

Test name New for Gen4? Notes
Transmitter Signal Quality Automated using SigTest
Transmitter Pulse Width Jitter Yes New for 4.0 – still being defined
Transmitter Preset Automated using SigTest
Transmitter initial Tx Equalization
Transmitter Link Equalization response
Lane Margining Timing Yes New for 4.0 – still being defined
Lane Margining Voltage Yes New for 4.0 – still being defined
Receiver Link Equalization Replaces receiver test from Gen3
PLL Bandwidth Only tested for add-in card
PCB Impedance Informative only – VNA test

(PCI Express 4.0 PHY test spec, Rev 0.5)

5
PCI Express 4.0 test equipment
Oscilloscope (all tests)
 High-bandwidth real-time oscilloscope
 25 GHz bandwidth is required for Gen4
 13 GHz bandwidth was required for
Gen3
 Integrated eye diagram and jitter
analysis tools
 Channel embedding and fixture de-
embedding
 PCI Express decoding with waveform
annotation and tabular analysis
 ProtoSync for high-level protocol
decode
 Server-class CPU with 20 cores gives
a significant speed advantage in PCIe Teledyne LeCroy LabMaster 10Zi-A
Oscilloscope models from 25 GHz to 100 GHz,
Gen4 testing
to support 16 Gb/s, 32 Gb/s and beyond

7
BERT (all tests except Tx tests)
Anritsu MP1900A SQA-R
 Multi-channel BERT from 2.4 Gbit/s to 32.1 Gbit/s
 Max 16-ch 32G NRZ or 8-ch 64G PAM4
 Link Training/Equalization and LTSSM Analysis
 Signal Integrity (low intrinsic jitter and fast Tr/Tf)
 Maximum 10 Tap +/-20dB Emphasis function
 12 dB CTLE and clock recovery functions
 Jitter and noise generation

Applications
 PCIe Gen1 to 5, Thunderbolt 3, USB3.1 Gen1/2
 IEEE 100/200/400 GbE, CEI-25/28/56/112G
 InfiniBand EDR/HDR, Fibre Channel
 Optical Module, SERDES, AOC, High-Speed Interconnect

8
Built-in PCI Express Link Training and LTSSM Analysis Functions
 MP1900A series supports Physical layer evaluations
 PCIe Gen1 to Gen4 and future Gen5 receiver tests
 Analyzing LTSSM (Link Training Status State Machine)
 Tx/Rx Link Equalization Response Test
 Rx Link Equalization Test
 Receiver Jitter Tolerance Test
 Identify the root of the Link Failure problem.

9
Test automation software
QualiPHY PCIe4-Tx-Rx
 Can be run on LabMaster 10Zi-A
or external PC
 Automates all testing
 Collection and analysis of
waveforms for Tx test
 BERT calibration and jitter
tolerance for Rx test
 Automated Link EQ testing
 Connection diagrams
 Report generation
 Can be integrated with external
test automation through simple,
powerful Host Program Control
feature

10
Additional automation options
TF-PCIe4-CTRL
 PCIe Gen4 Compliance Base
Board will feature dedicated
automation control headers for:
 Board power on/off
 DUT reset
 Tx preset toggle
 Teledyne LeCroy TF-PCIe4-CTRL
CBB automation controller
connects directly to the
oscilloscope
 Fully integrated into QPHY-PCIe4-
Tx-Rx to enable total test
automation for PCIe Gen4 testing

11
Test system components and the specification
Teledyne LeCroy Anritsu
LabMaster 10Zi-A MP1900A SQA-R

compliance software

Channel embedding

2.4 – 32.1 Gb/s ED

25 GHz bandwidth
TF-PCIE4-CTRL

LTSSM Analysis
Integrated noise
2.4 – 32.1 Gb/s

Integrated jitter
PCIe decode

Link Training
automation

processing

ProtoSync
Multi-core

Emphasis
QualiPHY

sources

sources
PPG
Transmitter Signal Quality X X X X X
Transmitter Pulse Width Jitter Specification still in progress
Transmitter Preset X X X X
Transmitter initial Tx Equalization X X X X X X X X X
Transmitter Link Equalization response X X X X X X X X X
Lane Margining Timing Specification still in progress
Lane Margining Voltage Specification still in progress
Receiver Link Equalization X X X X X X X X X X X X
PLL Bandwidth X X X X

12
PCI Express 4.0 test overview
Transmitter electrical testing
Transmitter electrical testing

Test name New for Gen4? Notes

Transmitter Signal Quality Automated using SigTest
Transmitter Pulse Width Jitter Yes New for 4.0 – still being defined
Transmitter Preset Automated using SigTest
Transmitter initial Tx Equalization
Transmitter Link Equalization response
Lane Margining Timing Yes New for 4.0 – still being defined
Lane Margining Voltage Yes New for 4.0 – still being defined
Receiver Link Equalization Replaces receiver test from Gen3
PLL Bandwidth Only tested for add-in card
PCB Impedance Informative only – VNA test

(PCI Express 4.0 PHY test spec, Rev 0.5)

14
Transmitter testing overview
 Two basic transmitter tests:
 Preset test: check that each
transmitter emphasis preset is
within limits
 Signal quality test: eye
diagram, jitter etc.

 Transmitter tests are performed

using SigTest software from PCI-
SIG
 SigTest for PCIe Gen4 is still in
development and not available
outside test development
subgroup

15
 PCIe 4.0 nominal channel

28 dB total system loss

All loss values specified at 8 GHz (Nyquist frequency for 16 Gb/s)

3dB package loss

Package
DIE
Endpoint

Rx
Tx
5dB package loss

Add-in card
DIE Tx
Rx

Package

8dB loss
(incl 3 dB package loss)

20 dB loss (incl 5 dB package loss)

Root complex CEM connector
System Board

16
Add-in card transmitter test

Device under test

Endpoint

Add-in card
We want to measure the Tx signal here:
Root-complex Rx after worst-case (20 dB) loss

20 dB loss (incl 5 dB package loss) Signal

Root complex CEM connector
System Board

17
Add-in card transmitter test: connection schematic
• 5dB is in the Add-in card 5dB package model
root-complex under test
package,
which we
emulate by
embedding an
s4p file using
the scope’s
software

• The other 15dB channel

15dB comes
from the ISI
board

18
Add-in card transmitter test: connection schematic
 Transmitter
test uses the
same setup
for both
preset and
signal quality
tests

19
Device preset selection
 For Gen4 testing, it’s necessary to
acquire at least one 1.6MUI
waveform at each preset

 Use compliance toggle button

(circled) or TF-PCIe4-CTRL to
toggle through presets:
 Gen1: 1 x 2.5 Gb/s preset
 Gen2: 2 x 5 Gb/s presets
 Gen3: 10 x 8 Gb/s presets
 Gen4: At least 11 presets:
P0-P10 compliance waveforms
Toggle (1010) pattern on all lanes
Toggle on each Tx lane with traffic
on all other lanes

20
Transmitter test execution

PCIE 4.0 Test Report

 QPHY-PCIe4-Tx-Rx runs SigTest preset and signal quality tests on

acquired waveforms
 Signal Quality test can be time-consuming
 QPHY-PCIe4-Tx-Rx leverages the 20-core processor in the LabMaster 10Zi-
A to execute many SigTest instances in parallel, reducing test time

21
Add-in card test procedure: Preset test results
 SigTest runs the preset test on
all acquired waveforms

 It measures the pre-shoot and

de-emphasis on the P4 (0dB,
0dB) waveform and tests all
others relative to that

22
Add-in card test procedure: Signal Quality test
 Signal Quality test is performed
only on one preset – typically P5
or P6 for PCIe Gen4

23
Add-in card test procedure: Signal Quality results

24
Test system components and the specification
Teledyne LeCroy Anritsu
LabMaster 10Zi-A MP1900A SQA-R

compliance software

Channel embedding

2.4 – 32.1 Gb/s ED

25 GHz bandwidth
TF-PCIE4-CTRL

LTSSM Analysis
Integrated noise
2.4 – 32.1 Gb/s

Integrated jitter
PCIe decode

Link Training
automation

processing

ProtoSync
Multi-core

Emphasis
QualiPHY

sources

sources
PPG
Transmitter Signal Quality X X X X X
Transmitter Pulse Width Jitter Specification still in progress
Transmitter Preset X X X X
Transmitter initial Tx Equalization X X X X X X X X X
Transmitter Link Equalization response X X X X X X X X X
Lane Margining Timing Specification still in progress
Lane Margining Voltage Specification still in progress
Receiver Link Equalization X X X X X X X X X X X X
PLL Bandwidth X X X X

25
PCI Express 4.0 test overview
Transmitter Link Equalization testing
Transmitter Link Equalization

Test name New for Gen4? Notes

Transmitter Signal Quality Automated using SigTest
Transmitter Pulse Width Jitter Yes New for 4.0 – still being defined
Transmitter Preset Automated using SigTest
Transmitter initial Tx Equalization
Transmitter Link Equalization response
Lane Margining Timing Yes New for 4.0 – still being defined
Lane Margining Voltage Yes New for 4.0 – still being defined
Receiver Link Equalization Replaces receiver test from Gen3
PLL Bandwidth Only tested for add-in card
PCB Impedance Informative only – VNA test

(PCI Express 4.0 PHY test spec, Rev 0.5)

27
 Transmitter Link Equalization testing
 Capture signals with all transmitter emphasis presets, and test using SigTest
 Preset changes are initiated through protocol request from the test equipment
 Not switched manually using fixture as for Tx electrical tests

Device under test

1.
Preset
request Measure
response
time
Add-in card

Trigger

3.
2.
Requested
Verify
preset
presets

CBB

28
Example test: response time
 “The test verifies that the add-in card will respond correctly to
transmitter equalization commands sent via the link protocol.”

 BERT negotiates DUT to change its transmitter emphasis preset

 Time, t, between request for preset change (protocol layer) and actual
preset transition is measured using the oscilloscope:
 t < 500ns: Pass
 500ns < t <1us: Pass with warning
 t > 1us: Fail

29
Transmitter Link Equalization – test setup
 Signal from
BERT to DUT
and from DUT
to BERT are
split to the
oscilloscope

 Trigger signal
from BERT
enables
scope to be
triggered at
any point in
the link
training
sequence

30
Example test: response time
 BERT
requests DUT
change its Tx
emphasis
preset from
P7 to P4

 BERT sends
trigger to
oscilloscope
at the time of
preset
change
request

31
Example test: response time
 Oscilloscope
acquires
both sides of Downstream signal
transaction (from BERT)

Upstream signal
(from DUT)

32
Example test: response time
 The emphasis
change is
clearly visible:
this is the end-
point of the
measurement

 But we need to
determine the ?
exact timing of
the protocol-
layer request
so we know
where to start
the
measurement

33
Example test: response time
 The oscilloscope
decodes the
downstream signal into
digital data and passes
it to Teledyne LeCroy
protocol analysis
software

34
Example test: response time
 The
oscilloscope
trace is time-
correlated
with the
protocol
analysis

 The packet
can be easily
identified in
the
waveform

35
Example test: response time
 Now it’s trivial
to measure
the response
time from
protocol
request to
physical
emphasis
change

 This device’s
response time
is 81.18ns –
an easy pass

36
Test system components and the specification
Teledyne LeCroy Anritsu
LabMaster 10Zi-A MP1900A SQA-R

compliance software

Channel embedding

2.4 – 32.1 Gb/s ED

25 GHz bandwidth
TF-PCIE4-CTRL

LTSSM Analysis
Integrated noise
2.4 – 32.1 Gb/s

Integrated jitter
PCIe decode

Link Training
automation

processing

ProtoSync
Multi-core

Emphasis
QualiPHY

sources

sources
PPG
Transmitter Signal Quality X X X X X
Transmitter Pulse Width Jitter Specification still in progress
Transmitter Preset X X X X
Transmitter initial Tx Equalization X X X X X X X X X
Transmitter Link Equalization response X X X X X X X X X
Lane Margining Timing Specification still in progress
Lane Margining Voltage Specification still in progress
Receiver Link Equalization X X X X X X X X X X X X
PLL Bandwidth X X X X

37
PCI Express 4.0 test overview
Receiver Link Equalization testing
Receiver electrical testing

Test name New for Gen4? Notes

Transmitter Signal Quality Automated using SigTest
Transmitter Pulse Width Jitter Yes New for 4.0 – still being defined
Transmitter Preset Automated using SigTest
Transmitter initial Tx Equalization
Transmitter Link Equalization response
Lane Margining Timing Yes New for 4.0 – still being defined
Lane Margining Voltage Yes New for 4.0 – still being defined
Receiver Link Equalization Replaces receiver test from Gen3
PLL Bandwidth Only tested for add-in card
PCB Impedance Informative only – VNA test

(PCI Express 4.0 PHY test spec, Rev 0.5)

39
Add-in card receiver test

Device under test

Endpoint

Add-in card
We want to calibrate the Rx signal here:
Root-complex Tx after worst-case (20 dB) loss

Signal

20 dB loss (incl 5 dB package loss)

Root complex CEM connector
System Board

40
Jitter and Noise Injection
Jitter Injection
• Dual Tone SJ: 1UI @ 250MHz
• Random Jitter (RJ): 0.5UIpp (2.2ps RMS @16GHz)
• BUJ and Half Period Jitter (Even/Odd Jitter)
• SSC
Noise Injection
• Common mode noise frequency: 0.1 GHz to 6 GHz
• Differential mode noise frequency: 2 GHz to 10 GHz
• White noise band: 10 GHz; Crest Factor: >5

Sinusoidal Jitter(SJ) Random Jitter(RJ) CM/DM Noise White Noise

41
Add-in card receiver test, Step 1:
Preset, Rj, Sj calibration

Preset calibration

Jitter calibration

42
Add-in card receiver test, Step 2:
DMI, CMI, initial preset/CTLE selection
 Differential and common-mode noise are calibrated with a 27 dB channel
 Optimal Tx preset + Rx (scope) CTLE combination is established with 27dB channel
Scope with embedded
3dB package model

BERT

43
DMI, CMI, Eye calibration: Connection schematic
3dB package model

Jitter
ED
PPG
Noise

5dB channel

19-22dB channel

1m SMA-SMA cables
44
Add-in card receiver test, Step 2:
DMI, CMI, initial preset/CTLE selection

Preset 5 or Preset 6
CTLE from 7.5dB – 9dB

 The next steps in Rx calibration should be performed with the best combination of:
 Tx emphasis Preset 5 or Preset 6
 One of 9 CTLE peaking values available for SigTest
 At least 5 waveforms of each preset should be analyzed for averaging
 5 waveforms x 2 presets x 9 CTLE presets = 90 SigTest runs at ~2 minutes each =
3 hours for this stage of calibration if done sequentially
 LabMaster 10Zi-A can process up to 20 waveforms in parallel, substantially
reducing test time for this step to approx. 20 minutes

45
Add-in card receiver test, Step 3:
Find marginal channel
 Increase total channel loss in 0.5dB increments
 Check eye width and height, stop incrementing if final calibration targets are violated
 Otherwise, stop at 30dB channel
Scope with embedded
3dB package model

BERT

46
Add-in card receiver test, Step 3:
Find marginal channel

Preset 5 or Preset 6
CTLE from 7dB – 9dB

 At least 5 waveforms at each channel should be analyzed for averaging

 When the new channel is established, optimal preset/CTLE must be
established AGAIN
 5 waveforms x 2 presets x 9 CTLE presets = 90 SigTest runs at ~2 minutes
each = 3 hours for this stage of calibration if done sequentially
 LabMaster 10Zi-A can process up to 20 waveforms in parallel, substantially
reducing test time for this step to approx. 20 minutes
47
Add-in card receiver test, Step 5:
Final Eye calibration
 Using final channel, converge to target eye height and width values by
varying amplitude, DM interference, and Sj
Scope with embedded
3dB package model

BERT

48
Add-in card receiver test, Step 6:
Negotiate into loopback
 The DUT must now be negotiated
into loopback by the BERT
 This must happen through the
“Recovery path”, where the device
performs link training and requests
the optimal transmitter preset from
the BERT

49
Add-in card receiver test, Step 7:
Perform BER test
 Once in loopback, a BER test is
run
 A PASS is defined as no more than
one error in 1012 bits

50
Jitter Tolerance Test

51
Test system components and the specification
Teledyne LeCroy Anritsu
LabMaster 10Zi-A MP1900A SQA-R

compliance software

Channel embedding

2.4 – 32.1 Gb/s ED

25 GHz bandwidth
TF-PCIE4-CTRL

LTSSM Analysis
Integrated noise
2.4 – 32.1 Gb/s

Integrated jitter
PCIe decode

Link Training
automation

processing

ProtoSync
Multi-core

Emphasis
QualiPHY

sources

sources
PPG
Transmitter Signal Quality X X X X X
Transmitter Pulse Width Jitter Specification still in progress
Transmitter Preset X X X X
Transmitter initial Tx Equalization X X X X X X X X X
Transmitter Link Equalization response X X X X X X X X X
Lane Margining Timing Specification still in progress
Lane Margining Voltage Specification still in progress
Receiver Link Equalization X X X X X X X X X X X X
PLL Bandwidth X X X X

52
PCI Express 4.0 test overview
PLL Bandwidth
PLL Bandwidth

Test name New for Gen4? Notes

Transmitter Signal Quality Automated using SigTest
Transmitter Pulse Width Jitter Yes New for 4.0 – still being defined
Transmitter Preset Automated using SigTest
Transmitter initial Tx Equalization
Transmitter Link Equalization response
Lane Margining Timing Yes New for 4.0 – still being defined
Lane Margining Voltage Yes New for 4.0 – still being defined
Receiver Link Equalization Replaces receiver test from Gen3
PLL Bandwidth Only tested for add-in card
PCB Impedance Informative only – VNA test

(PCI Express 4.0 PHY test spec, Rev 0.5)

54
PLL Bandwidth overview
 “The test verifies that the add-in card PLL bandwidth and peaking are
within the limits allowed by the PCI Express specifications.”

 This is essentially a jitter transfer function measurement, with the

intention of checking:
 That the -3dB point of the DUT’s jitter transfer function is within an
acceptable frequency range
 That the DUT’s jitter transfer function does not have excessive peaking

 To perform this test, we use the BERT’s subrate clock output to

intentionally apply calibrated jitter to the reference clock used by the
DUT

55
PLL Bandwidth: Calibration
 Apply a defined amount of
sinusoidal jitter (Sj) across the
PLL bandwidth measurement
range to a 100MHz subrate
clock

 Measure the jitter at each

frequency using the oscilloscope

56
PLL Bandwidth: Test
 For each applied calibrated Sj
value, measure the periodic jitter
(Pj) at the device transmitter

 Plot a curve of the jitter transfer for

each frequency, and compare to
the specification limits

57
Test system components and the specification
Teledyne LeCroy Anritsu
LabMaster 10Zi-A MP1900A SQA-R

compliance software

Channel embedding

2.4 – 32.1 Gb/s ED

25 GHz bandwidth
TF-PCIE4-CTRL

LTSSM Analysis
Integrated noise
2.4 – 32.1 Gb/s

Integrated jitter
PCIe decode

Link Training
automation

processing

ProtoSync
Multi-core

Emphasis
QualiPHY

sources

sources
PPG
Transmitter Signal Quality X X X X X
Transmitter Pulse Width Jitter Specification still in progress
Transmitter Preset X X X X
Transmitter initial Tx Equalization X X X X X X X X X
Transmitter Link Equalization response X X X X X X X X X
Lane Margining Timing Specification still in progress
Lane Margining Voltage Specification still in progress
Receiver Link Equalization X X X X X X X X X X X X
PLL Bandwidth X X X X

58
Please visit LeCroy and Anritsu at Designcon 2018
 Anritsu live demos at booth 741:
 PCIe Gen4 Receiver Compliance Test
 High Speed Serial Bus RX automation Test (PCIe, Thunderbolt, USB)
 400G PAM4 Direct Attach Cupper (DAC) Cable BER Test
 100G Active Optical Cable (AOC) JTOL Test

 Teledyne LeCroy live demos at booth 515:

 PCIe Gen4 Link Equalization Compliance Test
 PCIe Gen4 Protocol Analysis
 USB 3.1 Type-C Compliance Solutions
 PAM4 Test, Analysis and Debug
 DDR Memory Testing

59
Thank you