Formal Verification:

Too Good to Miss

Jonathan Bromley, Jason Sprott

© Accellera Systems Initiative 1

Duration: 15 minutes

SECTION 1:
PROMISE AND PLANNING

© Accellera Systems Initiative and Verilab 2

 Formal Property Checking
You can even start
work without RTL

• Spec. captured as properties

FPC Assert
• Formal methods – no simulation
DUT Assume
RTL • Prove properties (e.g. SVA) hold
Prove Cover
• Exhaustive state space coverage
• Interactive development/debug
result debug • Some limitations

Start work without

many/any properties

© Accellera Systems Initiative and Verilab 3

 Benefits
• Another view on the specification
– encourages critical mindset
• Potentially exhaustive
• Finds gnarly bugs
• Useful at a very early stage
– even when RTL and TB are incomplete
• Focus on design behaviour (not stimulus)
• Focused debug (near-minimal CEX)

© Accellera Systems Initiative and Verilab 4

 Drawbacks
• Difficult to tell if your design is suitable for FV
• Can be costly of compute resources
• Time to closure is hard to predict
• Requires skill in all but the simplest cases
• Results not always easy to interpret

It's still worth it!

© Accellera Systems Initiative and Verilab 5

 Use Models – Shift Left
Closure

Formal Property Checking

Bring-up Develop Bounded/Full Proof

Detailed bug hunting

Early bring-up • Can begin before RTL and
• Find bugs earlier formal TB are complete
• Incomplete TB useful • FV finds concise CEX
• Improve specification
• Value from FV done on other
blocks, e.g. assume-guarantee

© Accellera Systems Initiative and Verilab 6

 Use Models – Block Signoff
Closure

Formal Property Checking

Bring-up Develop Bounded/Full Proof

Thorough checking and proof

Late design changes,
• TB and RTL complete rogue bug hunting
• Rare to get full proofs of everything
• Detailed review of formal TB is vital
• Apps, e.g. auto-generated properties

© Accellera Systems Initiative and Verilab 7

 Save Time Using Formal Apps
Formal Property Checking
Bring-up Develop Bounded/Full Proof

Auto-Property Generation Extract implementation detail properties

Register Access Generate reset, access policy and functional checks

End-to-end Checkers IP for hard to develop checker models, e.g. scoreboards

Unreachable Analysis Identify unreachable states and save manual analysis

Coverage Analysis Are we done? Results: code, COI, proof, functional

X Propagation

Clock Domain Crossing And so on … with apps working across the flow

SoC Connectivity

© Accellera Systems Initiative and Verilab 8

 Use Models – Architecture
• Replace the RTL design with “assume property”
directives
• Can point to architectural issues:
– deadlocks
– latency limits
–…

© Accellera Systems Initiative and Verilab 9

 The Big Questions

Q1: Can I use FV on this block?

Q2: How do I get started?

© Accellera Systems Initiative and Verilab 10

 FV Planning
• Start with expectations
– Full/bounded proofs
– Early bring-up
– Identify goals, e.g. tricky to verify with simulation
• Analyze the actual design
• Repeatable, maintainable, sign-off auditable
• Closure
– Feed into the overall verification plan
– Dovetail with simulation

© Accellera Systems Initiative and Verilab 11

 Start With Expectations
It's not all about full proofs!

• Bug absence assured

• Difficult to predict if achievable
Full Proofs • Depends on size/type of design
• Takes time

• AKA "bounded"
Undetermined • Valuable
• Validate depth by analysis?
• Can be quick

© Accellera Systems Initiative and Verilab 12

 Plan For Early Bring-Up
• Essential for initial design analysis
• Designers can start on draft RTL
• Ask and capture simple questions
• Early/often drops, adding/fixing functionality
• Look for collateral
– Interface and protocol assertions
– Embedded properties
– FV end-to-end checkers
– Abstractions, e.g. counters, memories
• Results useful down the line

© Accellera Systems Initiative and Verilab 13

 Analyze: Complexity, States, Depth

Are all channels

identical?

Are all bits treated

the same way?

Can we simplify
by modeling?

© Accellera Systems Initiative and Verilab 14

 Closure
• Validate all assumptions
– Assume-guarantee in FV
– Run in simulation (property style must be considered)
– Review (sometimes the only way)
• Validate abstractions
– Are they "safe", i.e. do not over constrain?
• Gaps between simulation and FV
– Coverage needs looked at closely
– Extend confidence in bounded proof with simulation
– Small focused properties (typically better) can leave gaps
– Abstractions in FV can leave gaps

© Accellera Systems Initiative and Verilab 15

Duration: 45 min

SECTION 2: DELIVERING

© Accellera Systems Initiative and Verilab 16

 Start from the Spec
• Interface specifications:
– no grant without request…
– if VALID and not READY, everything should be stable…
– latency limits
– eventual response
• Protocol specifications:
– correct number of beats in burst, valid controls, …
• End-to-end specifications:
– transaction integrity, routing
– transaction ordering

© Accellera Systems Initiative and Verilab 17

 Analyze The Design?
• Structure
– Partitioning, size (e.g. I/O, flops)
• Functionality
– FSMs, counters, arithmetic, memory
– States, sequential depth
– Clocks, resets, asynchronous logic
• Properties (assert, assume, cover)
– Protocols, embedded, assume-guarantee
• Parameterization
– No shortcuts using FV
• Leverage
– Symmetry, abstractions

© Accellera Systems Initiative and Verilab 18

 An example
• Response steering/ordering block
• Loosely based on AXI AW-to-B channel relationships
• Massively simplified
– for tutorial clarity
– to avoid any confidentiality issues

© Accellera Systems Initiative and Verilab 19

 Example DUT
response_reorder
INITIATOR SLAVE 0
w_chan s_valid[0]
w_valid s_data[0]
w_ready s_ready[0]
w_data
w_id

SLAVE N-1
RESPONDER
s_valid[N-1]
b_valid FIFO s_data[N-1]
b_ready
s_ready[N-1]
b_id

© Accellera Systems Initiative and Verilab 20

 Planning the FV effort
• Interface specifications
– valid/ready handshake integrity Per-port checkers
– handshake eventually completes
• Protocol specifications
– no new same-ID write until Modelling and checker
required
previous ID acknowledged
• End-to-end specifications
– write goes to correct downstream port with correct data
– every write gets acknowledged in order FIFO-like checking

© Accellera Systems Initiative and Verilab 21

 Port protocol
• AXI-style valid/ready handshake
• Transfer when (VALID && READY)
– Payload and VALID must remain stable until READY

clock

PAYLOAD A B C D
VALID

READY

transfer A B C D

stability one xfr per cycle READY and payload

ignored if no VALID
© Accellera Systems Initiative and Verilab
22
 End-to-end specifications
• Responses strictly in initiator order
– but no limits on overlapping of
downstream activity
• Response cannot be given until transaction has
completed downstream
• No two same-ID transactions in flight together
– strict write observability
• Data exactly preserved from initiator to selected slave
• Initiator ordering preserved at each slave

© Accellera Systems Initiative and Verilab 23

 From spec to testbench
• Responses strictly in initiator order
– but no limits on overlapping of
downstream activity
• Response cannot be given until transaction has completed downstream
• No two same-ID transactions in flight together
– strict write observability
• Data exactly preserved from initiator to slave
• Initiator ordering preserved at each slave

• Don't think about stimulus!

– Focus on spec rules
• A surprisingly short list of key rules may be enough
– protocol on ports
– ordering among transactions
– ordering between ports
– transaction integrity
© Accellera Systems Initiative and Verilab 24
 Coding the properties
property p_payload_stable(VALID, READY, PAYLOAD);
disable iff (~areset_n) @(posedge clock)
reusable VALID && !READY |=> $stable(PAYLOAD);
endproperty
property
ast_b_id_stable: assert property (
property p_payload_stable(b_valid, b_ready, b_id)
instance );

asm_w_id_stable: assume property (

p_payload_stable(w_valid, w_ready, w_id)
);

• Assertion checks correct behaviour on DUT outputs

• Assumption enforces correct behaviour on inputs
– constrains TB to follow valid protocol

© Accellera Systems Initiative and Verilab 25

 Inside-out testbench structure
• Formal TB is a module module DUT #(parameter A = ...)
(input ... , output ...);
– ports match DUT module
bound_TB
• Bind TB into the DUT ...
dut_sva_TB

endmodule
module dut_sva_TB #(parameter A = ...)
(input ... , input ...);

... bind DUT

ast_DUT_xxxx: assert property .... dut_sva_TB #(.A(A), ...)
... bound_TB (.*);

endmodule
Vertical reuse is trivially easy
© Accellera Systems Initiative and Verilab 26
 Testbench structure
• Formal TB has only input ports
– Input ports are free to take any value at any time …
– … assumptions act as constraints, not drivers
• Inside-out structure unaltered works in simulation
– assumptions act as assertions
• FV tools allow for scripted addition of
– properties
– stopats
– modules
– …

© Accellera Systems Initiative and Verilab 27

 Unbounded latency
property p_eventually_ready(VALID, READY);
disable iff (~areset_n) @(posedge clock)
liveness property VALID |-> strong (##[0:$] READY);
endproperty
also SV-2009 eventually

SV-2009 keyword,
required for some tools

• Use in assumptions to guarantee forward progress

• Use in assertions to check no lockup

• Beware potential inefficiency / weakness in simulation

© Accellera Systems Initiative and Verilab 28

 Finding a real DUT bug
• Flawed early version of DUT can lose some responses

Failure shown in
assertions list

• Found in less than

1 sec runtime!

Looping trace shows

failure of liveness
property

© Accellera Systems Initiative and Verilab 29

 Finding a testbench bug
Violation: same ID written
before B completed

• Flawed assertion
“no duplicate ID in flight”
sample_written && !sample_responded
|->
(!(w_hsk && (w_id == sample_id)))
||
(b_hsk && (w_id == sample_id))

• Fix assertion to allow

same-ID W and B in the
same clock cycle

© Accellera Systems Initiative and Verilab 30

 Assume-guarantee
response_reorder
target device
SLAVE 0
s_valid[0] valid
s_data[0] data
s_ready[0] ready

assert property ( assume property (

p_payload_stable( p_payload_stable(
s_valid[0], valid,
s_ready[0], ready,
s_data[0]) data)
); );

• Powerful check on the properties themselves

– Flaws will cause false failures in one or other case
– Easily automated with parameters/generates
© Accellera Systems Initiative and Verilab 31
 Finding flawed properties
Too strict Too lax

DUT errors
assert False fails
• Cross-check missed

– by assume-guarantee
Some DUT fails
– or by simulation assume behaviours because of
not exercised bad input

• Review goals cover

Unreachable Coverage not
covers trustworthy

© Accellera Systems Initiative and Verilab 32

 Free variables, arbitrary tie-offs
• Default: undriven signals and inputs are free
– Can take any value at any time
– Constraints (assumptions) needed to enforce protocol

asm_fixed_but_arbitrary_cfg:
assume property (
disable iff (~areset_n) @(posedge clock)
$stable(cfg)
);

• Useful idiom: a free variable that's held constant

– proof must take account of all possible values
– but value doesn't change from one clock to the next

© Accellera Systems Initiative and Verilab 33

 Another useful modelling idiom
• Pulse for exactly one clock, just once,
at an arbitrary time during each trace
– don't try to simulate this!

assume property ( disable iff (~areset_n) @(posedge clock)

pulse ##[1:$] |-> !pulse;
);
Also allows for no pulse, ever

© Accellera Systems Initiative and Verilab 34

 Auxiliary logic (1)
• Combinational encapsulation
assign w_hsk = w_valid && w_ready;

• Event-has-happened recording
always @(posedge clock or negedge areset_n)
if (~areset_n)
push_seen <= 0;
else if (push)
push_seen <= 1;

© Accellera Systems Initiative and Verilab 35

 Specimen transactions
• Some attribute A (address, ID, mode, ...)
• Prove something for an arbitrary value of A ...
• ... then it is proven for all possible values of A
– because proof must consider all possible arbitrary values
– doesn't prevent other values appearing as well

• Compared with local property variables: Local property

– specimen transactions can be more efficient variables
may be better
– usually easier to understand and write in simulation
– value is available across multiple properties
© Accellera Systems Initiative and Verilab 36
 Auxiliary logic (2)
• Pending-transaction counter for a specimen ID value
assume property ( $stable(specimen_id) );

Pick an arbitrary ID assign push = w_hsk && (w_id == specimen_id);

to observe assign pop = b_hsk && (b_id == specimen_id);

always @(posedge clock or negedge areset_n)

if (~areset_n)
id_in_flight <= 0;
else
id_in_flight <= id_in_flight + push – pop;

• "Never two same-ID transactions in flight" is now easy:

Sample-ID proof is assert property ( id_in_flight <= 1 );
sufficient for all IDs

© Accellera Systems Initiative and Verilab 37

 Ordering
• Responses strictly in initiator order
– but no limits on overlapping of
downstream activity
• Response cannot be given until transaction has completed downstream
• No two same-ID transactions in flight together
– strict write observability
• Data exactly preserved from initiator to slave
• Initiator ordering preserved at each slave

• Full description of ordering:

– hard to write
– typically needs lots of auxiliary logic
– often very inefficient for formal tools
• Find a simplified description that captures all the
requirements

© Accellera Systems Initiative and Verilab 38

 FIFO ordering by counting
• Presented by Darbari and Singleton References
in several 2015 papers available

 Track how many transactions push_count – pop_count

are in the FIFO
 Pick some arbitrary transaction specimen transaction
 If track_count==N on push, then
this transaction must be the Nth to be popped
Complete description of FIFO in one simple assertion!

© Accellera Systems Initiative and Verilab 39

 Picking an arbitrary transaction
• Specimen-transaction is not sufficient
– id==specimen_id will always pick the first with that ID
• Add nondeterminism using a free variable
bit sample_accept;
assign push = sample_accept && (w_id == sample_id) && w_hsk;
assume property (sample_written |-> !sample_accept);

• Combine with sample-has-happened modelling

– exactly one specimen transaction

© Accellera Systems Initiative and Verilab 40

Duration: 20 min

SECTION 3: FOLLOWING THROUGH

© Accellera Systems Initiative and Verilab 41

 Reminder: Formal Apps
Formal Property Checking
Bring-up Develop Bounded/Full Proof

Auto-Property Generation Extract implementation detail properties

Register Access Generate reset, access policy and functional checks

End-to-end Checkers IP for hard to develop checker models, e.g. scoreboards

Unreachable Analysis Identify unreachable states and save manual analysis

Coverage Analysis Are we done? Results: code, COI, proof, functional

X Propagation

Clock Domain Crossing And so on … with apps working across the flow

SoC Connectivity

© Accellera Systems Initiative and Verilab 42

 Achieving proof and coverage closure
• Typical user experience:
– useful CEXs found very quickly
– as simple bugs are fixed, proof times get longer
– when RTL and TB are mature, some proofs don’t complete
• Reducing RTL design size (parameterization) can help
– Data widths can be very small
– FIFO depths, timeout counts, number of ports…
• Consider temporary constraints:
– one mode at a time

© Accellera Systems Initiative and Verilab 43

 Other techniques
• Abstraction
– replace counters, memories etc with abstraction
that exhibits critical behaviours without full modelling
– some tool automation
– skill and experience required in practice
• Invariants and helper assertions
– Use already-proven assertions as assumptions
to reduce state space (may be automatic in the tool)
– White-box assertions on internal structures

© Accellera Systems Initiative and Verilab 44

Confidence (or not) in bounded proofs
• Track counterexample lengths over project
– proof bound must exceed the largest CEX you’ve seen
• Reason about latency through the design
• Reason about cycles required to fill storage, etc
• Proof bound must exceed length of related covers
• Achieve 100% toggle coverage
• Track achieved bound as a function of tool runtime
– prioritize effort appropriately

© Accellera Systems Initiative and Verilab 45

 Covering multiple parameterizations
• Formal verifies only one parameterization per run
• Regression suite may need many parameterizations
• Consider pairwise or other test compression
• Logical proof of parameter relationships?
• Larger parameterizations are MUCH slower to prove
– but techniques exist to mitigate this

Some parameters can be converted into

pin-strap options

© Accellera Systems Initiative and Verilab 46

 Applying block-level formal TB in
subsystem-level simulation
• Cross-checking
– another pair of eyes on the formal TB
• Microarchitectural coverage
– auxiliary logic and formal covers make it easier to get
meaningful low-level coverage e.g. reordering counts
– hard to get accurately without low-level probing
– bind of formal TBs to RTL blocks simplifies this
• Incomplete exploration in formal
– especially for deep reordering, long FIFOs, timeouts…
– simulation can exercise long-running scenarios

© Accellera Systems Initiative and Verilab 47

 Formal-to-simulation challenges
• SVA is the same language for formal and simulation
• In principle, assertions are portable
– across vendor tools
– between verification modalities
• Some minor portability issues encountered, but...
The big problem:
inherent differences of approach
between formal and simulation

© Accellera Systems Initiative and Verilab 48

 Review process (1): code
• Code quality, clarity, comments/documentation
– even more important for a formal TB
• Mapping from spec points to assertions
– logical justification of how modelling+assertions checks a
given requirement in full
– justification of assumptions
• Parameterization
– justify your choices of parameters (usually smaller than RTL)
• Sanity cover properties for key use cases
• Good use of formal apps
© Accellera Systems Initiative and Verilab 49
 Hidden vacuity gotchas
• Beware multiple options in a single antecedent
assert property ( A || B |=> C ); Is this what you meant?

Either A or B assert property ( A |=> C );

sufficient to cover this assert property ( B |=> C );

• No failure if buggy constraints make B impossible

cover property ( A && !B );
cover property ( !A && B ); Confirm that all input
cover property ( !A && !B ); combinations are reachable?
cover property ( A && B );

• No failure if C is stuck true cover property ( !C );

© Accellera Systems Initiative and Verilab 50

 Review process (2): results
• Reproducibility
• All assertion preconditions covered
• All assertion and assumption preconditions covered in
simulation
• No CEXs or unreachable covers
• Agreed set of parameterizations
• Justification for acceptability of bounded proofs
• Justification for any removed/waived checks

© Accellera Systems Initiative and Verilab 51

Duration: 5 min + Q&A

CONCLUSIONS

© Accellera Systems Initiative and Verilab 52

 Is It For Me?
• Think why you need FV
– Early bring-up and critical thinking alone compelling
– Features tricky to verify using simulation
– Look at the types of bugs escaping
– Could you use apps, e.g. X-propagation?
• A lot depends on your design implementation
• Skills are required
– Current expertise (using assertions already?)
– In-house FV experts very useful
– Ability to learn (it'll take time and effort)
• You can start and get ROI quickly

Enables RTL designers to own more verification

© Accellera Systems Initiative and Verilab 53

 Where to go next
• There are good papers and presentations available

• Formal tools differ – take the vendor training

If you want find out about some useful

resources leave us your details or mail us

[email protected]
[email protected]

© Accellera Systems Initiative and Verilab 54

 Thank You!

Any Questions?

© Accellera Systems Initiative 55