RISC-V Summit 2019

Open Source RISC-V Processor

Verification Platform
Tao Liu, Richard Ho, Udi Jonnalagadda
Agenda
● RISCV-DV: Open source RISC-V instruction generator

● YAML based end-to-end verification flow

● Case study : ibex core verification

● Roadmap
RISCV-DV
SV/UVM based open source RISC-V instruction generator
https://github.com/google/riscv-dv

Google Cloud Platform Confidential & Proprietary 3

RISCV-DV
Generate a complete random RISC-V assembly program for processor verification

Random instructions
Page tables
PT 0 PT 2
PT 1 PT 3
PT 4

Trap handlers RISCV-DV

Data and stack sections
Data page 0
….
Data page n
Stack section
RISCV-DV new features
Initial release @01/2019 New features

ISA: RV32IMC, RV64IMC + ISA: F/D/A extensions

M/S/U privileged mode + Debug mode support
Virtual address translation + Functional coverage model
Regression suite + Hint/illegal instruction testing
Simulator: vcs, incisive + Handshake mechanism
ISS: spike + Vectored interrupt handling
Simple simulation flow + Bare program mode
+ Simulator: Questa, Metrics
+ ISS: ovpsim, whisper, sail-riscv
+ YAML based end-to-end flow
+ Improve performance
+ New regression suite
...
RISCV-DV new features
Illegal/HINT instruction support Configurable memory regions

mem_0
HINT Section mem_0

ILLEGAL Section mem_1 link

mem_1
HINT
Illegal instruction handler Section mem_2

mem_2
Type of illegal instructions:
Reserved opcode Use cases:
Unsupported instruction Access various internal memories
CSR access to unsupported CSR PMP verification
Operation in the wrong privileged mode Virtual address translation verification
...
 Debug mode support
Debug mode test scenarios
ebreak handler
Normal program
... > ebreak with dcsr.ebreakx enable/disable
ebreak
... > Enter debug mode through external debug request
External
debug mret
> Access non debug mode CSR in debug mode
request
> ebreak in debug ROM
ebreak Debug ROM
> Exception in debug ROM
dpc check
> Single step debug mode
randomize dcsr
> ….

?
save GPR
...
ebreak How to check the processor’s behavior if
... ISS cannot be put to debug mode the
same time as RTL design?
Restore GPR
dret
Handshake mechanism
Handshake packet definition CORE_STATUS : 0
TEST_RESULT: 1 Testbench Normal program Debug ROM
24 bits 8 bits
WRITE_GPR : 2
debug re
data cmd WRITE_CSR : 3 quest

data Variable length

data packets
data CORE_STATUS: IN_DEBUG_MODE
check
WRITE_CSR: dpc
Use cases:
Debug mode testing WRITE_CSR: dcsr
Interrupt testing dret
Memory access fault testing
Self-checked tests
...
Functional coverage flow

ISS log Independent of test

ISS
ISS log
log log2csv
spike log generation flow

Fast coverage closure

RISC-V ISA without RTL sim. No
ISS log ISS
ISS log
log SV functional RTL tracer needed
ISS
ISSlog
loglog log2csv Tracelog
ISS CSV
ovpsim coverage model
Flexible to support
different ISS

ISS
ISSlog
log log2csv
ISS loglog
whisper
Functional coverage model
● Cover all possible operands of each instruction
● Hazard conditions
● Corner cases like overflow, underflow, divide by zero
● Aligned/unaligned load/store
● Positive/negative immediate value
● Forward/backward branches, branch hit history
● Hint instruction
● Illegal instruction
● All opcode
● Access to all implemented privileged CSR
● Exception and interrupt
Generator performance improvement
test_0, batch 1, 20 iterations
35s
test_0, batch 2, 20 iterations
test_0, 100 iterations

test_0, batch 3, 20 iterations

test_1, 10 iterations

test_0, batch 4, 20 iterations

test_2, 5 iterations

… ... test_0, batch 5, 20 iterations

7s
Test_n, x iterations

Average test generation time (s) Run large regression in parallel with small batches
YAML based end-to-end verification flow
An ecosystem for RISC-V processor verification

Google Cloud Platform Confidential & Proprietary 12

 YAML based end-to-end verification flow

iss.yaml
simulator.yaml

ISS sim log log2csv csv

RISCV-DV RISC-V test
Compile test
source compiler RISC-V ELF compare
generator
code (gcc, llvm)
RTL sim log log2csv csv

Generate
RISC-V Assembly test
Assembly test
assembly Assembly test simulator.yaml
Assembly test
tests

testlist.yaml Add new RTL simulator, ISS, test to

the flow simply by modifying the
YAML configure file
YAML based end-to-end verification flow
ISS.yaml testlist.yaml

- iss: spike - import: <riscv_dv_root>/yaml/base_testlist.yaml

path_var: SPIKE_PATH
cmd: >
<path_var>/spike --isa=<variant> -l <elf> ... - test: riscv_machine_mode_rand_test
description: >
- iss: ovpsim Machine mode random instruction test
path_var: OVPSIM_PATH
iterations: 2
cmd: >
gen_test: riscv_instr_base_test
<path_var>/riscvOVPsim.exe
--controlfile <cfg_path>/riscvOVPsim.ic gen_opts: >
… +instr_cnt=10000
+num_of_sub_program=5
- iss: whisper
+boot_mode=m
path_var: WHISPER_ISS
rtl_test: core_base_test
cmd: >
<path_var> <elf> --log --xlen <xlen> --isa <variant> ...
YAML based end-to-end verification flow

# Run the generator with specific RTL simulator

python3 run.py --simulator vcs

# Run the generator with two ISS and cross compare the result
python3 run.py --iss ovpsim,spike

# Run the generator with RV64IMC variant

python3 run.py --target rv64imc

# Run a test with 10 iterations in verbose mode

python3 run.py --test_name riscv_rand_instr_test -i 10 -v
An ecosystem for RISC-V processor
verification
RTL simulator ISS Pre-defined targets

SPIKE RV32I

RV32IMC

RV64IMC

RV64GC

Sail-RISCV ...
Case study: Ibex core verification
Source code: https://github.com/lowRISC/ibex/tree/master/dv/uvm

Ibex features
RV32IMC + Debug mode +
User mode + PMP + ….
Case study : Ibex core verification

Categories of found bugs

Roadmap
Complete RISC-V verification platform

Complete ISA support Functional coverage model Reference testbench

Improve test library for Extend the current model to UVM environment framework
supported ISA extensions. support more ISA extensions, Test/sequence infrastructure
Vector extension
privileged spec coverage etc. Interface agent library
Bitmanip extension
Predefined targets for open
source RISC-V processors
Machine Learning for Verification Research Platform

Environment
Action
testlist.yaml Improve the generator controllability

Instruction Enhance coverage model to provide better

generator
differentiation of coverage results
ISS sim
Provide standard API to interact with ML models
RL Agent
Coverage model Benchmark platform for different ML algorithms
State/Reward
Unified configuration

riscv-config

Instruction Compliance Coverage

ISS ...
generator test model

A single configuration format for all flows

Backup Slides
 Generate program header Generate data/stack section

Privileged mode setup Generate page tables

Page table randomization Generate intr/trap handler

Generator flow
Initialization routine Test completion section

Generate main/sub programs Call stack randomization

Branch target assignment Apply directed instructions

 Benchmark flow
Processor candidates

Pulpino RI5CY:
4 stages, RV32-IMC, DSP extension
Pulpino Ariane :
6-stage, RV64-IMC, single issue,
in-order, support M/S/U privileged levels
Merlin:
Open Source RV32I[C] CPU
ISS simulator
Spike

Benchmark metrics
Bug hunting capability, test coverage
Flow integration effort, performance
Ariane core architecture
Bugs found
LSU
wr rd
Fence

Issue
Load SSTATUS mxr ... ALU
Store
...
Load MSTATUS mxr ...
Load Multiplier

Cache line access racing privileged CSR access FENCE operation failure

PT 0 PT 2
Load Branch Add ... Mult MULHSU
PT 1 PT 3
PT 4

page fault handling Incorrect branch execution ALU corner case bug
Handshake packet format

IN_DEBUG_MODE 0 CORE_STATUS

TEST_PASS 1 TEST_RESULT

MSTATUS 2
WRITE_GPR: MSTATUS
0x0000_1234