OpenSBI Deep Dive

Anup Patel <[email protected]>

Western Digital Research
Outline

• OpenSBI Introduction
– Overview and features

• OpenSBI Platform Specific Support

• OpenSBI Usage
– As a firmware: Reference Firmwares
– As a library: API

• Conclusion
OpenSBI Introduction

© 2019 Western Digital Corporation or its affiliates. All rights reserved. 6/12/2019 3
What is SBI ?

• SBI stands for RISC-V Supervisor Binary Interface

– System call style calling convention between Supervisor (S-mode OS) and Supervisor
Execution Environment (SEE)
App1 App2
• SEE can be:
– A M-mode RUNTIME firmware for OS/Hypervisor running in HS-mode ABI ABI
– A HS-mode Hypervisor for Guest OS running in VS-mode OS (S-Mode)
SBI
• SBI calls help:
SEE (M-Mode)
– Reduce duplicate platform code across OSes (Linux, FreeBSD, etc)
– Provide common drivers for an OS which can be shared by multiple platforms
– Provide an interface for direct access to hardware resources (M-mode only resources)
• Specifications being drafted by the Unix Platform Specification Working group
– Maintain and evolve the SBI specifications
– Currently, SBI v0.1 in-use and SBI v0.2 in draft stage

© 2019 Western Digital Corporation or its affiliates. All rights reserved. 6/12/2019 4
What is OpenSBI ?

• OpenSBI is an open-source implementation of the RISC-V Supervisor Binary Interface

(SBI) specifications
– Licensed under the terms of the BSD-2 clause license
– Helps to avoid SBI implementation fragmentation
• Aimed at providing RUNTIME services in M-mode
– Typically used in boot stage following ROM/LOADER
• Provides support for reference platforms
– Generic simple drivers included for M-mode to operate
 PLIC, CLINT, UART 8250
– Other platforms can reuse the common code and add needed drivers

© 2019 Western Digital Corporation or its affiliates. All rights reserved. 6/12/2019 5
Typical Boot Flow

Authenticate & Loads

• Runs from DDR
Jumps
• Typically open-source
• Runs from On-Chip SRAM • Filesystem support
• DDR initialization • Network booting
• Loads RUNTIME and • Boot configuration
BOOTLOADER • Lots of other features

RUNTIME
ROM LOADER BOOTLOADER OS
(OpenSBI)

• Runs from On-Chip • Runs from DDR

ROM • SOC security setup
• Uses On-Chip SRAM • Runtime services as-per
• SOC power-up and specifications
clock setup

© 2019 Western Digital Corporation or its affiliates. All rights reserved. 6/12/2019 6
Important Features

• Layered structure to accommodate various use cases

– Generic SBI library with platform abstraction
• Typically used with external firmware and bootloader
– EDK2 (UEFI implementation), Secure boot working group
OpenSBI Layers
– Platform specific library Platform Specific
• Similar to core library but including platform specific drivers Reference Firmware
– Platform specific reference firmware Platform Specific
• Three different types of RUNTIME firmware Library
• Wide range of hardware features supported SBI Library
– RV32 and RV64
– Misaligned load/store handling
– Missing CSR emulation
– Protects firmware using PMP support
• Well documented using Doxygen
© 2019 Western Digital Corporation or its affiliates. All rights reserved. 6/12/2019 7
OpenSBI Platform Specific Support

© 2019 Western Digital Corporation or its affiliates. All rights reserved. 6/12/2019 8
Why Platform Specific Support ?

• Any SBI implementation requires hardware dependent (platform-specific) methods

– Print a character to console
– Get an input character from console
– Inject an IPI to any given HART subset
– Get value of memory-mapped system timer
– Start timer event for a given HART
– … more to come …
• OpenSBI platform-specific support is implemented as a set of platform-specific hooks in
the form of a struct sbi_platform data structure instance
– Hooks are pointers to platform dependent functions
• Platform independent generic OpenSBI code is linked into a libsbi.a static library
• For every supported platform, we create a libplatsbi.a static library
– libplatsbi.a = libsbi.a + struct sbi_platform instance
© 2019 Western Digital Corporation or its affiliates. All rights reserved. 6/12/2019 9
Supported Platforms

• Supported platforms are available under /platform directory

in OpenSBI source code tree
OpenSBI Layers
• Currently:
– qemu/virt: QEMU RISC-V generic virtual machine Platform Specific Reference
(Refer, docs/platform/qemu_virt.md) Firmwares
– qemu/sifive_u: QEMU SiFive Unleashed virtual machine
(Refer, docs/platform/qemu_sifive_u.md) libplatsbi.a
– sifive/fu540: SiFive FU540 SOC
Per-platform
(Refer, docs/platform/sifive_fu540.md)
libsbi.a
– kendryte/k210: Kendryte K210 SOC
Generic
• More to come (platform independent)

© 2019 Western Digital Corporation or its affiliates. All rights reserved. 6/12/2019 10
Adding Support for New Platforms

• To add support for a new <xyz> platform

1. Create directory named <xyz> under /platform directory
2. Create platform configuration file <xyz>/config.mk
 config.mk will provide compiler flags, select common drivers, and select firmware options
 platform/template/config.mk can be used as reference for creating config.mk
3. Create platform objects file <xyz>/objects.mk for listing platform-specific objects to be compiled
 platform/template/objects.mk can be used as reference for creating objects.mk
4. Create platform source file <xyz>/platform.c providing “struct sbi_platform” instance
 platform/template/platform.c can be used as reference for creating platform.c

• The <xyz> platform support directory can also placed outside OpenSBI sources

© 2019 Western Digital Corporation or its affiliates. All rights reserved. 6/12/2019 11
Compilation Options for Platform Support

• CROSS_COMPILE environment variable need to be set for cross-compilation

• Build only generic OpenSBI (libsbi.a)
– make
• Build platform-specific OpenSBI (libplatsbi.a) for platform/<xyz> in OpenSBI sources
– make PLATFORM=<xyz>
• Build platform-specific OpenSBI (libplatsbi.a) for <xyz> not part of OpenSBI sources
– make PLAFORM_DIR=<path_to_<xyz>_directory>

© 2019 Western Digital Corporation or its affiliates. All rights reserved. 6/12/2019 12
Using OpenSBI As a Firmware

© 2019 Western Digital Corporation or its affiliates. All rights reserved. 6/12/2019 13
Reference Firmwares

• OpenSBI provides several types of reference firmware, all platform-specific

– FW_PAYLOAD: Firmware with the next booting stage as a payload
– FW_JUMP: Firmware with static jump address to the next booting stage
– FW_DYNAMIC: Firmware with dynamic information on the next booting stage
• SOC Vendors may choose:
– Use one of OpenSBI reference firmwares as their M-mode RUNTIME firmware
– Build M-mode RUNTIME firmware from scratch with OpenSBI as library
– Extend existing M-mode firmwares (U-Boot_M_mode/EDK2) with OpenSBI as library

© 2019 Western Digital Corporation or its affiliates. All rights reserved. 6/12/2019 14
FW_PAYLOAD

Loads
Jumps FW_PAYLOAD

ROM LOADER RUNTIME BOOTLOADER OS

(M-mode) (M-mode) (M-mode) (S-mode) (S-mode)
(ZSBL) (FSBL) (OpenSBI) (U-Boot) (Linux)

• OpenSBI firmware with the next booting stage as a payload

– Any S-mode BOOTLOADER/OS image as the payload to OpenSBI FW_PAYLOAD
– Allows overriding device tree blob (i.e. DTB)
– Very similar to BBL hence fits nicely in existing boot-flow of SiFive Unleashed board
• Down-side:
– We have to re-create FW_PAYLOAD image whenever OpenSBI or the BOOTLOADER (U-Boot) changes
– No mechanism to pass parameters from previous booting stage (i.e. LOADER) to FW_PAYLOAD

© 2019 Western Digital Corporation or its affiliates. All rights reserved. 6/12/2019 15
FW_JUMP

Loads
Jumps

ROM LOADER RUNTIME BOOTLOADER OS

(M-mode) (M-mode) (M-mode) (S-mode) (S-mode)
(ZSBL) (U-Boot_SPL/Coreboot/QEMU) (FW_JUMP) (U-Boot) (Linux)

• OpenSBI firmware with a fixed jump address to the next booting stage
– Next stage booting stage (i.e. BOOTLADER) and FW_JUMP are loaded by the previous booting stage
(i.e. LOADER)
– Very useful for QEMU because we can use pre-compiled FW_JUMP
• Down-side:
– Previous booting stage (i.e. LOADER) has to load next booting stage (i.e. BOOTLADER) at a fixed location
– No mechanism to pass parameters from pervious booting stage (i.e. LOADER) to FW_JUMP

© 2019 Western Digital Corporation or its affiliates. All rights reserved. 6/12/2019 16
FW_DYNAMIC

Loads
Jumps Passed via ‘a2’ register

ROM LOADER struct RUNTIME BOOTLOADER OS

(M-mode) (M-mode) fw_dynamic_info (M-mode) (S-mode) (S-mode)
(ZSBL) (U-Boot_SPL/Coreboot/QEMU) (FW_DYNAMIC) (U-Boot) (Linux)

• OpenSBI firmware with dynamic information about the next booting

stage struct fw_dynamic_info
– The next stage booting stage (i.e. BOOTLADER) and FW_DYNAMIC are loaded by unsigned long magic
the previous booting stage (i.e. LOADER) unsigned long version
– The previous booting stage (i.e. LOADER) passes the location of struct unsigned long next_addr
fw_dynamic_info to FW_DYNAMIC via ‘a2’ register unsigned long next_mode

• Down-side: unsigned long options

– Previous booting stage (i.e. LOADER) needs to be aware of struct fw_dynamic_info

© 2019 Western Digital Corporation or its affiliates. All rights reserved. 6/12/2019 17
Using OpenSBI As a Library

© 2019 Western Digital Corporation or its affiliates. All rights reserved. 6/12/2019 18
Typical use as Library
Loads

Jumps

External Firmware
(EDK2)

ROM RUNTIME BOOTLOADER OS

LOADER
(M-mode) (M-mode) (S-mode) (S-mode)
(M-mode)
(ZSBL) (OpenSBI) (GRUB/U-Boot) (Linux)

• External M-mode firmware linked to OpenSBI library

• Example: open-source EDK2 (UEFI implementation) OpenSBI integration
– HPE leading this effort (Ongoing)
– OpenSBI built with EDK2 build environment

© 2019 Western Digital Corporation or its affiliates. All rights reserved. 6/12/2019 19
Constraints on using OpenSBI Library

• Same GCC target options (i.e. -march, -mabi, and -mcmodel) need to be used for the
external firmware and OpenSBI sources
• External firmware must create per-HART non-overlapping:
1. Program Stack
2. OpenSBI scratch space (i.e. struct sbi_scratch instance with extra space above)
• Two constraints in calling any OpenSBI functions from external firmware:
1. MSCRATCH CSR of calling HART must be set to its own OpenSBI scratch space
2. SP register (i.e. the stack pointer) of calling HART must be set to its own stack
• External firmware must also ensure that:
– Interrupts are disabled in the MSTATUS and MIE CSRs when calling sbi_init()
– sbi_init() is called for each HART that is powered-up at boot-time or in response to a CPU hotplug event
– sbi_trap_handler() is called for M-mode interrupts and M-mode traps

© 2019 Western Digital Corporation or its affiliates. All rights reserved. 6/12/2019 20
Conclusion

© 2019 Western Digital Corporation or its affiliates. All rights reserved. 6/12/2019 21
Important Facts

• OpenSBI provides only RUNTIME firmware/library

• OpenSBI platform specific support makes OpenSBI easily extensible for new SOCs
• OpenSBI reference firmwares:
– Are optional and SOC vendors can choose to implement their own
– Don’t enforce any particular boot-flow

© 2019 Western Digital Corporation or its affiliates. All rights reserved. 6/12/2019 22
On-Going and Future Work

• SBI specifications
– SBI v0.2 specification
– SBI v0.2 HART power management extension
– SBI v0.2 remote fences extension (fence.i, sfence.vma, hfence.gvma, and hfence.bvma)
• OpenSBI
– RISC-V hypervisor extension support (We have a demo here !!!)
– SBI v0.2 support
– SBI v0.2 HART power management support
– SBI v0.2 remote fences support
– Support other M-mode bootloaders such as U-Boot_SPL/Coreboot
– Support RISC-V EDK2 integration
– More platforms support
• Need hardware !

© 2019 Western Digital Corporation or its affiliates. All rights reserved. 6/12/2019 23
© 2019 Western Digital Corporation or its affiliates. All rights reserved. 6/12/2019