DRM/KMS, FB and V4L2:

How to Select a Graphics and

Video API
Embedded Linux Conference Europe 2012

Laurent Pinchart
[email protected]
 Personal opinion

Flame war possible

Handle with care

Disclaimer
Problem Definition
Problem Definition
display / graphics /
video

Problem - Purpose
 format
memory / deep pipeline
device / CPU

Problem - Source
 rotation
scaling
composing

Problem - Processing
 X11
Wayland
DirectFB
Raw API

Problem - Stack
 DRM
FBDEV
V4L2

Problem - API
Origins
 D

2000
R
M
,L
in
u x
2.
2.
18

G
2008

EM
2009

KM

Origins DRM/KMS
S,
TT
M
D
um
b
Bu
2011 2012

Pl ffe
an
es rs
,D
M
AB
U
F
 V4

1999
L ,L
in
u x
2.
2.
0
V4

2002
L2
, Li
nu
x
2.
4.
0

2006
V4
L
de

Origins V4L2
pr
e ca
te
d
2008

V4
L2
su
bd
ev

M
vi ed
de ia
ob co
D uf2 ntro
2011 2012

M lle
AB r
U
F(
?)
 FB

1996
D
EV
, Li
n ux
Bl 1.

2000
an 3.
94
ki
ng

Origins FBDEV
4C
2012

C
Fo
rm
at
s
Features
 DRM FB V4L2

Dynamic Allocation Yes No Yes

Multiple Buffers Yes panning Yes

Import dmabuf No userptr

dmabuf
Export mmap mmap
mmap

Memory Management
 DRM FB V4L2
RGB
Formats 4CC 4CC
4CC

Enumeration Planes No Yes

Negotiation No No Yes

Atomicity Yes No No

Mode Setting
 DRM FB V4L2

Overlays Yes No Yes

Rotation Yes No Yes

Scaling Yes No Yes

Cropping/Panning Yes Yes Yes

Transformations
 drm
fb
v4l2

api core drivers

Documentation
 The DRM core exports several interfaces to applications,
generally intended to be used through corresponding libdrm
wrapper functions. In addition, drivers export device-specific
interfaces for use by userspace drivers & device-aware
applications through ioctls and sysfs files.

External interfaces include: memory mapping, context

management, DMA operations, AGP management, vblank
control, fence management, memory management, and output
management.

Cover generic ioctls and sysfs layout here. We only need high-
level info, since man pages should cover the rest.

DRM API Documentation

 drm
fb
v4l2

api core drivers

Documentation
 drm
fb
v4l2

api core drivers

Code Size
4000

3500

3000

2500 drm
fb
2000
v4l2
1500

1000

500 v4l2
fb

0 drm
v2.6.14 v2.6.18 v2.6.22 v2.6.26 v2.6.30 v2.6.34 v2.6.38 v3.2 v3.6-rc3
v2.6.12 v2.6.16 v2.6.20 v2.6.24 v2.6.28 v2.6.32 v2.6.36 v3.0 v3.4

Cumulative Changes - API

45000

40000

35000

30000
drm
25000 fb
20000 v4l2

15000

10000

v4l2
5000
fb

0 drm
v2.6.14 v2.6.18 v2.6.22 v2.6.26 v2.6.30 v2.6.34 v2.6.38 v3.2 v3.6-rc3
v2.6.12 v2.6.16 v2.6.20 v2.6.24 v2.6.28 v2.6.32 v2.6.36 v3.0 v3.4

Cumulative Changes - Core

1000000

900000

800000

700000

600000
drm
fb
500000
v4l2
400000

300000

200000
v4l2
100000
fb
0 drm
v2.6.14 v2.6.18 v2.6.22 v2.6.26 v2.6.30 v2.6.34 v2.6.38 v3.2 v3.6-rc3
v2.6.12 v2.6.16 v2.6.20 v2.6.24 v2.6.28 v2.6.32 v2.6.36 v3.0 v3.4

Cumulative Changes - Drivers

1600

1400

1200

1000
drm
800
fb
v4l2
600 drm+bug

400

200

0
2010-01 2010-05 2010-09 2011-01 2011-05 2011-09 2012-01 2012-05
2009-11 2010-03 2010-07 2010-11 2011-03 2011-07 2011-11 2012-03 2012-07

Mailing List Traffic

 Frame Buffer Display
(memory) Device

Device Model FBDEV

Frame Buffer
CRTC Encoder Connector
(memory)

Plane
(memory)

Device Model DRM/KMS

 Buffers Display
Connector
(memory) Device

Device Model V4L2

V4L2
0
queue

0
V4L2 V4L2
2 0 1
subdev subdev
1

Device Model V4L2/MC

Device Model V4L2/MC
Use Cases
 2500

2000
2370

1500

1000

500

fbdev

FB vs. DRM - sloccount

 2500

2000
2370

1500
1659
1000

500

fbdev drm

FB vs. DRM - sloccount

Use Cases - FBDEV
 (that's it...)

Use Cases - FBDEV

 Video

Use Cases - V4L2

 Everything else

Use Cases DRM/KMS

Frame Buffer
CRTC Encoder Connector
(memory)

Plane
(memory)

Memory SoC Off-Chip

KMS Device Model

Frame Buffer

width
height
GEM format
Object(s) pitches CRTC
offsets

Memory Properties

KMS Frame Buffer

Frame Buffer

GEM Object

width
height CRTC
bpp Memory
pitch
size

Properties

DRM/KMS GEM Object

 Process A Process B
3
Send FD
SCM_RIGHTS

Local Global Global Local

1 2 4
Handle FD FD Handle

GEM
Object

DRM Handles
Source: http://www.flickr.com/photos/buckaroobay/3721809183/
panel & backlight
http://lwn.net/Articles/512363/

Work In Progress
 Panel
DISPC DPI/LVDS DSI/LVDS Panel
Controller

Panel Module
Transmitter

WIP Display Framework

 [email protected]
[email protected]
[email protected]

[email protected]

Contact
?!
thx.