WBU-ISOG FORUM 27 28 November 2012

HEVC: High-Efficiency Video Coding

Next generation video compression
Jrme VIERON
Advanced Research Manager
H.264/AVC HEVC
Confidential
2
Keep content looking great wherever its played
J rme Viron - November 2012 1080p@25fps at 2.7 Mbits/s
Outline
1. HEVC in a nutshell
2. HEVC status: standardization and market
3. Technical description
4. Compression performance
5. HEVC and Interlaced coding
6. ATEME & HEVC
Confidential
3
Keep content looking great wherever its played
J rme Viron - November 2012
Outline
1. HEVC in a nutshell
2. HEVC status: standardization and market
3. Technical description
4. Compression performance
5. HEVC and Interlaced coding
6. ATEME & HEVC
Confidential
4
Keep content looking great wherever its played
J rme Viron - November 2012
HEVC in a nutshell
Context
HEVC: High Efficiency Video Coding
J i t t d d f ISO IEC/MPEG d ITU T/ VCEG J CTVC J oint standard of ISO-IEC/MPEG and ITU-T/ VCEG: J CTVC
Successor of H.264/MPEG AVC
ITU- H.265 and ISO- MPEG H Part 2
Goals
Achieve a compression gain of 50% over H264/AVC Achieve a compression gain of 50% over H.264/AVC
x10 complexity max for encoder and x2/3 max for decoder
Status
Gains are already obtained
Standard will be finalized J anuary 2013 (First version)
Confidential
5
Keep content looking great wherever its played
J rme Viron - November 2012
Standard will be finalized J anuary 2013 (First version)
HEVC: Standardization status
Strong industrial and academic interests
Up to1000contributions per meeting (and still 400 at last meeting)
M i d t i l t i l d ( th H264/ AVC) Many industrial actors involved (more than H.264/AVC)
Confidential
6
Keep content looking great wherever its played
J rme Viron - November 2012
HEVC: Standardization status
Standardization Timeline
J uly 2012: DIS (Draft International Standard)
J 2013 FDIS (Fi l D ft I t ti l St d d) J anuary 2013: FDIS (Final Draft International Standard)
Extensions /amendments are considered for mid-2013 mid 2014:
Scalable, 3D, 4:X:X, bit-depths >10 bits, color , (Interlace ?)
On-going standardization process
Profiles definitions/refinement : Main 10 bits Still picture Profiles definitions/refinement : Main, 10 bits, Still picture
Standard draft text V9 (J CTVC-K1003-v9)
HM 9: reference software (http://hevc.kw.bbc.co.uk/trac/browser)
Further tools investigations
3D activity in MPEG and Scalable activity in J CTVC
Confidential
7
Keep content looking great wherever its played
J rme Viron - November 2012
HEVC: Transport/System Layer Integration
On going definitions(MPEG, IETF,,DVB): benefitfromH.264/AVC
MPEG Media Transport (MMT) ?
Confidential
8
Keep content looking great wherever its played
J rme Viron - November 2012
HEVC: Potential applications
What becomes possible with 50% video rate reduction?
Existing applications and usage scenarios g pp g
IPTV over DSL : Large shift in IPTV eligibility
Facilitated deployment of OTT and multi-screen services
More customers on the same infrastructure: most IP traffic is video More customers on the same infrastructure: most IP traffic is video
More archiving facilities
Future services
1080p60/ 50 with bitrates comparable to 1080i
Immersive viewing experience: Ultra-HD (4K 8K) Immersive viewing experience: Ultra HD (4K, 8K)
Premium services (sports, live music, live events,): home theater,
Bars venue, mobile
HD 3DTV Full frame per view at todays HD delivery rates
Confidential
9
Keep content looking great wherever its played
J rme Viron - November 2012
HD 3DTV Full frame per view at today s HD delivery rates
HEVC: Market Timeline
Tentative Timeline?
2010 2011 2012 2013
IS early 2013
2014 2015
HEVC services
HEVC
HEVC Extensions
Offline S/W enc.
S/W dec.
Catchup TV - VOD
Live H/W enc.
IPTV-Broadcast
Live S/W enc.
H/W dec.
WebTV -
IPTV
HEVC Extensions
(3D, Scalability, Professional)
Extensions services
Confidential
10
Keep content looking great wherever its played
J rme Viron - November 2012
Outline
1. HEVC in a nutshell
2. HEVC status: standardization and market
3. Technical description
4. Compression performance
5. HEVC and Interlaced coding
6. ATEME & HEVC
Confidential
11
Keep content looking great wherever its played
J rme Viron - November 2012
HEVC: Yet Another Hybrid Codec?
Main structure very similarto conventional hybrid codecs
Confidential
12
But many differences in terms of flexibility: significant gain
Coding Structure: H.264/AVC vs HEVC
H264/AVC H.264/AVC
HEVC
16x16 Macroblock
Coding Unit
64x64 to 8x8 64x64 to 8x8
Quadtree coding structure Block coding structure
3 Intra partitioning
4 Inter partitioning
PU
TU
+4 sub-partitioning 8x8
2 Transformsizes:
4x4 8x8
Prediction Unit and TransformUnit partitioning
Multiples sizes/forms: 64x64 to 4x4
TU
4x4, 8x8
Up to 9 Intra prediction directions
35 Intra prediction directions
17 16 15 14 13 12 11 10 9 8 7 6 5 4 3
2
Confidential
13
Keep content looking great wherever its played
J rme Viron - November 2012
Efficient spatio-temporal mv prediction
Coding Structure: CU and PU
Coding Unit (CU)
8x8 to 64x64
PredictionUnit (PU)
Intra:
Square: 2Nx2N, NxN Squa e: ,
Inter: Inter:
2Nx2N, NxN, dNxN, NxdN.
AMP (Asymmetric Motion Partitioning)
AMP
Confidential
14
Keep content looking great wherever its played
J rme Viron - November 2012
Coding Structure: TU
TransformUnit (TU)
DCT(Discrete Cosine Tranform)
Varioussizes from4x4 to 32x32
To adapt the transformto the signal frequency characteristics
TU can overlap PU TU can overlap PU
Intra
- DCT+ DST(Discrete Sine Transform) on Luma intra 4x4
Inter
DCT only - DCT only
Confidential
15
Keep content looking great wherever its played
J rme Viron - November 2012
On Quadtree structure representation and coding
Quadtree
Clean and efficient structure for multi-partitioning codecs
Ch i li i Cheap signalization
Fast decision algorithmwell-known (pruning)
0
1 2
PU
3 4
5 6
9
0
1 2 3 4
7 8
10 11
12 13
14
15
TU
Confidential
16
Keep content looking great wherever its played
J rme Viron - November 2012
Intra prediction (1/2)
Luma: 35 prediction directions (33 + Planar + DC)
18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34
H.264/AVC
HEVC
0 : Intra_Planar
1 : Intra_DC
35: Intra_FromLuma
Planarpredictor:
H.264/AVC
HEVC
Planarpredictor:
Confidential
17
Keep content looking great wherever its played
J rme Viron - November 2012
Intra prediction (2/2)
3 Most Probable Modes (MPM): derived fromneighboring
Three coefficient scanning ordersdepending on the mode
Diagonal up-right Horizontal Vertical
Confidential
18
Keep content looking great wherever its played
J rme Viron - November 2012
Inter prediction (1/2)
Adaptive Motion VectorPrediction: AMVP
Motion vector predictor competition (spatial and temporal)
3 MV candidates 3 MV candidates
Fig. 1
AMVP
Merge
Motion copy mode + Residual pixel coding
Correction of Quadtree redundancies Correction of Quadtree redundancies
Extension of H.264/AVC Direct mode
5 PU candidates (spatial and temporal)
Ski h id l
Fig. 2
M
Confidential
19
Keep content looking great wherever its played
J rme Viron - November 2012
Skip when no residual
Merge
Inter prediction (2/2)
7 or 8-tap interpolation filterfor luma: Pel
4 t i t l ti filt f h P l 4-tap interpolation filterfor chroma: 1/8 Pel
Confidential
20
Keep content looking great wherever its played
J rme Viron - November 2012
Filtering
Deblocking Filter
Along CU boundaries as in H.264/AVC
Parallelizable Parallelizable
Sample Adaptive Offset: SAO
After deblocking
Add offset to pixels depending on their categorization (band, edge)
Up to 6% bitrate savings Up to 6% bitrate savings.
Confidential
21
Keep content looking great wherever its played
J rme Viron - November 2012
High level parallelism tools
Slices
WavefrontParallel processing: WPP
1 thread per CU line
X
X
1% loss compared to a
X
X
Pixel and MV dependency
Probabilties dependency
CABAC probabilitites
KEY
1% loss compared to a
non parallel bistream
Tiles
1thread per Tile
Probabilties dependency
X
Block(s) being encoded
1 2 3 10 11 16 17 18
2 7%loss compared to a
1thread per Tile
Example of Frame partitioning:
4 5 6 12
7
25
8
26
9
27
14
31
13 19 20 21
15
32
22
35
23
36
24
37
28 29 30 33 34 38 39 40
2,7% loss compared to a
non parallel bistream
Confidential
22
Keep content looking great wherever its played
J rme Viron - November 2012
p p g
41 42 43 44 45 46 47 48
Other tools and features
One unique entropy coder: CABAC witha more sophisticated context
management scheme
Improved compression efficiency p p y
Improved throughput speed while reducing its context memory requirements
High Level Syntax: (very close to H264/AVC) High Level Syntax: (very close to H.264/AVC)
SPS, PPS (Sequence/Picture Parameter Set) and new VPS(Video Parameter Set)
NAL (Network Abstraction Layer)
SEI (Supplemental Enhancement Information ) metadata SEI (Supplemental Enhancement Information ) metadata
VUI (Video Usability Information) metadata
No interlace coding tool
Field coding only
Only SEI message Picture timing (pic_struct)
Confidential
23
Keep content looking great wherever its played
J rme Viron - November 2012
HEVC toolset versus H.264/AVC and MPEG-2
MPEG 2 H264/AVC HEVC MPEG-2 H.264/AVC HEVC
Partitionsize Macroblock 16x16 Macroblock 16x16 (Large) Coding Unit
8x8 to 64x64
Partitioning Inter 16x8, Intra 8x8 Sub-block down to
4x4
Prediction Unit
Quadtree down to 4x4 4x4 Quadtree down to 4x4
Square, symmetric and
asymmetric
(only square for intra)
Transform Floating DCT Integer DCT
8x8 4x4
Transform Unit square
IDCT from 32x32to 4x4 8x8, 4x4 IDCT from 32x32to 4x4
+ DST Luma Intra 4x4
Intra prediction DC predictor Up to 9 predictors 35predictors
Motion prediction Vector fromone Spatial Median Advanced Motion
Neighbor (3 blocks) Vector Prediction AMVP
(spatial + temporal)
Motion-copy
mode
/ Direct mode Merge mode
Motion precision Pixel bilinear Pixel 6 tap Pixel 7or 8 tap Motion precision Pixel bilinear Pixel 6-tap,
Pixel bi-linear
Pixel 7or 8-tap,
1/8 Pixel 4-tap chroma
Entropy coding VLC CABAC, CAVLC CABAC
Filters /
Deblocking Filter
Deblocking Filter
Sample Adaptive Offset
Confidential
24
Keep content looking great wherever its played
J rme Viron - November 2012
g p p
Main Profile: Levels and Tiers
Confidential
25
Keep content looking great wherever its played
J rme Viron - November 2012
Outline
1. HEVC in a nutshell
2. HEVC status: standardization and market
3. Technical description
4. Compression performance
5. HEVC and Interlaced coding
6. ATEME & HEVC
Confidential
26
Keep content looking great wherever its played
J rme Viron - November 2012
HEVC compression performance
Comparison withH.264/AVC
J M 18.3 and 18.4 (High Profile) vs HM 7 and 8 (Main Profile)
WVGA to Full HD video sequences WVGA to Full HD video sequences
Objective quality (PSNR-Bjontegaard)
Average bitrate savings of 35%for entertainment applications (RA)
Coding performance increases with resolutions: >39% for HD and beyond
Average bitrate savings of 40 % low delay applications (Low delay)
Confirmed with ATEMEs in-house encoders
Subjective quality (Mean Opinion Score) Subjective quality (Mean Opinion Score)
Average bitrate savings of 50% for equivalent perceived quality (RA)
From 30% up to 67% on certain 1080p content
Confidential
27
Keep content looking great wherever its played
J rme Viron - November 2012
HEVC compression performance for 4K broadcast
Performance assessment on 4K native content
Both H.264/AVC and HEVC encoders (demonstrated at IBC 2012)
CBR same GOP structure and comparable configuration CBR, same GOP structure and comparable configuration
4:2:0 sampling and 8-bit material
4K t t 4K content
Digital Cameras: Red EPIC, J VC, Sony F65
Captures performed by FranceTelevisionsand Orange: 4EVER consortium
Scanned films: SVT reference sequences
38% to 50% bitrate savings observed (Bjontegaard) 38% 50% g ( j g )
Further results in SMPTE technical paper October 2012
Observed gain would allow the broadcast of 4KTV (50/60p) at video
bitrates below 13Mbps (Confirmed by subjective testing)
Confidential
28
Keep content looking great wherever its played
J rme Viron - November 2012
bitrates below 13Mbps (Confirmed by subjective testing)
Outline
1. HEVC in a nutshell
2. HEVC status: standardization and market
3. Technical description
4. Compression performance
5. HEVC and Interlaced coding
6. ATEME & HEVC
Confidential
29
Keep content looking great wherever its played
J rme Viron - November 2012
HEVC and Interlaced
So far, interlaced only supported with SEI message
Pic_struct (AVC like) => Picture timing SEI message
Assuming field only coding Assuming field only coding
No specific coding tool (neither PAFF or MBAFF)
L i t l d id t t t till l t Legacy interlaced video content represent still a very large part:
Low-Level Interlaced coding tools support was expressed
Contributions from Broadcasters and service providers
Contributions from National Bodies (US, J apan, France,)
EBU Liaison to MPEG in last Shanghai meeting (October 2012)
Contributions reported assessment of HEVC interlaced coding
Limited performance against H.264/AVC were highlighted
Major issue regarding Chroma misalignment
Confidential
30
Keep content looking great wherever its played
J rme Viron - November 2012
Major issue regarding Chroma misalignment
HEVC and Interlaced
MPEG AHG on Study of interlace coding
Created for next 6 months (http://lists.uni-klu.ac.at/mailman/listinfo/interlace)
Mandates:
Define sequencestest set
Highlight HEVC interlaced use cases g g
Evaluate current HM performance over H.264/AVC
Evaluate improvement (normative and non-normative)
Interlaced profile for 2014? Interlaced profile for 2014?
Participants:
ATEME ( h i ) C t C bl L b M t l Z ATEME (co-chair), Comcast, CableLabs, Motorola, Zenverge,
Sony, Mitsubishi, Orange, Technicolor, Harmonic, NBC, DirecTv,
U t d f id !
Confidential
31
Keep content looking great wherever its played
J rme Viron - November 2012
Urgent need for video sequences!
Conclusion
Overview of future HEVC video compression standard
Performance is more than promising: 50% bit-rate savings on HD and beyond
First version will be delivered in January 2013 (up to 10 bits, 4:2:0)
HEVC extensions are already considered for next year : Scalable, 3D, professional
Interlaced coding not fully supported (only SEI message)
ATEME is working on its own HEVC encoding core technology
targeting demos and field operation for H1 2013
French research consortium 4EVER for Enhanced Video ExpeRience French research consortium 4EVER for Enhanced Video ExpeRience
3 year collaborative project multi-million project
Research, develop and promote an Enhanced quality Television Experience.
HEVC technology development and codec prototyping
Confidential
32
Keep content looking great wherever its played
J rme Viron - November 2012
- HEVC technology development and codec prototyping
- Ultra High Definition - 4KTV, format definition and qualification.
M i Th k Y Merci, danke, gracias, grazie, xixie, chokrane, Thank You,
dankie, faleminderit, Barak Allahu fiik, chnorakaloutioun, ox sag olun,
a ni ki, bedankt, waita, eskerrik, dhanyabaad, dziakuju, thint ko, kyay
t ti t i ki t bl d i i l t k h tzu tin pa te, a ni ki, trugr, blagodaria, grcies, salamat, kam sah
hamnida, grazie,msi, tak, dankon, akp, kiitos, gracie,tapadh leat,
go raibhmaithagat, diolch, efharisto, aguyj, meherbani, mahalo, toda,
tau, dhanyavad, ksznm, terima kasih, takk, arigat, tanemirt, hvala, , y , , , , g , , ,
dhanyavadagalu, akun, murakoze, sobodi, tenki, spas, merc, khob
chai, gratias ago, paldies, choukrane, aciu, blagodaram, terima kasih,
misaotra, nizzik hajr, aabhari aahe, bayarlalaa, dank u wel, merc,
tack shukriya danki motashakkeram mamnun dzikuj obrigado tack, shukriya, danki, motashakkeram, mamnun, dzikuj, obrigado,
dakujem, multumesc, spacibo, marci, faafetai lava, gratzias, ahsante,
salamatpo, mauruuru, nanedri, rahmat, dkuji, kop khunkrap, yekeniele,
tesekkur ederim, sagolun, diakuiu, cm n, grces, djiere dieuf,
enkosi, a dank, ngiyabonga, siyabonga, ngiyabonga kakhulu
Jrme VIERON
Advanced Research Manager
[email protected] j @