An Introduction to DMS Integration

Matthew Jerrard
Last updated: 12th October, 2009.

Confidential ©NDS Ltd 2009. All rights reserved.

 Introduction to DMS

• The course consists of a mixture of slides, explanations

and demonstrations.
• You can keep the slides, which cover most of the
content, but you may also want to take notes.
• Main sections:
– Middleware architecture
– DMS Testing Strategy and Environment
– The build tree and running a build
– A walk through of a typical DMS scenario
– Running a DMS scenario
– Continuous integration
• Feel free to ask questions as we go along

Confidential ©NDS Ltd 2009. All rights reserved.

 Fusion is Split into Processes

• Processes give us Application Engine Platform

(per terminal)
Including
App Engine Mgmnt,
- Separation of functionality Window Mgmnt,
App Platform Components,
Graphics stack.

- Memory protection
• IPC
– POSIX FIFOs Middleware Services
Including:
Information Services, System Security Services

– Serialisation Resource Management,

Media Services.

– Network byte order

– Asynchronous
System Control and
Diagnostics
Including
System Init,
Diagnostics Control,
Software Download Management.

Confidential ©NDS Ltd 2009. All rights reserved.

 Fusion is Split into Components

• Standalone building blocks for middleware

• Component can be:
– Pure library (e.g. SYSTEMCONFIG)
– Library with threads (e.g. IPC)
– Server
(note that a Client library can be in a process separate from the
server)

Confidential ©NDS Ltd 2009. All rights reserved.

 Fusion Architecture
Application (Master Application/EPG or Interactive Application)

Platform Component API (per engine language) Fusion-API (per engine language)

Application Engine Run-Time Environment

FSION-DSN-1182
Platform Component Adaptor

Application Engine
Application Engine

Platform Components API Fusion-API (native)

System Platform Components Platform Components

Resource Mgmnt Download Player Planner Fusion-API Shim
Key to process colours
PCAT Plugin
Plugin Management
System Platform System User Activity Guide Object FileSystem
Mgmnt Recording Mgmnt Monitor

Application Process
System Booking Shared Content Application Engine
Preferences
Mgmnt Management Management

Middleware Process Common Middleware Services API

System Resource Management Information Services Presentation Services Caption

Security Services Process Resource

Management
Peripheral
Device Mgmt
System
Service
Program
Scheduler
File
Delivery
Graphics
Engine
Window
Mgmnt
Pres.

Audio
Framework Mgmnt Mgmt
Engine

System Control and Media Services

Program
Catalogue
Misc Libraries
Diagnostics Process Viewing and Recording Content Transfer Database
SI Mgmt
Security Services
Security Services HTTP RTSP

Management Management Engine BinXML UPNP

CA Kernel BDOM SSDP
Software Section
Media Connection Management
Download Filter ZLIB
Serial Port
Comms
Mgmt Mgmt
TFTP OIG Client
Front Back Media Cap-
End End Presentation tion SSL
Mgmt Mgmt Mgmnt Proc.

CA System Libraries
System Control & Comms Services
Media Streaming Management Module Diagnostics Net Cfg Mgmnt
SYSTEM SYSTEM SYSTEM
OBJECT TIME LOCATOR
Mgmt
SYSTEM
System Init PPPD MEMMAN
ITC
IPC

Modem Mgmnt SYSTEM SYSTEM SYSTEM

ALARM CONFIG LANG
Media Device Management Diagnostics
DHCP SYSTEM SYSTEM SYSTEM
UTIL COMMON STRING

CDI/POSIX

OS Helpers USB Filesystems

Boot Basic
Network Storage Main
OS Helper Helper Loader class
Drivers Stack RAM ROM NVRAM XTVFS EXT3
Hotplug (BSL) drivers
Helper

Confidential ©NDS Ltd 2009. All rights reserved.

 Layered Architecture

Application (Master Application/EPG or Interactive Application)

• Platform component adaptors Platform Component API (per engine language) Fusion-API (per engine language)

Application Engine Run-Time Environment

• Application engine(s) Platform Component Adaptor

• DMS testing takes place across

Application Engine
Application Engine

the Platform
Platform Components API Fusion-API (native)

System Platform Components Platform Components

Components/Fusion API
Resource Mgmnt Download Player Planner Fusion-API Shim
PCAT Plugin
Plugin Management
System Platform System User Activity Guide Object FileSystem
Mgmnt Recording Mgmnt Monitor

• Platform Components
System Booking Shared Content Application Engine
Preferences
Mgmnt Management Management

• System Platform Components

Common Middleware Services API
System Resource Management Information Services Presentation Services Caption
System File Pres.
Resource Program
Peripheral Graphics Window
Service

• FusionOS
Management Delivery
Device Mgmt Scheduler Engine Mgmnt Audio
Framework Mgmnt Mgmt
Engine
Program
Media Services Catalogue

– CDI
SI Mgmt Misc Libraries
Security Services HTTP RTSP
Viewing and Recording Content Transfer Database Security Services
Management Management Engine BinXML UPNP
CA Kernel BDOM SSDP

– POSIX
Software Section
Media Connection Management
Download Filter ZLIB
Serial Port
Comms
Mgmt Mgmt
TFTP OIG Client
Front Back Media Cap-
End End Presentation tion SSL

– VFS
Mgmt Mgmt Mgmnt Proc.

CA System Libraries
System Control & Comms Services
Media Streaming Management SYSTEM SYSTEM SYSTEM

– Helpers
Module Diagnostics Net Cfg Mgmnt OBJECT TIME LOCATOR
Mgmt
SYSTEM
System Init PPPD MEMMAN
ITC
IPC

Modem Mgmnt SYSTEM SYSTEM SYSTEM

ALARM CONFIG LANG
Media Device Management

• Component dependencies are

Diagnostics
DHCP SYSTEM SYSTEM SYSTEM
UTIL COMMON STRING

CDI/POSIX

always downwards
OS Helpers USB Filesystems
Boot Basic
Network Storage Main
OS Helper Helper Loader class
Drivers Stack RAM ROM NVRAM XTVFS EXT3
Hotplug (BSL) drivers
Helper

Confidential ©NDS Ltd 2009. All rights reserved.

 FusionOS and CDI

• FusionOS implements a POSIX subset

• The Middleware is a “trusted client” of the CDI
(Common Driver Interface) drivers
– This means that the Middleware must NEVER call the CDI with
invalid values, and must NEVER call a device in an invalid state.
• Memory
– Only MEMMAN (and DIAG) may use malloc(), free() etc.
– Since the Hound kernel was introduced, the kernel allocates a
stack of fixed size when requested to do so. The kernel will not
grow the stack automatically if the fixed size is exceeded.
– Memory in the Middleware is organised into Pools. If a request
to allocate memory from a pool exceeds the available memory,
then that request will fail. The kernel should not do anything in
this case – this is handled by the Middleware.

Confidential ©NDS Ltd 2009. All rights reserved.

 Fusion Components
Platform Components API

System Platform Components Platform Components

Resource Mgmnt Download
PCAT Plugin Player Planner VOD Client
Plugin Management
System Platform System User Activity Guide
Mgmnt Recording Mgmnt Monitor
System Booking Shared Content Anytime Guide Object FileSystem
Mgmnt Preferences
Management
Remote Application Engine
Remote Booking
Notification Client Management

Common Middleware Services API

System Resource Management Information Services Presentation Services Caption
System File Pres.
Resource Program
Peripheral Graphics Window
Management Service Delivery
Device Mgmt Scheduler Engine Mgmnt
Framework Mgmnt Mgmt

Key to process Media Services

Program
Catalogue
colours SI Mgmt Security Services Misc Libraries
Viewing and Recording Content Transfer Database HTTP RTSP
Application Management Management Engine VideoGuard
BinXML UPNP
Process Software Section
CA Kernel
Media Connection Management Download Filter BDOM SSDP
Middleware Mgmt Mgmt
SERIAL PORT
ZLIB
Front Back Media
Process End End Presentation
Cap-
tion
COMMS

TFTP OIG Client

Mgmt Mgmt Mgmnt Proc.
Security
Services CA System Control & Comms Services
System Libraries

Process Media Streaming Management Module

Mgmt
Diagnostics Net Cfg Mgmnt
SYSTEM
OBJECT
SYSTEM
TIME
SYSTEM
LOCATOR
SYSTEM
System Init PPPD MEMMAN IPC
System Control SYSTEM
ITC
SYSTEM SYSTEM
Modem Mgmnt
and Diagnostics Media Device Management Diagnostics
ALARM CONFIG LANG
DHCP SYSTEM SYSTEM SYSTEM
Process UTIL COMMON STRING

Confidential ©NDS Ltd 2009. All rights reserved.

 System Platform Components

• Resource Management Plugin (RMP)

– Found in dms/components/darwin_system_platform_components/darwin_rmfplugin
– Contains project-specific and business logic for resource allocation and conflict
management.
• Service Plan Management (SPM)
– Found in dms/components/darwin_system_platform_components/darwin_spm
– Discovers broadcast services, builds channel list, and applies business rules applicable to
the service plan.
• Remote Booking (REMBOOK)
– Found in dms/components/darwin_system_platform_components/darwin_rembook
– Allows bookings to be made by a user remote from the STB.
• System Recording Management (SRM)
– Found in dms/components/darwin_system_platform_components/darwin_srm
– Applies project specific and business rules to booked reminders and recordings, e.g.
starting and stopping booked jobs.
• System Booking Management (SBM)
– Found in dms/components/darwin_system_platform_components/darwin_sbm
– Manages automatic bookings made by the broadcaster or an interactive application, e.g.
Push VOD.

Confidential ©NDS Ltd 2009. All rights reserved.

 Platform Components

• Player
– Found in dms/components/darwin_platform_components/darwin_player
– Allows applications to view content from various sources (broadcast, VOD, recorded, etc.).
• Guide
– Found in dms/components/darwin_platform_components/darwin_guide
– Allows applications to access service information.
• Preferences (PREFS)
– Found in dms/components/darwin_platform_components/darwin_prefs
– Allows storage and retrieval of user preferences in NVRAM. Also manages the configuration
of output ports at start up.
• Planner
– Found in dms/components/darwin_platform_components/darwin_planner
– Allows applications to book recordings and reminders and to access the catalogue of
bookings and recordings.
• Object File System (OFS)
– Found in dms/components/darwin_platform_components/darwin_ofs
– Allows clients to access objects (such as media files) regardless of the delivery mechanism.
• VideoGuard Client
– Found beneath dms/components/security_services/vgv/ and delivered as a precompiled
binary library
– Application process interface to the verifier.

Confidential ©NDS Ltd 2009. All rights reserved.

 System Control and Diagnostics

• System Init (SYSINIT)

– Found in
dms/components/system_control_and_diagnostics/sysinit
– Controls start up of processes and components.
• Diagnostics (DIAG)
– Found in dms/components/system_libraries/diag
– Collects and formats debug log information. No function in the
production build.

Confidential ©NDS Ltd 2009. All rights reserved.

 System Resource Management

• Resource Management Framework (RMF)

– Found in dms/components/system_resource_management/rmf
– Manages resources and resolves resource conflicts
• Peripheral Device Management (PDM)
– Found in dms/components/system_resource_management/pdm
– Manages peripheral devices, e.g. storage (disk) and front panel
LEDs
• System Service Management (SSM)
– Found in dms/components/system_resource_management/ssm
– All servers register with SSM at start up. Client libraries use this
to locate their server.

Confidential ©NDS Ltd 2009. All rights reserved.

 Media Services - 1

• Media Device Management (MDM)

– Found in dms/components/media_services/mdm
– Allows clients to query devices for a media connection.
– Allows clients to query whether simultaneous media connections are possible.
• Media Streaming Management (MSM)
– Found in dms/components/media_services/msm
– Allows content to be streamed from a source to an output.
• CA Module Management (CAMM)
– Found in dms/components/media_services/camm
– Interface to the verifier. Allows us to run different CA kernels.
• Front End Management (FEM)
– Found in dms/components/media_services/fem
– Management and control of front end devices (e.g. tuners).
• Back End Management (BEM)
– Found in dms/components/media_services/bem
– Control and management of all back end devices (SCART, HDMI, etc.).

Confidential ©NDS Ltd 2009. All rights reserved.

 Media Services - 2

• Media Presentation Management (MPM)

– Found in dms/components/media_services/mpm
– Audio and video decoding and demux functionality.
• Caption Processing (CAPPROC)
– Found in dms/components/media_services/capproc
– Acquires and processes elementary streams carrying caption data.
• Media Connection Management (MCM)
– Found in dms/components/media_services/mcm
– Controls media connections (the chain of devices required to stream
content).
• Viewing and Recording Management (VRM)
– Found in dms/components/media_services/vrm
– Controls viewing and recording jobs.

Confidential ©NDS Ltd 2009. All rights reserved.

 Information Services

• Program Scheduler (PSCHED)

– Found in dms/components/information_services
– Allows clients to schedule a recording or reminder.
• Program Catalogue (PCAT)
– Found in dms/components/information_services
– Maintains a database of all booked and recorded programmes on the
disk.
• File Delivery Management (FDM)
– Found in dms/components/information_services
– Acquisition of broadcast file systems.
• SI Management (SIM)
– Found in dms/components/information_services
– Handles all service information.
• Section Filter Management (SFM)
– Found in dms/components/information_services
– Controls the section filters used to monitor DVB/XSI table sections.

Confidential ©NDS Ltd 2009. All rights reserved.

 Comms Services

• DHCP
– Found in dms/components/comms_services
– Passes config information to hosts over TCP/IP; allows
automatic allocation of network addresses.
• Modem Management (MODEM)
– Found in dms/components/comms_services
– Controls the modem.
• PPPD
– Found in dms/components/comms_services/pppd
– Controls network connections over point to point protocol.

Confidential ©NDS Ltd 2009. All rights reserved.

 Platform Management

• Application Engine Management (AEM)

– Found in dms/components/platform_management/aem
– Provides a common interface for application developers to
control the application engine.

Confidential ©NDS Ltd 2009. All rights reserved.

 Presentation Services

• Graphics Engine (GFX)

– Found in dms/components/presentation_services/gfx
– Provides graphics functionality
• Window Management (WIN)
– Found in dms/components/presentation_services/win
– Allows windows to be drawn on the display
• Caption Presentation (CAPPRES)
– Found in dms/components/presentation_services/cappres
– Allows display of subtitles

Confidential ©NDS Ltd 2009. All rights reserved.

 Security Services

• CA Kernel
– Found below dms/components/security_services/vgv and
delivered as a prebuilt binary library.
– Determines if a subscriber has the appropriate entitlements to
watch encrypted content and decodes it if they do.

Confidential ©NDS Ltd 2009. All rights reserved.

 System Libraries

• Memory Manager (MEMMAN)

– Found in dms/components/system_libraries/memman
– Controls allocation and freeing of memory (organised into
memory pools) in the system.
• Inter-process communication (IPC)
– Found in dms/components/system_libraries/ipc
– Provides POSIX FIFOs for communication between processes.
• SYSTEM libraries
– Found in dms/components/system_libraries/ipc
– Various functions and definitions used by all components.

Confidential ©NDS Ltd 2009. All rights reserved.

 Miscellaneous Libraries

• HTTP, RTSP, ZLIB, Serial Port

– Found in dms/components/misc_libraries
– Various functions that could include open source libraries.

Confidential ©NDS Ltd 2009. All rights reserved.

 Media Services – Setting up Device
Chains (1)

Live TV
Surface
Blitter Viewport
(OSD)
Blender
TS TPID Vid
Tuner ES Filter Surface Viewport
Channel Filter Decoder

Colour Scheme Key – showing MediaHighway component ownership of CDI devices

Device owned by Device owned by Device owned by Device owned by

Device owned by
Front End Mgmt Media Presentation Back End Mgmt Media Streaming
Graphics (GFX)
(FEM) Mgmt (MPM) (BEM) Mgmt (MSM)

Confidential ©NDS Ltd 2009. All rights reserved.

Media Services – Setting up Device
Chains (2)

Confidential ©NDS Ltd 2009. All rights reserved.

Media Services – Setting up Device
Chains (3)

Confidential ©NDS Ltd 2009. All rights reserved.

 DMS Test Strategy

• Currently around 1000 test scenarios

• DMS scenarios do test use cases
- DMS scenarios don’t test every API, every edge and failure case
- This compliments the component test strategy
• Three types of test:
- Functional tests
- Soak tests
- Performance tests

Confidential ©NDS Ltd 2009. All rights reserved.

 DMS Test Harnesses

• DMS_Test_Harness (CUnit)
– Basic functionality

• DMS_PVR_Test_Harness (CUnit)
– PVR functionality

• DMS_Performance_Test_Harness (CUnit)
– Performance tests – including stress tests and timing information for
key functionality

• DMS_HFL_PVR_Test_Harness (HFL)
– PVR functionality, but based on the HFL framework rather than CUnit.

• DMS_Soak_Test_Harness (CUnit)
– Soak tests – continuous, repeated operations simulating normal use
over a long period.

Confidential ©NDS Ltd 2009. All rights reserved.

 Terminology and Questions

• What is CUnit?
– Open source testing framework
– Organises a series of TESTS into SUITES
– Gives us assertions
– Gives us an XML formatted results summary as output
– http://cunit.sourceforge.net/

Confidential ©NDS Ltd 2009. All rights reserved.

 Terminology and Questions

• What is “build type”?

– RELEASE
– DEBUG
– RELEASE_DBG

• Which test harness will I find scenario DMS_xxxx in?

– See the .ini file – the keyword “binaries” gives the test harness
binary to use.

Confidential ©NDS Ltd 2009. All rights reserved.

 DMS Test Environment

• Linux server for building/NFS mount

• Around 36 stream sets, but most tests require one of
two stream sets (TIG Recorded, TIG Generated)
– (Aside: How can you tell which stream is required for a test?
See the ini file or the Test Release and Execution document.)
– Each stream is a few GB in size – around 5 GB is typical.
• Stream players playing out main streams continuously
• Stream players for ad hoc use (special streams)
• Continuous integration system

Confidential ©NDS Ltd 2009. All rights reserved.

 DMS Test Environment – Stream Players

Network of three
streams (1 EPG Fed to desks
Stream player transponder, two and
– TIG other continuous
RECORDED transponders integration
carrying
services)

Confidential ©NDS Ltd 2009. All rights reserved.

 A Note on Cabling Infrastructure

• Note that one physical wire can generally only carry

one network (usually of three streams). This is because
streams of similar frequency cannot be resolved from
each other by the tuners.
• TIG Generated and TIG Recorded are a special case,
though. We can run these over the same wire even
though the EPG transponder for each has the same
frequency. This is because the polarisation of each of
the EPG transponders is different, so the signals can be
seperated from each other.

Confidential ©NDS Ltd 2009. All rights reserved.

 Lead and Follow on Boxes

• There are currently four hardware platforms.

– The lead box – Samsung, Broadcom 7401 chipset (skpd-
sam0203)
– The follow on boxes:
• Amstrad DRX890, Broadcom 7335 chipset (skpd-
ams0201)
• Amstrad DRX780, NXP ARM9 chipset (skpd-
ams0102)
• Pace, Broadcom 7401 chipset (skpd-pac0102)

Confidential ©NDS Ltd 2009. All rights reserved.

 Key Parts of the DMS Build Tree

• components/
– The middleware components
• integration/dms/scenarios/
– .dat files and makfiles for each DMS test harness
• integration/dms/tests/
– Test files.
- There is often one test file per scenario, but this is certainly not
always true. Sometimes a test file might be reused across several
scenarios, or a single scenario might use several test files.
• integration/dms/services/
– Useful functions for use in DMS tests
• integration/dms/building_blocks/
– DEPRECATED service-like functions

Confidential ©NDS Ltd 2009. All rights reserved.

 Service Functions

• The service functions are a set of functions designed to

make writing tests easier. They only exist in the DMS
test harness – not the middleware, itself.
• They encapsulate several calls across the DMS API,
combining them into an easy to use function call – for
example, to tune to a channel.

Confidential ©NDS Ltd 2009. All rights reserved.

 An Example Service Header
/**
* @brief Tunes to a service represented by the service locator.
*
* Serivce locator should be obtained from the GUIDE using the DMS_GUIDE and
DMS_GUIDEAPI services
* If a NULL locator is given to this function it will only stop the current live job.
* This service will wait for a notification that the tuning occured or until timeout
reached.
*
* @param[in] player_session_handle - handle to the player session
* @param[in] service_locator - locator of the service
* @param[in] timeout - time to wait for the tuning to complete
* @param[out] job_number - a unique number associated with the live tuning.
* @param[out] actual_time_to_tune - if not NULL this will return the actual time it
took to tune to the channel.
*
* @return DMS_STATUS_OK on success,
* DMS_STATUS_TIMEOUT_PAT if tuning times out because no PAT is acquired,
* DMS_STATUS_TIMEOUT_PMT if tuning times out because no PMT is acquired,
* DMS_STATUS_TIMEOUT if tuning times out for any other reason,
* DMS_STATUS_SIGNAL_LOSS if tuning fails because of signal loss,
* DMS_STATUS_ERROR if tuning fails for any other reason.
*/
SYSTEM_STATUS DMS_PLAYER_TuneToChannel(PLAYER_API_SessionHandle player_session_handle,
SYSTEMLOCATOR_Handle service_locator,
uint32_t timeout,
DMS_PLAYER_Job_Number * job_number,
uint32_t * actual_time_to_tune);

Confidential ©NDS Ltd 2009. All rights reserved.

 Services versus Building Blocks

• Building Blocks were an early attempt to make a set of

functions like the service functions.
• Building Blocks are DEPRECATED. Some tests still use
them – these are being rewritten to use Service
functions instead (ongoing).
• New tests should use Services – not Building Blocks.

Confidential ©NDS Ltd 2009. All rights reserved.

 Other Processes used in DMS

• Stream Sync process – tunes to the stream and

monitors the time reported on the stream. If a stream
window is specified in the ini file, then the stream sync
process waits for that time on the stream and then
exits. If no stream window is specified, then the stream
sync process waits for the stream to wrap around then
exits.
• monitor_dms.sh – checks to see if the test has ended,
either by generating a results file, exceeding the
execution time limit, or generating a core file.

Confidential ©NDS Ltd 2009. All rights reserved.

 Build System

make_<platform_name>_scenario.sh

make_scenario.sh

dms_common.mk

makefile (processes)

List of libraries looked up in libraries.mk

Confidential ©NDS Ltd 2009. All rights reserved.

 Build system (2)

• Generic makefiles:

• setenv.mk – set environment and paths

• platform.mk – platform-specific rules

• process.mk - rules for creating processes

• library.mk – rules for creating libraries

• common.mk – common rules

Confidential ©NDS Ltd 2009. All rights reserved.

 Component Configuration (.cfg) File

• Read at run time

• Allow a component to be customised per platform/per
project
• Can contain a wide range of information depending on
the functionality of the component. This usually
includes memory pool information.

Confidential ©NDS Ltd 2009. All rights reserved.

 config.xml

• DMS manages the set of .cfg files, and their various per
platform/per project customisations, with the
config.xml file.
• This describes all the information in all the .cfg files –
including all possible project/platform variations.
• The basic .cfg files are described in the BASE SET.
• The possible variations are described in the PROFILES.
• The ini file defines which PROFILES are used in a test
(under keyword “config-profiles”).
• During test start up, config.xml is expanded out into
the complete set of .cfg files required for the
particular PROFILE.

Confidential ©NDS Ltd 2009. All rights reserved.

 DMS Scenario .ini Files

• Read during the start of a test

• Also used by continuous integration
• Includes a common part, which includes the following
information:
– Test harness binary
– Platforms the scenario should run on
• Includes several platform-specific sections, which
include the following information:
– Stream the scenario requires
– Stream window required
– Profile to be extracted from config.xml

Confidential ©NDS Ltd 2009. All rights reserved.

 How to Enable DIAG

• Modify diag.cfg. This is part of config.xml in DMS.

• Modify dms/projects/dms/build_options.mk - uncomment last 5
lines:
ifeq ($(strip
$(NDS_BUILD_TYPE_$(COMPONENT_NAME))),release_dbg)
COMPONENT_GLOBAL_DEFINES_REMOVE:=DIAG_RELEASE
COMPONENT_GLOBAL_DEFINES_REMOVE+=DIAG_RELEASE_DBG
GLOBAL_DEFINES += DIAG_DEBUG
endif
• Clean build

Confidential ©NDS Ltd 2009. All rights reserved.

 DIAG Levels in diag.cfg

DIAG Level What it means

F FATAL
E ERROR
W WARNING
R RETURN
P PARAMETER
I INFORMATION
< Function EXIT
> Function ENTRY
! EVENT

Confidential ©NDS Ltd 2009. All rights reserved.

 Interpreting DIAG output

• A typical line of DIAG output contains:

- Timestamp
- DIAG level
- Component generating the message
- Process name, thread id, thread name
- Function and line number
- A message

NDS: ^0946685105.658151 !INFO -DMS_T < P:TST t:0000043c

T:DMS_MAE_APPLICATION_THREAD F:DMS_DCA_Zapper L:00179 >
Starting zap 1/8 (LCN=101, name=BBC 1 London)

Confidential ©NDS Ltd 2009. All rights reserved.

 Redirecting DIAG

• To specify that DIAG is directed to stdout, use the

following line when starting a test:
/host/set_diag_output.sh "stdout:1024“
• To specify that DIAG is directed over UDP, use the
following line when starting a test:
/host/set_diag_output.sh “udp:<ip_address>:5555“
where the ip address is that of the PC where the DIAG log should
be sent.
• (See on for further description of the steps required to
run a test)

Confidential ©NDS Ltd 2009. All rights reserved.

 EZ Log

• The main reason for redirecting DIAG log is so that EZ

Log can use it.

Confidential ©NDS Ltd 2009. All rights reserved.

 Other Debugging Tools

• DIAG is the main debugging tool at present.

• Other solutions are being developed – e.g. GDB and hardware debuggers –
but these are not currently in a state where people can just pick them and
play with them easily. L
• If a core file is generated during a test run it can be analysed with the
version of GDB supplied with the toolset.
– Core file is automatically copied to the host when unified_stop.sh is run (at the
end of the test).
– This is a very brief description of how to invoke gdb – details of using gdb are
beyond the scope of this course.
– Unpack the zipped core file, and run the appropriate version of gdb. The
following is an example line for the Samsung platform. It would need to be
adapted to use a different GDB if the platform concerned uses a different
toolset.

<path_to_build_tree>/vobs/SYSTEM_BIN/mips4k_gcc_x86_linux_
04/bin/mips-linux-gdb
<path_to_build_output_tree>/<process_name>/DMS_PVR_Test_
Harness/SAM_BCM_MIPS4K_LNUX_DARWIN_03/BigEndian/fs/NDS/b
in/<process_name> <corefile>

Confidential ©NDS Ltd 2009. All rights reserved.

 Running a DMS Test

• Use the instructions printed at the end of a build, i.e.:

To run this scenario, type the following at the 7401c1 STB
login prompt, replacing NNNN by the scenario number:
root
mkdir /host
mount skeeter:/home/NDS-
UK/jerrardm/snapshots/dms/vobs/DARWIN_PLATFORM/dms/integ
ration/dms/DMS_PVR_Test_Harness/SAM_BCM_MIPS4K_LNUX_DARWIN
_02/BigEndian /host
/host/set_diag_output.sh "stdout:1024"
/host/set_stb_id.sh "0A2ECCBF"
/host/dms_startup.sh skpd-sam0203 NNNN
• Edit /home/NDS-UK/<username>/.bashrc before
running the build to populate the parameters in
set_diag_output.sh and set_stb_id.sh.

Confidential ©NDS Ltd 2009. All rights reserved.

 Running a DMS Test

• Note – the above example sets the DIAG output to go to

stdout. To redirect it over UDP, change the call
parameter passed to set_diag_output.sh:
/host/set_diag_output.sh “udp:<ip_address>:5555“
• (See previous slides for more information on redirecting
DIAG over UDP, and the use of EZ Log.)

Confidential ©NDS Ltd 2009. All rights reserved.

 Common Runtime Options

• It is sometimes useful (and necessary) to add another

runtime parameter to the dms_startup.sh script.
• These include:
– FORMAT – recreates the partition table and reformats the disk
partitions. (This is the normal parameter used by the
continuous integration system.)
– NOCLEAN – retains data stored on the disk and in NVRAM after
the last test run. (If this is option is not used, this data is
removed at the start of the test.)
– See unified_startup.sh for full list of options
– Example of using the FORMAT option:
/host/dms_startup.sh skpd-sam0203 999 FORMAT

Confidential ©NDS Ltd 2009. All rights reserved.

 Startup and Stop Scripts

• set_diag_output.sh – generates diagctl.cfg file in accordance with

environment variable setting. Allows output to stdout (for a
terminal), to a file, or to redirection over UDP (see previous
slides).
• set_stb_id.sh – generates a sysinit_component.cfg file using the
STB ID provided as a parameter. All our CA cards are paired to a
particular (real) box, so we set the STB ID to be the same as that
on the CA card for the purpose of testing.
• dms_startup.sh – Run the unified_startup.sh script, identify the
test to run (passed in as a parameter), read the ini file, check for a
stream window, copy config files, start Stream Sync process, start
the monitor_dms.sh script and run the binary.
• unified_startup.sh – partition/format the disk (if requested), load
kernel modules, create directory structure, copy files across to
RAM. Core file generation prerequisites (ulimit –c unlimited).
• unified_stop.sh – runs automatically when a test ends. Copies the
results file back to the host and reboots the box.

Confidential ©NDS Ltd 2009. All rights reserved.

 Middleware Startup

• Run the SCD Process (/NDS/bin/SCD_Process)

• SYSINIT reads sysinit.cfg
• Reads PROCESS entries and paths – starts processes
• Starts components in ring level order
• At a particular ring level, each component goes through
four states: Initialised, Prepared, Active, PVR.
• After all components at a given ring level have gone to
PVR state, components at the next ring level up are
started.

Confidential ©NDS Ltd 2009. All rights reserved.

 Test Output

• The Results.xml file is the first indication of a test pass

or failure.

• Some tests also produce a file with details about

changes in the memory pool sizes during the test.

Confidential ©NDS Ltd 2009. All rights reserved.

 A Typical DMS Scenario

• DMS_Test_Harness.c defines relationship between a test and the

.dat file
extern char *g_DMS_TEST_DATA_FILE; /* defined in
suite */
• The test file uses DIAG:
DIAG_DECLARE_FUNCTION("DMS_DCA_Zapper");
call_status =
PREFS_API_CreateSession(&prefs_session);
if (PREFS_API_STATUS_OK != call_status)
{
DIAG_LOG_ERROR(g_DMS_DIAG_SEG,
("PREFS_API_CreateSession failed (0x%X)",
call_status));
DMS_FAIL_FATAL("Failed to create PREFS
session");
}

Confidential ©NDS Ltd 2009. All rights reserved.

 A Typical DMS Scenario

• CUnit calls are in the test but hidden:

#define DMS_FAIL_FATAL(msg)\
do { (void) CU_assertImplementation(false, __LINE__,
(msg), __FILE__, "", true); }while(0);

Confidential ©NDS Ltd 2009. All rights reserved.

 A Typical DMS Scenario

• (Almost) All component API calls and DMS service calls

return a SYSTEM_STATUS value
- e.g. DMS_STATUS_OK, BEM_STATUS_FAIL

• SYSTEM_STATUS is built up of several bit fields:

– Component ID
– Severity
– Component interface
– Status (if it ends in zero then status is OK)

Confidential ©NDS Ltd 2009. All rights reserved.

 A Typical DMS Scenario

• The set top box starts in standby state. To bring it out

of standby:
call_status =
DMS_PREFS_EnterLocalActiveState(prefs_sess
ion, 20U);
• Commonly, the first call to a component gets a handle
that is used in later API calls:
call_status =
PLAYER_API_CreateSession(&player_session);

Confidential ©NDS Ltd 2009. All rights reserved.

 A Typical DMS Scenario
Asynchronous Call Handling Pattern

PREFS_API_TERMINAL_RegisterOperatingModeListener
(operatingModeListener);

operatingModeListener()
SYSTEMITC_API_ReadEvent()

Read
SYSTEMITC_API_PendEvent() SYSTEMITC_API_PostEvent()

Pend Post data

queue
Create queue
SYSTEMITC_API_CreateEventQueue()

Confidential ©NDS Ltd 2009. All rights reserved.

 A Typical DMS Scenario

• DMS_GRAPHICS_CreateApplicationWindow() is a typical
DMS service function.
call_status =
DMS_GRAPHICS_CreateApplicationWindow(playe
r_session, "epg", 1280, 720,
"/NDS/etc/arial.ttf",
&application_window);
• Creates an on-screen graphics window that text can be
written to
• Wraps 10+ calls to GFX component – much easier to use
than individual APIs!

Confidential ©NDS Ltd 2009. All rights reserved.

 A Typical DMS Scenario

• DMS_GUIDEAPI_SessionCreate()
– Creates a GUIDE session
– Blocks until service acquisition is complete (SPM)
– Sets up queues to handle asynchronous messages from GUIDE
– GUIDE_API_STATUS_SI_DATA_READY is returned to
DMS_GUIDEAPI_SessionCreate() when enough service
information has been acquired

Confidential ©NDS Ltd 2009. All rights reserved.

 A Typical DMS Scenario

• Test data is read from the .dat file:

DMS_TOOLBOX_ReadUint32(g_DMS_TEST_DATA_FILE,
"DMS_DCA_Zapper", 0U, "NUM_ZAPS", 0U, &num_zaps)
• DMS_0013_Service_List.dat:
…
[DMS_DCA_Zapper]
NUM_ZAPS=8
ZAP_SLEEP_TIME=5 # seconds
NUM_SERVICES=8
SERVICE_LCN.0 = 101 # BBC1 (clear channel)
SERVICE_LCN.1 = 106 # Sky1 (non-
subscription scrambled channel)
…

Confidential ©NDS Ltd 2009. All rights reserved.

 A Typical DMS Scenario

• The live job is started by the service function

DMS_PLAYER_TuneToChannel()
• This sets parameters (including the channel to tune to)
and then starts the live job with the call to Player:
PLAYER_API_Session_Play()
• To zap to the next channel there is no need to
explicitly destroy the existing live job. Just start a new
live job.

Confidential ©NDS Ltd 2009. All rights reserved.

 A Typical DMS Scenario

• At the end of the test, always go through clean up

actions.
• Destroy existing jobs
• Destroy session handles
• Free memory

Confidential ©NDS Ltd 2009. All rights reserved.

Fusion Architecture
Application (Master Application/EPG or Interactive Application)

Platform Component API (per engine language) Fusion-API (per engine language)

Application Engine Run-Time Environment

Platform Component Adaptor

Application Engine
Application Engine

Platform Components API Fusion-API (native)

System Platform Components Platform Components

Resource Mgmnt Download Player Planner Fusion-API Shim
PCAT Plugin
Plugin Management
System Platform System User Activity Guide Object FileSystem
Mgmnt Recording Mgmnt Monitor
System Booking Shared Content Application Engine
Preferences
Mgmnt Management Management

Common Middleware Services API

System Resource Management Information Services Presentation Services Caption
System File Pres.
Resource Program
Peripheral Graphics Window
Management Service Delivery
Device Mgmt Scheduler Engine Mgmnt Audio
Framework Mgmnt Mgmt
Engine
Program
Media Services Catalogue
SI Mgmt Misc Libraries
Security Services HTTP RTSP
Viewing and Recording Content Transfer Database Security Services
Management Management Engine BinXML UPNP
CA Kernel BDOM SSDP
Software Section
Media Connection Management
Download Filter ZLIB
Serial Port
Comms
Mgmt Mgmt
TFTP OIG Client
Front Back Media Cap-
End End Presentation tion SSL
Mgmt Mgmt Mgmnt Proc.

CA System Libraries
System Control & Comms Services
Media Streaming Management Module Diagnostics Net Cfg Mgmnt
SYSTEM SYSTEM SYSTEM
OBJECT TIME LOCATOR
Mgmt
SYSTEM
System Init PPPD MEMMAN
ITC
IPC

Modem Mgmnt SYSTEM SYSTEM SYSTEM

ALARM CONFIG LANG
Media Device Management Diagnostics
DHCP SYSTEM SYSTEM SYSTEM
UTIL COMMON STRING

CDI/POSIX

OS Helpers USB Filesystems

Boot Basic
Network Storage Main
OS Helper Helper Loader class
Drivers Stack RAM ROM NVRAM XTVFS EXT3
Hotplug (BSL) drivers
Helper

Confidential ©NDS Ltd 2009. All rights reserved.

DMS GM Project Hierarchy in ClearCase

Confidential ©NDS Ltd 2009. All rights reserved.

 FST Projects

• FST Projects can have three uses:

– Mustering regression fixes for testing before they are delivered
to the GM.
– New component release baseline testing.
– Delivering a set of Work Packages related to a single Functional
implementation .

Confidential ©NDS Ltd 2009. All rights reserved.

DMS Project in ClearCase

Confidential ©NDS Ltd 2009. All rights reserved.

 DMS Branching Strategy

• Work Package Branch

• Work Package Implementation- Way of Working

Confidential ©NDS Ltd 2009. All rights reserved.

 BKSYB Customer Branch

• The BSKYB branch is a branch off of Grand Master. It is

run more cautiously than Grand Master to promote
stability, and it does not take all the changes that go in
to Grand Master

• Customer stabilization Branch

• Way Customer stabilization Branch- way of working

Confidential ©NDS Ltd 2009. All rights reserved.

 Fusion Configuration Management
Documents

Process Documentation
• FSION-PPD-1358-Fusion Configuration Management Strategy
• FSION-PPD-1541-Fusion Naming Conventions for ClearCase
• FSION-PPD-582-Fusion Configuration Management Process

Procedure Document
• FSION-PPD-1692-Using Merge Manager in ClearCase
• FSION-PPD-1693-Using Terminal Server for Posted Delivery

VOB Architecture
• FSION-PPD-580 FUSION ClearCase VOB Structure

Confidential ©NDS Ltd 2009. All rights reserved.

 Contact

Use Mantis to log all ClearCase related issues

The url is : http://mantis J

Confidential ©NDS Ltd 2009. All rights reserved.

 Continuous Integration

• 50-60 lead boxes running at all times, smaller numbers

of follow on boxes are in continuous integration now.
• With around 1000 tests, it’s essential that as many as
possible run automatically!
• CI servers control the builds, queue the test on the set
top boxes, present and store results.
• For most tests, every build is a clean build, and no
assumptions are made about the previous test.
– Note that there are a few tests that deliberately break this rule
of thumb! These specifically rely on what happened in the
previous test. See the ini file, keyword ‘sequence’ for the order
of tests to be run and the parameters to pass to the
dms_startup.sh script in each case.

Confidential ©NDS Ltd 2009. All rights reserved.

 Continuous Integration

• Restarts test run after every new baseline by default

• Stores test results and all output logs for each scenario
• Rolling history of test runs on previous baselines
• Tests organised into groups reflecting their importance

Confidential ©NDS Ltd 2009. All rights reserved.

 Headlines Page

Confidential ©NDS Ltd 2009. All rights reserved.

 Headlines Page

• This shows the current status of the lead and follow-on

boxes on the BSkyB branch.
• Note division into test categories – e.g. Builds, Must
Pass, and Levels 1 to 5.
• Can also access Functional Sub-Tracks, Work Package
streams, and the Product streams from this page.

Confidential ©NDS Ltd 2009. All rights reserved.

Results Page – Feature Group

Confidential ©NDS Ltd 2009. All rights reserved.

 Results Page

• This shows the results of individual tests on a stream.

• There are several ways to view this – these are
selectable from a drop down menu. If you want to
check a particular test result, Flat List is the most
useful view.

Confidential ©NDS Ltd 2009. All rights reserved.

Flat List with a Comparison Between
Branch and GM

Confidential ©NDS Ltd 2009. All rights reserved.

 Test Result

Confidential ©NDS Ltd 2009. All rights reserved.

 Picklist

Confidential ©NDS Ltd 2009. All rights reserved.

 Using the Picklist

• The picklist is used to select which tests will run on a

particular stream in CI.
• The set of tests selected on GM, Branch Master and the
FSTs is very static.
• The set of tests required to be run on a WP stream can
vary. Often it is all of Level 1, but it could be more or
less depending on the functionality.

Confidential ©NDS Ltd 2009. All rights reserved.

 Regression Tracker

Confidential ©NDS Ltd 2009. All rights reserved.

 Regression Tracker

• Used to store information on regressions on Branch

Master and Grand Master
• Shows:
– the pass/fail history of each test
– CQs raised
– The current state of progress on the regression
– The owner, integrator and developer involved
• Allows user to extract metrics such as total number of
regressions

Confidential ©NDS Ltd 2009. All rights reserved.

 Important documents

• DMS Test and Execution Procedure (FSION-RPT-1491)

– How to build the code for each platform
– How to run the code for each platform
– Stream playout parameters and the streams required by each test
– Details of how to run manual tests
• DMS Test Plan (FSION-PLN-1486)
– A description of what each test does. This is the specification used by
integrators to write the test code.
• Fusion Architectural Rules (FSION-PPD-1190)
– Rules for design and coding of Fusion components. Essential reading
for integrators as well as component owners.
• Many other essential documents on the Fusion web page.
• http://shareatnds.nds.com/global/Fusion/Pages/Default.aspx

Confidential ©NDS Ltd 2009. All rights reserved.