Embedded Systems

Prof. Dr. Faruk Bağcı

Prof. Dr. Mesut Güneş
 CHAPTER OVERVIEW
• Introduction to Embedded Systems
• Embedded & Real-time Operating Systems
• Linux Kernel
• Peripheral Access
• Threads & Processes
• Memory
• Inter-process Communication
• Real-time Scheduling
• Interrupt Handling
• Advanced Topics
MANAGING MEMORY

• Virtual memory basics

• Kernel space memory layout
• User space memory layout
• The process memory map
• Swapping
• Mapping memory with mmap
• How much memory does my application use?
• Per-process memory usage
• Identifying memory leaks
• Running out of memory
• Memory mapped I/O
VIRTUAL MEMORY BASICS

• Linux configures the memory management unit (MMU) of the CPU to present a
virtual address space to a running program
• Begins at zero and ends at the highest address, 0xffffffff, on a 32-bit processor
• Divided into pages of 4 KiB (there are rare examples of systems using other page sizes)
• Linux divides this virtual address space into an area for applications, called user
space, and an area for the kernel, called kernel space
• The split between the two is set by a kernel configuration parameter named PAGE_OFFSET
• In a typical 32-bit embedded system, PAGE_OFFSET is 0xc0000000
• lower 3 gigabytes to user space and the top gigabyte to kernel space
• The user address space is allocated per process so that each process runs in a
sandbox, separated from the others
• The kernel address space is the same for all processes: there is only one kernel
• Pages in this virtual address space are mapped to physical addresses by the
MMU, which uses page tables to perform the mapping
 VIRTUAL MEMORY BASICS (CONTD.)

• Each page of virtual memory may be:

• Unmapped, so that trying to access these addresses will result in a SIGSEGV
• Mapped to a page of physical memory that is private to the process
• Mapped to a page of physical memory that is shared with other processes
• Mapped and shared with a copy on write (CoW) flag set:
• A write is trapped in the kernel, which makes a copy of the page and maps it to the process in
place of the original page before allowing the write to take place
• Mapped to a page of physical memory that is used by the kernel
• The kernel may additionally map pages to reserved memory regions, for
example, to access registers and memory buffers in device drivers
 WHY VIRTUAL MEMORY?

• There are numerous advantages to virtual memory:

• Invalid memory accesses are trapped and applications are alerted by SIGSEGV
• Processes run in their own memory space, isolated from others
• Efficient use of memory through the sharing of common code and data, for example,
in libraries
• The possibility of increasing the apparent amount of physical memory by adding
swap files, although swapping on embedded targets is rare
• Main disadvantage:
• It is difficult to determine the actual memory budget of an application
• Out-of-memory situations happen
• Delays introduced by the memory management code in handling exceptions—page
faults—make the system less deterministic
 KERNEL SPACE MEMORY LAYOUT
• Kernel memory is managed in a fairly straightforward way
• It is not demand paged
• Means that for every allocation using kmalloc()or a similar function, there is real physical
memory
• Kernel memory is never discarded or paged out
• Some architectures show a summary of the memory mapping at boot time in the
kernel log messages
 HOW MUCH MEMORY DOES THE KERNEL USE?

• Hard to answer exactly, but can

be approximated
• Use the size command
• You can get more information
about memory usage by reading
/proc/meminfo
USER SPACE MEMORY LAYOUT

• Linux employs a lazy allocation strategy for user space, only mapping
physical pages of memory when the program accesses it
• For example, allocating a buffer of 1 MiB using malloc()returns a pointer
to a block of memory addresses but no actual physical memory
• A flag is set in the page table entries such that any read or write access is
trapped by the kernel
• This is known as a page fault
• Only at this point does the kernel attempt to find a page of physical memory
and add it to the page table mapping for the process
#include <stdio.h>
#include <stdlib.h> int main (int argc, char *argv[])
#include <string.h>
{
#include <sys/resource.h> unsigned char *p; Output:
#define BUFFER_SIZE (1024 * 1024) printf("Initial state\n"); Initial state

print_pgfaults(); Major page faults 0

void print_pgfaults(void) p = malloc(BUFFER_SIZE); Minor page faults 172
{ printf("After malloc\n"); After malloc
int ret; Major page faults 0
print_pgfaults();
struct rusage usage; memset(p, 0x42, BUFFER_SIZE); Minor page faults 186
ret = getrusage(RUSAGE_SELF, &usage); printf("After memset\n"); After memset
if (ret == -1) { Major page faults 0
print_pgfaults();
perror("getrusage"); memset(p, 0x42, BUFFER_SIZE); Minor page faults 442
} else { printf("After 2nd memset\n"); After 2nd memset
printf ("Major page faults %ld\n", Major page faults 0
print_pgfaults();
usage.ru_majflt);
return 0; Minor page faults 442
printf ("Minor page faults %ld\n",
usage.ru_minflt); }
}
}
PROCESS MEMORY MAP

• You can see the memory map for a process through the proc filesystem
• As an example, here is the map for the init process, PID 1:
SWAPPING

• The idea of swapping is to reserve some storage

• Kernel can place pages of memory that are not mapped to a file so that it can free up
the memory for other uses
• It increases the effective size of physical memory by the size of the swap file
• There is a cost to copying pages to and from a swap file
• Becomes apparent on a system that has too little real memory for the workload it is
carrying and so swapping becomes the main activity
• This is sometimes known as disk thrashing
• Swap is seldom used on embedded devices
• It does not work well with flash storage, where constant writing would wear it out
quickly
• However, you may want to consider swapping to compressed RAM (zram)
 COMPRESSED MEMORY (ZRAM)

• The zram driver creates RAM-based block devices named /dev/zram0,

/dev/zram1, etc.
• Pages written to these devices are compressed before being stored
• With compression ratios in the range of 30% to 50%:
• Can expect an overall increase in free memory of about 10% at the
expense of more processing and a corresponding increase in power usage
MAPPING MEMORY WITH MMAP

• A process starts with certain amount of memory mapped to the text (thevcode)
and data segments of the program file, together with the shared libraries that it is
linked with
• It can allocate memory on its heap at runtime using malloc()and on the stack through
locally scoped variables and memory allocated through alloca()
• It may also load libraries dynamically at runtime using dlopen(3)
• Process can also manipulate its memory map in an explicit way using mmap():
• void *mmap(void *addr, size_t length, int prot, int flags, int
fd, off_t offset);
• This function maps length bytes of memory from the file with the descriptor fd, starting at
offset in the file, and returns a pointer to the mapping, assuming it is successful
• Since the underlying hardware works in pages, length is rounded up to the nearest whole
number of pages
• The protection parameter, prot, is a combination of read, write, and execute permissions and
the flags parameter contains at least MAP_SHARED or MAP_PRIVATE
USING MMAP TO ALLOCATE PRIVATE MEMORY

• You can use mmap to allocate an area of private memory

• by setting MAP_ANONYMOUS in the flags parameter and setting the file descriptor fd to -1.
• This is similar to allocating memory from the heap using malloc, except that the
memory is pagealigned and in multiples of pages
• The memory is allocated in the same area as that used for libraries
• In fact, this area is referred to by some as the mmap area for this reason
• Anonymous mappings are better for large allocations because they do not pin
down the heap with chunks of memory, which would make fragmentation more
likely
• Interestingly, you will find that malloc (in glibc at least) stops allocating
memory from the heap for requests over 128 KiB and uses mmap in this way, so in
most cases, just using malloc is the right thing to do
• The system will choose the best way of satisfying the request
 USING MMAP TO SHARE MEMORY

• POSIX shared memory requires mmap to access the memory segment

• In this case, you set the MAP_SHARED flag and use the file descriptor
from shm_open():
USING MMAP TO ACCESS DEVICE MEMORY

• It is possible for a driver to allow its device node to be mmaped and share some
of the device memory with an application
• The exact implementation is dependent on the driver
• One example is the Linux framebuffer, /dev/fb0
• The interface is defined in /usr/include/linux/fb.h, including an ioctl function
to get the size of the display and the bits per pixel
• You can then use mmap to ask the video driver to share the framebuffer with the
application and read and write pixels
HOW MUCH MEMORY DOES MY APPLICATION USE?

• As with kernel space, the different ways of allocating, mapping, and sharing user
space memory make it quite difficult to answer this seemingly simple question
• To begin, you can ask the kernel how much memory it thinks is available, which
you can do using the free command:

• You can force the kernel to free up caches by writing a number between 1 and 3
to /proc/sys/vm/drop_caches:
PER-PROCESS MEMORY USAGE
• There are several metrics to measure the amount of memory a
process is using.
• Two that are easiest to obtain: the virtual set size (vss) and the
resident memory size (rss):
• Vss: Called VSZ in the ps command and VIRT in top
• Total amount of memory mapped by a process
• It is the sum of all the regions shown in /proc/<PID>/map
• This number is of limited interest since only part of the virtual memory is committed to
physical memory at any time
• Rss: Called RSS in ps and RES in top
• Sum of memory that is mapped to physical pages of memory
• This gets closer to the actual memory budget of the process, but there is a problem:
• if you add the Rss of all the processes, you will get an overestimate of the memory in use
because some pages will be shared
IDENTIFYING MEMORY LEAKS

• A memory leak occurs when memory is allocated but not freed when
it is no longer needed
• Memory leakage is not unique to embedded systems
• But it becomes an issue because targets don't have much memory and they
often run for long periods of time without rebooting
• You will realize that there is a leak when you run free or top and
see that free memory is continually going down even if you drop
caches
• There are several tools to identify memory leaks in a program
• Here: mtrace
MTRACE

• mtrace is a component of glibc that traces calls to malloc, free,

and related functions, and identifies areas of memory not freed when
the program exits
• Call the mtrace()function from within the program to begin tracing and
then at runtime, write a path name to the MALLOC_TRACE environment
variable in which the trace information is written
• If MALLOC_TRACE does not exist or if the file cannot be opened, the mtrace
hooks are not installed
• While the trace information is written in ASCII, it is usual to use the mtrace
command to view it.
 MTRACE EXAMPLE

#include <mcheck.h>
#include <stdlib.h>
#include <stdio.h>
int main(int argc, char *argv[]) $ export MALLOC_TRACE=mtrace.log
{ $ ./mtrace-example
$ mtrace mtrace-example mtrace.log
int j;
mtrace(); Memory not freed:
-----------------
for (j = 0; j < 2; j++) Address Size Caller
malloc(100); /* Never freed:a memory leak */ 0x0000000001479460 0x64 at /home/chris/mtrace-example.c:11
calloc(16, 16); /* Never freed:a memory leak */ 0x00000000014794d0 0x64 at /home/chris/mtrace-example.c:11
0x0000000001479540 0x100 at /home/chris/mtrace-example.c:15
exit(EXIT_SUCCESS);
}
RUNNING OUT OF MEMORY

• The standard memory allocation policy is to over-commit

• Kernel will allow more memory to be allocated by applications than there is physical memory
• Most of the time, this works fine because it is common for applications to request
more memory than they really need
• However, there is always the possibility that a process demands more than there
really is
• This is an out of memory situation, or OOM
• At this point, there is no other alternative but to kill off processes until the
problem goes away
• This is the job of the out of memory killer
• There is a tuning parameter for kernel allocations in
/proc/sys/vm/overcommit_memory, which you can set to the following:
• 0: Heuristic over-commit
• 1: Always over-commit; never check
• 2: Always check; never over-commit
MEMORY MAPPED I/O

• I/O memory is simply a region of RAM-like locations that the device makes
available to the processor over the bus
• This memory can be used for a number of purposes, such as holding video
data or Ethernet packets, as well as implementing device registers that
behave just like I/O ports
• According to the computer platform and bus being used, I/O memory may
or may not be accessed through page tables
• Device memory regions must be allocated prior to use
• int check_mem_region(unsigned long start, unsigned long len);
• void request_mem_region(unsigned long start, unsigned long len,
char *name);
• void release_mem_region(unsigned long start, unsigned long
len);