Outline

In this tutorial, I will introduce

Create process from user mode
Create process from kernel mode
fork() in the linux
Process
 User Mode
 Kernel Mode

Complete access to hardware

9/27/2022 3
Process
 Process: Program in execution
Multiple processes: Concurrent vs Parallel. Processor vs CPU vs Core
 In multitasking operating systems, processes (running programs) need a way to create
new processes, e.g. to run a program.
Process state:
◦ new: The process is being created
◦ running: Instructions are being executed
◦ waiting: The process is waiting for some event to occur
◦ ready: The process is waiting to be assigned to a processor
◦ terminated: The process has finished execution

9/27/2022 4
Process
 Each process is named by a process ID number. Generally, process is identified and
managed via a process identifier (pid)

 A unique process ID is allocated to each process when it is created.

 The lifetime of a process ends when its termination is reported to its parent process.
At that time, all of the process resources, including its process ID, are freed.

9/27/2022 5
Process Creation
 Processes are created with the fork system call (so the operation of creating a new
process is sometimes called forking a process).

 The child process created by fork is an exact clone of the original parent process,
except that it has its own process ID.

9/27/2022 6
Process Creation
 Functions (Unix-like system):
◦ pid_t fork(void) system call creates new process

 Return value
◦fork() returns -1, the creation of a child process was unsuccessful.
◦fork() returns a zero to the newly created child process.
◦fork() returns a positive value, the process ID of the child process, to the
parent.

 The returned process ID is of type pid_t defined in ”sys/types.h”.

9/27/2022 7
Parent and Child Process
 If the call to fork() is executed successfully, Unix will
◦make two identical copies of address spaces, one for the parent and the
other for the child.
◦Both processes will start their execution at the next statement following
the fork() call.

9/27/2022 8
 Parent and Child Process 1 - pid
Both parent and
child process start
execution from
next statement
after fork call.

9/27/2022 9
Parent and Child Process 2

• getpid() returns PID of

calling system.
• getppid() returns PID of
the parent of calling
system.

Let parent process

to sleep for 3s, and
then print messages.

9/27/2022 10
Parent and Child Process 2

Why?
Parent process terminates, and the
child process is inherited by init
process, whose pid is 1378 in my
example. It may change every time
rebooting the OS.
When testing in Mac OS, it might be
Parent and child process -1.
runs concurrently after
forking.

9/27/2022 11
Parent and Child Process

9/27/2022 12
Executing a file
 exec is a functionality of an operating system that runs an executable file in the
context of an already existing process, replacing the previous executable. This act is
also referred to as an overlay.  

 Exec function family

◦ int execl (const char *filename, const char *arg0, ...)
◦ int execve (const char *filename, char *const argv[], char *const env[])
◦ int execle (const char *filename, const char *arg0, char *const env[], ...)
◦ int execvp (const char *filename, char *const argv[])
◦ int execlp (const char *filename, const char *arg0, ...)

9/27/2022 13
Executing a file
 New process will not be created, the original PID does not change, but the machine
code, data, heap and stack of the process are replaced by those of the new program. 

 Return value
◦ A successful exec replaces the current process image, so it cannot return anything to the
program that made the call.
◦ If an exec function does return to the calling program, an error occurs, the return value is −1

9/27/2022 14
Executing a file – execve()

Check if the test program is replacing

child process but will not create new
process

path of the
In child process, executed file
execute a new file.

Check if the child process

is replaced by new
program

9/27/2022 15
Process Termination
 Process executes last statement and asks the operating system to delete it (exit())
◦ Output data from child to parent (via wait())
◦ Process’ resources are deallocated by operating system

 If no parent waiting, then terminated process is a zombie

 If parent terminated, processes are orphans

9/27/2022 16
Process Termination 1 - orphan

Let parent process

to terminates
ahead of child
process Parent process terminates, and
the child process becomes
orphans, and will be adopted by
init process.

9/27/2022 17
Process Termination 2
0 / WNOHANG / WUNTRACED
 Functions: (defined in “sys/wait.h”) • 0 skips the option, and keeps waiting till the specified
◦ pid_t wait (int *status_ptr) child process terminates.
• WNOHANG demands status information immediately.
◦ pid_t waitpid (pid_t pid, int *status_ptr, int options) If status information is immediately available on an
appropriate child process, waitpid() returns this
information. Otherwise, waitpid() returns immediately
with an error code indicating that the information was
 Differences not available. In other words, it checks child processes
◦ wait() requests status for any child process without causing the caller to be suspended.
• WUNTRACED reports on stopped child processes as
◦ waitpid() requests status for specific child process. well as terminated ones.
◦ The waitpid() function behaves the same as wait() if the pid argument is (pid_t) -1 and
the options argument is 0.

https://stackoverflow.com/questions/33508997/waitp
id-wnohang-wuntraced-how-do-i-use-
these/34845669
https://linux.die.net/man/2/waitpid

9/27/2022 18
Process Termination 2
 Return value
◦ wait() or waitpid() returns PID of child process when the status of a child process is available.
◦ If unsuccessful, wait() or waitpid() returns -1.

 When waitpid() returns with a valid process ID (pid), below macros can analyze the
status referenced by the status argument.
◦ int WIFEXITED (int status)
◦ int WIFSIGNALED (int status)
◦ int WIFSTOPPED (int status)
◦ etc.

9/27/2022 19
Process Termination 2 – waitpid()

Child process sleeps 10s and parent process will

suspend until status information of child process is
available.

9/27/2022 20
Process Signals
 Linux supports the standard signals listed below.
◦ SIGQUIT 3
◦ SIGKILL 9
◦ SIGTERM 15
◦ SIGSTOP 19
◦ etc.

https://stackoverflow.com/questions/50299429/c-reporting-which-signal-terminated-a-
child

9/27/2022 21
Process Signals
 Send a signal to caller
◦ int raise (int sig)

 Evaluate child process’s status (zero or non-zero)

◦ int WIFEXITED (int status)
◦ int WIFSIGNALED (int status)
◦ int WIFSTOPPED (int status)

 Evaluate child process’s returned value of status argument(exact values)

◦ int WEXITSTATUS (int status)
◦ int WTERMSIG (int status)
◦ int WSTOPSIG (int status)

9/27/2022 22
 Process Signals 1 – wait()

Raise SIGCHLD in child

process

Check if child
process exits
normally Get status value of child
process

9/27/2022 23
 Process Signals 1 – wait()

Raise SIGKILL in child

process

Check if child
process received a Get status value for child progress’
terminating signal terminating signal

9/27/2022 24
Process creation

Search sys_fork() in Implemented via

sys_call_table[] calling do_fork()

9/27/2022 25
Process creation
https://tinylab.org/riscv-task-implementation/

9/27/2022 26
 Process creation (kernel_clone)
 Its implementation is defined in ‘/kernel/fork.c’:
 Call get_task_pid() to assign an available PID to the new task.
 Call copy_process() then either duplicates or shares open files, file system information, signal
handlers, process address space, and namespace.
 In copy_process, it calls dup_task_struct(), which creates a new kernel stack, thread_info
structure, and task_struct for the new process.
 wake_up_new_task() to wake up the child process and insert the child process to the running
queue.
 For more details
◦ https://elixir.bootlin.com/linux/v5.10/source/kernel/fork.c#L2415
◦ You should check basing on your own version

9/27/2022 27
Process creation (kernel_clone)
 kernel_clone:
extern pid_t kernel_clone(struct kernel_clone_args *kargs);

◦ Arguments:

9/27/2022 28
Process creation (kernel_clone)
 Arguments:
◦ flags: How to clone the child process. When executing fork(), it is set as SIGCHILD.
◦ stack: Specifies the location of the stack used by the child process.
◦ stack_size: Normally set as 0 because it is unused.
◦ parent_tid: Used for clone() to point to user space memory in parent process address space. It
is set as NULL when executing fork();
◦ child_tid: Used for clone() to point to user space memory in child process address space. It is
set as NULL when executing fork();
◦ tls: Set thread local storage.

9/27/2022 29
Process creation (kernel_clone)
 Return value:
◦ Fork successfully: pid of child process
◦ Failed: Err

9/27/2022 30
Program execution (do_execve)
 Its implementation is defined in ‘/fs/exec.c’:
 struct linux_binprm: This structure is used to hold the arguments that are used when loading
binaries. It is defined in ‘include/linux/binfmts.h’
 Use prepare_bprm_creds API to initialize the struct
 Open your program file and assign corresponding values into struct linux_binprm
 Use the bprm_mm_init and prepare_binprm to prepare execute your binary program
 Read the file content and copy corresponding information into your struct linux_binprm
 Use search_binary_handler to execute your binary program
 After execve succeeded, it calls acct_update_integrals and free the binary program by calling
free_brpm

9/27/2022 31
Program execution (do_execve)
 do_execve:
◦ int do_execve ( struct filename *filename,
const char __user *const __user *__argv,
const char __user *const __user *__envp);
 Parameters:
◦ filename: Filename of the executable file.
◦ argv: The arguments for executing the file.
◦ envp: System environment variable set.

 Return value:
◦ Execute successfully: 0
◦ Failed: Err

9/27/2022 32
Wait for signal (do_wait)
 In kernel mode, when system call do_wait() is executed, it is loading exit.ko module.
 Its implementation is defined in ‘/kernel/exit.c’:
 Create wait_opts structure and add it into wait queue by calling add_wait_queue.
 Use set_current_state to update state as TASK_INTERRUPTIBLE or TASK_RUNNING.
 When child process terminates, it calls wake_up_parent. Then parent process will go to
repeat scanning, so that it can get return of child process’ termination.

 For more details:

◦ https://elixir.bootlin.com/linux/v5.10/source/kernel/exit.c(do_wait)

9/27/2022 33
Wait for signal (do_wait)
 do_wait:
◦ long do_wait (struct wait_opts *wo);
 struct wait_opts:
◦ struct wait_opts { enum pid_type wo_type; //It is defined in ‘/include/linux/pid.h’.
int wo_flags; //Wait options. (0, WNOHANG, WEXITED, etc.)
struct pid *wo_pid; //Kernel's internal notion of a process identifier.
“Find_get_pid()”
struct siginfo __user *wo_info; //Singal information.
int __user *wo_stat; // Child process’s termination status
struct rusage __user *wo_rusage; //Resource usage
wait_queue_entry_t child_wait; //Task wait queue
int notask_error ;};

9/27/2022 34
Wait for signal (do_wait)

Look up a PID from hash

table and return with it’s
count evaluated.

Decrease the count and free

memory

9/27/2022 35
Handle signal (k_sigaction)
 For each process in the system, the kernel must keep track of what signals are
currently pending or masked. The kernel must keep track of how every thread group is
supposed to handle every signal.
 To do this, the kernel uses several data structures accessible from the process
descriptor. The most significant data structures related to signal handling：

9/27/2022 36
Handle signal (k_sigaction)
 The signal field of the process descriptor points to a signal descriptor, a signal_struct
structure that keeps track of the shared pending signals.
 The of properties of a signal are stored in a k_sigaction structure, which contains the
signal properties.
 The kernel noticed the arrival of a signal and invoked function to prepare the process
descriptor of the process that is supposed to receive the signal. (do_signal())
 If a handler has been extablished for the signal, the do_signal() function must enforce
its execution. (handle_signal())

9/27/2022 37
Kernel Thread
 Kthread creation:
◦ struct task_struct *kthread_create( int (*threadfn)(void *data),
void *data,
const char *namefmt, ...);
◦ The data argument will simply be passed to the thread function.
◦ The thread will not start running immediately. It will start to execute when returned task_struct is
passed to wake_up_process().

 Kthread execution function:

◦ int thread_function(void *data);
◦ It can either call do_exit directly if it is a standalone thread for which no one will call
kthread_stop()
◦ Or return when 'kthread_should_stop' is true (which means kthread_stop has been called).

9/27/2022 38
Kernel Thread
 Return value:
◦ It returns task_struct when executes successfully.
◦ When fails, it returns ERR_PTR

 Kthred start execution with:

◦ int wake_up_process (struct task_struct * p);
◦ ERR_PTR

 A convenient function which creates and starts the thread:

◦ struct task_struct *kthread_run(int (*threadfn)(void *data),
void *data,
const char *namefmt, ...);
◦ Same as kthread_create() + wake_up_process()

9/27/2022 39
 Kernel Thread
GPL: General Public
License.
Loading a proprietary or
non-GPL-compatible LKM
will set a 'taint' flag in the
running kernel

Create a kernel
thread to
execute func

9/27/2022 40
Kernel Thread

9/27/2022 41
Kernel Object
 A loadable kernel module (or LKM) is an object file that contains code to extend the
running kernel, or so-called base kernel

 LKMs are typically used to add support for new hardware and/or file systems, or for
adding system calls.

 Most current Unix-like systems support loadable kernel modules, although they might
use a different name for them,
◦ for example: kernel extension (kext) in MacOS

9/27/2022 42
Kernel Object Compiling (Makefile)
 Makefile
http://www.cyberciti.biz/tips/compiling-linux-kernel-module.html

 Build kernel object

‘KM’ is name of the Kernel Object

Type
command
“Make”

If you type command “Make clean”, it will

clear all built files and leave original c file
and makefile .

9/27/2022 43
Insert and Remove Kernel Module
 Before insert the kernel object, you have to sign in the root account.
◦ $ sudo su

 Insert module
◦ $insmod MODULE_NAME.ko

 List the module you insert

◦ $lsmod
◦ $lsmod | grep MODULE_NAME

 Remember to remove your module

◦ $rmmod MODULE_NAME.ko

9/27/2022 44
Compile Kernel
 Download source code from
◦ http://www.kernel.org
◦ mirro: https://mirror.tuna.tsinghua.edu.cn/kernel/v5.x/

 Install Dependency and development tools

◦ sudo apt-get install libncurses-dev gawk flex bison openssl libssl-dev dkms libelf-dev libudev-
dev libpci-dev libiberty-dev autoconf llvm dwarves
 Extract the source file to /home/seed/work
◦ cp KERNEL_FILE.tar.xz /home/seed/work
◦ cd /home/seed/work
◦ $sudo tar xvf KERNEL_FILE.tar.xz

9/27/2022 45
Compile Kernel
 Copy config from /boot to /home/seed/work/KERNEL_FILE

 Login root account and go to kernel source directory

◦ $sudo su
◦ $cd /home/seed/work /KERNEL_FILE

9/27/2022 46
Compile Kernel
 Clean previous setting and start configuration
◦ $make mrproper
◦ $make clean
◦ $make menuconfig
◦ save the config and exit

 Build kernel Image and modules

◦ $make bzImage -j$(nproc)
◦ $make modules -j$(nproc)
◦ ~ 30 mins to finish
◦ $make –j$(nproc)
(you could use this command to replace above two commands)

9/27/2022 47
Compile Kernel
 Install kernel modules
◦ $make modules_install

 Install kernel
◦ $make install

 Reboot to load new kernel

◦ $reboot
(When rebooting, you should select the updated kernel)

9/27/2022 48
Compile Kernel
 Check exiting kernel version
◦ $uname -r

9/27/2022 49
Problem discussion for compiling
kernel
 Fatal error (No such file or directory)
◦ Ensure your source package is extracted within Linux, not in Windows or Mac.
◦ Filename is Linux is case sensitive, but it is case non-sensitive in Windows or Mac. It may
rename some files when extracting them, so that these files cannot be found when compiling.

9/27/2022 50
Problem discussion for compiling
kernel
 Space is not enough
◦ Copy the source file from original space(e.g., /usr/src) to extended space(e.g.,
/home/seed/work)
◦ Remove the source file from original space(e.g., /usr/src)
◦ In source file (stored in extended space), continue to installation.
(Do not start from ‘make clean’, which will remove all the previous built, start from previous
interrupted step)

9/27/2022 51
Problem discussion for compiling
kernel
 Do not compile kernel source code under shared folder.
◦ Copy the source code to extended space.
◦ Then start to compile.
◦ If you compiling it under shared folder, some files might be created failed, and it would lead to
error when executing install step.

9/27/2022 52
Problem discussion for compiling
kernel
 Share folder mounting does not work
◦ After creating a Share folder in Shared folder setting, open terminal and type command ‘sudo
adduser seed vboxsf’

◦ Reboot the machine

◦ You will see /media/sf_Share will auto mount to your share folder
◦ You could access this share folder via ‘cd /media/sf_Share’.

9/27/2022 53
 Export Symbol
 EXPORT_SYMBOL() provides API to be used in other module.

  Before using this symbol in another kernel module, should use ‘extern’ 
to clarify it. 

9/27/2022 54
 Export Symbol
 If you modify the kernel source code, you should rebuilt the kernel module and install the 
updated kernel. Then it takes effect.

  To save your time, when you rebuilt the kernel, start from ‘make bzImage’. It will only 
rebuild the updated modules.
  (Do not start from ‘make clean’, which will clean all previous built, and it takes hours to 
rebuild the kernel modules)

  Once the symbol is exported and re-built the kernel module, you will find it’s defined in 
“Module.symdvers” (auto generated under source code after run “make bzImage”)

9/27/2022 55
 Recompile your kernel
 If you’ve already compiled and installed the new kernel, then modify the source code and 
need to make the changes effect. Please start compile from command ‘make bzImage’. 
(Follow compile steps from Tutorial 2)

  To save your time, do not start from ‘make clean’ again, which will remove all your previous 
built.

  Starting from ‘make bzImage’, it will only build the changes.

9/27/2022 56
References
 Understanding the Linux KERNEL (3rd edition)
 do_fork() (Page 117 – 130)
 sigaction (Page 420 – 455)

 Linux Kernel Development (3rd edition)

 Process descriptor and task structure (Page 24 – 27)
 Current task_struct (Page 29 – 30)

9/27/2022 57
Thank you