Debugging

1. What is printk() function? a. It is kernels formatted print function i. It is robust ii. It can be called from anywhere in the kernel at anytime. iii. It can be called from both process context and interrupt context iv. It can be called while holding a lock v. It can be called simultaneously from multiple processors and do not require to hold a lock vi. It is a resilient function 2. What is the weakness of printk ()? a. printk () cann0t be used before a certain point (console initialization) in kernel boot process i. Some of the alternatives for such debugging To use serial ports and use other HW early_printk (), this is not available on all architectures 3. What are log levels? a. Kernel uses loglevel to decide whether to print message to the console i. Examples are as below printk(KERN_WARNING This is a warning!\n); printk(KERN_DEBUG This is a debug notice!\n); printk( I did not specify a log level!\n); 4. What are available loglevels? Log Level KERN_EMERG KERN_ALERT KERN_CRIT KERN_ERR KERN_WARNING KERN_NOTICE KERN_INFO KERN_DEBUG Description An emergency condition A problem that requires immediate attention A critical condition An error A warning A normal, perhaps noteworthy, condition An information message A debug message typically superfluous

5. What is Log Buffer? a. Kernel messages are stored in a circular buffer called log buffer. i. It is easy to simultaneously write to and read from a circular biffer. 6. What is the size of log buffer? a. Default value for uniprocessor machine is 16 KB i. It is configurable at compile time using LOG_BUF_LEN via option CONFIG_ LOG_BUF_SHIFT

7. What are syslogd and klogd in linux? a. klogd is a user space daemon, it retrieves the kernel messages from the log buffer and feeds them into the system log file via the syslogd daemon 8. What is Oops in linux kernel debugging? a. Oops is the kernel message to user that something bad has happened. i. If oops occurs in interrupt context, kernel panics and halts the system ii. If oops occurs in idle task or in init then kernel panics and halts the system iii. Oops, when back traced points to exact functions that caused error 9. What is ksymoops? a. Oops gives memory addresses information of hardware context. These addresses when back traced are converted into symbolic names using ksymoops command in conjunction with system.map 10. What is kallsyms? a. kallsyms is available since linux 2.5. It helps kernel to print pre-decoded back traces. i. kallsym feature is enabled via CONFIG_KALLSYMS. This option loads the symbolic kernel name of memory address mapping into the kernel image. 11. What are kernel debugging options? a. To test and debug multiple kernel debug options are available. It needs to be set in kernel hacking menu of the kernel configuration editor, during compile time. i. Some of the kernel debugging options that can be enabled are as below slab layer debugging high-memory debugging I/O mapping debugging Spin-lock debugging Stack-overflow checking Sleep-inside-spinlock checking 12. What is atomicity of debugging? a. Linux kernel has atomicity counter. Kernel can be set such that if a task sleeps while atomic or even does something that might sleep, kernel will print a warning and provides a back trace. i. Following are options available. CONFIG_PREEMPT-y CONFIG_DEBUG_KERNEL-y CONFIG_KALLSYSMS-y CONFIG_SPINLOCK_SLEEP-y ii. Potential bugs that can be detected are as below Calling schedule while holding a lock Issuing a blocking memory allocation while holding a lock Sleeping while holding a reference to per CPU data 13. How BUG() and BUG_ON(),can be used in debugging? a. BUG() and BUG_ON() when used in code issues oops, results in stack trace and an error message dumped to the kernel i. Using these function in situations that should not happen catches bugs 14. What is panic() function do?

a. Prints an error and It halts system. 15. Magic SysRq Key? a. Magic SysRq key is a configurable option via CONFIG_MAGIC_SYSRQ i. When enabled, special combination of keys enable us to communicate with the kernel regardless of what else it is doing. This allows performing some useful tasks in the face of a dying system.

Supporting SysRq Commands

SysRq b SysRq e SysRq h SysRq i SysRq k SysRq l SysRq m SysRq o SysRq p SysRq r SysRq s SysRq t SysRq - u Reboot the machine Send a SIGTERM to all processes except init Display SysRq help on the console Send a SIGKILL to all processes except init Secure access key: kill all programs on this console Send a SIGKILL to all processes including init Dump memory information to console Shutdown the machine Dump registers to console Turnoff key board raw mode Sync all filesystems to disk Dump task information to console Unmount all mounted filesystems

16. Kernel debuggers? a. Gdb b. Kgdb c. Kdb 17. Using UID as a conditional in debugging? a. UID 7777 is a special user ID for testing in user space. 18. Using condition variables for debugging? a. If the code under debugging is not in process context or want a more global method of controlling feature, then a variable can be declared as global variable and can be used in condition variable. 19. Using statistics for debugging? a. In certain situations it is desirable to know how often a specific event happens, then statistics can be collected and used. 20. Using rate limiting for debugging? a. In kernel, some functions are called numerous times per second. To sniff a problem in such situations, printk() if used will create lot of information as debug outpot. This can be controlled by using a technique called rate limiting. It enables debugging output every few seconds at an interval. 21. What is binary searching for debugging? a. If a bug appears in a version which is not available in any of previous versuions, then ac areful examination of new features added in the binary might give clues on debugging information 22. Kernel handling of core dumps? a. Core() may be called in kernel by any of the following.

23.

24.

25.

26.

27.

i. Sigclean() - invokes core() if the user has corrupted the user stack during execution of a signal handler ii. Psig() responsible for processing signal and will call core () if the disposition of signal is SIG_DFL iii. Kern_gpfault() - invokes core() if the user has corrupted the user stack during execution of a signal handler iv. U_cdir/core How core () works? a. Set the u_dirp field of the u_area to point to the string core b. If the effective user ID and group ID do not equal the real user ID and group ID, the core file cannot be dumped, so core() returns c. Ensure that the core file can be written and truncate if necessary d. Call findpreg() to obtain the sizes for the text, data and stack regions e. Determine how much data can be written. i. If the complete core file cannot be written, the stack takes precedence over the data region which takes the precedence over text f. Create a CORES_UAREA section header and call dumpcorecontents () to write out the u_area g. Create CORES_PREGION section header and call dumpcorecontents () to write out the data region h. Create CORES_PREGION section header and call dumpcorecontents () to write out the stack region i. Call dumpcoreheader () to write out section headers for the u_area, data and stack j. For each other region, call dumpcorecontents () and dumpcoreheader () k. Create a CORES_PROC section header and call dumpcorecontents () to write out the proc structure. Call dumpcoreheader () to write out the section header for CORES_PROC l. Create a CORES_ITIMERS section header and call dumpcorecontents () to write out itimer array. Call dumpcoreheader () to write out the section header for CORES_ITIMERS m. Create the coreoffsets structure and write at the end of the file. What are breakpoints? a. A breakpoint makes program to stop whenever a certain point in the program reached. i. Break command is used for setting breakpoint What are watchpoints? a. A watch point is a special breakpoint that stops program when the value of an expression changes i. watch command is used for setting watchpoint What are catchpoints? a. A catchpoint is another special breakpoint that stops program when a certain kind of event occurs. i. catch command is used for setting catchpoint Breakpoints can be set for? a. Line number, function name or exact address

28. How are break, clear, enable, disable used for breakpoints? a. Break is used for setting breakpoints b. Clear is used to delete breakpoints c. Enable is used to enable breakpoints d. Disable is used for disabling breakpoints 29. What are trace points? a. Tracepoints are set in code and used for debugging. An arbitrary expression is set at tracepoint to evaluate value of expression when tracepoint is reached during execution of code. b. Trace points are set using trace and collect command 30. Why tracepoints are used for debugging? a. Setting breakpoint would potentially alter the behavior the program under evaluation. Using tracepoint real-time behavior of the code can be checked 31. What is a stack frame and call frame? a. Each time program performs a function call, information about the call is generated. This information includes the location of call in program, arguments of the call, and the local variables of the function being called. This information is saved in a block of data called stack frame. b. Stack frames are allocated in a region of memory called call stack. 32. How to debug using stack frames? a. Frame args command allows to move from one stack frame to another and to print the selected stack frame. b. Select frame command allows to move from one stack frame to another without printing it. 33. What is a backtrace? a. A backtrace is a summary of how a program has reached where it is during execution.