HACKER'S TNE yea THE KERNEL * NETWORKS SERVERS » HARDWARE * SECURITY ENHANCE YOUR KNOWLEDGE WITH IN-DEPTH PROJECTS AND GUIDESRARARRARARARARARAAI vi i Maa baad di nik im 3 RERRREES | ah BSESSAMES . BAAR REASS » ih SSS oom | aa | . : » SERERREERE | : i hone SESS SS bla Reeeet\Wielcome tothe 2023 edtion of the Hackers Manual! You holdin your hands 148 pages of Linus hacking tutorials guides and features from the experts at Line Format magazine -the home of open source software. In this edition we've gone in hard for security Youll ind guides to securing servers, getting 2 grounding in hacking, using security toos suchas Kal and Fedora Security Lab alongside solid features explaining how to protect your privacy online using established tools ike Tals. But we shouldnt lve in ear Hacking is ‘monumental fun, Never mind setting up and playing with Liu, we takea look at hacking tablets, mea servers, virtual machines cloud serves, ‘mult-booting with Grub and much moe. youle alittle timid wihen it comes to the Terminal we even have a meaty reference section atthe back. So dive inand enjoy!4 L FUTURE 4 rHACKER'S MANUAL 2023Contents HACKER'S MANUAL 2023 Security Distros 10 Ubuntu 22.04 Get the lo down on the ver its sec test Ubuntu release 18 30 years of Linux How a 2l-year-old’s bedroom coding projec took over the world 24 Inside the Linux kernel How did Linux come to be? What makes it tick? We answer allthis and mor 32 Build the kernel Its the ultimate nerd credential, tune up y' wn personal kernel, here's how. 40 Rescatux repair Explore one of the most famous rescue and repair systems powered by Linux. 6 | The Hackers Manual 54 64 68 Protect your privacy eats are out there and what Secure chat clients online without anyone snooping in on what you have to say. Lock down Linux We outline the essentials of lacking doy your Linux boxes for secure networking, Data recovery Key management earn how to create a good GnuPG key and pit safe from online thieves.r the most powerful Linux fe and get using it. 78 OpenELEC Get to grips with the media system for desktops and embedded systems. 82 Virtual Box Ensure you get the best out of your virtual systems with our essential guide. 86 NextCloud The break away, all new cloud storage and document systems lve for al 98 Nagios Industry-level system monitoring so you can track all your Linux PCs. The terminal Feel ike a 1337 hacker and get to grips ith the powerful terminal 126 Get started The best way to use the terminal sto dive in with both feet and start using it. 128 Files and folders We explain how you can navigate the file system and start manipulating things. 130. Edit config files Discover how you can edit configuration files, from within the text terminal 132. System information interrogate the local system to discover all of its dirty little secrets. Contents Hacking Take your Linux skills to the next level and beyond. 96 Hacker's toolkit Discover the tricks used by hackers tohelp keep your systems safe, 104 Linux on a Linx tablet Get Linux up and running ona low-cost Windows tablet without the hassle. 108 Multi-boot Linux Discover the inner workings of Grub and boot lots of OSes from one PC. 112 Build your own custom Ubuntu distro Why settle for what the existing distrubutions have to otter? 116 LT Tng monitoring Get toknow what all your programs are up to by tracing Linux app activity 120. USB multi-boot ‘We explain how you can carry multiple distros ona single USB drive. 134. Drive partitions Control, edit and create hard drive partitions and permissions, 136 Remote access Set up and access remote GUI applications using X11 138. Display contro! Sticking with the world of XI1 we take some randr for resolution control. 140 Core commands 20 essential terminal commands that all Linux ‘web server admins should know. ‘The Hackers Manual | 7eer me) Cece ee eee 2 ce peoecien eee ee eee Co eee oe a al cfeisemeaiee SE emer Bre etacn : : e cnr t horpryreerer eo molec eed euet Tiley oP UE in Ioan esac ad P | | if @task update_attributes(params/:task)) forma Cet cee jon: = SOIR ee eet ena Se Soe ede Pysa : Se eae Ler ind Ce REL se sa a aa a ton ae Oe n- MS rpeanre pe cin cen i Ge ren rence rig NRO ng eRDistros Because if there was only one form of Linux, we'd be bored Ubuntu 22.04 Get the lowdown on the latest Ubuntu release and discover its secrets. 30 years of Linux How 2l-year-old's bedroom coding project took over the world Inside the Linux kernel How did Linux come to be? What makes it tick? We answer allthis and more, Build the kernel It the ultimate nerd credential, tune up your own personal kernel, here's how. Rescatux repai Explore one of the most famous rescue and repair systems powered by Linux. ‘The Hackers Manual | 9Bullet-proof Ubuntu 22.04 Walpurgis Night is nearly upon us, so cast aside your old OS and begin life anew with Ubuntu 22.04, says Jonni Bidwell. ee ee enone eee ee Pave ne eer ae ee a ee Perr ee ee ewes ge et eee ee Mee ee ees cor eee ured Be eee ee eee eo Seto nt ee eee ee pc Ne re Ct me erat artist ar Lo ae nett ee ee ee ey Pee ea Cee ee ee Me eee ere OIC este cra that reputation ofstabilty.butitvasrt Debian, understand open source software and _usngit for fve whole years. You won't find Iwas something special, ere eta ee 2 Pe ic ‘flack when it abandoned Unity and Ceo ees Canonical which promised thatit would the Ubuntu Phone. 7 ee eec aes Pewee Eiiese ie tenn a Cosmic tns ee te nt eee ca eer eee mea cad share” For mary, Linux's golden age vias Proton has enabled some: pense ett pert Ct ess) abr) i Flash forward to today, and youll see s that not all of those dreams came true. bc orien a 4 racer lomemeceieete td Rt Nebr te se) wiole is becoming a niche hobby, becausé — managing great deal of our browsing and eaBullet-proof Ubuntu 22.04 Of jams and jellies It's Ubuntu LTS time, so let's see what will be the shape of Ubuntu for the next few years... ¢ aay ok fovardtotryngout ane W iiticess sina tneocintre rt expatng ait reverted destop para rhe protrance ep The porous Crurt LS, Foea Poss ater evenly ody begins has been oa servant to many fur cnathnes avers surely Jyh ite a Worthy succtsor We'eloong foward to Gore 4 though there are some ose eso ere releases). 2 mre patched Wayand experencs anda wrk se how Canarias pushing head wt Sap Fiat There's oni one problem Atti of wring thas besnreleaed But hat okay because bey the time youread tis Arif wee uly tere we wot have missed ary atrnut addons or ures, We've been testing the daly ary Jlysh mages or 2 couple of months por tothe fal lease Minor niggles, begone! ‘be aware that there's light and dark: theme, and you'll suspect (rightly) that these have been ‘And we've seen quite abit of change in that time, particularly as parts othe recently released Gnome 42 start to find thei way in. Indeed, as we write this we'e stil ‘about a week away from launch day but since both the Feature and Ul Freeze have passed we don't expect any drastic changes. We do rather hope somne minor niggles (such as stuttering and occasional crashes while dain windows between monitors) get sorted cout. though. if you've used either of the interim releases (21.04 or further tweaked. To be frank, if you've been using Ubuntu 21.10, then there's not anything ground breaking in 22.04. But that dosnt mean you shouldn't upgrade. You should, because if nothing else your interim release is about to be EOLd. Oh. and if you're of the ik that gets excited by phases lke'modern design ‘trends; then check out the new logo. I's similar to the (ld Circle of Friends logo, but on a web3-friendly (stop baiting sensible readers! ~ Ed) rectangular background, I you just want to see what Ubunty is like, there's no need to instal tall Just follow our handy three-step guide to downloading, writing and booting an Ubunts USB stick or DVD i you must, 21,10) since the lasts LTS then youl be aware that now Ubuntu uses Wayland and (maybe) remember that Active Directory can be set up from the installer. You'l B Boot Ubuntu Your machine might enableyouto bringupa ‘boot men by pressing F12 or FIO atboat ‘etcherio)to write theimage toa USB stick. time. Ifsouse this tone time boot from the Don't remove the medium untilyou'e told Ubuntumedium. Otherwise youlineed to go i's safe todo so, Bad media will cause into the UEFV/BI0S setup interface and problems later. You could alsa (using change the boot order See the oficial docs at ‘iferent software) make DVD, but this will https-//ubuntu.com/tutorials/try-ubuntu be slower than using flash media, before-younstall. Download an ISO Go tohttps://ubuntu.com/desktop anc download the|S0 fie. I's 35GB soyoumay want o fetch acupf teafor while youweat youre intrested in trying another favour, suchas Kubuntuor Lubuntu, yun inks athttps://ubuntu.com/download/lavours. ‘You aso find links to the Server, Pand Core edttions her, B Write out a USB stick Use your favourite image uniter (or ‘download Balena Etcher from https:// ‘The Hacker's Manual | 11Escape Windows Whether you're a complete novice or Windows has driven you to seek out other operating systems, Ubuntu can help. indows has been oe out though the Wi sit icpamsres ste at indo 10 ses wi nave been flere the parade nal elood bythe tine youre reading ts There's nothing ike anew Windows lease for ratatng people to sth Linux So her's ick tiie oral of you recat Windows Spots Youmay be tempted dua boot Windows and Lunt together Ths might sound omenint butts ‘ehh pas sore some tous, Ubu wii lonsde Windows, bul tee’ teint Con the ine WindowsUpdate wi, ona wn, deed the Linxparto() solange necessary. Fortis reason we don commen nstaling both Sesto the same dvce 25068 S80 i el or you frst Linux ‘xperatons an youcan gel hs eo around £25 hr ext pride i of gare perhaos overly cautions, anda ite inconvenient, but its the only way to be sure Window's won't touch your Linux: unplug the ‘SSD prior to booting to Windows. Yup i's hard to remember and potentially avkward to carry out (if your ‘case is under the desk, say), but at least you can describe your install asairgapped from Windows! ‘You might instead want to install Ubuntu to an ‘external harddrive or USB stick, though if you're not using USB3 storage this wont be terribly performant, Ideally, you'd put ton a whole new machine, but not everyone has a spare, working machine, BIOS, meet UEFI Modern PCs use a newer rmwar, the Universal Extensible Firmware Interface rather than te traditional BIOS of yore. UEFI machines may havea classic IOS ermuation made, but you almost certainly shouldnt Get to know Ubuntu’s Gnome desktop Activities Clckon here press Super (Windows) to bring up the Activities Overview. This willshow youpreviews ‘of open windows, which you can drag ‘over tothe right. tomove them to ew virtual desktop. siteher here too, Dock Thisprovides easy access to popular programs. Running appications are Fost by areddot Rightelck to nr un applications from here. 12 | The Hacker’s Manual ‘Applications grid Ths deoys alleen staed applications pe ew charctersto narrow down the st Youcan asotnd moths way, they festyourbost, Onandthores arta desktop Desktop options Right-click tochange either the background or csply settings. You canals create desktop folders. Status icons Network, volume and power incicators. Cickto access Bluetooth, brightness and (forlaptops) power profle settings. The logout and shutdown options are aso her, Notification area: ‘Aerts, suchasnew softwarebeing avalableornewmedathat beng played areshawnhee, There's ‘alendartoo~thscaneither be used localy or connectedwithacloud service.enable it, Especially if you akeady have OSes installed in UEFI mode (they wil stop working). Note that PCs these days can be Incredibly fussy about getting into the UEFI setup or summoning a ‘boot menu, Precision timing, ‘multiple reboots, as well as digging ‘up manuals to find the appropriate shortcut key may be required. The Ubuntu EFI boot capsules are all signed by a Microsoft endorsed key. so there should be no need to disable Secure Boot, One thing you shouldbe aware of 's that one EF partition i required toboot a UEF! system, Sof you plan on installing Ubuntu on a separate drive, make sure the “Device fr bootloader installation” is set to the Original drive, and thatthe EFI partition s selected, This, drive wil nee to be pluggedin to boot ether OS, but youl be able to choose which from the UEFI (Once you've successfully booted Ubuntu, you'll be ‘asked whether you want to try Ubuntu or jump right in {and install it. We'd suggest trying it fst. f you haven't ‘already. This enables you to get a fee forthe operating system without it touching any of your storage. So you can try out bundled software, install new things and see fit’ right for you. The only downside is that it won't quite be as performant as the real thing. Oh and any Changes you make willbe lot after a reboot, of course ‘The annotation (below leff) shows you the rudiments of Ubuntu’s Gnome desktop, ust as in other OSes you'l find folders for your Documents, Phatos and Downloads. But uniike a least ‘one other OS you won't be bombarded with marketing, > Ifyou know what you want, partition wise then the ‘Something Else option nthe installer wil help. Become a Keyboard warrior In an age of QHD sereens and 4K displays, the mause cursors pilgrimage to the top-left corner can be an arduous ‘one. This journey can be saved through the magic of keyboard shortcuts. Apart {rom the all-important Super (Windows) lettrignt key to bring up the Activities view, your life in Gnome may be improved (May? — Ec) with te folowing knowiedge: CCtrl+Super Left/tight » tie window ‘Super+A2 show applications grid Bullet-proof Ubuntu 22.04 Cor voice assistants trying to help you. The Dock area on > Don't know the left-hand side is anod to Ubuntu’s old Unity what to install desktop, The new desktop has been based on Gnome 3 first? since Ubuntu 18.0, but with some usability beaks, Let the Snap Gnome 3 was seen by some as too ambitious, Stor nei ou ‘occasionally buggy, and a memory hog when it was get ly buggy. y hog iets introduced in 2008 (this commentator even used the ‘phrase “hypermodern”), but these days the fact itforms the basis for so many desktops is testament tots Using a rock-solid gnome “The new desktop has been based on Gnome 3 since Ubuntu 18.04, but with some usability tweaks.” solcty I you imagine the dock was gone youll see What alt of traditionalists’ main problem with Gnome 's: There'sno obvious menu to launch applications. The applications grid provided by the Dock isnt quite the ‘same thing, but if you're inthe habit of using a mouse to ‘open traditionally placed applications menu, then your ‘muscle memory wll mare likely bring you here than to the Activities view (On a clean install the dock area has shortcuts to Firefox, Thunderbird and LibreOtfice Writer. The {question mark icon will tke you to the desktop guide, which hopefully answers any questions you may have. You'llso find links to Ubuntu Software (the shopping bag icon), in case you want to install more software. a5 well as the venerable Rhythmbox music player. Internal ‘and external drives wil also show up here, plus there isa Rubbish Bin from whence deleted files can be retrieved. ‘Super+PgUp/PeDown > switch virtual desktops Shift+Super+PgUp/PgDown » move window to prev/next workspace Shift+ Super Left/Right move active window to prev/next display 4 ‘The Hacker's Manual | 13Distros Customise Ubuntu Discover new software. Change settings. Install a new desktop (or three). buntu (and most other desktop Linux flavours) [ J have been designed tobe intuitive and easy to learn. However, sooner or later you'll probably want to change some things around, For example, we think Rhythmbor is great. It's been the default music player in Ubuntu since the very beginning (with only a brief sabbatical while Banshee took its place in 2011), But with ts Client Side Decorated window i looks. dated, and cannot connect to popular (albeit proprietary) streaming services so we might want to look at alternatives. By this point we're assuming ‘you've installed Ubuntu, and enjoyed its new look Flutter-buit installer Fire up the Ubunt Software application, soll dawn to thelist of categories and select Music and Aucio,You'l see a selection of audio programs, most of which we've Desktop choices There are mulipe favours of Ubuntu 22.08 that inde the same rock sli foundation asthe flagship but witha diferent desktop environment. Iyoulke Lunt but don ike medern Gnome, hen Kubuntu, Ubuntu MATE (npr bbyGnome 2) o thelightweigt LXQt- powered ubuntu are well worth your time, But rather than instal a whole new *buntu you might prefer oust ade a ‘ew desktop to the curretinstalation. Ths unlikely to break anything but the session packages we'linstallinclue each desktops core appcations. So \youright end up with two (or more he managers, text eitors andthe ke. ‘Some desktons come wth their on login manager too. So for example t ‘youlnstall te Kubuntu-desktop package you ibe asked you vat to stk vith Gneme's GDM3 or switch to SDDM (which s bul using Qt so looks more DEI), There's no right or wrong answer, anc you can change your mind with sudo dle reconfigure m3. The other deskton package are named ‘iar, there's ubuntu-mate-desktop and wubuntu-desktop, Most of these havea more simmed-down version - for example, kubuntu-core vill stall amare minimal et of appications > Instaling the whole Kubuntu desktop package makes for a ment ‘that, unsurprisingly, sich in tems that begin with K. 14 | The Hacker’s Manual never heard of You will, however, ind theofficial Spotty and Audible programs as well as unofficial players for Deezer YouTube, Google Play Music and Apple Music. If youprefer something even more nostagi, you'llalso find Foobar2000, DeaD Beets (a minimal GTK player ‘and glorious hex reference) as well as myriad text based music players. Install Spobfy(or whatever else takes ‘your fancy) by hitting the green button Most applications inthe Software application are shipped as Snap packages. You can see the delivery mechanism in the Source box inthe topight. Snap is Canonica's self-contained packaging format which (ike Flatpak, which isa similar etfort) enables developers toeasly ship software without having to worry about distro-specifc packaging and which versions of which libraries o ship. Snaps aso run ina confined sandbox (unless you give them permission to otherwise) so they ‘can't access any files or hardware they don't need to. Life on the bleeding edge Occasionally in the Source box youl see a variety of dierent ‘channels’ are available fora given Snap. These often enable youto grab abeta or development release, incase you want to lve onthe bleeding edge System packages are sil nstaled as. deb packages and there arestil tens of thousands of these traditional packages you can install from the command ine with Apt These ro fonger show up inthe Software application butt you install Synapticyou can browse these graphically Canonical has put alot o fr nto making sure popular applications are avaliable in Snap form. Besides ‘Spotify youl ind Telegram, Slack Blender, GIMPand the FB Pete covers sack © © Am ad blocker and Mozilla's container programs are ‘essential for the modern Web, And switching the default search to DuckDuckGo.> What’s up, dock? Here we've put the Dock atthe bottom, removed various cutter, and made it shorter. PyCharm development environment for Python. There's ‘also open source versions of some classic games, including Prince of Persia(SDLPOP), Qpen Jed! Knight ‘and Widelands (a Settlers clone). ‘Abig change in this Ubuntu outings that Firefoxis only avaliable as 2 Snap. This comes directly rom Mozila, saving Canonical a packaging burden (and forcing derivatives such as Linux Mint to build and package their own Firefox) nour testing, there was a delay of about 0s each time Firefox was started from {clean login. This is milly annoying since the web ‘browser soften the fist thing one opens postogin. but hopefully Snap startup times willbe worked on in future. ifthe slow stating Firefox (or Snaps in general) bother you, then you can always use the Mazila PPA to installa traditional package (see https://launchpad. ‘net/~mozilateam/ ‘archive ubuntu/ppa).Or ‘download a tarball from its FTP ste. Or you could switeh tothe ather side of the madern packaging formats {debate and instal Flatpak and Firefox from the Flathub. Your fist act wil ikely be to install Block Origin 2s well ‘as Mozlla's Facebook Container and Multraccount Container adc-ons, ‘Aurile ago we looked at how Firefox worked on Ubuntu 21.10 (and Fedora 35), and found that the Snap version det work at al with VAVAPI video acceleration, Happily, we were able to get it working in the new version, though some extra configuration is required, Goto about:config (noting the warning) and set media ffmpeg.vaapienabled to true Later you may also want to ‘et media navigatormediadatadecoder_vpx enabled as wel which will accelerate WebRTC (for example, Zoom Taams, is) sessions. In our testing (in Firefox 98, 99.and 100by way of Snap channels) we had to disable te RDD ‘sandbox to make it work. Since thisis @ security risk we ‘won't tell you how to doit here (but we'e sure you can DuckDuckGo it), In that feature we also saw that both Snap and Flatpak versions of Firefox and Chromium and Edge) can't handle extensions which use Native Messaging, Thisis stil true, so password manager extensions (as ‘well as things like hardware authentication tokens) don't currently work here. Both packaging formats should soon see a host messaging portal soon, but unt then these add-ons will only work with rationally packaged browsers. Ona related tangent, KeePassXCinstalled as a ‘Snap (or Flatpak) willintegrate with such browsers, but you'll ned to run a script, as described on its website, ‘Another consequence of contained browsers is that the old https://extensions gnome.org (EGO) website won't work correctly Even f you install chrome-grome- Bullet-proof Ubuntu 22.04 ‘shell andthe browser plugin, That's okay though, for now you can use a third-party tool, such as Extension “Manager to do this. Ths tools available as a Flatpak. so ‘well need to install that and set up the FlatHub repo first ‘You might want todo this even if you dont care about Gnome extensions since it gives you a whole other avenue (and tool by which more software can be accessed. So open a terminal and un: $ sudo apt install fatale gnome software plugin fatpak gnome-software § sudo fatal temote-add ~itnot exists fathub https! Alathub orgtepofiathub fatpalrepo ‘This add support for Flatpak programs in the Gnome. Software GUI, aso installed by the frst command. So you'll be able to search for Extension Manager thereafter reboot. Note that Grome Softwares distinct from the usual Ubuntu Software tool. I's just called Software and has a shopping bag icon. Alternatively, f you're enjoying the terminal the required incantation for installing and running (sans need to reboot) is $flatpak install commatfakeman ExtensionManager $ flatpak run com matjakeman ExtensionManager You'll see that Ubuntu uses three Gnome extensions (for desktop icons. appindicators andthe dock) and that ‘woof these can be configured. And if you navigate to the Browse tab you can find many mare. You might already have some favourite Gnome extensions, and hopefully most of those have been updated to support version 42, Extensions Manager will display ‘Unsupported if not. The popular “Blur my Shel” is available. Likewise GSConnect. a Gnome-centric take on the popular KDE Connect utlty fr talking to your phone from the desktop. ‘The shortcut baron the lft sn’t to everyone's taste, although fans caimit is more efficient than having it along the bottom. You might prefer to get rid oft altogether and make the desktop more like the varia Gnome you'd find inthe ikes of Fedora. Wherever you ‘want your dock, it canbe configured by starting the Settings application (either from the menu at the too Tight or from the Activites Overview) and navigating to the Appearance section. The screenshot shows siahty more orthodox Ubunta arrangement. except there dossnt seem tobe awayto _canmale move the Appleations rd shortcut tthe eft, which is -Yourvaious were traditionalists might prefer to nde thing wich Workepaces closely resembles acassical applica The workhow sou most closely resembles a classical apaication menu. The Work ew you new DarkTheme (which now should wok universally) MaMREhem tO can aso be enabled irom the Appearance section. one The Hacker's Manual | 15Distros Tweaking and rewiring Some final edits to perfect your installation, plus a little Ubuntu nostalgia. ayland by default was tested in Ubuntu 1710, but [ J that was perraps a tle ambitious. Now the technology has matured and Canonicalis, confident that's ~to dredge up an irksome phrase ~ ready for prime time" Extensive testing has taken place and the team are confident thatthe Wayland experience willbe good for all, Yep, even those using Nvidia harcvare. Hit snot, wel that’ fine, Te old XII session is stil there ‘Wayland has been faity misrepresented in the press, (who, me?-Ed) historical. The most egregious falsehoods were that remote desktop sessions. screen sharing and even humble sereenshatting ae impossible with Wayland, Do not believe such myths. The problem wasn't Wayland, twas programs that didnt support. Al the soreenshots inthis feature would not behereif that were the case. We're in Gnome’s golden age “The stutters and memory leaks that dogged Ubuntu Gnome’s performance for so long are well and truly gone.” ‘One change mulled for 22.04 but which in the end ‘ever made its the replacement of PulseAucho with PipeWire. The latter isa whole new multimedia framework wich. as it happens, enables desktop sharing and screen recording on Wayland. Programs may sil depend on PipeWire (particularly web browsers), but venerable PulseAudio remains the default sound server. I you want tachange this (for example ifyou are having dificulty with Bluetooth headsets), you can instal the PipeWire session with > One thing that was quite hard to screenshot (but for once not because of Wayland) was the new screenshot tool Oh the irony! 16 | The Hacker's Manual ' sudo apt install pipewire-pulse. Then i you log out and back in and run the command: ‘pact info you should see (among other lines) ‘Server Name: PulseAudio (on PipeWie 0322) ‘Additional libraries may be required for some Bluetooth audio codecs. Try ‘sudo apt install Iibiak-aace libldacbt-fabrenc}2 libopenapixo if yourun into dificutes. Alternatively, seek more upto: date documentation, we ae unfortunately static! Ubuntu has used Gnome as its default desktop since 1804 LTS. you pine forthe Unity desktop, then you might be interested in the Extended Security Maintenance (ESM) that's avaiable forthe previous LTS, Ubuntu 16.04 (Xenia Xerus), The oficial support period {or this expired in May 2021 but since this versions stil widely deployed Canonical offers paid-for support to ‘organisations. Tis fs achieved through its Ubuntu Advantage for Infrastructure program. Personal users are allowed ESM on upto three machines for fee, sof you want to keep the Xerus alive you can now do so ina safe {and (semi) supported manner. ‘We were feoing nostalgic, so we fred up Ubuntu 1604 ‘on our XPS. This hadt been booted for some time, and had problems seeing our nave fangled USB-C dock (or the network cable plugged therein). But once we'd updated i, enrolled the machine in Ubuntu Advantage and updated ‘again everything worked more or less fine, Don't et anyone tell you nostalgia isnot a good reason for running (ld software. Especially when you'e entitled to run tree instances for your own pleasure. I you were looking for ‘actual phone and ticket suppor, then this starts at $150/ year fora single desktop installation or $750/year fora ‘server It's not really intended to help beginners get their printers or WiFi working Ask rcely on httpst// Lubuntuforums.org or https://askubuntu.com for that sort of support ‘Booting back into the new release was much ‘uicker and smoother by comparison. which to be honest you'd expect after si years of Ul development. This release might not have the kind ground-breaking features that we used to erioy, but that’s probably a good thing. ll those features and breaking changes we used to love veto 15-years ago were a 4 ‘consequence of desktop Linux stil being rather new. Now that Ubuntu’ desktop is established, like itor not it doesn't make sense to go changing it Instead, we should take comfort inthe fact that after four years of using Gnome for its flagship desktop, t ‘experience is now second tonone. The stutters and memory leaks that dogged Ubuntu Gnome’s performance for so long ae well and truly gone, Acommon Theme Geme themes have come under the spotlight since the introduction of GTKé (inaugurated with Grome 40), Did we say themes? Ah, we meant theme, because custom theming of Gnome applications snow verboten The default GTS theme was called Adwata,a Sanskrit word often translated asthe ol oe; tray ‘not two), But it wasn really the only one, because developers could happily write their own CSS stylings. InGTKA this theme has been promoted toa plato rary Iadwaita wich Gnome developers say wil guarantee conformance wit ther Human Interface Guidelines. And (ike the characters often say in Fighinder ther canbe ony one, GTK3 applications wil stilrespect custom themes, but GTK4 ones ill only suppor thelimited changes (for example, backround and acent colours) permitted by Wadwaita For Ubuntu 2204 ths might be bad news, because at presenti uses amix of Gnome 42 applications {libadwaita based) lus some from older releases (such as Fes which s based on GTKS and andy) This tmay change pir to release ctherwise there are going to be some cosmetic inconsistencies this bothers you then you might want to runaway from Groen 42 for the nettle while, in which case there are some suggestions inthe box see below), The old Grome Tweaks tools til avaiable inthe repo, utlke the EGO webstit can no longer manage Gnome extensions, Tats okay, because ican do mest @ mic everything else. including cleaning up the mess our ‘Gnome fonts ended up in post installation of KDE. Plasma, Tieaks also enables you to manage startup programs, change ttlebar button visibility (or move them tothe left, MacOS style) and adjust legacy theming You Ccaninstall Tweaks with: ‘sudo apt install gnome tweaks This will install a diferent Extensions tool, curently in beta form. At the time of writing this doesn't et you install new extensions, otherwise we could do away with ‘the previous tool, For even more tweakabiity try Just Perfection, found in Extension Manager It allows for parts ‘ofthe shell theme tobe overruled (including removal of the top bar) to make matters more minimal. We don't go {for Gnome extensions ourselves (despite having two programs for managing them) let us know wat we're missing out on. Enjoy Ubuntu 22.04! Bullet-proof Ubuntu 22.04 > Menus in ‘tlre. Amazon search results in| the HUD, Ubuntu 16004 had some crazy ideas! Looking elsewhere’ Latterly there seems to have been ait fa tren for Unuxcleaning social mea chanel o announce they/re“no Inger recommending bunt orathe such things. Reasons ae varied we suppose but the ‘umuirate of Snaps. Wayland and Gnome dent seem to be to everyone's taste. ‘Wed stl hearty recommend Ubuntu to anyone, begin a ‘otherwise ~ 35st works Evenfyou don Ike, as weve seenit ‘an be customised estended or orice bashed around to our taste Los ofthe divas these channels recommend in Ubuntu’ stead ‘are themselves based on Ubuntu for example Linux Min, Pop_OS ‘and Elementary OS. Al grat estos that fer something which shard torereate on Ubuntu Linx, but uiimately distros that depend on ts packages inrastructure and documentation Unt now: perhaps. nt’ latest Debian Exton (LES) s api gaining traction. Fopl_OS has moved is PPA repostories away om Launch and is working on new Rust powered desktop erwironmant (wth view to moving away trom Gnome) And Elementary OS has had its own app store fra while anc has ikewse sided with Flatpaks oer Snaps. ‘summary. Ubunty doesnt dt fr you thee are plenty of derivatives you can ste to without having to learn a hole new way of working Were excted to see more people trying Fedora i's now more accessible, parteuiarly as regards instaling non-free software Together with its api release cele this makes it» great platfor er gaming Wel werth checking cut if bunt sro longer serving you > Fedoras and the distribution of that name are ll he rage right now. ‘The Hacker's Manual | 17Celebrate _ a0 \( in z Ak i x ss uj nr l How a 21-year-old’s bedroom coding project took over Runt N Vendo R= laleR-MNTMOLUn mtn ln -oe-lONUNR ANA me30 years of Linux inux only exists because of Christmas ‘On January 5, 1991, a 2-year-old computer science student who was curently ling with his mum, ‘rudd through the (we assume) snow-covered streets of Helsinki, with his pockets stuffed full of Christmas git ‘money, Linus Torvalds wandered up to hs local PC store and purchased his ist PC. an Intel 386 0X33. with 4MB of memory nd 3 40M hard rive. On this stalwart machine he ‘would write the fst eve version of Linux, From this meament ‘on. the history of Linux becomes a love story about community collaboration, open-source dewlopmert, software freedom and open platforms, Previous to walking into that computer store, Linus “Torvalds had tinkered onthe obscure (UK-designed) Sinclar QL (Quantum Leap) and the far better-known ‘Commadare VIC-20. Fine home computers, but neither was going to bith a world straddling kernel. Aboy needs standards to make something that willbe adopted worldwide, and an IBM-compatible PC was a perfect place to start. But we're sure Torvalds’ mind \was focused more on having fun with Prince of Persia, at that point than specifically developing a Microsoft: ‘conquering kernel Let’ be clear: a 21-year-old, barely able to afford an Intel 386 DX33, was about to start a development process that would support a software ecosystem, \which in turn would run most of the smart devices in the world, a majority of the internet al ofthe work's fastest ‘supercomputers, chunks of Hollywood's special effects industry, SpaceX rockets, NASA Mars probes, sel driving cats, tens of millions of SBC lke the Pi and a ‘whole bunch of other stuf. How the heck did that happen? Turn the page to find out, “A 21-year-old, barely able to afford an Intel 386 DX33, was about to start a development process that would support a software ecosystem. ” ‘The Hackers Manwal | 19Distros Pre-Linux development Discover how Unix and GNU became the foundation of Linus Torvalds’ brainchild. understand how Linux got started, you need to ' I Tcl Ute Bete teas ae established operating system standard through the 19606 into the 170s. twas aready powering mainframes bul by the hes of BM, HP and ATA. Wore not aking smal ‘rythen~ they were mega corporations sling their products ‘round the gabe. If we look at the development of Unix, you'l see certain parallels with Linux: freethinking academic types who were given free rein to develop what they want. But Whereas Unix was ultimately boxed into closed-source Ccorporatism, tied toa fixed and cwindling developrnent team, eroded by profit margins and lawyers’ fees, ‘groups that followed Linux embraced a more strict > Ken Thomas (left) and Dennis Ritchie are credited with largely ereting much ‘ofthe original UNIX family of operating systems, while Ritchie aso created the language. ‘open approach. This enabled free experimentation, development and collaboration on a worldwide scale. Yeah, yea, you get the point! Back to Unix. which isan operating system standard that started development in academia atthe end of the 1960s as part of MIT, Bell Labs and then part of AT&T. The italy single or uni-processing OS, spawned from the Multics OS, was dubbed Unies, wth an assembler, editor and the 8 programming language. At some point that "es" was swapped to an", probably because it was cooler, dude. ‘At some point, someone needed a text editor to run ‘on aDEC POP-I1 machine. So, the Unix team obliged and developed ro and troft, the fst digital typesetting system. Such unfettered functionality demanded documentation, so the "man" system (still used to this day) was created with the frst Unix Programming Manual in November 1971. This was alla stroke of luck, because the DEC POP-1 was the mast popular mini ‘mainframe ofits day, and everyone focused on the neatly documented and openly shared Unix system, In 1973, version of Unix was rewritten in portable C, though it would be five years until anyone tried running Unicon anything but a PDP.I. At this point, a copy of the Unix source code cost almost $100,000 in current ‘money to licence from ATET, so commercial use was limited during the 70s. However, by the early 80s costs had rapialy dropped and widespread use at Bell Labs, ‘ATE, and among computer science students propelled the use of Unix. twas considered a universal OS standard, and inthe mici1980s the POSIX standard was proposed by the IEEE, backed by the US government This makes any operating system folowing POSIX at least partly f not largely compatible with other versions Linux runs everythin Developing software fo supercomputers is expensive, During the 1980s, Cray was spending as much on softvare evelopment ast was ont hardware In trend that would ony gra, Crayntily ‘hited to UNIX System then BSD based OS, and eventual, n 2004, SUSE Linc to pom it supercomputer. This was matched aeross the sector. and the top 500 supercomputers (ww topS00.0rg) rnowallrun Lin, Internet services have lo allbeen developed to un on Unicaystems. Meosoft ‘and B50 systems do retain a good sie of ‘serces but ever 50 percent of web servers are pomered by Linux Recent moves to vitual series with container based deployment are 20 | The Hacker’ Manual Linubosed. Merosoft's cloud Service Azure report that Linuxisits largest deployment 0S and, more to the point, Google uses Linxto power most of ite series, 28 many other service suppliers aka AWS, ‘Androids mile OS share dropped in 2020 toust 84 per cont its powered ty Lun Google bought the startup that was ‘developing Anco in 2005, LineageOS (https:/lineageos.org) swe ‘maintained fork of eid and supports most, popula handsets wellatter their ‘manufactoers abandon them ‘Space was tought tobe Linux’ fal frontier because isnot a erties ‘eterminste OS, which the gold standard foc real-time OSes in misslon-rtical situations Turns out that SpaceX rockets use Lunuxto power tha fight systems, using 2 ‘ple redundancy system, while NASA has sent Linux te Mars ins helicopter drone, Ingenuity Tes aso reportedly running Luin ts crs, [LUnux ha also been a the heart of Holywood special effects since 1997 Titanic used a nu server farm of DEC Alphas at Digtal Drain to create its CL Dreamorks' Sek in 2001 was the fist fam ‘that was entirely created on Linux systems Meanwhile, Psae ported its Renderman system to Linu rom SGI ad Sun se ‘around 2000. ntime to produce Fing Nemo n 2008,[At the end of the 1980s, the Unix story got messy, with commercial infighting, competing standards and closing off of standards often dubbed Unix Wars. While ATG, Sun Microsystems, Oracle, SCO, and others argued, a Finnish boy was about to start university. We GNU that Before we dive into the early world of Linux, there's another part ofthe puzzle ofits success that we need to ‘utn place: the GNU Project, estabished by Richard ‘Stallman, Stallman was a product ofthe 1970s development environment: freethinking, academic, hippy type. One day, he couldn't use a printer, and ‘because the compary refused to supply the source ‘code, he couldn't fix the issue - supplying source code ‘was quite normal atthe time, He went apoplectic and established a free software development revolution: an entire free OS ecosystem, free software licence and philosophy that's stil going strong Take that proprietary software! Linux Format Interv ‘the GNU free software movement, in 201. 30 years of Linux Linus Tvalds being interviewed by Linux Format, back in 2012. The GNU Project was established by Stallman in 1983, with GNU being a hilarious (to hackers atleast) recursive acronym for "GNU is Not Unix” Geddt? Its ‘aim was to establish a free OS ecosystem with all the tools and services a fully functioning OS requires. Do keep in mind that most of the tools created then are still being used and maintained today. By 1987, GNU had established its own compiler, GCC, the Emacs editor the basis ofthe GNU Core Utes (basic file manipulation tools such as ist copy delete ‘and so on), a rucimentary kernel anda chess engine (Gee LXF273). But more importanty, Stallman had ‘cemented his deal of software freedom with the 1989 “He established a free software development revolution: an entire free OS ecosystem, free software licence and philosophy that’s still going strong.” “copyleft” GPL software licence, and his manifesto setting out the four software freedoms enabling users torun, study, modify and distribute any software — including the source ~ for any reason The GPL remains the strongest copyleft icence, and ‘while ithas perhaps fallen out of vogue. i's stil regarded 2 the best cence for true open-source development, ‘and cements mast Linux distros. GCC still an industry standard, Emacs remains a feature ich development environment, and the GNU Core Utities are stil widely Used in certain POSIX systems and most Linux distros. ‘You could argue that without the GNU Project being established, Linux would never have taken off The GPL licence (adopted early on in Linux development) forces all developers to share back their enhancements to the source code. I's feedback loop that promotes shared Improvements, Alternative open-source licences enable corporations to take source cade and never share back improvements, meaning the base code is more likely to remain static. This was backed by a generation of ‘The Hackers Manwal | 21Distros >We have to mention Tux, the mascot of, Lr, because a penguin once bit Linus Tue story! developers that grew up studying and using Unix, looking to contribute toa truly freed open-source OS. Let's all Freax out! We're getting ahead of ourselves. Linus Torvalds had his Intel 386, was studying computer science at the University of Helsinks, and was using the MINIX 16-bit (5 and Kernel. MINIX is @ POSIX-compatible Unin-ike (0S and miro-kernel. In 1991, it hada liberal ence, costing just $69, offering the source code but restricted ‘moaification and redistribution ‘We imagine the 16-bit imitation spurred Torvalds to create his own 32-bit kernel, but he states the licence restrictions were also key. So, on 25 August, 1991, he posted to compos. rminix that he was developing his own free OS, He said that it was "nothing professional ike GNU’ anditd only support aT ‘disks, as that's all he had. This was developed on a MINIXsystern, compiled on GNU GCC, and he'd ported ‘GNU bash, Torvalds had planned tocallhis OS Freax, combining Free" "Freak," and Unix” but once he'd uploaded it toftpfunetsfi ‘volunteer admin (Ac Lemme) renamed i Linux, as he thoughtit was better. So, version 01 > Minx for all ofits creator's protestations to its superiority has ceased development, even though itruns Intot's CPU Management Engine. ‘of Linux was released to the world in September 1991, One teling part of the release notes states: "Akerel by itself gets you nowhere, To get a working system you need a shel ‘compilers, a library, ec...Most ofthe tools used with Linux are GNU software and are under the GNU Copyleft. These tools arent inthe distribution ~ ask me (6r GNU) for more info” Importantly, tis outlines Linux’s reliance on other GPLicenced tools, and shows the use ofthe term ‘distrbution” naw shortened to “distro As Torvalds points out, an operating system isn'ta kernel alone: i's, ‘collection of tools, scripts, contigs, drivers, services and a kernel, lumped together in an easier form for Users to install and use, ‘A for the licence, Torvalds intially used his ovn, which restricted commercial use, but by January 1992 he'd been asked to adopt the GPL, and had stated the kemel licence would change to align it with the other tools being used. It was December 1992, and for the release of v0.99, the Linux kernel was GPLv2-licenced, This cemented the legal clause that anyone using the eee! source has to contribute back any changes used in published code. Birth of The Linux Foundation L Development Labs was setun at the tum ofthe mileneium to, arnong other things, get Linuxinto datacentres and carhmunication networks. Tey became Torval's (and his reh-hand man Andrew Merton's) employer 2003. Price otis was employed by Tansmeta, who permitted THE LINUX FOUNDATION Include Facebook, Microsoft Te intel Al 22 | The Hacker’ Manual hi to continue kernel development alongside his other work Fe years previous anater consortium, the Free Standards Group hag ‘ben setup. y 2007 its work was mostly kivng people to smtch to Lira an the two groups merged to form the Linux Foundation (LE) Today te LFs Platinum members nM and wher. contibute (besides the al Armlion dollars required for Platnum status) great deal of code to the Kemal. In 2012, ‘ahen Microsoft wanted to get Linux working conits Azure clo, they were for atime the bianest contributor Besides funding the Kernel. the LFhost hundreds of ther open source projects, nluding Let's Enerypt, the (Open Foundation and the Co Infrastructure litive, which ams to secure he sftware which underpins theinternet ‘Thee’ conferences to, andi’ thanks tothe Lun Foundation that we've been able to vide interviews andcoversee rom the annual Open Source Summit We ook forward to-conterence season resuming. 0 back o the snack bars and cots30 years of Linux Early kernel development Refining the very heart of Linux hasn't been an easy ride over the years... re yuarecent in cane who ato Dy reciente ie. mbehenng aves ater bt ores? Thon spores eur oe ea apes whose beareprs and recive ures bled el cartenporn detop Lina Up ent compare recent ns entre posible tedster monitor Sreue) eat ergtiemon be aa roe Gr? ying tou wane an Riaeat doninmedhan rouniomsioon non il rece eer oto aks of rua re tame stoaseiicesoney trot pred mde mer ven eras Sat pte Xserve tna consred ogee enough td was pete Aon i shicrchermseyevhe owe yerse Sowrie S000 om ined Sout ows Me vag tnt tnd cesta a BOS your age Uurusorweuld po try nr etahtin poss carpet Een yl otastage woud tala opto pps ne OS nodbnt Sacer Ravers Soecbon was ayer cxstert afew eves at habanero Tuc ret werent rection galy som the prea perstel mary were ofthe mist tat € hepleatins Commer OOM Ble Est ew Places Help ol Desktop fuser | 1tem Fee space:86.G8 preferred the DOS way of working, which began to be Sidelined as the millennium approached. Windows users. were having their fles abstracted away ~My Computer epitomises this movement. In January 2001 Kernel 24 was released and with it ‘came support for USB and the exciting new Pentium IV processors. among other things. It was of particular importance to desktop users thanks to is unified treatment of PCI, ISA, PC Card and PnP devices as well ‘as ACPI support. The dot-com bubble was ust about to burst, but all the excitement and speculation around it ‘meant that many computer enthusiasts had a broadband connection in their home, some even enjoyed the luxury of owning more than one computer. User-unfriendly Linux. This sted some majr entry barriers to Linux people could rw downicadt much mor easy, uptodate documentation was easy accessible and when Linux saw fit tocisappear one's internet connection (or render the system Lnbootate, the ther machine cou be used to sok suidance But the user experience was tion the whole woefully inhaspitabe While some instal had eed saphca capabilites these more often than not were mere trouble than they were worth Users were expectedto Lsderstand the ns and ots ot Sun 6 jul, 08:41) ‘The Hackers Manual | 23Distros ‘disk pattioning and beable to discern which packages they required from often ese description. ‘Windows XP was released in October 2001, and while this was seen asa vast improvement over its predecessor, many users found that their machines weren't up torunningit. After alt required 64MB RAM and a whopping LSGB of disk space. Remember that BB|OSes had only recently gained the abiity to address large crves (there were various limits, depending on the BOS, 21, 42 and 8468 were common barriers). So ‘many people couldnt install ton their hardware. and ‘many that met the minimum specs found the performance rapidly degraded once the usual pantheon of ofice suites and runtime libraries were installed ‘This provided the motivation for another minor exodus to Linux, and the retro-hardware contingent ‘continue to make up a key part ofthe Linux userbase (and berate us for not including 32-bit distros). Before 2006 all Macs had PowerPC processors. and many of these (as well as early Intel Macs), long-berett of software updates from Apple, now run Linux too, Gnome makes an appearance The Gnome 2 desktop envionment was released in2002 and this would become a desktop silent that some sil seek whether out frost tam orcurmudgeonly ‘dike of modem altratives) to reproduce it taimed to be as simple. tweakable and intutve. and its rato argue against its achieving allt these adjectives, Oe ofthe major ‘enablers was its strict adherence tothe Gnome Hurnan Interface Guideines, which set out some key principles for _anpication designers, This meant the desktop was consistent not just interaly, but in respect tal the GTX apps that people would goon to write frit ‘Also released was KDE 3, which vaguely resembled Windows ~ in tha it was cosmeticaly similar and lightly more resource-demanding than Gnome. People ‘and distributions sided with one or the other. SUSE Linux (predecessor of openSUSE) always aimed to be desktop agnostic, but went KDE-only in 2009, Today it caters to both Gnome and KDE, Inlate 2002, ‘DVD’ Jon Johansen was charged ‘over the 1999 release ofthe DeCSS software for circumventing the Content Scrambling System (CSS) used on commercial DVDs. This software enabled Linux users to play DVDs, a feat they had been hitherto unable todo since DVD software required licence key from the DVD Copy Control Agency, ane ofthe plaintifs in ‘the suit I later emerged that CSS could be broken much more trivially and Johansen was eventually ‘acquitted. By this time Pods and piracy meant that MP3 files were commonplace. These were dogged by patent issues with a number of bodies asserting ‘ownership of various parts ofthe underlying algorithm, ‘As arresult, many distros shipped without patent encumbered multimedia codecs. The law is murky Big Business vs Linux Being the root ofl ev whenever moneys invohed things can turn nasty. So hen the bigplayers inthe enterprise and business markets began tosee Linux dstros asa threat. lamers were called ‘series flaked Microsoft memes rom ‘August 198, known asthe Haloween [Documents forthe date they were released, etaled Merosoft’s private worries tat Linux ‘and open-source development in genera. was a irct treat its business, along wth ways to-combat is uptake. This private view wasin ‘rect confict withthe company’s pubic ine con the matter though Steve Balmer intarousty called Lira cancerin 2001, The ‘documents are availabe af wcat, ‘org/~esr/nalawwen and in them Microsoft predicted that“Lnux's ontrack to eventually ‘the x86 UNIX market Iwas correct. There was ite Merosot could do tocombat Linus, a it coun tbe Dough ‘The documents suggested extending open protocols with Mlrosots on proprietary ‘extensions (that didnt wort), and seeding the markt with er uncertainty and doubt UD) ‘350 fale, There was another angle however: help a compary that’s suing oer copyright infringement ofthe sourcecode In 2008, a company caled $00 claimed par ofits UND. System sourcecode was being used within Linux, making an unauthorised derwative of UNDG SCO sued IBM for SLbilion (among many other companies). and demandecend users pay a Linux hence fe, Microsoft leaped into action and pa SCO $106 milion, 2s detaledin a leaked and verified SCO meme, After ears of legal arguments, code aut found there tobe no ‘evidence of copied UNI code nthe Lux kemel $CO went bankrupt in 2008, but pats ofthe laws stumble on. o £ @ = = 24 |The Hacker’ Manual SERIO i ee NOVEMBER 1991 5 JANUARY I ;‘though. and rights holders have shown restraint i fling sult against FOSS implementations of these codes. Most distros are prudent and leave it up tothe user to install these although Ubuntu and derivatives wil do 30 ifyou tick a box. The MP3 patent expired in 2017, though it doesn't really matter ~ we have plenty of open formats ‘and codecs now (OGG. FLAC, VPx and x264). I's still technically a DMCA violation to use libdudess (a modem {and much more efficient way of eracking CSS, used by the majority of media players on Linux) to watch a DVD, bt that only applies in some countries and to date. no ‘one has challenged its use Early kernel development As Linux gained traction frst among academics and hhobyjsts and then by the mid-90s, when businesses started to form around it, the number of contributors bloomed. On take from Linus himsels that once the X Windows System was working on Linux (vith v0.95) it ‘became much more appealing. So one could infer that even in 1982 people were afraid a the command ine ‘This popularity led tothe estabishment of the ‘maintainer heiarchy so that patches submitted could be reviewed and promoted efciertly to Lius’ source ‘ree. Though that fist version ofthe MAINTAINERS file descr Linus as “buried alvin emai’ ‘Te emai-centric development process sti followed today, except thatthe Official Linux Kernel Mailing List was setup in 1997, and now Gitis used for vetsion control. So's lat easier to make sure you're working on an up-to-date branch. rather than having to ‘wat for floppies i the mal Patches are still generated Using elf-u to show which ines have been changed in whic fies. Before Git. the proprietary BitKeeper concurrent versioning system (CVS) was used. And when this arrangement came to an end (helped by ‘Andrew Tridge's reverse engineering mischief), Torvalds gothacking and 10 days later there was Git ‘After two yearsin development Kernel 26 was released in 2008, This was a vastly different beast to 2 featuring scheduler enhancements, improved ‘support for multiprocessor systems (incuding hyperteading, NPTL and NUMA suppert), faster /O and ahuge amount of extra hardware support. We also saw the Physical Adress Extension (PAE) so that 32-t ‘machines could adress up to 64GB of RAM (before they wer limited to about 3268), Also introduced was 12.0 Linux 95, 30 years of Linux Srey rey Cee ero Cees tees peer are peers fy “The venerable Advanced Linux Sound Architecture (ALSA) subsystem enabled (almost) out- of-the-box functionality for popular sound cards.” the venerable Advanced Linux Sound Architecture (ALSA) subsystem, This enabled (almost) out ofthe box functionality for popular sound cards, as well as ‘support for multiple devices, hardware mixing, full: duplex operation and MIDI, The most far-reaching new feature was the old device management subsystem, devs, being superceded by udev. This didn't appear Lntil2.613 (November 2003), at which point the /dew directory ceased to be alist of (many. mary) static ‘nodes and became a dynamic reflection ofthe devices ‘actually connected tothe system. The subsystem udev also handled firmware loading, and userspace events ‘and contributed to a much mare corwenient experience {for desktop users. Although you still relied on such arcana as HAL and ivman in order to automount a USB. stick withthe correct permissions. Linux (having already been ported to non-x86 64 bit processors) supported the Itanium’s IAG4 instruction when it was ‘leased in 2001. This architecture was doomed to fal though, and intel eventually moved tothe more conservative AMD64 (or x86-64) architecture, which has been around since 2003, EET i Soe eto’ MP) is ad ‘Sept 2002 Blender ‘The Hackers Manual | 25Distros Thanks to open source development, Linux users were running 64-bit desktops right away, while Windows Users would have to wait unt 2008 forthe x64 release Cf XP Various proprietary applications (notably Steam and latsits games) run in 32-bit mode, which provides some motivation for distributions to maintain atleast some 32-bit ibrares, Debian 1 will support 32-bit x86 in some form until 2026. but most other distros have abandoned it Eventually such machines will go the way ofthe 386,n0 longer supported on Linux since 2013. Enter the archetype 12004, sound server called Polypaudio was released by abitherto unknown developer called Lennart Poettering and some others. At ths time desktop environments relied on sound servers to overcome shortcomings in ALSA'sdimix syste: Gnome was using the Enlightened Sound Daemon (ESD) and KDE was Using the analogue Realtime synthesizer (ats). Polypaudio was designed to bea dropin replacement for ESD. providing much more advanced features, such as per-appliation volume control and network transparency. In 2006 the project citing criticism that ‘nobody wants polyps, renamed itself PulseAudio (it was in factnamed attr the sea dweling creature). Asus EeePC Withits nev name and increased demand for a LUnuxwas based sound system comparable with that of OSX or the ‘2eXandos nen leased (and much maligned) Windows Vista, leeWM, PulseAudio enjoyed substantial development and began ee oye a begar tobe considered fr inclusion in many distros. Asis traditional, Fedora was the fist to adopt, incorporating, itas the default in version 8, released in late 2007, professionals Just replace it S Debian is the dstro that launched more distros than any other! stability, high level of testing, dedication to software freedom, and being a rigorously well-run organisation.” ) The late lan Murdock founded the influential Linux distribution Debian in 1993, Linux Format spent time talking with hie in 2007 Brest Chart the (CREDIT: Bareon the nuendo GNU ree Dasimentton Leen 3. nt/gthvacom/FabeLala/Lnainelne/re/mie ‘The Hackers Manual | 29Distros > With big bucks, comes big offices! Here's the Red Hat He sporting Its old logo. (CREDIT B34, CCBYSA 3 hns en kanes or/ki/ RedHat med resdquates at Rallgh North Carina US One early example was Mandrake Linux, in mid 1998. fork of Rea Hat it was crazily aimed at making Linux ‘easy to use for new users, using the new Kool Desktop Environment (KDE), The French/Braziian development team gained lot of attention but, ultimately financial problems closed the project in 2011 However its spirit continues inthe excellent but less well-known Mageia ‘and OpenMandriva projects Adistro with humanity in mind ‘With Mandrake iting the way he cary 2000s sat an ‘explosion of stro releases, Now that the Debian project at thispoint was wel established, wall regarded and wellknown itbecame the bsisfor hundreds of Linux distros. But wel ‘onlymenton one: Ubunt, eleasedin 2006 by South rican mlbonaie Mark Stuttleworth, who jokingly als rise the selt appointed benevelent dictator fre. The Ubunty Foundation was created in 2005 a pantropic project ~ Uunts Zulu wore meaning humanity to prove quality ‘open source softvare with Canonical as the supporting ‘omeecia am, Ubuntu abranch of Debian hassel sen over 80, éistos fork rom it while Ubuntu has the highest share ‘ofall desktop Linux instals though this is notoriously see Sanaoeeen 9 Novenber 2015)98 hard to measure ~ when users are polled. Why Ubuntu became so popular is hard t fully pinpoint, Key is just lke Mandrake before it, Ubuntu set out to make desktop. Linux easy for first-time users, It also offered the distro ‘on free CDs viaits Shipt service unt 2011, alongside fast, relable server downloads. Furthermore, t was based on the popular Debian itjumpedon the new, slick Gnome desktop, andit set out a regular six-month release cycle, with a Long Term Support release every ‘wo years. Support was for 18 months (now nine months) for regular releases, nd 36 months for LTS ‘ones (now five years), Ubuntu aso offered great forums and help sites, ‘along with a community counel, and suppor for forks ‘such as Xubuntu, Lubuntu and many others. Ithad sane defaults too, and made it easier to install display drivers {an absolute pain 10-plus years ago), while offering a huge catalogue of tested, ready-to-run open-source software and dedicated server bullds. We guess when yu say allthis out loud it sounds pretty cornpeling! Two core release branches well quickly mention are [rch Linux and Gentoo, both released around 2000. Gentoo (named after the fastest penguin in the word) is ‘a built-rom-source distro compiled with speciic ‘optimisations for the hardware it’s going to run on. This is very clever, but also very time-consuming, Google (Chrome 0S is derived from Gentoo. In early 2002, Arch Linux was released, devised as a minimalist cistro, where the user does much of the installation work to Create an OS with just the parts required. This DIY approach was partly why Arch s renowned for its ‘amazing documentation and fr rolling out the earliest release of new versions of software. [At the height ofthe distro madness (around 2010), there were almost 300 Linux aistros, we'd argue an ‘unsustainable number, with many just repeating basic desktop functionality aiready avaiable in core root distros. Progressing into the 2000s, and with increasing Get your Linux game on ‘Theres always been ariche interest ingaming con Linx. but ths was mostly done tough Wine, which hasbeen around sine the mi 80s and frankly ays felt thea sticking Plaster to enabe Word of Warerat or whatever the current Windows game of choice was tobe played on Linu 30 | The Hacker’ Manual Things started to changed when Valve ported its Source engine to Linx alongwith releasing its Steam for Linux cent in 2012. ‘This opened the gate for Sourcerbased native Lax game distribution, Inastion a the end of 2013 Va announced itwas creating Steam0S a edicated Debi based dro for running ts ‘Steam cnt, Ths was tote in ater wth ts faled attempt at creating a Steam Machine ecosystem, Today there are over 7000 native Lua games avalable on Steam, outof ‘around 14000 in total Perhaps mare significant is that Vahe ‘never stopped developing Steam0S, despite its Steam Machine fare In 2018 Valve released its oun internal fal of Wine caled Proton that was integrated into Steam isl and propelled Linux support fr Windows games toa ne lve ith curenty a ported 50 percent of games oenng Platinum compat But wy allthis work just to nelp one per contof Steams Linut using gamers? Ths summer Valve revealed its Steam Deck, 2 Li powered hand-held PC consol, which it promised would runall Windows games via its ‘Steam Proton layer. Perhaps 2021is year of the Linux desktop ater alcomplexity in maintaining a ‘modern 0S, the number of Linux distros started to reduce, but that didn't stop wel ‘organised groups creating popular new distro forks when they felt a need, ‘Agood example is Raspberry Pi OS, a rebrand of Raspbian, ise fork of Debian, The new Arm-based hardware platform needed a dedicated operating system, so picking up Debian and refitting it forthe Raspberry Pi, including educational software libraries {or its GPIO access, and tailored tools to configure its hardware, made absolute sense, Linux hardware specialist System76 was ticed of niggling software issues associated with using other distros, and wanted direct control. So, it introduced Popl_OS, a fork of Ubuntu, tonat only directly support its laptops and desktop hardware, but also its customers’ needs t's @ slick, modern distro, with support for popular software and hardware, Linux Mint started in 2006 as a small personal Ubuntu fork project. When Ubuntu changed to its ‘modern” Unity desktop design in 2011, many users revolted The Linux Mint project created its own classic” desktop, called Cinnamon. in 2012, andit brought many former Ubuntu users with it. The Linux Mint project has stuck with ts “user fist” design approach, and evolved remarkably well This doesn't even touch upon commercially focused unbelievably. giving users what they wanted! 30 years of Linux > Google's Android (not a Aistro) is frowned ‘upon in the Linux world, but you can't deny the effec it ad on the market. distros, such as Android, Chrome OS, Inte’ ClearOS, Google's Wear 0S, Sash OS, and the host of server- specific distros. Even today there are well over 200 ‘active Linux distros, and they'e as diverse. interesting, ‘and wonderful asthe communities that use them, Looking forward But what ofthe future? Technology predictions are notoriously tricky, but why would we ever lt that stop us? WLTux til be active in 30 years? We'd say thats a safe bet: even ill development stoped now, people would keep on using’ for years if not fr decades. “There are retro computer systems that are stil ticking cover almost as long ater. andthe Linux kemelis far ‘more functional than they ever were. ‘Amore likely scenario fs Google, as an example moving to an alternative kernel - Fuschia, say - though this would ly just befor Android ands oT devices Yet event Google moved literally everything it runs to Fuschia, the Linux kerelis used so widely elsewhere that it would just keep on trucking ‘AS we've Seen, the Linux world is larger than justits kernel. An OSis a whole ecosystem of interconnected systems that have to be developed, tested and packaged in an orchestrated manner. Linux was built on GNU tools andits licence this widened the appeal of Linux and enabled the kernel wth suitable cstros to be deployed in such vasty ciffering devices rom the fastest super computerin the world to lowly $4 Pi “The Linux kernel is tied to the success of any one corporation, Sure there's the Linux Foundation and Torvalds himselt, but succession has already been put into place to keep kernel development going if Torvalds should step down, And while the Linux Foundation isn't necessary t's certainly handy to orchestrate and handle funding and trademarks, Put allo that aside, the reason Linux has succeeded 's that t's damn good atts jab and everyone can Contribute. It's the single greatest software development project of modern times. which doesn't mean is perfect i's software ater all but t's contiually improved and enhanced. it's strong copyleft open ouroe fostered a fabulous community andit's given usall endless opportunites. So keep on enjoying it ‘The Hackers Manual | 31‘inux, if you want to be annoyingly. of the operating system, as well as some things you might think belong elsewhere This includes drivers, filesystems, network protocols, schedulers (process and 1/0), memory allocators and al kinds of other treasure. Inthe early days of| regularly have to face compiling their own kernels, sometimes to get new hardware support, sometimes to reconfigure their way out of| ‘and som improve performance. Users of Linux from ch or Gentoo days willbe familiar with the kernel compilation process, since pux, users would its more or less mandatory there, But it's ‘an option for any flavour of Linux, owe thought we'd have ago at making building emels more accessible. AAs well iscover, the Linux Kernel is hhuge and complicated. It may be tempting for some to pore overt with a fine-toothed comb, trying to "optimise" it for speed or size, But tis is forthe most part hopeless and mare likaly to result in breakage than anything else. I you want responsive kernel, then a better approach is touse one of the mary custom etforts that are avallable.Grasp the Build the Kernel kernel basics Just what is a kernel and why is it telling my computer what to do? penyouthikot n operating yt (5) & WC incense tat sresportlo tr nye compet does afer he BOSE roto tetotcadarrrpondrytarsochs Wows anted0s fseay oboe ge hy Thasnochocet tops noclents tatoo (a) conan ie so ea wa tree coe took: (ue err asa), Ones Tattags noch moreno the pregessen om Vern oofote otra ol seen ch mo Sorat ysteauhyoucen on tye trometer ands vee teat tom (oreanpe sender nih Wtitansaatot ory Spertgoom ios ra Thetascoed tat nx sbsed on BSD (perpetuated by lazy journos and Mac users who like to einer OSt mero op thn serene Darwin OS yon wih bced Davin op se snapary SR toed tte browse ote ee parter NOMTETEP rcs Dot ype ino) orwn asso rel cled XN on Suomen hot Us) nce Woda Chea a cpenad toa mona tlo s bexedon elas eel orgy aesech poet tao eral WN BSD hcl sed otsnata SO sosed ara) Thenoresty eso nas tor eA Gl te Gon/Cartn hrarend Cares apcatonseien ae al epty adhe voting core Driving the deal The XNU kernel has its own ceiver API called OK, and Windows’ kere! has the less imagintivey titled ‘Windows Driver Model Drvers (programs that talk to hardware) are perhaps the easiest component of kernel to get your head around inthe sense its easy to see why they need tobe there. The only troubles, mast ‘modern Linux distros have very few rivers bakediinto their kernels Instead the prefered approachis to include crivers compiled as external modules, which can be loaded early onin the boot process by Udev ‘when the corresponding hardware detected. Modules plug straight ito the kernel and forthe most pat act asifthey were apart oft Other kernel components can be modularised' and loaded on demand too, but certain lowevel systems (or example, the one fr loading ‘modules inthe fist place), have tobe buit in, Sot Linux Kernel contains drivers fr every bit ot hhardware supported by Linux Well almost. Modern hardware (particularly Wi-Fi and GPUs) often requires firmware to be uploaded tit on every boot, otherwise it \won't work This firmware is generally not included in distribution’ kernel source packages (since isnot source code and sometimes proprietary), but ather shipped in a separate ine frmware package. >The kernel | modular masterpiece ‘hat ean run everything from ‘old AGP graphics tothe Large Hadron Colisr. Since this package is required for al the drivers in the kemel (usually packaged as linux-image-..) to work, we can't forget its siz (ib/firmware occupies over 700MB con our system). Drivers themselves, occupy fairly negligible space. ‘ver the page wack at comping 2 trivaly modtied Ubuntu kere Canonical apply patches and backportd ies to their kernels with the elect that an Ubuntu kernel bearing the version number 5130. say (which we gt by running nama on our Ubunts 20.08 VM) maybe constitutional very diferent rom a" manne” Kernel 513 that you woul downlead fom Wtps// ermelorg, You might want to bul your aan mainline kernel, tor exami you suspect a bug has been fied upstream (or indeed, you think that an Ubunta patch introduced the bug). Tats easy ust extract the source tree and flow the ineructions oer the page. ‘Wiha’ even easier using the pre-packaged mainline kernels produced by the Uount team. First reac the blur at tps://wikiubuntu.com/Kernel/ MainlineBuilds then flew the ine tere to the Manin Kernel Archive You'l soe that while Ubunty 21.10 and LTS t's kites out withthe Haare Enablement tac) are runing Kernel 513, the current ir-deelopment series (contig also caled"mailine") is 517 We dont advise trying out these release-candaate (RC) kere at est, But you ‘may have reasonto ry the Longterm branch, which curently is 5.15 As of now this branch is used by Popl_OS and it's what te next LS of Unt wil be based on The Linux Kernel Archives A | 1 Asmorgasbord of kernels can be found at http://KerneLorg, rom bleeding edge-RCs, to the SLTS 4.4 series, Manual | 33Distros Compiling a kernel Get straight to business and build your own Ubuntu-esque kernel — re! a ‘compile your ovn kernel using unt I (or any distro that uses Ubuntu kernels for example Mint elementary OS, but not Pop!_OS) the fist step isto get hald ofthe kernel sources, ‘The official channel for vanilla kernel sources is hhttps:/“kernelong, but this isnt necessarily the best place to start Instead, we'll use an Ubuntu kernel, which includes numerous patches and backported features, It alsohas the advantage of coming with a configuration very close to what you're currently running (in fact is identical if you have the same version). So if we make only small changes there, then we'd hope the resulting kernel stil has all the functionality of the old Besides the kernel source files, we also need the required build tools. f you've ever compiled anything before you'll probably have all these, But f not, get them withthe folowing: ' sudo apt build dep linux limx-image-S(uname 1) Using. uname lke this ensures the build tools you're ‘about to dovnlgad correspond to the kernel you're running, and that both are new. fyou updated your system before this command, you might want to reboot (in case a new kernel was available) and then fetch the build tools (and kernel sources). I the command complains about not being able to find source packages, youmay need to uncomment the lines beginning deb-src in /ete/apt/sources.list or bath the main ‘and updates repositories. Alternatively, since we've got abit of a Perl flavour going on this issue, this ite snippet wil do it for you $ sudo perl -Lbak-pe"s/*¥ (debste." Ssh _release cs) (updates)? main restricted)" fto/apt/sources list Those dependencies are plentiful and include Java ‘and LaTeX), but our story isnot yet done. The (slightly outdated) Ubuntu wiki at https:/wikiubuntu.com/ Kernel/BuildYourOwnKernel mentions thatthe previous command doesn't instal evrything eequired ‘And that statement is sil corect. The folowing packages should fillin any gaps, but if not youl be told at's missing: ' sudo apt instal lincurses-dev fakeroot kernel package You may see a warring about updating kernelimg. Cont, which case you should choose to install the package maintaine's version In any event, let's get hold of those sources > Switching rom ZSTD to LZ4 compression reduced the time taken to load the kernel on our virtual machine, although not by anything significant. +s Manuel $ apt source linux image unsigned {uname +) Hf you're running the HWE kernel, which you probably ate you're running an up-to-date desktop edition of Ubuntu 22.04, then the sources wil land in te limux- hhwe-513-5.130/ directory. Yul also be told thatthe HWE kernel package is developed on Git. and that more Upto-date sources can be acquired va $ git clone giti/gtJaunchpad net/—uibunty kernel ‘ubunt/*soureeinux/*gitfocal Either way, once we have the sources we may as well lsable the source repositories. fyou used the Per! ‘command earlier you can do this with $ sudo mv + /etfapt/sources ist bale} Incidentally the apt source directive will get the source code to absolutely any package in the Ubuntu (or whatever apt-based distro your using). Neat. We didn't use sudo in the apt command because sources ate downloaded to the current directory, which we carefully made sure was the home directory. The full kernel sources are large (12GB for Ubuntu 21.10) and ‘apt wl fail if there's insufficient space. If this happens Fit CtrhC, delete the linuxchwe* files in your home directory, and try again somewhere with plenty of space Time to compile Having successfully downloaded the sources. e's configure and camps them. Wen you compile the Linux kernel. you're fre to include or exclude whatever components you nant. radktionaly these were selected froma glonous text-based menu accessed by running ‘make menuconfig. Tings have moved on now and there's amodern configuration interface. Dont get too cited though ~ is stil ext base, but now i's powered by neursesand nas ahacker style black background Summon swith $d linxchwe-5.135.130/ make nconfig See? Glorious. Navigate through the menus using the cursor keys and Enter. Options are selected or deselected (or selected to be bult as modules) using Space, bt dont change anything just yet (yourcan aways quit. by pressing Escape without saving and run the last command again to return, nstead let shave a look at what's in your kernel Everything's organised into menus, one of which is Device Drivers which weve already talked about. ‘oul also see sections for Memory Management, Networking Virtualisation anda few others that all right be expected to appear in an OS. Sections that house futher options have => atthe end ‘Well start by making a single, simple change. Go tothe General setup category Inside youl see lots of confusing options. some of which have been selected according to Ubuntu's default configuration. The option touse ZSTD compression, you might recal, became the detain Ubuntu 20.04We can find out more about a particular option by highlighting it and pressing F2. Doing this in the kernel ‘compression mode option tell us something about \which compression options work best on what syst ‘as well as 2 considered note about who to conta ifcertain options don’t work. Let's switch to LZ4 compression and see if we can spot a difference See what's on the SLAB if you the ode sounding acronyms, have alook nthe Choose SLAB allcator menu (towards the bottom ofthe General setup menu). Before you get dragged deepinto the rabbit's wazen of kemel options, ext by pressing F9 anc meke sure you save te configuration “oulbe tod it stored ina fe named. config. One slight quirk with using the Ubuntu kernels that it’ configured by default to use Canonical's Trusted System Keys, Because these arent included inthe kere! sources the build wil fai without some intervention. We can ether manually disable the keys (they're deep in the Cryptographic Services menu), ‘or we can use the helpful Debian seripts 'scriptsiconfig~set str SYSTEM REVOCATION KEYS 8 scriptslconig~setstr SYSTEM TRUSTED. KEYS, This sts these to blank strings. The contig rogram alsohas a =disable argument but ifyou use that here itbreaks because string value required here PEBasuersece Build the Kernel > Comping ermels wal really put your (CPU through its paces. Our laptop {got too hot to type on (now there's an excuse for late copy! Ea). —> —! Note this is documented precisely nowhere. Finally let's build the thing $ fakeroot make-kpkg + “S(npree)” intel -appendi. version="*hx” kernel image kernel headers Now make acup of tes because the stock Uountu kernels rather ful featured. You'l see fils being sted as they're being compiled, linked and generated. I's reasonably pleasing to watch, but youmay need anather ‘cup of tea, On aur XPSI3 it took most of alunch hour (iho said you were alowed an hour for lunch? ~ Eo finish up, ovr the course of which it got rather hot, Even In Covid-fee draughty coridors of Future Towers. All going wel you should see two deb packages in your home directory (nt the source directory): one Containing the kernel image. and one containing the headers. We've been careful to only do avery trivial ‘modification (so, only changing the compression format) for our first outing. And with good reason, ‘These packages are going to replace the current kernel packages on the system. This shouldn't really be a problem since Ubuntu always has a fallback kernel on hand, but we don't really want our fist effort to result ina kernel panic. See how you fare with $$ sudo dpkg linux" deb The kernel shouldbe installed as Grub’s new default Check after areboot withthe uname-a command, and if necessary hold down Shift to bring up a boat menu, Colonel Kurtz advises. Using the Ubuntu (tecniealy Debian) tools to bul kernels al el and good. But ifyoureust hacking away ona kernel or personal sage you might not want o goto the ther of packaging al. And It'S certany inconvenient todo as we've done ad replace the current kernel packages with custom anes. since those paclages vil be cveriten as soon as Ubuntu release 2 new kere it's not recommended to try and bul these packages with ferent nares, andi’ ely to brea things if you try to se any other kindof Workaround here Instead you can bud ker using the manne tools andnot worry about packaging tom. You can run le neon from the sources rectory tous the (modern) ncurses confeurator. ‘The ance you've saved tun malt to bul it and sido male edule instal to stall the modes, Fal, youl want to copy the arel image to /boot wits sido inatal id we say final? Not qute- we stil eet run BERG to enumerate our modules, andthen (inal) update ru wth do pdate-grib >It you ke endless options, settings and frobe all arranged interface then you've this. The Hacker's Manual | 35,Distros Kernel minification Perfection is reached not when there's nothing left to add, but rather when there's nothing left to take away... look around the config menu and have noticed there's an awl lt of stuffin there you don’t ‘eed. This is probably true, and aot of people are interested in making the kernel smaller. People building or embedded systems, where storage space ancl memory are scarce, naturally wil want to pull as ‘much stuff as possible out of the kernel. We've mentioned that therein relly a performance or memory ht from having so many unused drivers compiled as modules, However, there is ack space hit although perhaps not as large as you might think (see bo, Below). B 1 this point the chances are you've already had a ‘Another reason for removing things is demonstrat by the Linuxlibe Kernel. Here, drivers that require proprietary firmware or microcade have alleen removed. As has firmuvare and microcode itself, which if youtake alook at your /ib/fiemware directory (ours was over 700MB) adds up toa substantial saving, OF course, removing firmware files should be done carefully since any device depending on them will cease to function. There's also the annoyance of carefully removing unneeded fies, only for anew linux-fimware package to appear and not only replace them, but add yet more unnecessary crt At frst glance, ‘theres alt of superfluous rivers the Kernel. But you never know winen youl need ‘tape access. Spotting superfluous fireware Depending on your hardware it might be easy to see ‘which firmware fies you need, or atleast which ones you don'tneed. For example you don't have an ‘ANI graphics card then you can remove the SEMB of firmware in imware amu. or indeed the TMB of bits in firmware /radeon for older car. There's small ‘volume of firmware forthe Nouveau driver, the open source offering for Nua cards, butts nat really worth worrying about at this stage. To get some dea about which fmware has been loaded on your system, run Journalt-b to bring up the Journal forthe curent boot. Press / to instigate a search, {and search for firware, On our XPS we found the follwing lines (and then some irrelevant matches to do with UEFT keys) Lore) (oT B.GiherA-Ce hs ols Space atleast the space taken up by the er) on modern hardware shouldbe 8 concer The compressed kernalimage 7 bbootmlinuz on our Pop! OS system ceeupies a mere IMB, And the accompanying modules, ound in by ‘modiles/515./ occupies around 425M, Zay other Ubuntu-base system wl have ‘ria statistics (unless youhave been tweaking things already) hala eeabyte is enportantto you which t might be ond hardnare) then recompiling alaner kernel (ith ent the requied ives) would be one ‘way to achieve tat. Natural that comes with the downside that ityou eer acqure erent harware wert work with the imine kerma. Furthor investigating inthe modules/ rectory reveas thatthe network, mec and (GPU device cnvers together occupy the most space (165MB) I you examine further you see that AMDGPUis the largest graphics chiver at around 20M. Noweau, the FOSS liver for Nvidia cards isby comparison airy 3AMB, Song whichever ofthese you don't need isa start to slimming down your kernel Divers on Linux ae smal you might have noticed, Even AMDGPU iy compared to ‘he equivalent massive (-450MB) downland ‘onWindows The litference is that very tle cf that Windows packages eriver: Most of itis emware, ch and shiny but useless (it pot ‘outright annoying) garmeware applications. So realy removing ders one by one sa fairy thankess exercise (he here configuation interface doesnt make this easy other). I's untkel tolberate more than ‘couple of une megabytes Of couse theres pent le to remove to, but again ‘each component is smal and sometimes tmuch mor ertica thant sounds So by remouing swathes of thems ikely going toto fend you up wth a broken, (but undeniably leaner kernel The kernel sources directory gets big once you start compiling. Here the Intel graphics bits are occupying half a gigabyte.[arm] Finished loading DMC firmware i91S/kbL dme_ ‘verl_ot bin (v4 _ath0k_pei 0000:34000: firmware ver WLAN RM4.4.1. 00157. QCARMSWFZ.1/ ‘So our Inte graphics need to run DMC (rice) frmware ‘and our wireless chip needs something too. The ater doesn't explicitly tll us which frmware fle itneeds, but if \we ook at the output of Isp we can obtain a model ‘number (QCA6I74).Andit turns out there's a /lb/ firmware ath1Ok/QCAGI7A directory so that's probably what we want. ‘Again, unless you're seriously strained for space it's not really worth picking your firmware directory apart like this, Atleast it wasn't on aur XPS, But we do ‘maintain a growing collection of old hardwaee, including the undying Eee PC trom 2007, With only 268 of SSD, there can be no wasted bytes Indeed, such old harcare doesn't need any firmware at all Naturally. we've slimmed down the kernel to next to nothing as wel I youre making a kernel thats talloed for particular hardware, then a common technique is to build any required drivers ito the kernel image as opposed to building modules. The kernel configuration interface ‘even has its own mechanism for selecting modules ‘based on what's currently loaded. Touse this, make sure any hardware you'd ike the ‘new kernel to suppor is plugged in and working. '$ make localmodconfig This generates a configuration based on the current kernel configuration together with any currently loaded ‘modules. In general, this results in a kernel that sil has a ‘number of extraneous modules, butf you have the patience to iteratively compile test and remove then eventually you'l find your way to kere! iss Itcan be taxing to manually load every possible diver, and in some cases it's hard to resolve hardware names to ‘module names. t's particulary frustrating when you ‘miss a key module (yes, you do need SCSI disk support believe it or not) and have to rebuild the whole thing, One tool that can help you with this is Modorabeckl (see https://github.com/graysky2/modprobed-db). which keeps a database o all the modules ever loaded ‘onthe system. This increases the chances that hardware yuhaven't plugged into the system for ages wil stil ‘Work withthe new kernel Where to compile your drivers? ityou'd rather have those drives compiled int the kernel, as opposed to as modules, you can use the command loealyesconfg insteed. a previous configuration from an older kernel exists, youl be asked about new drivers that have een added tothe new sources, Usually its okay to accept the defaults here ‘Once the new configurations witten out wecan usea handy script to see any ditferences ke so: $ scriptsdfconig config.ld config ‘ou should notice alot of ines inthe new fle (prefixed by +) arernow set to negative. Lets try building our det sized kernel wth a simple ‘make $S(np100) ‘The mproe incantation (used on the previous page to) ensures compilation i divided into paral obs, one for each CPU core. This might make your machine citcutto work wth wilt’ all ulin, but wil certainly speed up complation time We're not using Build the Kernel ‘too. Once the image builds we need to follow the Graysky has instructions in the Colonel Kurtz Advises box. Let's Heseraad on assume you've done that, and installed your modules _ 7 Scare Those steps should build a coresponding initrd (inital ramsk/remfs image) and putit in Zboot where itcan be found by GRUB If you'e not using GRUB, for ‘example you're on Pop!_OS which uses systernd-boot ther extra work may be required to get the bootloader to Give modules the heave-ho “If you’d rather have those drivers compiled into the kernel, as opposed to as modules, you can use the command localyesconfig instead” pick up the new kernel. We were pleasantly surprised this all worked almost without a hitch on our Pop!_OS laptop. But do study the documentation fr the emelstub utility before goingin with all guns blazing. ‘Systema-boot requires the whole kernel to ive onthe EFI Partition, and you don't want to accidentally fil this up, {As you become more involved with bulding kernels, you'll grow more and more familiar withthe Configuration interface and where particular options hide, You'l also find tha. contrary to widely misheld belie, Linux does in fact support a huge amount of hardware. From HAM radio to steering wheel controllers (new in 515) to ISDN modems (remember them?) There's aso a good amount of weirdness. If younosey ‘deep enough into the kernel sources themselves then youlcan find plenty of interesting stutt. The areh/x6/ [purgatory subdirectory for example contains \VMspecific code. Specifcaly, purgatory is an object that sits between running kernels (for example, when 2 ‘new kemelis about tobe loaded). I the process goes ‘wrong the kernel hangs with friendly “min purgatory” message It's possible to disable purgatory, but this won't absolve you of your sins The Hacker's ManualDistros Popular patches Forget trawling through configs — use a pre-rolled patchset to set the rules. reshays impressed by Cobra bsed W Beers cmetitentoti Nate covet legos harare anda fe cetepemteerh sao putin att ‘fort tealight canes re stings he serknascntdin the Sten sche sere wih teat theres OFS (Corte Fa Schedles forte tency enon pores Youde ren fae tobe ing Pol 0 ale Serotage ot youre atraaly pore fregrurdapteaton as elas coneon Scstoprossse suas Go Solan Processes playing nice “If two processes competing for CPU time both have the same priority, then the process with the lowest niceness takes Get Gard kernel goodness without the fon arises, with wake 38 | Theli precedence.” Garuda Linux is making a lot of heacines lately. The slick, eye-candy heavy, Arch-based distro includes a couple of daemons that ought to improve responsiveness First there's Ananicy (ANother Auto NiCe daemon), which automatically renices (gives greater affinity) processes. The idea here (as with any scheduling tweak) isrt to magically give you more speed. Rather. is concerned with tweaking priorities so that heavy tasks (Ike compilation or indexing) don't interfere with things happening inthe foreground, icing is feature ofthe Linux kernel itself, and you can set program iceness’ manually using the renice command, However this needs to be set every time a process is spawried, so the advantage of nancy (and others lke it) is that multiple nicenesses (niceties?) can be set automatically. All the user needs todo is start a systemd service Interpreting levels of niceness Technical, ieness is distinct rom priory, inthe sense that processes have a value for both Prot can take ay value from rt (ealtme) to-51 (very high privity) to 20 (arma preity). By default, user processes (and mast system processes) ae given apron of 20, important system processes have a peoity of andeally important ones have even lower. two processes competing fr CPU time both have the same priority, then the process wth the lowest ricenes takes precedence. The Pulseaucio daeran, fr example, by default runs asa ser process wth ance level of so that other use processes don't make fr choppy audio output ‘One ofthe earliest and mast popular patchsets forthe kernels Con Kolvas "CK" efor. Starting out 28 set of tweaks to improve desktop performance, it evolved to include a series of new CPU schedulers, culminating in BFS (the middle letter of which stands for something tude) in 2003, Kolvas has in the past voiced concerns about kerne developers ack of interest n the desktop. But with BFS and much ot his scheduler work) the intention was never to getit mainlned t's not general purpose enough for the kernel and the kernel has only one scheduler anyway (the Completely Fair Scheduler, CFS). BFShas since been retired in favour of ‘MuQSS (Multiple Queue Skipist Scheduler). which was introduced in 2016. Ifyou want to try out MUQSS, you can gra the patches diectiy and apply them toa vanilla source tree (coe https://githubscom/ckolivas linux io instructions). Alternatively, MuQSShas been included in various custom kernels, namely Liquor (htps// liquorizet) and Linwx-TK6, The latter includes a choice of several schedulers, as well as some patches from Inte pertormance-focused Clear Linux LinuxcTKG also enables you tocornpiethe whole kernel with CPU: native optimisations. You might have noticed the vanilla kernel has some CPU options (under the Processor ‘ype and Features) but these only target a particular family. rather than a given microarchtecture such as Zen 2for madern Ryzen processors. LincTKG (see https: //github.com/Frogging- Family/tiunt) is included in Garuda Linux, but you cantry it out on any Arct-based distro, or any distro with Arch’s Pacman package manager installed. In theory it l work anywhere, but ths is offially sanctioned, Let's try Youll need to install Gitandt fetch the sources:> Any excuse to feature a Pop!_OS background, They also ‘make bespoke scheduler tweaks now ‘sudo apt instal it git clone bttpe/githuh com Frogging Famiyinuxtg ait ‘Sling ‘There's 2 helpful installation script which well run in ‘moment, but do read the documentation before doing so, There's also a config file customization that can influence the script's behaviour. Like make-kokg the script wll build DEB (or RPM if you're using Fedora) packages which canbe installed like any other software, See how you get on with ' instal sh install ‘You'llbe asked which distro you're using (there's a ‘generic option, but in genera if you choose something ‘lose to your distro it should work), Next you'l be asked ‘which kernel version you want to install. They're listed newest ist, from the bleeding edge-RCS to the strong ‘and stable 54 series. We figured wed go for 2516 version, because that seemed newer and shiner than What we were currently running. You'll then be offered a choice of CPU scheduler (we chose Project C/BMQ) ‘and given a choice of compiler, Tools of the Linux trade When Linux was announced 1991 Linus noted that he rad ported GNU Bashand GCC to work witht. He also noted these tools werent part of Linux proper, so.as ‘lstbutions'strted bundling GNU tools withthe Linux Kernel. the GNU/Linux conjunction was bor. Todey's <éstrbutions stil bundle lts of GNU tools (bash, emacs 3), but also las of other non-GNU tools. Those other Build the Kernel tools dont ask tobe included in the Linux tage, anit they did we would probably mock them, Ifyou choose Ciang (see bor, below), you can choose toenable Link Time Optimisations, but this wll use alot ‘of memory and take along time, Researching Clang Using Clang requires (at east the elang, um and id packages tobe instaled. Andif youre on Ubuntu 2004 youl have to work around the older (version 10 saris) in {he repos. DuckDuckGo is your fiend here. ‘oul be asked several more questions, and some of them have defaults which you should probably stick to at frst. Kernel Timer frequency has been the subject of same debate over the years and TKG's default of SOOHr seems oft sensibly in the mide ofthe generic kernel etait of 25042 and the ,000Hz setting curently used by some loulatency kernels Fnaly youllbe asked ifyou want to run nconfig (or ay ofthe ther kernel configuration interfaces) for any final tweaks ‘Once again now is a good time to make a cup of tea, ‘as kernel sources have tobe cloned, patched and configured. Any kernels generated this way must be removed manual, but the script can help with that. In general, any custom kernel you build as 2 DEB package should be easy to remeve, but ahvays keep an eye on your Aboot rectory for ancient artefacts. We'd dearly love to cover mare kemel patches. in particular Xarmod, butt looks ike we'l have to leave these explorations to you. Dolet usknow how you get on! clang 3.5 is here. 6 ee Stes onsen Pein Grown we ne ® ye > if you use Ubuntu 2004 youl have todo some Apt magick to make ‘ang 35 work. Some distros such a5 Aline and Aackoid, include very few GNU components (or ‘example, muslibe and Bion ae sed, respectively stead of the GNUC ira). Most distros are stil using GCCto compile ‘ther kernels but for mary years now thas been possible to use the LM Clg compiler instead. Ancoid and ChromeOs do this. an so does OpenMandrva LLVM (the Low Level Virtua Machine) #3 tole based around C++ objets, and Clangs 8 ‘rontenl to LLVM that supports (the language in which most ofthe kerelis wetter) as wel asthe GNU Cextersons. “There ar technical reasons fr using Cong as oppose to GOC First. makes ‘compilation for diferent platforms easy. A binary compiled with Clang(n what's calles the LLVM Intermediate Representation) can target. after being processed by the appropeateLLVM backend matple architectures. Curent. bul the kernels ‘only support for ARM an 86 targets, but ‘others (MIPS, RISC, PowerPC) ae avalabe Furthrmoce.a binary compiled with Clang canbe investigated with advanced static and dynamic analyss tools rom the LVM site These can ep find bugs. And yes, his means that using Glangto compile the kernel can resultnaperormance inrease As well 3s. this, sine Linux 5.2 February 2021) the kernel has Supported LTO (inktime optimisations) with Gang This exer enables the kernel tobe optimised as a whole, nstead ain the context, individu source is, ‘The Hacker's Manual