System Programming Essentials with Go: System calls, networking, efficiency, and security practices with practical projects in Golang
By Alex Rios
()
Related to System Programming Essentials with Go
Related ebooks
Mastering the Art of Go Programming: Unraveling the Secrets of Expert-Level Programming Rating: 0 out of 5 stars0 ratingsGo Debugging from Scratch: A Practical Guide with Examples Rating: 0 out of 5 stars0 ratingsMastering Go: Navigating the World of Concurrent Programming Rating: 0 out of 5 stars0 ratingsGetting Started with Go: A Practical Guide with Examples Rating: 0 out of 5 stars0 ratingsGo File Handling for New Coders: A Practical Guide with Examples Rating: 0 out of 5 stars0 ratingsMastering Go Programming: From Basics to Expert Proficiency Rating: 0 out of 5 stars0 ratingsGo Programming Essentials: From Zero to Production-Ready Applications Rating: 0 out of 5 stars0 ratingsGo Recipes for Developers: Top techniques and practical solutions for real-life Go programming problems Rating: 0 out of 5 stars0 ratingsModern Web Development with Go: Build real-world, fast, efficient, and scalable web server apps using Go programming language Rating: 0 out of 5 stars0 ratingsMastering Go A Practical Guide to Developers: A Practical Guide to Developers Rating: 0 out of 5 stars0 ratingsGo Programming Essentials: A Comprehensive Guide for Developers Rating: 0 out of 5 stars0 ratingsConcurrency in Go Programming: Methods and Tools for Efficient Coding Rating: 0 out of 5 stars0 ratingsGo Functional Programming Simplified: A Practical Guide with Examples Rating: 0 out of 5 stars0 ratingsThe Way to Go: A Thorough Introduction to the Go Programming Language Rating: 3 out of 5 stars3/5Mastering Concurrent Programming in Go: A Comprehensive Guide Rating: 0 out of 5 stars0 ratingsGo Exception Handling Made Easy: A Practical Guide with Examples Rating: 0 out of 5 stars0 ratingsGo Programming Cookbook Rating: 0 out of 5 stars0 ratingsThe Go Programming Language Reference: Definitive Reference for Developers and Engineers Rating: 0 out of 5 stars0 ratingsRust Programming Basics: A Practical Guide with Examples Rating: 0 out of 5 stars0 ratingsGo Algorithms for Beginners: A Practical Guide with Examples Rating: 0 out of 5 stars0 ratingsWeb Programming with Go: Building and Scaling Interactive Web Applications with Go's Robust Ecosystem Rating: 0 out of 5 stars0 ratingsCrafting Scalable Web Solutions: Harnessing Go and Docker for Modern Development Rating: 0 out of 5 stars0 ratingsSystems Programming with C# and .NET: Building robust system solutions with C# 12 and .NET 8 Rating: 0 out of 5 stars0 ratingsPractical Go: Building Scalable Network and Non-Network Applications Rating: 0 out of 5 stars0 ratingsThe Complete Handbook of Golang Microservices: Best Practices and Techniques Rating: 0 out of 5 stars0 ratingsGolang Mini Reference: A Hitchhiker's Guide to the Modern Programming Languages, #1 Rating: 0 out of 5 stars0 ratingsMastering the Art of Network Programming: Unraveling the Secrets of Expert-Level Programming Rating: 0 out of 5 stars0 ratings
Programming For You
SQL QuickStart Guide: The Simplified Beginner's Guide to Managing, Analyzing, and Manipulating Data With SQL Rating: 4 out of 5 stars4/5Python: Learn Python in 24 Hours Rating: 4 out of 5 stars4/5Coding All-in-One For Dummies Rating: 4 out of 5 stars4/5PYTHON PROGRAMMING Rating: 4 out of 5 stars4/5Python: For Beginners A Crash Course Guide To Learn Python in 1 Week Rating: 4 out of 5 stars4/5SQL All-in-One For Dummies Rating: 3 out of 5 stars3/5Excel : The Ultimate Comprehensive Step-By-Step Guide to the Basics of Excel Programming: 1 Rating: 5 out of 5 stars5/5SQL: For Beginners: Your Guide To Easily Learn SQL Programming in 7 Days Rating: 5 out of 5 stars5/5Python Data Structures and Algorithms Rating: 5 out of 5 stars5/5Python Programming : How to Code Python Fast In Just 24 Hours With 7 Simple Steps Rating: 4 out of 5 stars4/5JavaScript All-in-One For Dummies Rating: 5 out of 5 stars5/5Python 3 Object Oriented Programming Rating: 4 out of 5 stars4/5Python for Data Science For Dummies Rating: 0 out of 5 stars0 ratingsExcel 101: A Beginner's & Intermediate's Guide for Mastering the Quintessence of Microsoft Excel (2010-2019 & 365) in no time! Rating: 0 out of 5 stars0 ratingsCoding All-in-One For Dummies Rating: 0 out of 5 stars0 ratingsPYTHON: Practical Python Programming For Beginners & Experts With Hands-on Project Rating: 5 out of 5 stars5/5Learn to Code. Get a Job. The Ultimate Guide to Learning and Getting Hired as a Developer. Rating: 5 out of 5 stars5/5Linux: Learn in 24 Hours Rating: 5 out of 5 stars5/5Microsoft Azure For Dummies Rating: 0 out of 5 stars0 ratings
Reviews for System Programming Essentials with Go
0 ratings0 reviews
Book preview
System Programming Essentials with Go - Alex Rios
System Programming Essentials with Go
Copyright © 2024 Packt Publishing
All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.
Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.
Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.
Group Product Manager: Kunal Sawant
Publishing Product Manager: Akash Sharma
Book Project Manager: Prajakta Naik
Senior Editor: Kinnari Chohan
Technical Editor: Rajdeep Chakraborty
Copy Editor: Safis Editing
Proofreader: Kinnari Chohan
Indexer: Tejal Soni
Production Designer: Vijay Kamble
DevRel Marketing Coordinator: Sonia Chauhan
First published: June 2024
Production reference: 1070624
Published by Packt Publishing Ltd.
Grosvenor House
11 St Paul’s Square
Birmingham
B3 1RB, UK
ISBN 978-1-83763-413-2
www.packtpub.com
For Erika, my life partner, who supported me through the long hours and extra effort needed to complete this work. Your love and encouragement have been my constant source of strength.
Contributors
About the author
Alex Rios is an established Brazilian software engineer with a 15-year track record of success in large-scale solution development. He specializes in Go and creates high-throughput systems that address diverse needs across fintech, telecom, and gaming industries. As a staff engineer at Stone Co., Alex applies his expertise using unconventional system designs, ensuring top-notch delivery. Also, he uses his expertise to evaluate books and publications as a technical reviewer. He is an enthusiastic community member, actively participating in its growth and development as Curitiba’s Go meetup organizer. His dedication is evident in his regular presence as a speaker at major national tech events, such as GopherCon Brazil.
I would like to express my gratitude to Elton Minetto for opening this door and making the push for me to write my very first book. I am also immensely grateful to my editor, Kinnari Chohan, for her patience and meticulous attention to detail during the editing process. Your expertise and dedication have greatly enhanced the quality of this book. Lastly, a heartfelt thanks to the Go community for creating such an inspiring and supportive environment.
About the reviewer
Natan Streppel has over a decade of experience in the IT field and, in this period, has worked with a range of different technologies. He discovered Golang in 2017 and immediately fell for the language’s simplicity and expressiveness. Since then, he has been using the language and its ecosystem for both professional and non-professional projects. He thrives on tackling challenging problems in distributed systems, loves contributing to the open-source community, and enjoys exploring innovative solutions.
Table of Contents
Preface
Part 1: Introduction
1
Why Go?
Choosing Go
Concurrency and goroutines
Concurrency
Goroutines
CSP-inspired model
Share by communication
Interacting with the OS
Tooling
go build
go test
go run
go vet
go fmt
Cross-platform development with Go
Summary
2
Refreshing Concurrency and Parallelism
Technical requirements
Understanding goroutines
WaitGroup
Changing shared state
Managing data races
Atomic operations
Mutexes
Making sense of channels
How to use channels
An unbuffered channel
Buffered channels
The guarantee of delivery
Latency
State and signaling
State
Signaling
Choosing your synchronization mechanism
Summary
Part 2: Interaction with the OS
3
Understanding System Calls
Technical requirements
Introduction to system calls
The catalog of services and identification
Information exchange
The syscall package
A closer look at the os and x/sys packages
x/sys package – low-level system calls
Operating system functionality
Portability
Everyday system calls
Tracing system calls
Tracing specific system calls
Developing and testing a CLI program
Standard streams
File descriptors
Creating a CLI application
Redirections and standard streams
Making it testable
Summary
4
File and Directory Operations
Technical requirements
Identifying unsafe file and directory permissions
Files and permissions
Scanning directories in Go
Understanding file paths
Using the path/filepath package
Traversing directories
Symbolic links and unlinking files
Symbolic links – the shortcut of the file world
Unlinking files – the great escape act
Calculating directory size
Finding duplicate files
Optimizing filesystem operations
Summary
5
Working with System Events
Managing system events
What are signals?
The os/signal package
Task scheduling in Go
Why schedule?
Basic scheduling
Handling timer signals
File monitoring
Inotify
fsnotify
File rotation
Process management
Execution and timeouts
Execute and control process execution time
Building a distributed lock manager in Go
Summary
6
Understanding Pipes in Inter-Process Communication
Technical requirements
What are pipes in IPC?
Why are pipes important?
Pipes in Golang
The mechanics of anonymous pipes
Navigating named pipes (Mkfifo())
Best practices – guidelines for using pipes
Efficient data handling
Error handling and resource management
Security considerations
Performance optimization
Developing a log processing tool
Summary
7
Unix Sockets
Introduction to Unix sockets
Creating a Unix socket
Going a little deeper into socket creation
Creating the client
Inspecting the socket with lsof
Building a chat server
The complete chat client
Serving HTTP under UNIX domain sockets
Client
HTTP request line
HTTP request header
Empty line signifying end of headers
The textproto package
Performance
Other common use cases
Summary
Part 3: Performance
8
Memory Management
Technical requirements
Garbage collection
Stack and heap allocation
The GC algorithm
GOGC
GC pacer
GODEBUG
Memory ballast
GOMEMLIMIT
Memory arenas
Using memory arenas
Summary
9
Analyzing Performance
Escape analysis
Stack and pointers
Pointers
Stack
Heap
How can we analyze?
Benchmarking your code
Writing your first benchmark
Memory allocations
Common pitfalls
CPU profiling
Memory profiling
Profiling memory over time
Preparing to explore the trade-offs
Summary
Part 4: Connected Apps
10
Networking
The net package
TCP sockets
HTTP servers and clients
HTTP verbs
HTTP status codes
Putting it all together
Securing the connection
Certificates
Advanced networking
UDP versus TCP
Summary
11
Telemetry
Technical requirements
Logs
Zap versus slog
Logging for debugging or monitoring?
What to log?
What not to log?
Traces
Effective tracing
Distributed tracing
Metrics
What metric should we use?
The OTel project
OTel
Summary
12
Distributing Your Apps
Technical requirements
Go Modules
The routine using modules
CI
Caching
Static analysis
Releasing your application
Summary
Part 5: Going Beyond
13
Capstone Project – Distributed Cache
Technical requirements
Understanding distributed caching
System requirements
Requirements
Design and trade-offs
Creating the project
Thread safety
Choosing the right approach
Adding thread safety
The interface
TCP
HTTP
Others
Eviction policies
Sharding
Summary
14
Effective Coding Practices
Technical requirements
Reusing resources
Using sync.Pool in a network server
Using sync.Pool for JSON marshaling
Executing tasks once
singleflight
Effective memory mapping
API usage
Advanced usage with protection and mapping flags
Avoiding common performance pitfalls
Leaking with time.After
Defer in for loops
Maps management
Resource management
Handling HTTP bodies
Channel mismanagement
Summary
15
Stay Sharp with System Programming
Real-world applications
Dropbox’s leap of faith
HashiCorp – Go from day one
Grafana Labs – visualizing success with Go
Docker – building a container revolution with Go
SoundCloud – from Ruby to Go
Navigating the system programming landscape
Go release notes and blog
Community
Contribution
Experimentation
Resources for continued learning
Advanced Programming in the UNIX Environment by W. Richard Stevens
Learn C Programming - Second Edition: A beginner’s guide to learning the most powerful and general-purpose programming language with ease
Linux Kernel Programming - Second Edition: A comprehensive and practical guide to kernel internals, writing modules, and kernel synchronization
Linux System Programming Techniques: Become a proficient Linux system programmer using expert recipes and techniques
Operating Systems: Design and Implementation by Andrew S. Tanenbaum
Unix Network Programming by W. Richard Stevens
Linux System Programming Techniques: Become a proficient Linux system programmer using expert recipes and techniques
Mastering Embedded Linux Programming - Third Edition: Create fast and reliable embedded solutions with Linux 5.4 and the Yocto Project 3.1 (Dunfell)
Modern Operating Systems by Andrew S. Tanenbaum
The Art of UNIX Programming by Eric S. Raymond
Your system programming journey
Appendix
Hardware Automation
Automation in system programming
USB
Application
The goal
The /proc/mounts file
Reading the files on the flash drive
Partitions versus blocks versus devices versus disks
Open source to the rescue!
Interacting with USB events
Bluetooth
Detecting the smartwatch
Locking the screen
XDG dilemma
The Wayland conundrum
Summary
Index
Other Books You May Enjoy
Preface
System programming is a critical area of knowledge in software engineering. It matters most for professionals who want to write efficient, low-level code that interacts closely with the operating system. System Programming Essentials with Go is designed to guide you through the principles and practices necessary to stand up in system programming using Go. This book covers a broad range of topics, from basic system programming concepts to advanced techniques, providing a comprehensive toolkit for tackling real-world system programming challenges.
Who this book is for
This book is tailored for software engineers, architects, and developers who possess a basic understanding of programming and seek to deepen their knowledge of system design. It is ideal for those addressing complex design problems at work or simply interested in enhancing their skills with low-level programming in general. A foundational understanding of programming concepts and experience with at least one programming language is required.
What this book covers
Chapter 1
, Why Go?, provides an overview of Go’s suitability for building efficient and high-performance system software, providing you with the necessary knowledge and skills to leverage Go’s capabilities for system-level development. The chapter covers Go’s concurrency model, networking and I/O, low-level control, system calls, cross-platform support, and tooling, offering practical insights and examples for building robust system programs.
Chapter 2
, Refreshing Concurrency and Parallelism, provides an overview of the core aspects of Goroutines, data races, channels, and their interplay in the Go programming language. Understanding these principles is critical for implementing efficient concurrency, managing shared resources, and ensuring effective inter-goroutine communication.
Chapter 3
, Understanding System Calls, provides an overview of system calls and their practical applications. You will learn how to create symbolic links, unlink files, and manipulate filename paths. Also, you will better understand package OS and syscall in Go and learn how to develop and test a CLI program.
Chapter 4
, File and Directory Operations, provides an overview of handling filesystems in Go, focusing on detecting unsafe permissions, calculating directory sizes, and identifying duplicate files.
Chapter 5
, Working with System Events, provides comprehensive insights into building advanced and efficient system tools using Go, focusing on task scheduling, file monitoring, process management, and distributed locking.
Chapter 6
, Understanding Pipes in Inter-Process Communication, provides an exploration of the concept of pipes in Inter-Process Communication (IPC). It provides comprehensive information about anonymous pipes, named pipes (mkfifo), and how pipes interact with other programs.
Chapter 7
, Unix Sockets, provides an understanding of how UNIX sockets function, their types, and their role in IPC on UNIX and UNIX-like operating systems such as Linux.
Chapter 8
, Memory Management, focuses on the mechanisms and strategies underpinning garbage collection. We’ll explore the evolution of Go’s garbage collection, the distinctions between stack and heap memory allocations, and advanced techniques for efficient memory management.
Chapter 9
, Analyzing Performance, covers key optimization techniques for Go applications, including escape analysis, benchmarking, CPU profiling, and memory profiling. It explains how to improve memory usage with escape analysis, measure and compare code performance through benchmarking, identify hotspots with CPU profiling, and detect memory leaks using memory profiling.
Chapter 10
, Networking, delves into the fascinating world of Go network programming. Networking is essential to system programming, and Go provides powerful primitives for handling network communications. You will gain the expertise needed to create robust networked applications by exploring TCP, HTTP, and additional relevant protocols.
Chapter 11
, Telemetry, dives into how you can leverage industry tools to implement effective telemetry practices. From logs to traces and metrics, you’ll explore the tools and guidelines necessary to monitor your application efficiently.
Chapter 12
, Distributing Your Apps, explores the key concepts and practical applications of distributing applications using Go modules, continuous integration, and release strategies.
Chapter 13
, Capstone Project - Distributed Cache, guides you through the Capstone Project. This project will build a distributed cache system in Go with features such as Memcached or Redis. It will cover sharding strategies, eviction policies, consistency models, and technology choices, all while navigating the trade-offs that come with each decision.
Chapter 14
, Effective Coding Practices, explores the principles and techniques of efficient resource management in Go programming, specifically focusing on avoiding common pitfalls that can lead to performance issues and hinder overall efficiency. It dives into the intricacies of optimizing resource usage using the Go standard library, providing strategies for developers seeking to enhance the effectiveness of their Go applications.
Chapter 15
, Stay Sharp with System Programming, provides a continuous learning path to Go-based system programming based on real-world case studies. By gaining insight into how Go is utilized in actual applications, you can apply these lessons to their projects.
Appendix, Hardware Automation, explores how to utilize various tools to automate mundane tasks with USB drives and Bluetooth devices and monitor peripheral events. By understanding how to automate these processes, you will save valuable time and increase productivity in your daily lives.
To get the most out of this book
You will need to have an understanding of the basics of Golang.
If you are using the digital version of this book, we advise you to type the code yourself or access the code from the book’s GitHub repository (a link is available in the next section). Doing so will help you avoid any potential errors related to the copying and pasting of code.
Download the example code files
You can download the example code files for this book from GitHub at https://github.com/PacktPublishing/System-Programming-Essentials-with-Go
. If there’s an update to the code, it will be updated in the GitHub repository.
We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing/
. Check them out!
Conventions used
There are a number of text conventions used throughout this book.
Code in text: Indicates code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles. Here is an example: Lastly, update the main function to create a cache with a specified capacity and test the TTL and LRU features.
A block of code is set as follows:
func main() {
cache := NewCache(5) // Setting capacity to 5 for LRU
cache.startEvictionTicker(1 * time.Minute)
}
Any command-line input or output is written as follows:
go run main.go -port=:8080 -peers=http://localhost:8081
Tips or important notes
Appear like this.
Get in touch
Feedback from our readers is always welcome.
General feedback: If you have questions about any aspect of this book, email us at [email protected]
and mention the book title in the subject of your message.
Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you have found a mistake in this book, we would be grateful if you would report this to us. Please visit www.packtpub.com/support/errata
and fill in the form.
Piracy: If you come across any illegal copies of our works in any form on the internet, we would be grateful if you would provide us with the location address or website name. Please contact us at [email protected]
with a link to the material.
If you are interested in becoming an author: If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, please visit authors.packtpub.com
.
Share Your Thoughts
Once you’ve read System Programming Essentials with Go, we’d love to hear your thoughts! Please click here to go straight to the Amazon review page
for this book and share your feedback.
Your review is important to us and the tech community and will help us make sure we’re delivering excellent quality content.
Download a free PDF copy of this book
Thanks for purchasing this book!
Do you like to read on the go but are unable to carry your print books everywhere?
Is your eBook purchase not compatible with the device of your choice?
Don’t worry, now with every Packt book you get a DRM-free PDF version of that book at no cost.
Read anywhere, any place, on any device. Search, copy, and paste code from your favorite technical books directly into your application.
The perks don’t stop there, you can get exclusive access to discounts, newsletters, and great free content in your inbox daily
Follow these simple steps to get the benefits:
Scan the QR code or visit the link below
https://packt.link/free-ebook/9781837634132
Submit your proof of purchase
That’s it! We’ll send your free PDF and other benefits to your email directly
Part 1: Introduction
In this part, we will explore the foundational aspects of using Go for system programming. You will learn about the best practices in managing concurrency, and ensuring efficient cross-platform development. This section provides a closer look at why Go is a powerful choice for building high-performance system software and how to leverage its features to support real-world scenarios.
This part has the following chapters:
Chapter 1
, Why Go?
Chapter 2
, Refreshing Concurrency and Parallelism
1
Why Go?
At some point in your programming journey, your programs performed I/O-related tasks such as creating and removing files and directories. They may have orchestrated the creation of new processes and the execution of other programs or even facilitated communication between threads and processes on the same computer and between processes on different computers connected via a network.
When our programs center on using a low-level set of tasks, we categorize them as system programming.
It is alleged that system programming is tedious. But I do not see it this way at all! In fact, it is quite the opposite – an enjoyable and entertaining experience. It is like being a magician. You get to control the operating system and hardware, and you can make things happen that would be impossible in other languages.
In this chapter, we discuss why Go is an excellent fit for building efficient, high-performance system software to support real-world scenarios.
In this chapter, we are going to cover the following main topics:
Choosing Go
Concurrency and goroutines
Interacting with the OS
Tooling
Cross-platform development with Go
By the end of this chapter, you will have a grasp of where Go is in the ecosystem of system programming, the importance of the Go concurrency model to build efficient and high-performance system software, how Go chooses to interact with the OS, the Go approach to cross-platform development, and the main commands in the Go built-in tooling.
Choosing Go
There are plenty of languages in the system programming space nowadays: some are well established, such as C and C++; some form a wave of newcomers, such as Zig, Rust, and Odin; and others claim the title of C/C++ killer,
with their pledges of impressive performance.
Sure, we can use all of them and achieve outstanding results. Still, we could fall into hidden traps such as a steep learning curve, high cognitive load, a lack of community and support, inconsistent APIs with constant breaking changes, and a lack of adoption.
Go’s design philosophy emphasizes simplicity, expressiveness, robustness, and efficiency. Its support for concurrency and strong dependency management, as well as its focus on composition, make it a compelling choice for system programming. Its creators aimed to build a language that provides powerful building blocks without unnecessary complexity, which makes writing, reading, understanding, and maintaining system-level code easier. People with programming experience usually take two weeks to get acquainted with Go. While they may not be considered experts, they can confidently read standard Go code and write basic to medium-complexity programs without struggle.
Also, Go is excellent for system programming because the language has a Unix-minded design by checking all the boxes for simplicity. Many programmers who are proficient in Python and Ruby often transition to Go, as it allows them to retain their level of expressiveness while achieving improved performance and the capability to work with concurrency.
It is worth noting that Go’s philosophy doesn’t prioritize zero cost in terms of CPU usage. Instead, the language aims to reduce the effort demanded from programmers, which is considered more significant and, as a by-product, makes the experience enjoyable.
One of the leading criticisms of using Go for system programming is the garbage collector (GC), specifically its pauses and explicit memory limits. If you still have this pet peeve with Go, don’t worry. In Chapter 6
, we’ll see that more granular memory management is available from Go 1.20 and above.
Note
In a GC pause worst-case scenario, the stop-the-world time is typically less than 100 microseconds.
Concurrency and goroutines
One of the most essential features of Go is its concurrency model. Concurrency is the ability to run multiple tasks at the same time. In system programming, executing many tasks in parallel is essential for improving the performance and responsiveness of our programs.
Concurrency
Real-time systems demand precision, with concurrency being a pivotal factor. These systems coordinate tasks with exceptional timing, particularly in scenarios where even milliseconds matter. Concurrency offers significant advantages by increasing throughput (a measure of how many units of information a system can process in a given amount of time) while decreasing task completion times. Real-life instances show how concurrency improves responsiveness, making systems more flexible and tasks more efficient. Moreover, concurrency’s isolation abilities guarantee data integrity by preventing interference.
System programming involves a diverse range of tasks, from CPU-bound to I/O-bound. Concurrency orchestrates this diversity by allowing CPU-bound tasks to progress while I/O-bound tasks await resources.
Later, in Chapter 10
, when we discuss distributed systems, the importance of concurrency will shine. It orchestrates tasks across an application or even different nodes in the network, which is ideal for managing large-scale concurrency.
Goroutines
Go’s concurrency model relies on goroutines and channels. Goroutines are lightweight execution threads, often referred to as green threads. Creating them is cost-effective. Unlike conventional threads, they exhibit remarkable efficiency, enabling thousands of goroutines to run simultaneously on just a few OS threads.
Channels, on the other hand, provide a mechanism for goroutines to communicate and synchronize without resorting to locks. This approach is inspired by the Communicating Sequential Process (CSP) (https://www.cs.cmu.edu/~crary/819-f09/Hoare78.pdf
) formalism, emphasizing coordinated interactions between concurrent components.
Diverging from numerous other programming languages that depend on external libraries or threading constructs for concurrency, Go incorporates concurrency seamlessly into its core language design. This design decision leads to code that is not only easier to comprehend but also less susceptible to errors, as the complexities of threading are abstracted.
CSP-inspired model
Go’s concurrency model draws inspiration from CSP, a formal language for describing concurrent systems. CSP focuses on communication and synchronization between concurrently executing entities. Unlike traditional multi-threaded programming, CSP and Go prioritize communication through channels instead of shared memory, reducing complexity and potential hazards. Synchronization and coordination are essential, with CSP using channels for process synchronization and Go using similar channels to coordinate goroutines. Safety and isolation are key, as both languages ensure safe interaction through channels, enhancing predictability and reliability. Go’s channels directly realize CSP’s communication-based approach, providing a safe way for goroutines to exchange data without the pitfalls of shared memory and locks.
Share by communication
The famous Go proverb, Don’t communicate by sharing memory, share memory by communicating
is often a source of discussion and misinterpretation. Still, reading it as Share by communicating, not by locking
would be more precise, mainly because mainstream languages often rely on locks to protect shared data, leading to potential issues such as deadlocks and race conditions. Go encourages a different paradigm: sharing data through channels by sending and receiving messages. This share by communicating
philosophy reduces the need for explicit locks and promotes a safer concurrency environment.
If you are into functional programming, I have great news for you. In Go, data is not implicitly shared between goroutines. In other words, the data is copied. Did you see the concern with data immutability? This stands in contrast to languages, where shared memory is the default mode of communication between threads. Go’s emphasis on explicit communication via channels helps avoid the unintended data sharing and race conditions that can arise in traditional threading models. Another benefit of this model is that there is no callback hell since every interaction with concurrent code is often read in a procedural manner. A regular Go function can be used in procedural code without tying the signatures with extra keywords.
Note
Callback hell, also known as the pyramid of doom,
is a term used in programming to describe a situation where nested and interdependent callback functions make the code difficult to read, understand, and maintain. This typically occurs in asynchronous programming environments, such as JavaScript, where callbacks are used to handle asynchronous operations.
In the next chapter, we will refresh all concepts of concurrency and its building blocks to prepare you for interacting with the OS interfaces.
In addition to its concurrency model, Go also provides a way to interact with the operating system at a low level. This is essential for system programming, where you often need to control the OS and hardware.
Interacting with the OS
Go’s approach to system calls is designed to be safe and efficient, especially in the context of its concurrency model.
In Go, system calls are comparatively lower-level in comparison to certain other programming languages. It can be helpful if you need fine-grained control over system resources, but it also means you’re dealing with more low-level details.
Making system calls often requires understanding the underlying operating system APIs and conventions. The side effect is that it can introduce a steeper learning curve if you are new to systems programming or lower-level development.
Not familiar with system calls? Fear not! The book’s second part will explore and experiment with them in detail to cover the main aspects we need to progress in our system programming journey.
Tooling
Go is like a toolbox. It has everything we need to build great software, so we don’t need anything more than its standard tools to create our programs.
Let’s explore the principal tools that facilitate building, testing, running, error-checking, and code formatting.
go build
The go build command is used to compile Go code into an executable binary that you can run.
Let’s see an example.
Assume you have a Go source file named main.go containing the following code:
package main
import fmt
func main() {
fmt.Println(Hello, Go!
)
}
You can compile it using the go build command:
go build main.go
This will generate an executable binary named main (or main.exe on Windows). You can then run the binary to see the output:
./main
go test
The go test command is used to run tests on your Go code. It automatically finds test files and runs the associated test functions.
Here’s an example.
Assume you have a Go source file named math.go containing a function to add two numbers:
package math
func Add(a, b int) int {
return a + b
}
You can create a test file named math_test.go to write tests for the Add function:
package math
import testing
func TestAdd(t *testing.T) {
result := Add(2, 3)
if result != 5 {
t.Errorf(Expected 5, but got %d
, result)
}
}
Run the tests using the go test command:
go test
go run
The go run command allows you to run Go code directly without explicitly compiling it into an executable.
Let’s see this using an example.
Assume you have a Go source file named hello.go containing the following code:
package main
import fmt
func main() {
fmt.Println(Hello, Go!
)
}
You can directly run the code using the go run command:
go run hello.go
This will execute the code and print Hello, Go! to the console.
go vet
We use the go vet command to check our Go code for potential errors or suspicious constructs. It employs heuristics that may not ensure all reports are actual issues, but it can uncover errors not caught by the compilers.
Here’s an example.
Assume you have a Go source file named error.go containing the following code with an intentional error:
package main
import fmt
func main() {
movie_year := 1999
movie_title := The Matrix
fmt.Printf(In %s, %s was released.\n
, movie_year, movie_title)
}
You can use the go vet command to check for errors:
go vet error.go
It might report a warning such as this: Printf format %s has arg 1999 of wrong type int.
go fmt
The go fmt command is used to format your Go code according to the Go programming style guidelines. It automatically adjusts code indentation, spacing, and more.
Let’s see an example for this too.
Assume you have a Go source file named unformatted.go containing improperly formatted code:
package main
import fmt
func main() {
msg:=Hello
fmt.Println(msg)
}
You can format the code using the go fmt command:
go fmt unformatted.go
It will update the code to match the standard formatting conventions:
package main
import fmt
func main() {
msg := Hello
fmt.Println(msg)
}
Now that we have a good grasp of the basic tools, we can start familiarizing ourselves with Go’s cross-platform capabilities.
Cross-platform development with Go
Cross-platform development with Go is a breeze. You can write code running on various operating systems and architectures with ease.
Cross-platform development with Go can be achieved by using the GOOS and GOARCH environment variables. The GOOS environment variable specifies the OS you want to target, and the GOARCH environment variable specifies your target architecture.
For example, assume you have a Go source file named main.go:
package main
import fmt
func main() {
fmt.Println(This program runs in any OS!
)
}
To compile code for