2 dl The UNIX Programming Environment Brian W. Kernighan Rob Pike Bell Laboratories Murray Hill, New Jersey PRENTICE-HALL, INC. Englewood Cliffs, New Jersey 07632 * UNIX is a Trademark of Bell LaboratoriesLibrary of Congress Catalog Card Number 8 3 -6 285 1 Prentice-Hall Software Series Brian W. Kernighan, Advisor Editorial/production supervision: Ros Herion Cover design: Photo Plus Art, Celine Brandes Manufacturing buyer: Gordon Osbourne Copyright © 1984 by Bell Telephone Laboratories, Incorporated. Alll rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopy- ing, recording, or otherwise, without the prior written permission of the publisher. Printed in the United States of America. Published simultaneously in Canada. This book was typeset in Times Roman and Courier by the authors, using a Mer- genthaler Linotron 202 phototypesetter driven by a VAX-11/750 running the 8th Edition of the UNIX operating system. UNIX is a trademark of Bell Laboratories. DEC, PDP and VAX are trademarks of Digital Equipment Corporation. 20:19 18 17 16 1S 14 ISBN O-13-937b99-2 ISBN O-13-937?b81-X {PBK} PRENTICE-HALL INTERNATIONAL, INC., London PRENTICE-HALL OF AUSTRALIA PTY. LIMITED, Sydney EDITORA PRENTICE-HALL DO BRASIL, LTDA., Rio de Janeiro PRENTICE-HALL CANADA INC., Toronto PRENTICE-HALL OF INDIA PRIVATE LIMITED, New Delhi PRENTICE-HALL OF JAPAN, INC., Tokyo PRENTICE-HALL OF SOUTHEAST ASIA PTE. LTD., Singapore WHITEHALL BOOKS LIMITED, Wellington, New Zealand3. 4. Preface UNIX for Beginners 1.1 Getting started 1.2. Day-to-day use: files and common commands 1.3 More about files: directories 1.4 The shell 1.5 The rest of the UNIX system The File System 2.1. The basics of files 2.2 What’s in a file? 2.3. Directories and filenames 2.4. Permissions 2.5. Inodes 2.6. The directory hierarchy 2.7 Devices Using the Shell 3.1 Command line structure 3.2 Metacharacters 3.3 Creating new commands 3.4 Command arguments and parameters 3.5 Program output as arguments 3.6 Shell variables 3.7 More on I/O redirection 3.8 Looping in shell programs 3.9 bundle: putting it all together 3.10 Why a programmable shell? Filters 4.1 The grep family 4.2. Other filters iii CONTENTS 101 102 106< CONTENTS. 4.3 The stream editor sea 108 4.4 The awk pattern scanning and processing language 114 4.5 Good files and good filters 130 5. Shell Programming 133 5.1 Customizing the cal command 133 5.2 Which command is which? 138 5.3 while and until loops: watching for things 144 5.4 Traps: catching interrupts 150 5.5 Replacing a file: overwrite 152 5.6 zap: killing processes by name 156 5.7 The pick command: blanks vs. arguments 159 5.8 The news command: community service messages 162 5.9 get and put: tracking file changes 165 5.10 A look back 169 6. Programming with Standard /O 171 6.1 Standard input and output: vis 172 6.2 Program arguments: vis version 2 174 6.3 File access: vis version 3 176 6.4 A screen-at-a-time printer: p 180 6.5 Anexample: pick 186 6.6 On bugs and debugging 187 6.7 Anexample: zap 190 6.8 An interactive file comparison program: idiff 192 6.9 Accessing the environment 199 7 UNIX System Calls 201 7.1 Low-level /O 201 7.2 File system: directories 208 7.3 File system: inodes 214 7.4 Processes 220 7.5 Signals and interrupts 225 8. Program Development 233 8.1 Stage 1: A four-function calculator 234 8.2 Stage 2: Variables and error recovery 242 8.3 Stage 3: Arbitrary variable names; built-in functions 245 8.4 Stage 4: Compilation into a machine 258 8.5 Stage 5: Control flow and relational operators 266 8.6 Stage 6: Functions and procedures; input/output 273 8.7 Performance evaluation 284 8.8 A look back 28610. Document Preparation 9.1 The ms macro package 9.2 The troff level 9.3 The tbl and eqn preprocessors 9.4 The manual page 9.5 Other document preparation tools Epilog Appendix 1: Editor Summary Appendix 2: hoc Manual Appendix 3: hoc Listing Index CONIENLS. v 289 290 297 301 308 313 318 319 329 335 349= 43 44 4.5 CONTENTS The stream editor sed The awk pattern scanning and processing language Good files and good filters 5. Shell Programming 5.1 5.2 5.3 5.4 5.5 5.6 5.7 5.8 5.9 Customizing the cal command Which command is which? while and until loops: watching for things Traps: catching interrupts Replacing a file: overwrite zap: killing processes by name The pick command: blanks vs. arguments The news command: community service messages get and put: tracking file changes 5.10 A look back 6. Programming with Standard 1/0 Standard input and output: vis Program arguments: vis version 2 File access: vis version 3 A screen-at-a-time printer: p An example: pick On bugs and debugging An example: zap An interactive file comparison program: idiff Accessing the environment 7. UNIX System Calls el 7.2 73 74 75 Low-level 1/0 File system: directories File system: inodes Processes Signals and interrupts 8. Program Development 8.1 8.2 8.3 8.4 8.5 8.6 8.7 8.8 Stage 1: A four-function calculator Stage 2: Variables and error recovery Stage 3: Arbitrary variable names; built-in functions Stage 4: Compilation into a machine Stage 5: Control flow and relational operators Stage 6: Functions and procedures; input/output Performance evaluation A look back 108 130 133 133 13810. Document Preparation 9.1 The ms macro package 9.2 The trofé level 9.3 The tbi and eqn preprocessors 9.4 The manual page 9.5. Other document preparation tools Epilog Appendix 1: Editor Summary Appendix 2: hoc Manual Appendix 3: hoc Listing Index CONTENTS. v 289 290 297 301 308 313 315 319 329 335 349PREFACE “The number of UNIX installations has grown to 10, with more expected.” (The UNIx Programmer's Manual, 2nd Edition, June, 1972.) The UNIXt operating system started on a cast-off DEC PDP-7 at Bell Labora- tories in 1969. Ken Thompson, with ideas and support from Rudd Canaday, Doug Mcliroy, Joe Ossanna, and Dennis Ritchie, wrote a small general- purpose time-sharing system comfortable enough to attract enthusiastic users and eventually enough credibility for the purchase of a larger machine — a PDP-11/20. One of the early users was Ritchie, who helped move the system to the PDP-11 in 1970. Ritchie also designed and wrote a compiler for the C programming language. In 1973, Ritchie and Thompson rewrote the UNIX ker- nel in C, breaking from the tradition that system software is written in assem- bly language. With that rewrite, the system became essentially what it is today. Around 1974 it was licensed to universities “for educational purposes” and a few years later became available for commercial use. During this time, UNIX systems prospered at Bell Labs, finding their way into laboratories, software development projects, word processing centers, and operations support systems in telephone companies. Since then, it has spread world-wide, with tens of thousands of systems installed, from microcomputers to the largest main- frames. What makes the UNIX system so successful? We can discern several rea- sons. First, because it is written in C, it is portable — UNIX systems run on a range of computers from microprocessors to the largest mainframes; this is a strong commercial advantage. Second, the source code is available and written in a high-level language, which makes the system easy to adapt to particular requirements. Finally, and most important, it is a good operating system, + UNIX is a trademark of Bell Laboratories. “UNIX” is not an acronym, but a weak pun on MUL- TICS, the operating system that Thompson and Ritchie worked on before UNIX. viiPREFACE especially for programmers. The UNIX programming environment is unusually rich and productive. Even though the UNIX system introduces a number of innovative programs and techniques, no single program or idea makes it work well. Instead, what makes it effective is an approach to programming, a philosophy of using the computer. Although that philosophy can’t be written down in a single sen- tence, at its heart is the idea that the power of a system comes more from the relationships among programs than from the programs themselves. Many UNIX programs do quite trivial tasks in isolation, but, combined with other pro- grams, become general and useful tools. Our goal in this book is to communicate the UNIX programming philosophy. Because the philosophy is based on the relationships between programs, we must devote most of the space to discussions about the individual tools, but throughout run the themes of combining programs and of using programs to build programs. To use the UNIX system and its components well, you must understand not only how to use the programs, but also how they fit into the environment. As the UNIX system has spread, the fraction of its users who are skilled in its application has decreased. Time and again, we have seen experienced users, ourselves included, find only clumsy solutions to a problem, or write programs to do jobs that existing tools handle easily. Of course, the elegant solutions are not easy to see without some experience and understanding. We hope that by reading this book you will develop the understanding to make your use of the system — whether you are a new or seasoned user — effective and enjoyable. We want you to use the UNIX system well. We are aiming at individual programmers, in the hope that, by making their work more productive, we can in turn make the work of groups more productive. Although our main target is programmers, the first four or five chapters do not require programming experience to be understood, so they should be helpful to other users as well. Wherever possible we have tried to make our points with real examples rather than artificial ones. Although some programs began as examples for the book, they have since become part of our own set of everyday programs. All examples have been tested directly from the text, which is in machine-readable form. The book is organized as follows. Chapter | is an introduction to the most basic use of the system. It covers logging in, mail, the file system, commonly- used commands, and the rudiments of the command interpreter. Experienced users can skip this chapter. Chapter 2 is a discussion of the UNIX file system. The file system is central to the operation and use of the system, so you must understand it to use the system well. This chapter describes files and directories, permissions and file modes, and inodes. It concludes with a tour of the file system hierarchy and an explanation of device files.