Harley Hahn's Guide to Unix and Linux - PDF Free Download (2024)

A PERSONAL NOTE FROM HARLEY HAHN

This book will change your life. That’s a strange thing to say about a computer book but, as sure as you are reading this introduction, your life will be different by the time you ﬁnish the book. You will think differently and you will approach problems differently. You see, your computer is not a lifeless piece of machinery. It is a dynamic tool that interacts with your very thought processes. Whenever you use a computer it becomes, for better or for worse, an extension of your mind. This means that, over an extended period of time, the computer system you use changes how you think. Indeed, we might classify systems as mentally “good” or “bad” depending on how they affect the minds of their users. In this sense, Unix is, without a doubt, the very best computer system ever invented (and Linux is a type of Unix). When you use Unix, you are not working with a machine. You are working with the people who designed Unix. Every line and every picture you see on your monitor was put there by a person. Every tool you use was invented by a person. Every technical term and every concept you learn was created by a person. When you use Unix, you are interacting with these people, just as surely as you are interacting with me as you read this page. Unix and Linux are wonderful because they were developed by bright, creative people who delighted in thinking well. These people were not only very, very smart; they knew what they were doing and they loved their work. This means that, whenever you use a Unix or Linux system, you are forging a mental relationship with some of the smartest, most accomplished (and satisﬁed) programmers and computer scientists who ever lived. Such a partnership can’t help but have a positive effect on you. The fact is it really doesn’t matter why you want to learn Unix or Linux, or why you picked up this book. Perhaps you love computers and you have a burning desire to learn. Perhaps you are taking a class and this will be your textbook. Perhaps you have a job and you are required to use Unix or Linux. It doesn’t matter.

83977_fm_i_xxxiv.indd i

1/9/2008 12:51:58 PM

You are about to begin a long, complex, and very rewarding journey. In the days, weeks, and months to come, you will encounter new ideas and build new skills, far beyond anything you can imagine at this moment. As you do, your mind will change for the better, your thought processes will improve, and your way of looking at the world and at yourself will change. This is not your average computer book. (I’m sure you realize that by now.) Aside from a large amount of technical material, there are hints, jokes and a lot of plain-spoken advice. I did the very best I could to show what you really need to know. This is not a computer manual. This is not a compendium of impersonal details. This is one person (me) talking to another person (you). I will make you a promise. As you teach yourself Unix, I will be with you, every step of the way. What you are holding in your hand is my guide to learning Unix and Linux, and now it is yours. Are you ready? Good. Turn to page 1 and start reading.

83977_fm_i_xxxiv.indd ii

1/9/2008 12:51:59 PM

G U I D E

T O

Unix and Linux Harley Hahn

83977_fm_i_xxxiv.indd iii

1/9/2008 12:51:59 PM

HARLEY HAHN’S GUIDE TO UNIX AND LINUX Published by McGraw-Hill, a business unit of The McGraw-Hill Companies, Inc., 1221 Avenue of the Americas, New York, NY 10020. Copyright 2009 by Harley Hahn. All rights reserved. No part of this publication may be reproduced or distributed in any form or by any means, or stored in a database or retrieval system, without the prior written consent of The McGraw-Hill Companies, Inc., including, but not limited to, in any network or other electronic storage or transmission, or broadcast for distance learning. Some ancillaries, including electronic and print components, may not be available to customers outside the United States. The name “Harley Hahn” is a registered trademark of Harley Hahn. This book is printed on acid-free paper. 1 2 3 4 5 6 7 8 9 0 DOC/DOC 0 9 8 ISBN 978-0-07-128397-7 MHID 0-07-128397-8 www.mhhe.com

hah83977_fm_i_xxxiv.indd iv

1/11/2008 10:33:31 AM

ABOUT THE AUTHOR

Harley Hahn is a writer, computer expert, philosopher, humorist, artist and musician. In all, he has written 32 books, which have sold more than 2 million copies. This is his 7th Unix book. The book Harley Hahn’s Internet Yellow Pages was the ﬁrst Internet book in history to sell more than 1,000,000 copies. Two of his other books, Harley Hahn’s Internet Insecurity and Harley Hahn’s Internet Advisor have been nominated for a Pulitzer Prize. These books, along with others, have made Hahn the best-selling Internet author of all time. Hahn has written numerous articles, essays and stories on a wide variety of topics, including romance, philosophy, economics, culture, medicine and money. Much of his writing is available on his Web site www.harley.com. Hahn’s work — including a complete set of his books — is archived by the Special Collections Department of the library at the University of California at Santa Barbara. Hahn has a degree in Mathematics and Computer Science from the University of Waterloo (Canada), and a graduate degree in Computer Science from the University of California at San Diego. He also studied medicine at the University of Toronto Medical School. Hahn has been the recipient of a number of honors and awards, including a National Research Council (Canada) post-graduate scholarship and the 1974 George Forsythe Award from the ACM (Association for Computing Machinery). Hahn enjoys writing computer books, because “I get to sleep in, and I like telling people what to do.”

83977_fm_i_xxxiv.indd v

1/9/2008 12:51:59 PM

To Linda, for patience and encouragement.

83977_fm_i_xxxiv.indd vi

1/9/2008 12:51:59 PM

LIST OF CHAPTERS AND APPENDIXES

CHAPTER CHAPTER CHAPTER CHAPTER CHAPTER CHAPTER CHAPTER CHAPTER CHAPTER CHAPTER CHAPTER CHAPTER CHAPTER

1: 2: 3: 4: 5: 6: 7: 8: 9: 10: 11: 12: 13:

CHAPTER CHAPTER CHAPTER CHAPTER CHAPTER CHAPTER

14: 15: 16: 17: 18: 19:

CHAPTER CHAPTER CHAPTER CHAPTER CHAPTER CHAPTER CHAPTER

20: 21: 22: 23: 24: 25: 26:

Introduction to Unix . . . . . . . . . . . . . . . What Is Unix? What is Linux?. . . . . . . . . . . The Unix Connection . . . . . . . . . . . . . . . Starting to Use Unix . . . . . . . . . . . . . . . . GUIs: Graphical User Interfaces . . . . . . . . The Unix Work Environment . . . . . . . . . . Using the Keyboard With Unix . . . . . . . . . Programs to Use Right Away . . . . . . . . . . . Documentation: The Unix Manual and Info . Command Syntax . . . . . . . . . . . . . . . . . . The Shell . . . . . . . . . . . . . . . . . . . . . . . Using the Shell: Variables and Options . . . Using the Shell: Commands and Customization . . . . . . . . . . . . . . . Using the Shell: Initialization Files . . . . . . Standard I/O, Redirection, and Pipes . . . . . Filters: Introduction and Basic Operations . Filters: Comparing and Extracting . . . . . . Filters: Counting and Formatting . . . . . . . Filters: Selecting, Sorting, Combining, and Changing . . . . . . . . . . . . . . . . . . . Regular Expressions . . . . . . . . . . . . . . . . Displaying Files . . . . . . . . . . . . . . . . . . . The vi Text Editor . . . . . . . . . . . . . . . . . The Unix Filesystem . . . . . . . . . . . . . . . . Working With Directories . . . . . . . . . . . . Working With Files . . . . . . . . . . . . . . . . Processes and Job Control . . . . . . . . . . . .

. . . . . . . . . . . .

. 1 . 9 . 37 . 55 . 73 . 93 131 161 189 223 239 255

. . . . . .

277 327 345 373 395 421

. . . . . . . .

447 497 521 559 627 659 715 767

List of Chapters and Appendixes

83977_fm_i_xxxiv.indd vii

vii

1/9/2008 12:51:59 PM

APPENDIX A

Summary of Unix Commands Covered in This Book . . 817

APPENDIX B

Summary of Unix Commands by Category . . . . . . . . 821

APPENDIX C

Summary of vi Commands . . . . . . . . . . . . . . . . . . 827

APPENDIX D

The ASCII Code . . . . . . . . . . . . . . . . . . . . . . . . . 833

APPENDIX E

What to Do If You Forget the Root Password . . . . . 838

APPENDIX F

Time Zones and 24-Hour Time. . . . . . . . . . . . . . . . 841

APPENDIX G

Shell Options and Shell Variables . . . . . . . . . . . . 846

GLOSSARY

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 851

. . . . . . . . . . . . . . . . . 891 GENERAL INDEX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 895 QUICK INDEX OF UNIX COMMANDS . . . . . . . . . Inside back cover QUICK INDEX FOR THE VI TEXT EDITOR

viii Harley Hahn’s Guide to Unix and Linux

83977_fm_i_xxxiv.indd viii

1/9/2008 12:51:59 PM

TA B L E O F C O N T E N T S

List of Figures . . . . . . . . . A Note to Teachers . . . . . . . Acknowledgments . . . . . . . How This Book Was Developed

CHAPTER 1:

. . . .

. . xxii . .xxvi . . xxx . xxxiv

Introduction to Unix . . . . . . . . . . . . . . . . . . 1 Why Use Unix? . . . . . . . . . . . . . . . . . . The Unix Language . . . . . . . . . . . . . . . . Hints for Learning Unix . . . . . . . . . . . . . People Who Don’t Know They Are Using Unix . People Who Do Know They Are Using Unix . . Getting the Most from This Book . . . . . . . . What I Assume in This Book . . . . . . . . . . . What I Do Not Assume in This Book . . . . . . How to Use This Book . . . . . . . . . . . . . .

CHAPTER 2:

. . . .

. . . . . . . . .

.2 .3 .4 .4 .5 .5 .6 .6 .6

What Is Unix? What is Linux?. . . . . . . . . . . . . . 9 What Is an Operating System? . . . . . . . . . What Is the Kernel? . . . . . . . . . . . . . . . Unix = Kernel + Utilities . . . . . . . . . . . . “Unix” Used to Be a Speciﬁc Name . . . . . . “Unix” Is Now a Generic Name . . . . . . . . The Free Software Foundation . . . . . . . . . Excerpts from the GNU Manifesto . . . . . . . The GPL and Open Source Software . . . . . . Unix in the 1970s: From Bell Labs to Berkeley Unix in the 1980s: BSD and System V . . . . . Unix in 1991: Waiting for... . . . . . . . . . . . ...Mr. Right, Linus Torvalds . . . . . . . . . . . Linux Distributions . . . . . . . . . . . . . . . BSD Distributions . . . . . . . . . . . . . . . . What Type of Unix Should You Use?. . . . . . How Do You Get Linux or FreeBSD? . . . . . What Is Unix? What Is Linux? . . . . . . . . .

. . . . . . . . . . . . . . . . .

. .9 . 11 . 12 . 13 . 14 . 14 . 16 . 18 . 19 . 20 . 22 . 24 . 28 . 29 . 30 . 32 . 35

Table of Contents

83977_fm_i_xxxiv.indd ix

1/9/2008 12:51:59 PM

CHAPTER 3:

The Unix Connection . . . . . . . . . . . . . . . . . .37 Humans, Machines and Aliens . . . . . . . . . . In the Olden Days, Computers Were Expensive . Host and Terminals . . . . . . . . . . . . . . . . Terminal Rooms and Terminal Servers . . . . . The Console . . . . . . . . . . . . . . . . . . . . The Unix Connection . . . . . . . . . . . . . . . Hosts Without Consoles . . . . . . . . . . . . . The Client/Server Relationship . . . . . . . . . . What Happens When You Press a Key? . . . . . Character Terminals and Graphics Terminals . . The Most Common Types of Terminals . . . . .

CHAPTER 4:

. . . . . . . . . . .

. 37 . 38 . 41 . 43 . 45 . 46 . 48 . 49 . 50 . 52 . 53

Starting to Use Unix . . . . . . . . . . . . . . . . . . .55 The System Administrator . . . . . . . . . . . . . . . . . . . . . . . 55 Userids and Passwords . . . . . . . . . . . . . . . . . . . . . . . . . 56 Logging In (Starting Work with Unix) . . . . . . . . . . . . . . . . . 57 What Happens After You Log In? . . . . . . . . . . . . . . . . . . . 59 Getting Down to Work: The Shell Prompt . . . . . . . . . . . . . . 61 Logging Out (Stopping Work with Unix): logout, exit, login . 62 Upper- and Lowercase . . . . . . . . . . . . . . . . . . . . . . . . . 63 A Sample Session with Unix . . . . . . . . . . . . . . . . . . . . . . 64 Changing Your Password: passwd . . . . . . . . . . . . . . . . . . 66 Choosing a Password . . . . . . . . . . . . . . . . . . . . . . . . . . 67 Checking If Someone Has Been Using Your Unix Account: last. . . . . . . . . . . . . . . . . . . . . . 69 Userids and Users . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70 The Superuser Userid: root . . . . . . . . . . . . . . . . . . . . . . 70 Having Fun While Practicing Safe Computing . . . . . . . . . . . . 71

CHAPTER 5:

GUIs: Graphical User Interfaces . . . . . . . . . . .73 What is a GUI? . . . . . . . . . . . . X Window . . . . . . . . . . . . . . Who Is in Charge of X Window? . . Layers of Abstraction . . . . . . . . The Window Manager . . . . . . . The Desktop Environment . . . . . Layers of Abstraction: Revisited . . How the Unix Companies Blew It . KDE and Gnome . . . . . . . . . . CDE and Total Cost of Ownership . Choosing a Desktop Environment . The Grandmother Machine . . . .

83977_fm_i_xxxiv.indd x

. . . . . . . . . . . .

. 73 . 75 . 77 . 78 . 78 . 79 . 81 . 81 . 82 . 85 . 87 . 90

Harley Hahn’s Guide to Unix and Linux

1/9/2008 12:52:00 PM

CHAPTER 6:

The Unix Work Environment . . . . . . . . . . . . .93 Doing More Than One Thing at a Time: Part I . . . . . . . . . The GUI and the CLI . . . . . . . . . . . . . . . . . . . . . . . Logging In and Logging Out with a GUI . . . . . . . . . . . . Runlevels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Does Microsoft Windows Have Runlevels? . . . . . . . . . . . Learning to Use a GUI . . . . . . . . . . . . . . . . . . . . . . Of Mice and Menus . . . . . . . . . . . . . . . . . . . . . . . . Resizing, Minimizing, Maximizing and Closing Windows . . . Controlling the Focus: Task Switching . . . . . . . . . . . . . . Multiple Desktops / Workspaces . . . . . . . . . . . . . . . . . Terminal Windows . . . . . . . . . . . . . . . . . . . . . . . . Virtual Consoles . . . . . . . . . . . . . . . . . . . . . . . . . . The One and Only Console . . . . . . . . . . . . . . . . . . . . Selecting and Inserting . . . . . . . . . . . . . . . . . . . . . . Copying and Pasting . . . . . . . . . . . . . . . . . . . . . . . Working as Superuser: su . . . . . . . . . . . . . . . . . . . . Entering a Single Command as Superuser: sudo . . . . . . . Conﬁguration Files . . . . . . . . . . . . . . . . . . . . . . . . Looking Inside a Conﬁguration File . . . . . . . . . . . . . . . Shutting Down and Rebooting: init, reboot, shutdown What Happens When the System Starts or Stops? dmesg . . . Doing More Than One Thing at a Time: Part II . . . . . . . .

CHAPTER 7:

. . . . . . . . . . . . . . . . . . . . . .

. 93 . 96 . 97 . 98 100 101 102 104 107 108 110 113 116 116 117 118 121 122 124 125 126 127

Using the Keyboard With Unix . . . . . . . . . . . 131 The First Unix Terminals . . . . . . . . . . . . . . . . . . . . . . . Teletypes and the Unix Culture. . . . . . . . . . . . . . . . . . . . Termcap, Terminfo and curses . . . . . . . How Does Unix Know What Type of Terminal You Are Using? . . The Modiﬁer Keys; The Key . . . . . . . . . . . . . . . . . The Unix Keyboard Signals . . . . . . . . . . . . . . . . . . . . . . Signals to Use While Typing: erase, werase, kill . . . . . . The Strangeness of and . . . . . . . . . . The Case of the Mysterious ^H . . . . . . . . . . . . . . . . . . . . Stopping a Program: intr . . . . . . . . . . . . . . . . . . . . . . Another Way to Stop a Program: quit . . . . . . . . . . . . . . . Pausing the Display: stop, start . . . . . . . . . . . . . . . . . The End of File Signal: eof . . . . . . . . . . . . . . . . . . . . . The Shell and the eof Signal . . . . . . . . . . . . . . . . . . . . . Bash: Trapping the eof Signal . . . . . . . . . . . . . . . . . . . . Korn Shell: Trapping the eof Signal. . . . . . . . . . . . . . . . . C-Shell: Trapping the eof Signal . . . . . . . . . . . . . . . . . . Displaying Key Mappings: stty -a . . . . . . . . . . . . . . . .

131 133 134 137 138 139 140 142 142 145 146 147 148 149 149 150 150 151

Table of Contents

83977_fm_i_xxxiv.indd xi

1/9/2008 12:52:00 PM

Changing Key Mappings: stty . . . . . . . . . . Command Line Editing . . . . . . . . . . . . . . . RETURN AnD LINEFEED . . . . . . . . . . . . . The Importance oF NEWLINE . . . . . . . . . . . An Important Use for ^J: stty sane, reset . The Fable of the Programmer and the Princess . . CHAPTER 8:

. . . . . .

Programs to Use Right Away . . . . . . . . . . . . . 161 Finding a Program on Your System: which,type, whence . . . How Do You Stop a Program? . . . . . . . . . . . . . . . . . . . . Displaying the Time and Date: date . . . . . . . . . . . . . . . . Displaying a Calendar: cal . . . . . . . . . . . . . . . . . . . . . The Unix Reminder Service: calendar . . . . . . . . . . . . . . Information About Your System: uptime, hostname, uname . Information About You: whoami, quota . . . . . . . . . . . . . Information About Other Users: users, who, w . . . . . . . . . Locking Your Terminal Temporarily: lock . . . . . . . . . . . . . Asking Unix to Remind You When to Leave: leave . . . . . . . . A Built-In Calculator: bc . . . . . . . . . . . . . . . . . . . . . . . Using bc for Calculations . . . . . . . . . . . . . . . . . . . . . . Using Variables With bc . . . . . . . . . . . . . . . . . . . . . . . Using bc with Different Bases . . . . . . . . . . . . . . . . . . . . Reverse Polish Notation . . . . . . . . . . . . . . . . . . . . . . . . The Stack-Based Calculator: dc . . . . . . . . . . . . . . . . . . .

CHAPTER 9:

83977_fm_i_xxxiv.indd xii

158 164 164 165 167 168 169 170 172 174 175 176 179 180 182 184

Documentation: The Unix Manual and Info . . . 189 The Unix Tradition of Teaching Yourself . . . . . . . . . . . . . . RTFM. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . What Is the Unix Manual? man . . . . . . . . . . . . . . . . . . . Man Pages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Displaying Man Pages . . . . . . . . . . . . . . . . . . . . . . . . . Two Useful Man Page Techniques . . . . . . . . . . . . . . . . . . Alternatives to man: xman and the Web . . . . . . . . . . . . . . How the Unix Manual Is Organized . . . . . . . . . . . . . . . . . Specifying the Section Number When Using the man Command . How Man Pages Are Referenced . . . . . . . . . . . . . . . . . . . The Format of a Manual Page . . . . . . . . . . . . . . . . . . . . A Quick Way to Find Out What a Command Does: whatis . . . Searching For a Command: apropos . . . . . . . . . . . . . . . Foo, Bar and Foobar . . . . . . . . . . . . . . . . . . . . . . . . . . The Info System . . . . . . . . . . . . . . . . . . . . . . . . . . . . Info and Trees . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Starting Info: info . . . . . . . . . . . . . . . . . . . . . . . . . . Learning About Info. . . . . . . . . . . . . . . . . . . . . . . . . .

xii

151 153 155 156 157 158

189 190 192 193 193 196 198 199 202 203 204 208 209 210 211 213 214 215

Harley Hahn’s Guide to Unix and Linux

1/9/2008 12:52:00 PM

Reading an Info File . . . . . . . . . . . . . . . . . . . . . . . . . . 216 Jumping From One Node to Another . . . . . . . . . . . . . . . . 219 CHAPTER 10:

Command Syntax . . . . . . . . . . . . . . . . . . . . 223 Entering More Than One Command at a Time . . . What Happens When You Enter a Command? . . . Command Syntax . . . . . . . . . . . . . . . . . . . Options. . . . . . . . . . . . . . . . . . . . . . . . . Dash Options and Dash-Dash Options . . . . . . . Arguments . . . . . . . . . . . . . . . . . . . . . . . Whitespace . . . . . . . . . . . . . . . . . . . . . . . One or More; Zero or More . . . . . . . . . . . . . The Formal Description of a Command: Syntax . Learning Command Syntax From the Unix Manual How Can You Learn So Many Options? . . . . . . .

CHAPTER 11:

. . . . . . . . . . .

. . . . . .

. . . . . . . . . . .

223 224 225 226 227 229 230 231 232 235 235

. . . . . .

239 240 244 247 249 251

Using the Shell: Variables and Options . . . . . 255 Interactive and Non-interactive Shells . . . . . . . . . . . . . The Environment, Processes and Variables . . . . . . . . . . Environment Variables and Shell Variables . . . . . . . . . . Displaying Environment Variables: env, printenv . . . . Displaying Shell Variables: set . . . . . . . . . . . . . . . . Displaying and Using the Value of a Variable: echo, print Bourne Shell Family: Using Variables: export, unset . . Shell Options: set -o, set +o . . . . . . . . . . . . . . . . Displaying Shell Options . . . . . . . . . . . . . . . . . . . . Machine-readable, Human-readable . . . . . . . . . . . . . .

CHAPTER 13:

. . . . . . . . . . .

The Shell . . . . . . . . . . . . . . . . . . . . . . . . . 239 What is a Shell? . . . . . . . . . . . . . . . . . . . . . . The Bourne Shell Family: sh, ksh, bash . . . . . . . The C-Shell Family: csh, tcsh . . . . . . . . . . . . . Which Shell Should You Use? . . . . . . . . . . . . . . . Changing Your Shell Temporarily . . . . . . . . . . . . The Password File: Changing Your Login Shell: chsh .

CHAPTER 12:

. . . . . . . . . . .

. . . . . . . . . .

256 257 259 262 264 264 267 271 273 274

Using the Shell: Commands and Customization . . . . . . . . . . . . . . . . . 277 Metacharacters. . . . . . . . . . . . . . . . . . . Quoting and Escaping . . . . . . . . . . . . . . Strong and Weak Quotes . . . . . . . . . . . . . Commands That Are Built into the Shell: type Learning About Builtin Commands . . . . . . . External Commands and the Search Path . . . .

. . . . . .

277 279 283 284 286 287

Table of Contents xiii

83977_fm_i_xxxiv.indd xiii

1/9/2008 12:52:00 PM

Modifying Your Search Path . . . . . . . . . . . . . . . . . How a Hacker Can Use the Search Path . . . . . . . . . . . The Shell Prompt . . . . . . . . . . . . . . . . . . . . . . . Modifying the Shell Prompt . . . . . . . . . . . . . . . . . Using the Value of a Variable . . . . . . . . . . . . . . . . . Which Quotes to Use When Quoting Variables . . . . . . . Special Codes That Use an Escape Character . . . . . . . . Command Substitution . . . . . . . . . . . . . . . . . . . . Typing Commands and Making Changes . . . . . . . . . . The History List: fc, history . . . . . . . . . . . . . . . History List: Setting the Size . . . . . . . . . . . . . . . . . History List Example: Avoid Deleting the Wrong Files . . . Displaying Event Number & Working Directory in Your Shell Prompt . . . . . . . . . . . . . . . . . . . . . Autocompletion . . . . . . . . . . . . . . . . . . . . . . . . Autocompletion: Beyond the Basics . . . . . . . . . . . . . Using Autocompletion for Fun and Proﬁt . . . . . . . . . . Command Line Editing: bindkey . . . . . . . . . . . . . Aliases: alias, unalias . . . . . . . . . . . . . . . . . . Suspending an Alias Temporarily . . . . . . . . . . . . . . Alias Example: Avoid Deleting the Wrong Files . . . . . . . Alias Example: Reusing Commands From the History List Alias Example: Displaying Name of Working Directory in Shell Prompt. . . . . . . . . . . . . . . . . . . . . . . CHAPTER 14:

. . . . . . . . . . . .

289 291 292 293 294 296 297 299 301 302 305 306

. . . . . . . . .

307 309 312 314 314 316 318 319 320

. . . . 322

. . . . . . . . . . . .

327 329 330 331 332 333 334 335 335 336 337 341

Standard I/O, Redirection, and Pipes . . . . . . . 345 The Unix Philosophy . . . . . . . . . . . . . . . . . . . The New Unix Philosophy . . . . . . . . . . . . . . . . Standard Input, Standard Output and Standard Error . Redirecting Standard Output . . . . . . . . . . . . . . .

xiv

. . . . . . . . . . . .

Using the Shell: Initialization Files . . . . . . . . 327 Initialization Files and Logout Files . . . . . . . . . . . . Names of Initialization and Logout Files . . . . . . . . . Dotﬁles and rc Files . . . . . . . . . . . . . . . . . . . . Using a Simple Text Editor . . . . . . . . . . . . . . . . . Login Shells and Non-Login Shells. . . . . . . . . . . . . When Are Initialization Files Executed? . . . . . . . . . . A Quick History of Shell Initialization Files . . . . . . . . What to Put in Your Initialization Files . . . . . . . . . . Displaying, Creating and Editing Your Initialization Files Comments in Shell Scripts . . . . . . . . . . . . . . . . . Bourne Shell Family: Sample Initialization Files . . . . . C-Shell Family: Sample Initialization Files . . . . . . . .

CHAPTER 15:

. . . . . . . . . . . .

. . . .

345 346 348 349

Harley Hahn’s Guide to Unix and Linux

Openmirrors.com 83977_fm_i_xxxiv.indd xiv

1/9/2008 12:52:00 PM

Preventing Files From Being Replaced or Created by Redirection . Redirecting Standard Input . . . . . . . . . . . . . . . . . . . . . . File Descriptors; Redirecting Standard Error . . . . . . . . . . . . With the Bourne Shell Family . . . . . . . . . . . . . . . . . . . . Subshells . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Redirecting Standard Error With the C-Shell Family . . . . . . . . Combining Standard Output and Standard Error . . . . . . . . . Throwing Away Output . . . . . . . . . . . . . . . . . . . . . . . . Redirection: Summaries and Experimenting . . . . . . . . . . . . Pipelines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Splitting a Pipeline: tee . . . . . . . . . . . . . . . . . . . . . . . The Importance of Pipelines . . . . . . . . . . . . . . . . . . . . . Conditional Execution . . . . . . . . . . . . . . . . . . . . . . . . CHAPTER 16:

Filters: Introduction and Basic Operations . . . 373 Variations of Commands and Options . . . . . . . . . . . . . . Filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Should You Create Your Own Filters? . . . . . . . . . . . . . . . The Problem Solving Process . . . . . . . . . . . . . . . . . . . . The Simplest Possible Filter: cat . . . . . . . . . . . . . . . . . Increasing the Power of Filters . . . . . . . . . . . . . . . . . . . A List of the Most Useful Filters . . . . . . . . . . . . . . . . . . Combining Files: cat. . . . . . . . . . . . . . . . . . . . . . . . Splitting Files: split. . . . . . . . . . . . . . . . . . . . . . . . Combining Files While Reversing Lines: tac . . . . . . . . . . . Reversing the Order of Characters: rev . . . . . . . . . . . . . . Select Lines From the Beginning or End of Data: head, tail . Deleting Columns of Data: colrm . . . . . . . . . . . . . . . .

CHAPTER 17:

. . . . . . . . . . . . .

373 374 375 376 377 380 382 382 385 388 389 391 392

Filters: Comparing and Extracting . . . . . . . . 395 Comparing Files . . . . . . . . . . . . . . . . . . . . . . Comparing Any Two Files: cmp . . . . . . . . . . . . . . Comparing Sorted Text Files: comm . . . . . . . . . . . . Comparing Unsorted Text Files: diff . . . . . . . . . . Options to Use With diff . . . . . . . . . . . . . . . . . Output Formats When Comparing Files: diff, sdiff Diffs and Patches . . . . . . . . . . . . . . . . . . . . . . Extracting Columns of Data: cut . . . . . . . . . . . . . Combining Columns of Data: paste . . . . . . . . . . .

CHAPTER 18:

350 352 353 353 355 358 359 360 362 365 367 369 370

. . . . . . . . .

395 396 397 399 403 404 408 410 415

Filters: Counting and Formatting . . . . . . . . . 421 Creating line numbers: nl . . . . . . . . . . . . . . . . . . . . . . 421 Counting Lines, Words and Characters: wc . . . . . . . . . . . . 424 How Unix Uses Tabs . . . . . . . . . . . . . . . . . . . . . . . . . 427

Table of Contents

83977_fm_i_xxxiv.indd xv

1/9/2008 12:52:00 PM

Visualizing Tabs and Spaces . . . . . . Changing Tabs to Spaces: expand . . Changing Spaces to Tabs: unexpand . Formatting lines: fold . . . . . . . . . The 80-Character Line . . . . . . . . . Formatting Paragraphs: fmt . . . . . . The Olden Days of Printing . . . . . . Formatting Text Into Pages: pr . . . . Formatting Text Into Columns: pr . . CHAPTER 19:

. . . . . . . . .

Filters: Selecting, Sorting, Combining, and Changing . . . . . . . . . . . . . . . . . . . . . 447 Selecting Lines That Contain a Speciﬁc Pattern: grep . . . . . 447 The Most Important grep Options . . . . . . . . . . . . . . . . . Variations of grep: fgrep, egrep . . . . . . . . . . . . . . . . Selecting Lines Beginning With a Speciﬁc Pattern: look . . . . . When Do You Use look and When Do You Use grep?. . . . . . Finding All the Words That Begin With a Speciﬁc Pattern: look . Sorting Data: sort . . . . . . . . . . . . . . . . . . . . . . . . . . Controlling the Order in Which Data Is Sorted: sort -dfn . . . Checking If Data Is Sorted: sort -c . . . . . . . . . . . . . . . . The ASCII Code; Collating Sequences . . . . . . . . . . . . . . . . Locales and Collating Sequences . . . . . . . . . . . . . . . . . . . Finding Duplicate Lines: uniq . . . . . . . . . . . . . . . . . . . Merging Sorted Data From Two Files: join . . . . . . . . . . . . Create a Total Ordering From Partial Orderings: tsort . . . . . Translating Characters: tr . . . . . . . . . . . . . . . . . . . . . . Translating Unprintable Characters . . . . . . . . . . . . . . . . . Translating Characters: Advanced Topics . . . . . . . . . . . . . . Non-interactive Text Editing: sed . . . . . . . . . . . . . . . . . . Using sed for Substitutions . . . . . . . . . . . . . . . . . . . . . Telling sed to Operate Only on Speciﬁc Lines . . . . . . . . . . . Using Very Long sed Commands . . . . . . . . . . . . . . . . . .

CHAPTER 20:

429 430 432 433 435 436 439 440 443

450 454 455 457 458 459 461 463 464 466 471 473 478 482 484 486 488 490 492 493

Regular Expressions . . . . . . . . . . . . . . . . . . 497 Introducing Regular Expressions . . . . . . . . . . . . . . . . . The Origin of Regular Expressions. . . . . . . . . . . . . . . . Basic and Extended Regular Expressions . . . . . . . . . . . . Matching Lines and Words . . . . . . . . . . . . . . . . . . . . Matching Characters; Character Classes . . . . . . . . . . . . . Predeﬁned Character Classes; Ranges . . . . . . . . . . . . . . Locales and Collating Sequences: locale; The ASCII Code . Using Ranges and Predeﬁned Character Classes . . . . . . . .

. . . . . . . .

497 498 500 502 505 506 507 510

xvi Harley Hahn’s Guide to Unix and Linux

Openmirrors.com 83977_fm_i_xxxiv.indd xvi

1/9/2008 12:52:00 PM

Repetition Operators . . . . . . . . . . . . . . . . . . . . . . . . . 511 How to Understand a Complex Regular Expression . . . . . . . . 514 Solving Three Interesting Puzzles; The Dictionary File. . . . . . . 514 CHAPTER 21:

Displaying Files . . . . . . . . . . . . . . . . . . . . . 521 Survey of Programs Used to Display Files . . . . . . . . Introduction to less: Starting, Stopping, Help . . . . The Story of less and more . . . . . . . . . . . . . . Using less . . . . . . . . . . . . . . . . . . . . . . . . Using less to Search Within a File . . . . . . . . . . . Options to Use With less . . . . . . . . . . . . . . . . When to Use less and When to Use cat . . . . . . . Using Environment Variables to Customize Your Pager Displaying Multiple Files With less . . . . . . . . . . Displaying a File Using more . . . . . . . . . . . . . . Displaying the Beginning of a File: head . . . . . . . . Displaying the End of a File: tail . . . . . . . . . . . Watching the End of a Growing File: tail -f . . . . . Binary, Octal and Hexadecimal. . . . . . . . . . . . . . Reading and Writing Binary, Octal and Hexadecimal . Why We Use Hexadecimal Rather Than Octal . . . . . Displaying Binary Files: hexdump, od . . . . . . . . . Why Does So Much Computer Terminology Come From Mathematics? . . . . . . . . . . . . . . . . . .

CHAPTER 22:

. . . . . . . . . . . . . . . . .

522 524 526 527 529 531 534 535 536 539 541 541 542 544 549 550 551

. . . . . . 556

The vi Text Editor . . . . . . . . . . . . . . . . . . . 559 Why Is vi So Important? . . . . . . . . . . . . . . A Quick History of vi . . . . . . . . . . . . . . . Vim: an Alternative to vi . . . . . . . . . . . . . . Starting vi . . . . . . . . . . . . . . . . . . . . . . Starting Vim: vim . . . . . . . . . . . . . . . . . . Command Mode and Input Mode . . . . . . . . . Knowing What Mode You Are In. . . . . . . . . . Starting vi as a Read-Only Editor: view, vi -R Recovering Data After a System Failure . . . . . . Stopping vi . . . . . . . . . . . . . . . . . . . . . How vi Uses the Screen . . . . . . . . . . . . . . Using vi and ex Commands . . . . . . . . . . . A Strategy for Learning vi Commands . . . . . . Creating a Practice File . . . . . . . . . . . . . . . Moving the Cursor . . . . . . . . . . . . . . . . . Moving Through the Editing Buffer . . . . . . . . Jumping to a Previous Location . . . . . . . . . .

. . . . . . . . . . . . . . . . .

560 560 564 565 566 568 570 571 571 572 573 574 575 577 577 581 582

Table of Contents xvii

83977_fm_i_xxxiv.indd xvii

1/9/2008 12:52:00 PM

See Also

Actor | TheresNothingNew

. . . . . . . . .

524 528 537 540 546 547 549 550 552

22-1 22-2 22-3 22-4 22-5 22-6 22-7 22-8 22-9 22-10 22-11 22-12

Bill Joy and Dennis Ritchie . . . . . . . . . . . . . The Lear Siegler ADM-3A terminal . . . . . . . . . Vim Startup Screen . . . . . . . . . . . . . . . . . . Keyboard layout of the ADM-3A terminal . . . . . How vi Displays Empty Lines . . . . . . . . . . . Keys to Use to Make Corrections While Using vi . The H, J, K and L keys on the ADM-3A terminal . Using regular expressions when searching with vi vi Options: Switches and Variables . . . . . . . . . Characters to use as vi and Vim macro names . . vi/Vim sample initialization ﬁle . . . . . . . . . . Vim: Enhancements over standard vi . . . . . . .

. . . . . . . . . . . .

561 562 567 569 574 575 578 585 605 617 620 623

23-1 23-2 23-3 23-4 23-5 23-6 23-7 23-8 23-9 23-10

An example of organizing with directories . . . The most interesting special ﬁles . . . . . . . . The most interesting Linux proc ﬁles . . . . . . The standard Linux ﬁlesystem . . . . . . . . . . The original Unix ﬁlesystem . . . . . . . . . . . Contents of the root directory . . . . . . . . . . Contents of the /usr directory . . . . . . . . . Directories that hold program ﬁles . . . . . . . A typical home directory-based tree structure . The most common ﬁlesystems . . . . . . . . .

. . . . . . . . . .

630 631 638 639 641 644 647 650 652 655

. . . . . . . . . .

. . . . .

. . . . . . . . . .

. . . . .

xxiv Harley Hahn’s Guide to Unix and Linux

Openmirrors.com 83977_fm_i_xxxiv.indd xxiv

1/9/2008 12:52:01 PM

24-1 24-2 24-3 24-4 24-5 24-6 24-7 24-8 24-9

A sample directory tree. . . . . . . . . . . . . . . Making a sample directory tree . . . . . . . . . . Directory stack commands . . . . . . . . . . . . Flags displayed by the ls -F command . . . . . Summary of wildcards used to specify ﬁlenames Wildcards: Predeﬁned character classes . . . . . . Dotﬁles used by the shells and by vi/Vim . . . . File type indicators used by ls -l . . . . . . . . An example of a ﬁle manager . . . . . . . . . . .

. . . . . . . . .

661 670 677 687 698 700 703 706 711

25-1 25-2 25-3 25-4 25-5 25-6

Characters that are safe to use in ﬁlenames . . . . Summary of ﬁle permissions . . . . . . . . . . . Numeric values for ﬁle permission combinations Contents of an inode (index node) . . . . . . . . The find program: Tests . . . . . . . . . . . . . The find program: Actions. . . . . . . . . . . .

. . . . . .

719 730 737 741 753 757

26-1 26-2 26-3 26-4 26-5 26-6 26-7 26-8 26-9 26-10

Commonly used system calls . . . . . Job control: Tools . . . . . . . . . . . . Job control: Specifying a job . . . . . . The ps program: UNIX options . . . The ps program: BSD options . . . . The ps program: Column headings . The ps program: Process state codes . The top program . . . . . . . . . . . Signals . . . . . . . . . . . . . . . . . . Daemons . . . . . . . . . . . . . . . .

. . . . . . . . . .

769 778 786 790 791 792 796 799 808 813

F-1 F-2 G-1 G-2 G-3

U.S. time zones . . . . . . . . . . . European and Indian time zones . Bourne Shell family: Shell options C-Shell family: Shell options. . . . C-Shell family: Shell variables . . .

. . . . .

842 843 847 848 849

. . . . .

List of Figures xxv

83977_fm_i_xxxiv.indd xxv

1/9/2008 12:52:01 PM

NOTE TO TEACHERS

I designed this book very carefully to help you teach Unix and Linux. My goal is to support your teaching no matter what the length of your course, and no matter which topics you choose to present to your class. To do so, I offer you a long book that covers every important topic a student needs to master to understand and use basic Unix. I designed the book as a comprehensive reference that can be read from beginning to end, in order. However, I also designed it to give you maximum ﬂexibility. My intention is for you to look at the Table of Contents and see what the book has to offer. Then choose which topics you want to teach directly, and which ones you want to assign for self-study. This approach will work well for you, because I wrote every chapter so it can be studied independently. Moreover, every section within a chapter is designed so that the student can teach himself. In order to make this possible, I used several techniques. First, we know that, in any area of study, one of the most important things a student must learn is the relevant terminology. There are 622 technical terms explained in this book, and each term is explained explicitly. Moreover, no term is used until it is explained. To support this effort, there is an extensive glossary at the end of the book. If you assign a chapter or section out of order, your students can simply use the glossary to look up concepts with which they are unfamiliar. (Please encourage them to do so.) For further help, each glossary deﬁnition is followed by the number of the chapter in which the student can ﬁnd a detailed discussion of the topic. Second, as the student reads, he or she is led from one idea to the next by the careful use of examples. Indeed, this book contains well over a thousand examples fully integrated into the text. Most commands and ideas are coupled with sample input and output, which allows the book to stand on its own. This makes it possible for the student to fully understand what he is reading, even if he is not working in front of his computer at the time. Third, all the examples in this book were tested on Linux, FreeBSD and Solaris systems. In most cases, each example was also tested under four different shells: Bash, the Korn Shell, the Tcsh, and the C-Shell. Thus, no matter which type of Unix or Linux your students use, what they read in the book will work for them. Where there are important exceptions, I note them. Thus, if the student is following along at his computer, what he sees will be similar to what is printed in the book.

xxvi Harley Hahn’s Guide to Unix and Linux

Openmirrors.com 83977_fm_i_xxxiv.indd xxvi

1/9/2008 12:52:01 PM

Finally, as a student reads a particular section, there is no assumption that he has read the previous section or that he will read the next section. This allows you to teach whatever you want in whichever order makes sense to you. (For more thoughts on choosing what to teach, see the discussion on the Unix Model Curriculum below.) What makes this possible is the liberal use of forward and backward references to other parts of the book. Thus, whenever a topic depends upon ideas discussed elsewhere, the student will ﬁnd it easy to take a moment and ﬁll the gap in his knowledge.

UNIX AS A PART OF COMPUTER SCIENCE One of the most interesting aspects of teaching Unix is that, unlike other areas of computer science, there is no standard curriculum. This, in spite of the fact that Unix is a mature area of study, having been taught for well over two decades. This seeming paradox is explained by the observation that, for many years, Unix was considered to be merely a technology, rather than a part of computer science. As such, instruction in Unix was conﬁned mostly to explaining how to carry out various tasks such as using the shell, entering commands, manipulating ﬁles, running programs, and so on. For programming students, Unix was presented only as a vehicle for writing and testing programs. To be sure, some operating systems teachers considered Unix to be a classical system, important enough to be studied from a historical point of view. To suggest, however, that Unix should be recognized as a legitimate topic within computer science was, for many years, considered to be far-fetched. Today, however, this viewpoint is changing with the realization that the study of Unix and Linux form an important part of the computer science curriculum. There are several reasons for this change. First, the history of Unix is the best example we have of a well-designed computing system that has evolved and survived for more than a (human) generation. There are, indeed, many people using Unix whose fathers and mothers used Unix. Second, most parts of Unix were designed by computer scientists or programmers well versed in basic computer science. Thus, a proper study of Unix affords the student a chance to see computer science in action. This naturally leads to the study of more mainstream topics, such as data structures and number systems. For example, see the discussion of trees in Chapters 9 and 23; of stacks in Chapter 8 and 24; and of the hexadecimal, octal and binary number systems in Chapter 21. Finally, the Unix culture was the crucible from which Linux and the Open Source movement emerged in the 1990s. Thus, the study of Unix affords the student the background necessary to understand, appreciate, and (perhaps) contribute to these important international efforts. To promote the teaching of Unix and Linux in this way, this book is structured around what is called the “Unix Model Curriculum”. The intention is that teachers will use this curriculum to help plan their courses. For more information, see the section called “Support for Teachers” below.

Note to Teachers xxvii

83977_fm_i_xxxiv.indd xxvii

1/9/2008 12:52:02 PM

A UNIX-NEUTRAL APPROACH One of the goals of this book is to ensure that students become comfortable using any type of Unix or Linux, in their own language, anywhere in the world. This goal is promoted in several ways. First, it is a core belief of mine that students should be educated generally enough as to be able to use any major type of Unix as well as the most important shells. Speciﬁcally, students should be comfortable, not only with Linux, but with System V-based Unix (such as Solaris), and BSD-based Unix (such as FreeBSD and Mac OS X). Moreover, students should understand the basic operation of the most important shells: Bash (the default Linux shell); the Korn shell (the modern version of the Bourne shell); and the Tcsh (the modern version of the C-Shell). After all, in the course of a lifetime, one will be called upon to use a variety of Unix and Linux systems. Thus, it behooves us to consider the student’s long-term needs, regardless of which system happens to be available at your particular school. Toward this end, this book introduces Unix and Linux by using a set of basic principles common to all Unix-like operating system. Where signiﬁcant differences exist, they are taught as variations of the standard, ensuring that the student becomes comfortable with the most important, most enduring concepts. A similar didactic approach is used with the shells. The student is introduced to the idea that there are the two main families of shells, each of which is explained in terms of the appropriate historical and technical background. The Korn shell and Bash are then introduced as members of the Bourne Shell family, while the C-Shell and the Tcsh are taught as being members of the C-Shell family. Because some of the details are complex, the book has numerous tables and explanatory notes that act as a reference, should the student need to switch from one operating system to another, or from one shell to another (as we all must do from time to time). The second way in which a Unix-neutral teaching environment is developed concerns internationalization. In the early days (1970s and 1980s), all Unix systems were derived from either System V or BSD (see Chapter 2), both of which were U.S.-centric systems, based on the ASCII code. Today, Unix and Linux systems are used widely, well beyond the United States. Indeed, the Linux kernel and the various Linux distributions are developed by volunteers from around the world. As a result, Unix has evolved into a true international operating system that supports much more than U.S. English and ASCII. For the beginner, the most signiﬁcant concepts related to internationalization are locales, collating sequences, and character classes. These topics are discussed in detail as part of the treatment of ﬁlters (Chapter 19) and regular expressions (Chapter 20). It is my feeling that establishing and maintaining a Unix-neutral approach in our teaching leads the student to internalize the idea that Unix and Linux are global systems. In this way, the student develops the knowledge and skills to become conversant with any type of Unix or Linux he or she may be called upon to use.

xxviii Harley Hahn’s Guide to Unix and Linux

Openmirrors.com 83977_fm_i_xxxiv.indd xxviii

1/9/2008 12:52:02 PM

SUPPORT FOR TEACHERS: THE UNIX MODEL CURRICULUM The Unix Model Curriculum is a detailed plan for teaching all the important concepts necessary for an introductory course for Unix or Linux. The Unix Model Curriculum was designed to help you decide which topics to teach and the order in which to teach them. You can learn about the Unix Model Curriculum by visiting either of the following Web sites. First, a special site I have set up for Unix teachers and students: www.harley.com/unix Second, a teaching support site sponsored by McGraw-Hill Higher Education speciﬁcally for teachers who use this book: www.mhhe.com/harleyhahn On this Web site, you will ﬁnd a great deal of useful material, including answers to all the exercises, a discussion of the Unix Model Curriculum, and a variety of teaching aids. To access the McGraw-Hill site, you will need a password, which you can obtain at no charge, either from your sales rep or from the McGraw-Hill Higher Education marketing department.

Note to Teachers xxix

83977_fm_i_xxxiv.indd xxix

1/9/2008 12:52:02 PM

ACKNOWLEDGEMENTS

This is a large book, and if you are going to read it all, you have a lot of work ahead of you. As such, you might be asking yourself if you should take the time to read the acknowledgements. I think you should, for two important reasons. First, if you are a student, this might be the only part of the book you will not cover in class. For a few minutes, then, you can relax and enjoy yourself, knowing that what you read here will never appear on a test. Second, although the information in this section is somewhat esoteric, you never know when it will come in handy. For example, later in this section, you will ﬁnd out that Professor Sanjiv Bhatia of the University of Missouri, St. Louis, was one of my reviewers. One day, you may be visiting St. Louis and, as you are walking around the university, you might actually run into Professor Bhatia. “Dr. Bhatia,” you will say, “it is such a pleasure to meet you. I hear you did an excellent job helping Harley Hahn with his book.” Upon which, he shakes your hand and invites you to the Faculty Club for tea. So are you ready? Good. Let’s push on.

MY TEAM (AND WELCOME TO IT) This book was a long time in the making and, during that time, a lot of people helped me. I produced the book myself, which means I hired and supervised the people who did most of the work so, to start, let me introduce the members of my team. First, Lydia Hearn. Lydia and I have worked together on books for a long time, and I have found her to be a truly remarkable person. Her skill, energy and enthusiasm is virtually boundless. For this book, Lydia did the copy editing and the page layout. However, to call Lydia a mere “copy editor” or “layout artist” is like calling the Grand Canyon a hole in the ground. The reason this book looks so good is that Lydia spent many, many hours painstakingly making every single page look as good as possible. She is a tireless perfectionist with a devotion to quality that would be difﬁcult to overpraise. And just between us, Lydia Hearn is the only person in the world I will let make changes in what I write. Lydia, by the way, is an accomplished academic in her own right. She is a tenured professor at De Anza College in Cupertino, California, where she teaches English. She has

xxx Harley Hahn’s Guide to Unix and Linux

Openmirrors.com 83977_fm_i_xxxiv.indd xxx

1/9/2008 12:52:02 PM

served as Academic Senate President and Dean of Language Arts, and has been recognized by the college as a Distinguished Educator. I am lucky to have her. Next, my art director, Lee Anne Dollison. Lee Anne brought her many years of publishing experience to the project. Over many months, she helped Lydia and me create an attractive, easy-to-read book. Lee Anne was responsible for the interior design, the front cover*, and the illustrations. She was also our resident layout expert, patiently ﬁnding answers to the many technical questions that come with producing a book of this complexity. Actually, except for one thing, Lee Anne would be perfect. She doesn’t have a PC; she only has a Macintosh. Still, maybe it’s all for the best. If she did get a PC, she would be perfect, and it is not easy to work with a perfect person. (Just ask any of my editors.) The indexes for the book (there are more than one) were created by Cheryl Lenser. Cheryl is a very talented, highly experienced indexer, who has indexed several of my books. She is always my ﬁrst choice, and I was lucky to get her for this project. Next, Alan Jones, one of the smartest people I know, prepared the production budget, and helped solve a variety of important problems. Finally, the snazzy photo of me and Little Weedly (my cat) on the back cover was taken by Dan Sullivan, a professional photographer based in Santa Barbara, California.

REVIEWERS As I wrote the book, I had help from a number of extremely smart people who read and critiqued every chapter as I ﬁnished it. I had two types of reviewers: professors who teach Unix courses and technical experts who use Unix or Linux in their work. Because there are 26 chapters, each of these people had to go through the process 26 times, which is a lot of work, so they all deserve a special mention. First, here are the professors: Ron Thomson: Don Southwell: Damarra Mulholland: Eugene Eberbach: James Heliotis: Ivan Bajic: Sanjiv Bhatia:

Central Michigan University Delta College, Michigan Delta College, Michigan (retired) Rensselaer Polytechnic Institute, Connecticut Rochester Institute of Technology, Rochester NY San Diego State University, California University of Missouri, St. Louis

The experts are: Andrew Eberbach: Susan Pierce: Michael Schuster: Tammy Cravit: Stephanie Lockwood-Childs:

IBM Software Engineer, Research Triangle Park Santa Barbara Linux Users Group (SBLUG) Solaris Kernel Engineer, Sun Microsystems Taylored Software VCT Labs; President, SBLUG

*In case you are wondering, the green design on the left side of the front cover was taken from one of my paintings. To see more, take a look at www.harley.com/art.

Acknowledgements xxxi

83977_fm_i_xxxiv.indd xxxi

1/9/2008 12:52:02 PM

TABLE OF CONTENTS COMMENTATORS This book was three years in the making. However, before I even started writing, I created a proposed Table of Contents for the book. The next group of people contributed by looking at my original plans for the book and sharing generously with their comments and suggestions: Dave Spence: Ronald Czik: Mark Hutchenreuther: John Connely: Seung Bae Im: Tulin Mangir: Jay Harris: Riccardo Pucella: Nathaniel Nystrom: Pamela Williams: Mike Smith: Ancelin Shah: Don Lopez: Weldon Hill: Ashvin Mahajan: Brian Davison: Peter McDervitt: Gordon Goodman: Ivan Bajic: Thuy Le: Marty Froomin: Ka-Cheong Leung: Adrienne Decker: Charlie Shub: Frank Ducrest: Robert Morris: Iren Valova: Fred Hosch:

BBC, England Boston University, Massachusetts Cal Poly, San Luis Obispo, California Cal Poly, San Luis Obispo, California California State University, Chico California State University, Long Beach Clemson University, South Carolina Cornell University, Ithaca, New York Cornell University, Ithaca, New York Fayetteville State University, North Carolina Harvard University, Cambridge, Massachusetts Houston Community College, Texas Idaho State University, College of Technology Idaho State University, College of Technology Iowa State University Lehigh University, Pennsylvania New Brunswick Community College, Canada Rochester Institute of Technology, Rochester NY San Diego State University, California San Jose State University, California San Jose State University, California Texas Tech University, Lubbock, Texas University at Buffalo, State University of NY University of Colorado, Colorado Springs University of Louisiana, Lafayette University of Massachusetts Boston University of Massachusetts, Dartmouth University of New Orleans, Louisiana

EXPERTS WHO ANSWERED QUESTIONS As I was working on the book, there were times when I needed to contact various experts to ask questions. Most everyone was kind enough to write me back and, in some cases, provide valuable historical information that you will not ﬁnd elsewhere. In particular, I am happy to thank:

xxxii Harley Hahn’s Guide to Unix and Linux

Openmirrors.com 83977_fm_i_xxxiv.indd xxxii

1/9/2008 12:52:02 PM

Richard Stallman, one of the seminal ﬁgures in the world of open source software. Stallman is the founder of the Free Software Foundation (see Chapter 2) and a highly accomplished programmer. Matthias Ettrich, founder of the KDE desktop environment project (see Chapter 5. ) John Mashey, who worked at Bell Labs in the early days of Unix. Mashey is the creator of the Mashey/PWB shell (see Chapter 11). Doug McIlroy, who also worked at Bell Labs on the early versions of Unix. It was McIlroy who ﬁrst suggested the idea of pipes (see Chapter 15). Mark Nudelman, the creator of the less pager (see Chapter 21). Charles Haley, who worked with Bill Joy at U.C. Berkeley in the mid-1970s (see Chapter 22). Haley and Joy created the ex text editor. Haley also contributed to the original version of the vi text editor. This is also a good place to mention the members of the Santa Barbara Linux Users Group, who patiently answered many questions: Stephanie Lockwood-Childs and Susan Peirce (both mentioned above), Ron Lockwood-Childs, Chad Page, Tom King, Nick Lockwood, Ron Jeffries and Marc Provett. Thanks also to Skona Brittain, who read and commented on an early version of the ﬁrst ﬁve chapters of the book.

PEOPLE WHO PROVIDED SERVICES As you read through this book, you will see well over a thousand examples. Each of these examples had to be tested on several different Unix and Linux systems using all four major shells. (Shells are explained in Chapter 11.) For access to such systems, I am grateful to the following people. First, Patrick Linstruth, co-founder of QNET, a high-quality, personalized Southern California Internet Service Provider. Pat arranged for me to have access to a FreeBSD system, as well as important Web and FTP services. Thanks also to Pat’s brother and QNET co-founder Chris Linstruth, and to the staff of the Austin Cafe: Lana, Melanie and Austin. Next, Helmut vom Sondern, a Service and Support Engineer for Sun Microsystems in Germany. Helmut very generously provided me with access to a state-of-the-art Solaris system, which I used for as a test system virtually every day for well over two years. For other Sun-related resources, I thank the following Sun Microsystems employees. In the Netherlands: Antoon Huiskens, Support Engineer. In California: Angel Camacho, Senior Technical Product Manager, and Neha Sampat, Group Manager in Product Marketing. At Apple, I am grateful to Ernie Prabhaker (Unix Product Manager) and Farman Syed (Software QA Engineer) for arranging for me to evaluate a Macintosh laptop computer running OS X (which runs on Unix). As I mentioned earlier, the front cover image was created by my art director, Lee Anne Dollison. The back cover and spine were created by Jenny Hobein, a cover designer who proved herself to be both talented and patient (two traits that are crucial for anyone working with me). Finally, Jake Warde of Warde Publishers, a pleasant, skillful, freelance developmental editor, assisted the original publisher, Alan Apt, during the ﬁrst part of this project.

Acknowledgements xxxiii

83977_fm_i_xxxiv.indd xxxiii

1/9/2008 12:52:02 PM

HOW THIS BOOK WAS DEVELOPED

Since this is such a long, complex book and since I supervised the production myself, it occurred to me that you might be interested in how such a book was created. To start, I created a preliminary Table of Contents, planning the chapters and some of the topics. As you might expect, in the course of the writing, many of the details changed. Still, one needs to start somewhere. Using the Web, I showed the preliminary Table of Contents to 28 Unix and Linux teachers from around the country. I asked them all to ﬁll in a form with their comments and suggestions. This helped me enhance my overall plan. I then began to write the book, one chapter at a time. For each chapter, I started doing my research and compiling notes. I then talked to Susan Pierce (a very experienced Unix/Linux expert) about my plans. Susan and I brainstormed various ideas. What should be in the chapter? What should be omitted? I then started to write the chapter, testing ideas and examples constantly. Indeed, almost every example you see in this book — and there are over 1,000 of them — was tested on Linux, Solaris and FreeBSD using a variety of shells. (Shells are explained in Chapter 11.) After the ﬁrst draft of the chapter was ﬁnished, I posted it on a secret Web site, and notiﬁed my reviewers. Each chapter was read by 12 different Unix professors and experts (see the Acknowledgments on page xxx). Each reviewer used a special Web page to answer questions about the chapter, send me their comments, make suggestions, and point out mistakes. Since there are 33 chapters and appendixes, the reviewers and I went through this time-consuming process 33 times. After each round, I would revise the chapter . I then sent the revised chapter to Lydia Hearn for copy editing, while I moved on to the next chapter. At the same time, my art director, Lee Anne Dollison, created the illustrations and tables for the chapter. After the editing was complete, Lydia (who is multi-talented) created the actual book pages. In this way, the entire book — chapters, appendixes, glossary, and indexes — was built carefully and slowly, one step at a time. You might ask, why did I decide to produce the book myself? Producing the book myself enabled Lydia, Lee Anne, and me to have complete control over the ﬁnal product. It was a lot of work but I hope, by the time you ﬁnish reading the book, you will agree that the extra effort was worth it.

xxxiv Harley Hahn’s Guide to Unix and Linux

Openmirrors.com 83977_fm_i_xxxiv.indd xxxiv

1/9/2008 12:52:02 PM

Introduction to Unix

This book is about using Unix: a family of operating systems that are used throughout the world and that run on virtually all types of computers. This book is also about Linux, a type of Unix. We’ll talk about the details in Chapter 2. For now, all you have to know is that an operating system is a master control program that runs a computer. There are many types of Unix: some are Linux; some aren’t. As a general rule, all types of Unix are similar enough so that, for practical purposes, if you know how to use one type of Unix, you know how to use them all. The ﬁrst Unix system was developed in 1969 by a programmer at AT&T’s Bell Labs so that he could run a program called Space Travel*. Today, Unix is nothing less than a worldwide culture, comprising many tools, ideas and customs. Modern Unix in its entirety is very large and complicated. Indeed, there is no single person who knows everything about Unix in general or even everything about one speciﬁc type of Unix. In fact, there is no single person who knows even most of Unix. I realize this might seem strange. After all, if Unix is that complex, how did it come to exist at all? Who creates and enhances Unix? And who changes it and ﬁxes problems when things go wrong? I’ll answer these questions in Chapter 2, when we talk a bit about the history of Unix and about how Unix is maintained today. For now, what I want you to appreciate is that Unix is much more than an operating system: it is nothing less than a culture. So as you read this book and think about what you are learning, realize that you are doing more than simply learning how to use yet another computer tool. You are becoming a member of the global Unix community, the largest collection of smart people in the history of the world. If you have never used Unix before, you are in for some pleasant surprises. Unix is not easy to learn, but it is well-designed, extremely powerful, and – once you get used to it – a great deal of fun. *Space Travel simulated the movements of the Sun and planets, as well as a spaceship that you could land in various locations. The programmer in question was Ken Thompson who, with various other people at Bell Labs, went on to develop the ﬁrst full-ﬂedged Unix operating system (presumably, after they got tired of Space Travel).

Introduction to Unix

33614_01_001_008.indd 1

1/9/2008 12:20:03 PM

Chapter 1 As with all computer systems, there will be times when you are puzzled by a problem whose solution is not obvious. There will also be times when you will be frustrated or discouraged. However, no matter what happens, I can promise you one thing: you will never be bored.

WHY USE UNIX? The Unix culture, which you are about to enter, contains an enormous number of tools for you to learn about and use. You can create and manipulate information – text ﬁles, documents, images, music, video, databases, spreadsheets, and so on – in more ways than you can imagine; you can access the Internet to use the Web, email, ﬁle transfer, and discussion groups; you can play games; you can design your own Web pages and even run your own Web server; and you can write computer programs using a variety of different languages and programming tools. Of course, you can do all these things using other operating systems, such as Windows, so why learn Unix? There are lots of reasons but, at this point, they will seem rather technical, so let me give the four most important reasons to learn Unix. The ﬁrst is choice. With Unix, you will decide how you want to use your computer and how deep you want to get into the details. You are not stuck with using your computer the way anyone else (such as Microsoft or IBM or your mother) thinks you should use it. You can customize your system as you see ﬁt, and you can choose from many well-designed tools and applications. Second, using Unix will change how you think, and for the better. I believe that if you learn how to read Shakespeare, listen to Mozart, or appreciate the paintings of Van Gogh, you will, in some sense, be a better person. The same is true for learning how to use Unix. At this point, I don’t blame you if you don’t believe me, but when you have ﬁnished the book, come back to this chapter, reread this section, and see if I am not right. Third, as a member of the global Unix community, you will learn how to use some of the best tools ever invented by human beings. In addition, you will be working with a computer system that can run for months without rebooting. You won’t have to worry about your computer crashing, freezing or stopping unexpectedly, and – unless you are administering a large network – you won’t need to care about such irritations as computer viruses, spyware, programs that run amok, or mysterious rituals that must be carried out to keep your computer running smoothly. Finally, if you are a programmer (or if you want to learn how to be a programmer), you will ﬁnd a wonderful selection of Unix-based tools to develop, test and run programs: text editors with language-speciﬁc plugins, script interpreters, compilers, cross-compilers, debuggers, emulators, parser generators, GUI builders, software conﬁguration managers, bug-tracking software, build managers, and documentation tools. Moreover, for most types of programming, there are active communities with their own Web sites, mailing lists and discussion groups, as well as comprehensive software archives.

Harley Hahn’s Guide to Unix and Linux

Openmirrors.com 33614_01_001_008.indd 2

1/9/2008 12:20:03 PM

Introduction to Unix Of course, I can’t teach you all the details of the Unix culture in one book. If I tried, both of us would be overwhelmed. Rather, I will teach you the basics. By the time you ﬁnish this book, you will understand the most important concepts, and you will be able to use the most important tools. You will also be able to teach yourself whatever else you want to know as the need arises. To start, all you need is access to any computer that runs some type of Unix (such as Linux), an Internet connection, and a lot of time and patience. And, oh yes, one more thing: in most cases, everything – including software upgrades – is free.

THE UNIX LANGUAGE Around the world, the ﬁrst language of Unix is American English. Nevertheless, Unix systems and documentation have been translated into many other languages, so it is not necessary to know English, as long as your system works in your language. However, as you explore the worldwide Unix-based community, you will ﬁnd that much of the information and many of the discussion groups are in English. In addition, the Unix community has introduced many new words of its own. In this book, I will pay particular attention to such words. Every time I introduce a new word, I will use CAPITAL LETTERS. I will make sure to explain the word and show you how it is used. For easy reference, all the deﬁnitions are collected into a glossary at the end of the book.When I come to a name with a particularly colorful history, I will give a special explanation like this. WHAT’S IN A NAME? Unix In the 1960s, a number of researchers from Bell Labs (a part of AT&T) worked at MIT on a project called Multics, an early time-sharing operating system. Multics was a collaborative effort involving programmers from MIT, GE and Bell Labs. The name Multics was an acronym for “Multiplexed Information and Computing Service”. (“Multiplex” refers to combining multiple electronic signals into a single signal.) By the late 1960s, the management at Bell Labs decided not to pursue Multics and moved their researchers back into the lab. In 1969, one of these researchers, Ken Thompson, developed a simple, small operating system for a PDP-7 minicomputer. In searching for a name, Thompson compared his new system to Multics. The goal of Multics was to offer many features to multiple users at the same time. Multics was large and unwieldy and had many problems. Thompson’s system was smaller, less ambitious and (at least at the beginning) was used by one person at a time. Moreover, each part of the system was designed to do only one thing and to do it well. Thompson decided to name his system Unics (the “Uni” meaning “one”, as in unicycle), which was soon changed to Unix. In other words, the name Unix is a pun on the name Multics.

Throughout the book, I print certain names in boldface, usually the names of commands. This allows you to see immediately that a word is a special Unix term. Here is an example:

The Unix Language

33614_01_001_008.indd 3

1/9/2008 12:20:03 PM

Chapter 1 “To copy a ﬁle, you use the Unix cp command. To remove (delete) a ﬁle, you use the rm command.”

HINTS FOR LEARNING UNIX As you read this book, you will notice many hints for learning Unix. These are ideas and shortcuts I have found to be important for newcomers and experienced users alike. To emphasize these hints, I present them in a special format that looks like this: HINT Unix is fun.

PEOPLE WHO DON’T KNOW THEY ARE USING UNIX What type of person uses Unix? Taken literally, there is no deﬁnitive answer to this question. Unix systems are used by a vast number of people around the world, so it would be very difﬁcult to generalize. However, that never stopped me, so let’s have a go at it. Broadly speaking, we can divide the world of Unix users into two parts: those who know they are using Unix and those who don’t. Most of the people who use Unix don’t know they are doing so. This is because Unix is used with many different types of computer systems, and those systems work so well that the people using them aren’t even aware they are using Unix. For example, most Web servers run some type of Unix. When you visit Web sites, more often than not you are using Unix, at least indirectly. Unix is also used by many businesses, schools, and organizations. When you have occasion to use their computer systems, say, to make a reservation, look up information, control a machine, register for a class, or do ofﬁce work, you are also using Unix without even knowing it. In addition, Unix is used to run all kinds of machines: not only computers of all sizes (from the largest mainframes to the smallest handheld devices), but embedded or realtime systems, such as appliances, cable modems, cell phones, robots, karaoke machines, cash registers, and so on. Finally, most of the machines that support the Internet run Unix. For example, the computers that pass data from one point to another (routers) all use some type of Unix, as do most mail servers (which store email) and Web servers (which send out Web pages). Once you understand Unix, a lot of the idiosyncrasies that you ﬁnd while using the Net will make sense. For example, you will understand why you must pay attention to case – that is, small letters and capital letters – when you type Web addresses. (Most Web servers run under some type of Unix and, as you will see, Unix is case sensitive.) To me, the most interesting group of people who use Unix without knowing it are the Macintosh users. Millions of people using a Mac with OS X have no idea that, underneath, OS X is actually based on a type of Unix called FreeBSD. (We’ll talk more about OS X

Harley Hahn’s Guide to Unix and Linux

Openmirrors.com 33614_01_001_008.indd 4

1/9/2008 12:20:04 PM

Introduction to Unix in Chapter 2.) This is one reason Macs are so reliable, especially when compared to PCs running Windows (which is deﬁnitely not based on Unix).

PEOPLE WHO DO KNOW THEY ARE USING UNIX What about the people who do know that they are running Unix? In other words, what type of person voluntarily chooses to learn Unix? In my experience, such people (like you and I) have four basic characteristics. First, Unix people are smarter than average, in most cases, a lot smarter than average. Second, Unix people are both lazy and industrious. They don’t like to do busy work for no reason, but when they have a problem that interests them, they will work hour after hour looking for a solution. Third, Unix people like to read. In a world in which most people have trouble mustering up an attention span longer than ﬁve minutes, Unix people will actually read the manual when they have a problem. Finally, when Unix people use a computer, they want to be in control of the experience; they don’t want to feel as if the computer is controlling them. Are you wondering about yourself? If so, don’t worry. The fact that you are reading this book and have gotten this far qualiﬁes you as a Unix person.

GETTING THE MOST FROM THIS BOOK I have designed this book to make it easy for you to ﬁnd what you need quickly. Before you start, take a moment to examine the various parts of the book. (I know when I say that, most of you won’t want to do it, but please do it anyway.) First, look at the Quick Index of Unix Commands on the inside back cover. This is a list of the most important commands covered in the book and where to look for the discussion and examples. Next, take a look at the glossary. This will give you an overview of the important concepts I will be teaching you in this book. Notice there is a lot to learn. Third, take a glance at the Quick Index for the vi Text Editor. Once you learn how to use the program (Chapter 22), you will ﬁnd this index especially helpful. Now take a look at the general index. Spend a few minutes skimming it (always a good idea with a new book). This will give you a rough feeling for the new ideas you will encounter as well as the concepts I will be emphasizing. Aside from the glossary and the indexes, there are two summaries of Unix commands, also at the back of the book. These summaries contain one-line descriptions of each command I cover in the book. One summary lists the commands in alphabetical order; the other summary groups the commands by category. These summaries are a good place to check if you want to do something and are not sure what command to use. Once you have found your command, check with the Quick Index of Unix Commands (inside back cover) to see what page to read. If you want to ﬁnd the discussion of a particular topic, you can, of course, use the general index. Alternatively, you can look up the appropriate term in the glossary. Along with each

Getting the Most From This Book

33614_01_001_008.indd 5

1/9/2008 12:20:04 PM

Chapter 1 deﬁnition, you will ﬁnd a reference to the chapter in which that term is explained. Once you know the chapter you want, a quick look at the table of contents will show you what section to read.

WHAT I ASSUME IN THIS BOOK In this book, I make two important assumptions as to what type of Unix system you are using. First, as you will see in Chapter 2, there are many versions of Unix. Today, the most popular Unix systems are of a type called Linux. Most Unix systems contain the same basic elements so, for the most part, it doesn’t matter what type of Unix you are using. However, at times, there will be a choice between Linux or non-Linux functionality. In such cases, I will lean toward the Linux conventions as these are the most popular. Second, as you will see in Chapter 4, the program that reads and interprets the commands you type is called the “shell”. In Chapter 11, I will explain that there are various shells you might choose to use. Almost all the time, it doesn’t really matter what shell you use. However, in those few places where it does matter, I will use a particular shell named “Bash”. If you want to use another shell, that is ﬁne. A few details may be different, but you won’t have any real problems.

WHAT I DO NOT ASSUME IN THIS BOOK If you are an experienced computer user who wants to learn about Unix, this book will get you started and provide you with a ﬁrm background in all the important areas. However, I do not assume that you have any prior experience. It’s okay if you have never really used a computer. You do not need to know anything about Unix. You do not need to be a programmer, nor do you need to know anything about electronics or mathematics. I will explain everything you need to know. Work at your own speed and enjoy yourself.

HOW TO USE THIS BOOK Before we start, it is important to realize that the world of Unix is bursting with information. To get started, read the ﬁrst seven chapters of the book. They will introduce you to Unix and teach you the basic skills. After you are oriented to Unix, and you know how to start and stop a work session, enter commands, and use the keyboard, you can read the rest of the book in any order you want. HINT It is impossible to learn everything about Unix. Concentrate on what you need and what you think you will enjoy.

Although every effort has been made to make each chapter as independent as possible, you should realize that each topic is dependent on other topics. There is no perfect place to start learning Unix and no perfect order in which to study the various topics.

Harley Hahn’s Guide to Unix and Linux

Openmirrors.com 33614_01_001_008.indd 6

1/9/2008 12:20:04 PM

Introduction to Unix For example, say that you want to customize your work environment. Naturally, it would make sense to start by reading about the Unix working environment (Chapter 6). You then need to understand what we call the “shell” (Chapter 11), as well as the details of using the shell (Chapters 12, 13 and 14). At this point, you can customize your work environment by modifying certain ﬁles. However, in order to modify ﬁles, it is handy to already know how to use a text editing program (Chapter 22). Since you need to save these ﬁles, you should already understand the ﬁle system (Chapter 23), the commands to display your ﬁles (Chapter 21), and the commands to manipulate your ﬁles (Chapters 24 and 25). And, of course, before you can type in messages, you need to understand how to start a work session (Chapters 4 and 5) and how to use the keyboard with Unix (Chapter 7). Obviously, this sort of approach leads nowhere fast, but it does underscore the most important principle that you need to understand at the outset: Unix was not designed to be learned; Unix was designed to be used. In other words, it can be confusing and time-consuming to learn Unix. However, once you have mastered the skills you need for whatever work you want to do, working with Unix is fast and easy. If you think back to when you learned how to drive a car, you will remember that it was anything but simple. Once you had some experience, though, your actions became smooth and automatic. By now, you can probably drive all day with one hand on the wheel as you listen to the radio and talk to other people. Let us embody this idea as the following hint: HINT Unix is easy to use, but difﬁcult to learn.

Remember, once you have read the ﬁrst few chapters of this book, you can teach yourself any topic in any order. If you come across an idea or skill you do not yet understand, you can either pause for a quick look at another chapter, or skip the part that confuses you and learn it later. This is how people learn Unix in real life: a bit at a time, depending on what they need at the moment. Don’t worry about memorizing every detail. In some chapters, I treat topics in depth. Learn what seems interesting and useful to you and just skim the rest. If you know the basics and you have an idea as to what is available, you can always return to the details when you need them. HINT Start by learning the basics. Then learn whatever you want, in whatever order you want.

How to Use This Book

33614_01_001_008.indd 7

1/9/2008 12:20:04 PM

Chapter 1

C H A P T E R

E X E R C I S E S

REVIEW QUESTIONS 1. When was the ﬁrst Unix system developed? Where was the work done? 2. What are four important reasons to learn Unix? 3. What is the origin of the name “Unix”?

FOR FURTHER THOUGHT 1. Prior to 2001, the operating system used by Apple for Macintosh desktop computers was completely proprietary. In 2001, Apple introduced a new operating system (OS X) based on Unix. What are three advantages to changing to a Unix-based operating system? What are three disadvantages?

Harley Hahn’s Guide to Unix and Linux

Openmirrors.com 33614_01_001_008.indd 8

1/9/2008 12:20:04 PM

What Is Linux? What Is Unix?

What is Unix and what is Linux? The short answer is Unix is a type of computer system, and Linux is the name of a particular family of Unix systems. That answer should be good enough if your grandmother happens to ask “What is all this Unix stuff you are learning?” For practical purposes, however, you need to know more. In this chapter, I’m going to explain the most important ideas that underlie Unix in general and Linux in particular, and I’m going to do it for two reasons. First, I think you will ﬁnd the discussion interesting. To me, Unix is the most wonderful computer system ever invented, and I’d like you to share that feeling. Second, once you have a basic understanding of Unix, you’ll have a context for what I will be teaching in this book. In that way, when we get into the heavy-duty technical stuff, you’ll ﬁnd it a lot easier to remember the details. By the end of the chapter, I will have given you a good, serviceable deﬁnition of both Unix and Linux. However, before I can do that, I’m going to have to explain a number of important concepts. To lay the groundwork, I’m going to start with a discussion of the fundamental component of any computer system: the operating system.

WHAT IS AN OPERATING SYSTEM? Computers perform tasks automatically by following instructions. A list of instructions is called a PROGRAM. As the computer follows the instructions, we say that it RUNS or EXECUTES the program. In general, programs are referred to as SOFTWARE, while the physical components of the computer are referred to as HARDWARE. The hardware includes the system board, disk drives, keyboard, mouse, display, screen, printers, and so on. An OPERATING SYSTEM (which is software) is the complex master control program that runs the computer. The principal function of an operating system is to make efﬁcient use of the hardware. To do so, the operating system acts as the primary interface to the hardware, both for you as you are working, and for your programs as they are executing. Whenever your computer is up and running, the operating system is there, waiting to serve you and to manage the resources of your computer.

What Is an Operating System?

33614_02_009_036.indd 9

1/9/2008 12:23:35 PM

Chapter 2 For example, say you type a command to display the names of your ﬁles. It is the operating system that handles the details of ﬁnding the names and displaying them on your screen. When you are running a program that needs to open a new ﬁle, it is the operating system that sets aside the storage space and handles all the details. More precisely, the most important functions of an operating system are to: • Take control of the computer and initialize itself each time the machine starts or restarts. This initialization is part of the BOOTING process. • Support the interface (text or graphical) that you use to interact with the computer. • Provide an interface for programs as they need to use the computer’s resources (disk space, ﬁle allocation, processing time, memory, and so on). • Manage the computer’s memory. • Maintain and manage a ﬁle system. • Schedule work to be done. • Provide accounting and security services. In addition, all operating systems are packaged with a large variety of programs for you to use. For example, there are programs to help you create, modify and manage ﬁles; to manage your work environment; to communicate with other people; to use the Internet; to write programs; and on and on. Unix comes with more than a thousand such programs, each of which is a tool to perform one speciﬁc job. As a family, Unix operating systems all share two important characteristics: multitasking and multiuser. MULTITASKING means that a Unix system can run more than one program at a time. MULTIUSER means that Unix can support more than one user at a time. (Microsoft Windows, by the way, is a multitasking, single-user operating system.) WHAT’S IN A NAME? Booting The term “booting” is short for bootstrapping, which refers to the old saying “pulling yourself up by the bootstraps”. For example, “After Bartholomew lost all his money, he was poor for a long time. However, by working hard, he was able to pull himself up by the bootstraps and become a successful groatcakes merchant.” Obviously, it is physically impossible to pull yourself up by straps that are attached to your own boots. The idea is that a difﬁcult, complex goal can often be accomplished by starting with one small action and then, using that as a foundation, building one step at a time until the desired goal is reached. A computer system starts in just this way. When you turn on the power (or restart the computer) a single, small program is run automatically. That program starts another, more elaborate program, and so on. Eventually, the operating system (a very complex program) takes control and ﬁnishes the initialization.

Harley Hahn’s Guide to Unix and Linux

Openmirrors.com 33614_02_009_036.indd 10

1/9/2008 12:23:35 PM

What Is Unix? What Is Linux? WHAT IS THE KERNEL? When a computer is started, it goes through a sequence of actions that comprise the booting process. The ﬁnal act in this process is to start a very complex program called the KERNEL. The job of the kernel is to take control of the computer and to act as the core of the operating system. As such, the kernel is always running. Indeed, it does not stop until you shut down the system. In this way, the kernel is constantly available to provide essential services as they are needed. The kernel, being the heart of the operating system, is very important, so I want to take a moment to tell you a few details. I’m going to get a bit technical, so if you are at all confused, just nod your head wisely and pretend you understand. Although the nature of a kernel can vary from one operating system to another, the essential services it provides are pretty much the same from one system to another. They are: • Memory management (virtual memory management, including paging) • Process management (process creation, termination, scheduling) • Interprocess communication (local, network) • Input/output (via DEVICE DRIVERS, programs that perform the actual communications with physical devices) • File management • Security and access control • Network access (such as TCP/IP) If these technical terms mean something to you, ﬁne. If not, don’t worry about it. Only the most nerd-like programmers actually care about the internal details of a kernel. For people like you and me, the crucial thing is to appreciate that the kernel is the most important part of the operating system. In fact, as we will see later, the main difference between Linux and other types of Unix is that Linux uses a particular kernel that is different from other Unix kernels. There are a variety of different types of kernels, but basically they can be divided into large ones called MONOLITHIC KERNELS and small ones called MICROKERNELS. A monolithic kernel consists of one very large program that does everything by itself. A microkernel is a much smaller program that can carry out only the most basic tasks. To perform the rest of its functions, a microkernel calls upon a set of other programs called SERVERS. The advantage of a monolithic kernel is that it is fast: everything is done within a single program, which is efﬁcient. The disadvantage is that monolithic kernels are large and unwieldy, which makes them difﬁcult to design and maintain. A microkernel is slower, because it must call upon servers to do most of its work, which is less efﬁcient. However, because of the modular design, microkernels are a lot easier for

What Is the Kernel?

33614_02_009_036.indd 11

1/9/2008 12:23:35 PM

Chapter 2 programmers to understand, and they can be modiﬁed to work on new systems relatively quickly. Microkernels also have the advantage in that they are easier to customize than are monolithic kernels. For example, say you are creating an operating system for a mobile phone or a robot. Storage space is at a premium, so a small kernel is better than a large one. Moreover, with a special purpose device, you may not need all the functionality of a monolithic kernel. In this case, a microkernel with a carefully selected set of servers may be the best bet. When a new operating system is being created, the designers have a choice. They can use either a single large monolithic kernel or a small minimal microkernel with servers. Most Unix systems use some type of monolithic kernel. However, as we will see, some Unixes, such as the Macintosh Unix (called OS X), use a microkernel. WHAT’S IN A NAME? Kernel Imagine a pistachio nut. The outside is a hard shell. The inside is a soft, edible seed which, in biology, is called the kernel. Thus, if we want to be precise, we can say that, when we eat pistachio nuts, we crack open the shell in order to eat the kernel. If we think of Unix as a nut, the inside would be the kernel, and the outside would be the shell; and indeed, that is the case. The kernel, which we just discussed, is what we call the core of the operating system. The SHELL (which we will discuss in Chapter 11) is the name we give to a special type of program (a command processor) that “surrounds” the kernel and acts as our personal interface into the system.

UNIX = KERNEL + UTILITIES I have explained that the kernel is the central part of the operating system. The kernel is always running, and its job is to perform the essential tasks. But what about everything else? With Unix, “everything else” consists of a large number of auxiliary programs that are included as part of the Unix package. These programs fall into a number of different categories. The most important programs are the ones that provide an interface for you to use the computer. They are the shells and GUIs. A shell is a program that provides a textbased interface: you type commands, one after another; the shell reads your commands and does what is necessary to carry them out. A GUI (graphical user interface) is a more elaborate program, which provides a graphical interface with windows, a mouse pointer, icons, and so on. We will talk about shells and GUIs in Chapters 5 and 6, and shells in detail in Chapter 11 so, for now, all I will do is mention that they exist. The other programs are called the Unix UTILITIES, of which there are hundreds. Each utility (that is, each program) is a separate tool. All Unix systems come with hundreds of such tools as part of the operating system. Some of these are for programmers, but

Harley Hahn’s Guide to Unix and Linux

Openmirrors.com 33614_02_009_036.indd 12

1/9/2008 12:23:36 PM

What Is Unix? What Is Linux? many of them are useful to anyone, even casual users. Moreover, as we will discuss in Chapter 15, Unix provides ways to combine existing tools to solve new problems as they arise. This wealth of tools, which is part of every Unix system, is the main reason why Unix is so powerful. Unix utilities work pretty much the same on every Unix system. This means that, for practical purposes, if you know how to use one type of Unix, you know how to use them all. Indeed, most of this book is devoted to teaching you how to use the most important Unix utilities. My goal is that, by the time you have ﬁnished this book, you will be familiar with the most important tools, which means you will feel comfortable using Unix on a day-by-day basis. So, what is Unix? A good answer would be that Unix is a type of operating system that uses a Unix kernel and comes with the Unix utilities and a Unix shell. In fact, most (but not all) Unixes come with a selection of shells and at least one GUI. In informal speech, we often use the term “utilities” to include the shell, so at this point, we can deﬁne Unix as follows: Unix = a Unix kernel + the Unix utilities

“UNIX” USED TO BE A SPECIFIC NAME In Chapter 1, I described how the ﬁrst primitive Unix system was developed in 1969 by a single programmer, Ken Thompson. The work was done at Bell Labs, the research arm of AT&T. Since then, a large number of people have developed Unix into a modern family of operating systems. But who owns the actual name “Unix”? For years, it was owned by AT&T, which insisted that Unix must always be spelled with capital letters: UNIX. More precisely, the AT&T lawyers speciﬁed that “The trademark UNIX must always appear in a form that is typographically distinct.” In addition, the lawyers decreed that you could not talk about UNIX in its own right. You had to talk about UNIX operating systems. (“The trademark UNIX may not be used as a noun, but must always be used as an adjective modifying a common noun as in ‘UNIX operating system.’”) For many years, Bell Labs remained one of the centers of Unix development, and the silliness continued. In 1990, AT&T formed a new organization to take over Unix. The new organization was called Unix Systems Laboratory (USL). In June 1993, AT&T sold USL to Novell Corporation. In October 1993, Novell transferred the rights to the name “UNIX” to an international standards organization called X/Open. Finally, in 1996, X/Open merged with the Open Software Foundation (a consortium of computer companies) to form The Open Group. Thus, the UNIX trademark is now owned by The Open Group. So what does The Open Group say that UNIX means? They say that “UNIX” is a brand that refers to any operating system that has been certiﬁed by them as complying with their so-called “Single UNIX Speciﬁcation”. The details are horrible, so let’s move on.

“Unix” Used to Be a Speciﬁc Name

33614_02_009_036.indd 13

1/9/2008 12:23:36 PM

Chapter 2 “UNIX” IS NOW A GENERIC NAME The Open Group (and AT&T and USL and Novell and X/Open) notwithstanding, the word “Unix” has, for years, been used informally to refer to any operating system that is Unix-like. In this sense, there are many, many different types of Unix, with more being developed every year. However, all of this begs the question, what does it mean to be Unix-like? There are two answers, neither of which is exact. The ﬁrst answer is that an operating system is Unix if (1) it consists of a Unix kernel and the Unix utilities and, (2) it can run Unix programs that run on other Unix systems. The second answer is that an operating system is Unix if people who understand Unix say that it is Unix. I realize that to a purist – such as Socrates or an Orthodox Rabbi – such an informal deﬁnition is not satisfying. However, in the real world (the place we live when we are not in school), it is not always possible to be completely precise. Consider, for example, the very interesting United States Supreme Court case of Jacobellis v. Ohio, 378 U.S. 184 (1964). This case involved an appeal by the manager of a movie theater who was convicted of showing an obscene ﬁlm. In the ofﬁcial written judgment, Mr. Justice Potter Stewart discusses the question of obscenity. What, in fact, is it? He writes, “I shall not today attempt further to deﬁne the kinds of [hard-core p*rnography] material I understand to be embraced within that shorthand description; and perhaps I could never succeed in intelligibly doing so. But I know it when I see it, and the motion picture involved in this case is not that.” So, I know I am in good company if I tell you the truth: If you want to get really technical, I can’t tell you what Unix is – perhaps no one can – but I know it when I see it. (And, one day, you will too.)

THE FREE SOFTWARE FOUNDATION Now that you know what Unix is, I’d like to answer the next question: What is Linux? To do so, I need to start by talking about the Free Software Foundation and the idea of open source software. Imagine you enjoy this book so much that you would like all your friends to have a copy. How could you do so? You could buy a whole set of books and give them out to your friends (which isn’t a bad idea, especially if you are trying to impress everyone you know). Of course, that would cost a lot of money. However, each person would receive an actual printed book, and at least you would feel you were getting something for your money. Alternatively, you could make copies of the book. For example, you could photocopy, say, 30 copies of the book and give away the copies. This would save you some money but, compared to the original, the copies would be far from perfect. Moreover, it would take a lot of time and effort to photocopy, collate, bind and distribute the copies, and when your friends received them, they would know they were getting an inferior product. Even worse, if a friend wanted to make copies of his own, the quality would be worse because a photocopy of a photocopy is not nearly as good as the original.

Harley Hahn’s Guide to Unix and Linux

Openmirrors.com 33614_02_009_036.indd 14

1/9/2008 12:23:36 PM

What Is Unix? What Is Linux? Now, suppose you were reading an electronic version of this book, and you wanted to share the book with your friends. All you would have to do is copy some ﬁles and email them to your friends or burn them onto a CD and give it away. Not only would it be cheap to do so (maybe even free), but the copies would be identical to the original. Furthermore, your friends could easily make copies of the copies. To be sure, it would be illegal to make such copies, but forget legal for a moment. Morally, is it right or wrong to copy and distribute electronic data (books, software, music, videos, and so on)? There is no easy answer here. It all depends on your point of view, and there are good arguments on each side. One thing I can tell you: because electronic copying is so cheap and reliable, we tend to devalue anything that is in electronic format. For example, think of what you paid for this book. Would you feel comfortable paying the same amount of money for a CD that contained the book? Or for access to a Web site where you could read the book online? Because it is so easy to copy (or steal) electronic data, people tend to feel that electronic data isn’t worth much. For this reason, software companies have traditionally been wary about releasing programs without strict license agreements that prohibit copying and modiﬁcation. However, what if you could convince a lot of talented programmers to distribute their software in such a way that it encouraged copying? Would it change the world for the better? In the early 1980s, a visionary named Richard Stallman asked himself that question. Stallman had been working in the MIT Artiﬁcial Intelligence (AI) Lab since 1971. The AI Lab had a long history of sharing software with anyone, not only inside the lab, but in other organizations. In 1981, however, conditions changed, and many of the AI Lab people left to join a startup company. The main computer was changed and the operating system was replaced with a proprietary system. Stallman now found himself working in an environment in which he and his coworkers had lost the right to look inside the operating system and modify it. It happened that Stallman was more than an expert programmer. He was also a thoughtful social critic, who saw the change to a proprietary operating system as a restriction on his social rights as a creator. In his words, a “proprietary-software social system, in which you are not allowed to share or change software”, was not only “antisocial”, it was “unethical” and “wrong”. Such systems, he observed, create unhealthy power struggles between programmers and software companies. The solution was to create a large body of well-written, useful software that, from the beginning, was designed to be freely distributed. In January 1984, Stallman quit his job to devote himself to the project. Within a short time, he had attracted a small number of other programmers and, in 1985, they started an organization called the FREE SOFTWARE FOUNDATION or FSF. Stallman’s guiding principle was “Computer users should be free to modify programs to ﬁt their needs, and free to share software, because helping other people is the basis of society.” Stallman believed that programmers, by their nature, like to share their work and, when they do, everyone beneﬁts.

The Free Software Foundation

33614_02_009_036.indd 15

1/9/2008 12:23:36 PM

Chapter 2 It is important to understand that when Stallman talked about “free software”, he was not referring to cost; he was referring to freedom. Free software can be examined, modiﬁed, shared and distributed by anyone. However, according to Stallman, there is nothing wrong if someone charges for their services or makes other people pay for software distribution. The way Stallman explained it is to think in terms of “free speech, not free beer”. Since the word “free” can refer to both freedom and cost, there is some ambiguity. So, to avoid any possible confusion, free software is now called OPEN SOURCE SOFTWARE. (The name comes from a programming term. When a programmer creates a program, the actual instructions he writes are called the SOURCE CODE or, more simply, the SOURCE or the CODE. (For example, let’s say that you are talking to the Queen of England, and she is complaining that the program she uses to rip music from CDs keeps crashing. “If I only had the code”, she says, “I could get Charles to ﬁx the bug.” “Don’t worry,” you answer. “I know where to get the source. I’ll email Chuck the URL.”) Since the heart of any computer is the operating system, Stallman’s ﬁrst major goal for the FSF was to create an operating system that could be shared freely and modiﬁed by anyone. In order for the new operating system to be freely distributable, Stallman realized that it would have to be written from scratch. Stallman wanted the FSF’s products to ﬁt smoothly into the prevailing programming culture, so he decided that the new operating system should be compatible with Unix in the sense that it should look just like Unix and should be able to run Unix programs. In the tradition of the programming community in which he worked, Stallman chose a whimsical name for the as-yet unbuilt operating system. He called it GNU. WHAT’S IN A NAME? GNU GNU is the name Richard Stallman chose to describe the Free Software Foundation’s project to develop a full Unix-like operating system. The name itself is an acronym meaning “GNU’s Not Unix” and is pronounced “ga-new”. (It rhymes with the sound that you make when you sneeze.) Notice that, within the expression “GNU’s Not Unix”, the word GNU can be expanded indeﬁnitely: GNU (GNU's Not Unix) ((GNU's Not Unix) Not Unix) (((GNU's Not Unix) Not Unix) Not Unix) ((((GNU's Not Unix) Not Unix) Not Unix) Not Unix) and so on. Thus, GNU is actually a recursive acronym. (RECURSIVE refers to something that is deﬁned in terms of itself.)

EXCERPTS FROM THE GNU MANIFESTO As I mentioned in the last section, Richard Stallman, the founder of the Free Software Foundation, was more than a programmer. He was an educated social critic, and his vision of the future was to have an enormous impact on the world.

Harley Hahn’s Guide to Unix and Linux

Openmirrors.com 33614_02_009_036.indd 16

1/9/2008 12:23:36 PM

What Is Unix? What Is Linux? Shortly after founding the FSF, Stallman wrote a short essay in which he explained his reasons for promoting the idea of free software. He called the essay the GNU MANIFESTO. His basic idea – that all software should be shared freely – is, at best, naïve. However, with the rise of the Internet, the development and distribution of open source software (what Stallman called “free software”) has become an important economic and social force in our world. There are literally tens of thousands of programs available for free, and their contribution to the world at large (and to the happiness of their programmers) is beyond measure. We will discuss this idea more in the next section. Before we do, I’d like to take a moment to show you a few passages from the original 1985 essay. As a philosopher, Stallman was not a heavyweight. His public declaration was not as sophisticated as other well-known manifestos, such as 95 Theses (Martin Luther, 1517), Manifesto of the Communist Party (Karl Marx and Frederick Engels, 1848), or The Playboy Philosophy (Hugh Hefner, 1962-1966). Still, the work of the Free Software Foundation was very important and, to this day, it continues to make an important contribution to our culture. For this reason, you may be interested in reading a few excerpts from Stallman’s 1985 essay. (Quick aside: When Stallman was working at MIT, he created the Emacs text editor. If you have access to a GNU version of Emacs, which is the case if you use Linux, you EXCERPTS FROM THE GNU MANIFESTO “I consider that the golden rule requires that if I like a program I must share it with other people who like it. Software sellers want to divide the users and conquer them, making each user agree not to share with others. I refuse to break solidarity with other users in this way. I cannot in good conscience sign a nondisclosure agreement or a software license agreement. For years I worked within the [MIT] Artiﬁcial Intelligence Lab to resist such tendencies and other inhospitalities, but eventually they had gone too far: I could not remain in an institution where such things are done for me against my will. “So that I can continue to use computers without dishonor, I have decided to put together a sufﬁcient body of free software so that I will be able to get along without any software that is not free. I have resigned from the AI lab to deny MIT any legal excuse to prevent me from giving GNU away... “Many programmers are unhappy about the commercialization of system software. It may enable them to make more money, but it requires them to feel in conﬂict with other programmers in general rather than feel as comrades. The fundamental act of friendship among programmers is the sharing of programs; marketing arrangements now typically used essentially forbid programmers to treat others as friends. The purchaser of software must choose between friendship and obeying the law. Naturally, many decide that friendship is more important. But those who believe in law often do not feel at ease with either choice. They become cynical and think that programming is just a way of making money... “Copying all or parts of a program is as natural to a programmer as breathing, and as productive. It ought to be as free... “In the long run, making programs free is a step toward the post-scarcity world, where nobody will have to work very hard just to make a living. People will be free to devote themselves to activities that are fun, such as programming, after spending the necessary ten hours a week on required tasks such as legislation, family counseling, robot repair and asteroid prospecting. There will be no need to be able to make a living from programming...”

Excerpts From the GNU Manifesto

33614_02_009_036.indd 17

1/9/2008 12:23:37 PM

Chapter 2 can display the entire GNU Manifesto by starting Emacs and entering the command .)

THE GPL AND OPEN SOURCE SOFTWARE Over the years, Stallman and the many other programmers who supported the Free Software Foundation worked hard to create a huge body of open source software. In fact, as I mentioned, Stallman was the original author of the Emacs text editor. Today, the FSF is not the only organization to promote open source software; indeed, there are many. However, the FSF has always been one of the leaders, not only with Emacs, but with a C compiler (gcc), a C++ compiler (g++), a powerful debugger (gdb), a Unix shell (Bash), and many, many other tools. All of this software – which is part of the GNU project – is used around the world and is considered to be of the highest quality. In the late 1980s, Stallman had some experiences that showed him if he was going to create a large body of free software, he was going to need an appropriate license under which he could distribute that software. For this purpose, he invented the idea of the COPYLEFT. (The name came from a friend of Stallman’s who, in the mid-1980s, sent Stallman a letter in which several witty sayings were written on the envelope. Among these was “Copyleft — all rights reversed.”) Within the software community, traditional copyrights were used to restrict the use of the software. The purpose of the copyleft was to do the opposite. In Stallman’s words, “The central idea of copyleft is that we give everyone permission to run the program, copy the program, modify the program, and distribute modiﬁed versions – but not permission to add restrictions of their own.” To implement the idea of copyleft, Stallman wrote the GENERAL PUBLIC LICENSE or GPL, which he released in 1989. The GPL itself is rather complicated, so I won’t go into the details. Basically, when applied to software, the GPL says that anyone may distribute the software, view the source code, modify it, and distribute the changes. Furthermore – and this is the crucial part – no one who redistributes the software, including modiﬁed versions, can take away any of the freedoms or add any restrictions of his own. The GPL ensures that, whenever anyone uses free software to create a new product, the new product cannot be distributed except under the GPL. In practical terms, this means that if someone starts with free software, changes it, and redistributes it, he must also release the source code. The GPL became popular and, over the years, a variety of similar licenses have been developed. Indeed, the combination of copyleft licenses and the Internet (allowing programmers around the world to share and to work together) has led to an enormous ﬂourishing of shared creativity, the so-called OPEN SOURCE MOVEMENT. The open source movement is so important that it would be difﬁcult to overstate its impact within the world of programming and on the world at large. By way of illustration, if all open source software were to vanish, the Internet as we know it would disappear in an instant. (For example, most of the Web servers in the world are run by an open source program called Apache.)

Harley Hahn’s Guide to Unix and Linux

Openmirrors.com 33614_02_009_036.indd 18

1/9/2008 12:23:37 PM

What Is Unix? What Is Linux? In the world of Unix, the effects of the open source movement have been momentous. There are many reasons for this, not least of which is that if it hadn’t have been for the FSF and the GPL, and the changes they made in the programming culture, you and I probably would never have heard of Linux.

UNIX IN THE 1970S: FROM BELL LABS TO BERKELEY At the beginning of Chapter 1, I explained that the ﬁrst version of Unix was developed at AT&T’s Bell Labs, in New Jersey, in 1969. In the early 1970s, Unix was rewritten and enhanced by a handful of people, all from Bell Labs. In 1973, a Unix development support group was formed. Later that year, Ken Thompson (one of the two principal Unix developers) delivered the very ﬁrst paper on Unix at a conference of computer professionals. This sparked an interest in Unix, and within six months, the number of sites running Unix increased from 16 to 48. In July 1974, Thompson and Dennis Ritchie (the other principal Unix developer) published a paper, “The UNIX Time-Sharing System”, in Communications of the ACM*, the most widely read computer science journal in the world. It was the ﬁrst time that Unix was described to the world at large, and the world at large began to respond. Academics at universities and researchers inside companies requested copies of Unix and began running it on their own computers. In some cases, they PORTED Unix (that is, they adapted it) to run on new types of hardware. Outside Bell Labs, the most important nexus of Unix development was the Computer Science Department at the University of California at Berkeley. In 1974, a professor at Berkeley, Bob Fabry, procured a copy of AT&T’s UNIX version 4, and Berkeley students started making major enhancements. In 1975, Ken Thompson went to Berkeley for a one-year sabbatical, which acted as a catalyst for Berkeley’s Unix development. In the same year, AT&T formally started licensing UNIX to universities. (As you will remember, AT&T insisted that “UNIX” must always be spelled in capital letters, and I will do just that when I am referring speciﬁcally to AT&T UNIX.) Around this time, Bill Joy, a Berkeley graduate student, became very interested in Unix. Although no one knew it at the time, Joy’s work was to have far-reaching effects. What Joy was doing set the stage for Berkeley to become a major player in the Unix world, creating and distributing their own version of Unix. Indeed, several of today’s important Unixes are direct descendents of Berkeley Unix and the work of Bill Joy. In addition, Joy was a skillful and proliﬁc programmer who, over the course of a few years, single-handedly created a great deal of important software. Even today, there *One month later, in August 1974, the same journal published my ﬁrst technical paper. At the time, I was an undergraduate student at the University of Waterloo, in Canada, and I had won the ACM’s George E. Forsythe Student Paper Competition. The name of the paper was “A New Technique for Compression and Storage of Data”. (In case you are wondering, the ACM is the principal professional association for computer scientists. The name, which dates back to 1947, stands for Association for Computing Machinery.) Here is something interesting. If you check out the back copies of the CACM (as the journal is usually called), you will ﬁnd that the July 1974 issue with the original UNIX paper is often missing, no doubt the result of overenthusiastic souvenir hunters. The August 1974 issue (the one with my paper), however, is always available.

Unix in the 1970s: From Bell Labs to Berkeley

33614_02_009_036.indd 19

1/9/2008 12:23:37 PM

Chapter 2 exists no Unix system in the world that is not signiﬁcantly inﬂuenced by the work Joy did in Berkeley from 1975 to 1982. In 1982, Joy became one of the cofounders of Sun Microsystems which, in its time, was one of the most important Unix companies in the world. Perhaps I can illustrate the importance of Joy’s work in this way. It has been more than 20 years since Joy worked at Berkeley, and yet, there are several chapters of this book in which we discuss programs that Joy developed at that time: the vi editor (Chapter 22) which was written in 1976, and the C-Shell (Chapters 11-14) which was written in 1978. (While I am musing, let me mention the startling fact that the two most popular Unix text editors, still in wide use today, were both written a very long time ago: vi by Bill Joy in 1976, and Emacs by Richard Stallman in 1975.) In 1977, Bill Joy shipped the ﬁrst version of Berkeley’s Unix. This was the same year in which I started using Unix. (At the time, I was a computer science graduate student at U.C. San Diego.) The system that Bill Joy shipped was referred to as the Berkeley Software Distribution, later abbreviated as BSD. In all, Joy ended up shipping about 30 copies to users outside of Berkeley. Although this seems like a small number, it was considered a success, and in mid-1978, Joy shipped the next version, 2BSD. In 1979, AT&T ﬁnally acknowledged the potential of UNIX and announced that they were going to start selling it as a commercial product. The ﬁrst commercial version was called UNIX System III (“System Three”). In time, it was replaced by UNIX System V (“System Five”). By 1979, all BSD users were required to buy a license from AT&T and, every year, AT&T increased the price of that license. More and more, the BSD programmers were chaﬁng under the yoke of the AT&T restrictions.

UNIX IN THE 1980S: BSD AND SYSTEM V The ﬁrst version of Linux was developed in 1991, and we’ll get to it in a moment. However, to understand how Linux ﬁts into the overall picture, we need to look at what happened to Unix in the 1980s. In particular, I want to explain how it came to pass that, by the end of the 1980s, there were two principal branches of Unix: BSD and System V. By 1980, there was a dichotomy between East Coast Unix (AT&T UNIX) and West Coast Unix (BSD), and it was growing quickly. The Berkeley programmers and BSD users resented having to pay money to AT&T just to install BSD. AT&T, on the other hand, was determined to make UNIX a successful commercial product, one that was oriented towards the needs of companies who were willing to pay a lot of money for a license. In 1980, Bob Fabry, the Berkeley professor I mentioned earlier, received a large contract from DARPA (the U.S. Defense Advanced Research Projects Agency) to develop Unix. DARPA had set up a nationwide computer network connecting all their major research centers. As such, they wanted an operating system that could run on different types of hardware. The purpose of the Berkeley grant was to develop Unix for this purpose.

Harley Hahn’s Guide to Unix and Linux

Openmirrors.com 33614_02_009_036.indd 20

1/9/2008 12:23:37 PM

What Is Unix? What Is Linux? (Quick aside: DARPA was the same agency that from 1965 through 1988 funded the research that led to the creation of the Internet. Before 1972, the name of the agency was ARPA. Thus, the ancestor of the Internet was called the Arpanet, a name you may have heard. The Arpanet as a separate entity was shut down in 1989.) Once Fabry got the DARPA contract, he set up an organization called the Computer Systems Research Group or CSRG. Although Fabry couldn’t have known it at the time, the CSRG was to last until 1994 and, during those years, was to have a major inﬂuence on BSD and on the growth of Unix throughout the world. In 1980, however, all Fabry cared about was developing BSD, and that was the task to which the CSRG devoted itself. Over the years, they released a number of versions, all of which were highly regarded within the academic and research communities. The number of BSD users began to grow. By 1982, 4.1BSD supported TCP/IP, the system which was to become the basis of the Internet. In 1983, 4.2BSD was released and was considered very popular, being used in almost 1,000 installations around the world. In the commercial world, AT&T was pushing in a different direction. AT&T’s goal was to market UNIX as a commercial product. In 1982, they released UNIX System III, the ﬁrst public release of UNIX outside of Bell Labs. In 1983, they released System V, the ﬁrst version that came with ofﬁcial support. At the same time, AT&T combined three internal groups to create the UNIX System Development Lab. By the end of the year, System V had an installed base of 45,000. In 1984, when System V Release 2 (SVR2) was introduced, there were 100,000 UNIX installations. Thus, by 1985, there were two main Unix streams of consciousness. To be sure there were other forms of Unix, but all of them were derived from either BSD or System V.* In the world of Unix, the late 1980s was characterized by two phenomena: the growth of Unix in general, and the proliferation of different types of Unix. Figure 2-1 shows the most important commercial Unixes that were used during the 1980s and beyond. (A few of these operating systems are still being sold today.) Every one of these Unixes, without exception, was based on either BSD or System V or both. Although all the important Unixes were derived from either BSD or System V, there came to be many different versions and, for several years, there was enormous inﬁghting and contention. During the last half of the 1980s, I was a writer and a consultant, and I remember the intrigue and the confusion. The world of Unix was characterized by the various companies making alliances, breaking alliances, forming consortiums, disbanding consortiums, and offering one technical proposal after another, all in an attempt to “standardize” Unix and dominate the market. In the meantime, in the universities and research institutions, Unix users were growing restless. Many of them were using some variant of BSD, and they were not happy that *Indeed, this situation was to last well into the 1990s, until the growing inﬂuence of Linux and the open source movement was to change the world of Unix permanently. For example, in the first two editions of this book (1993 and 1996), I taught that Unix had basically two variations: BSD and System V.

Unix in the 1980s: BSD and System V

33614_02_009_036.indd 21

1/9/2008 12:23:37 PM

Chapter 2 NAME

COMPANY

BSD OR SYSTEM V?

AIX

IBM

BSD + System V

AOS

IBM

BSD

A/UX

Apple

BSD + System V

BSD/OS

Berkeley Software Design

BSD

Coherent

Mark Williams Company

System V

Dynix

Sequent

BSD

HP-UX

Hewlett-Packard

System V

Irix

Silicon Graphics

BSD + System V

MachTen

Tenon Intersystems

BSD

Nextstep

Next Software

BSD

OSF/1

Digital Equipment Corp

System V

SCO Unix

Santa Cruz Operation (SCO)

System V

Solaris

Sun Microsystems

BSD + System V

SunOS

Sun Microsystems

BSD

Ultrix

Digital Equipment Corp

BSD + System V

Unicos

Cray Research

System V

UNIX

AT&T

System V

Unixware

Novell

System V

Xenix

Microsoft/SCO/Altos/Tandy

System V

FIGURE 2-1: The most important types of commercial Unix

AT&T had partial control over what they were doing. The solution was to rewrite, from scratch, the parts of BSD that were based on AT&T UNIX. It took a long time, but the Berkeley programmers worked diligently, replacing the UNIX parts of BSD one component at a time. Creating a completely independent version of BSD was a noble goal, but it was not to happen until 1992.

UNIX IN 1991: WAITING FOR... By 1991, PCs had been around for 10 years. Still, there was no PC operating system that was attractive to hackers, the type of programmers who like to take things apart and modify them just for fun. DOS (for PCs) was simple and unsatisfying. The Apple Macintosh operating system was better, but the machine itself was too expensive for hobbyists. Unix would have been ﬁne. After all, some of the best hackers in the world had been working on it for years. However, BSD (as yet) didn’t run on a PC, and commercial versions of Unix were prohibitively expensive. Computer science professors had a similar problem. AT&T UNIX would be a good tool to use for teaching operating systems courses except that, in 1979, AT&T changed

Harley Hahn’s Guide to Unix and Linux

Openmirrors.com 33614_02_009_036.indd 22

1/9/2008 12:23:38 PM

What Is Unix? What Is Linux? their policy. Starting with what they called UNIX Seventh Edition, no one outside AT&T was allowed to look at the UNIX source code. AT&T’s new policy meant that, unfortunately, UNIX could no longer be used for teaching, because operating systems students need to be able to look at the source code to see how the system works. What computer science professors needed was a free, readily available operating system, one that was suitable for teaching and for which source code was readily available. One such professor was Andrew Tanenbaum, who taught at the Vrije Universiteit in Amsterdam. He bought an IBM PC and set out to build his own operating system from scratch. The new operating system was a lot like Unix, but was much smaller, so Tanenbaum called it Minix (“minimal Unix”). Minix contained no AT&T source code whatsoever, which meant that Tanenbaum could distribute the new operating system as he wished. The ﬁrst version of Minix, released in 1987, was designed to be compatible with UNIX Seventh Edition. It was used primarily for teaching and, as teaching tools go, it was a good one (but not perfect). For several years, Minix was the best tool available for programmers who wanted to learn about operating systems and to experiment. People around the world began to use it, especially in universities. At the same time, a great number of enthusiastic volunteers began to modify Minix on their own. Still, Minix did not meet people’s expectations. A lot of programmers liked it and wanted to enhance the ofﬁcial version, but the copyright was held by Tanenbaum who vetoed most of the requests. Minix, he insisted, should remain a simple operating system, suitable for teaching. Minix users were disgruntled. When Richard Stallman had founded the Free Software Foundation in 1985, he had inspired programmers around the world, and they yearned for a robust, free operating system on which they could vent their collective energy. Now, Stallman had planned a free, Unix-compatible operating system as his ﬁrst major project. He even had a name for it, GNU. By 1991, Stallman and the FSF had created a great deal of high-quality, free software, but GNU itself was nowhere near ready. As we discussed earlier, there are two types of programs that make up Unix: the kernel and everything else, which we called the “utilities”. By the end of the 1980s, the FSF had programmed a lot of the important utilities, including a shell. However, they did not have a ﬁnished kernel (the core of the operating system). Stallman and the FSF programmers had been working for some time, but the GNU kernel, which he called HURD, was far from ready. In fact, work on HURD had really only started in 1990 and, for years, programmers around the world had been crying themselves to sleep every night, plaintively singing, “Someday, my HURD will come...” So to put it all together: By 1991, PCs had been available for 10 years, and thousands of programmers, students and computer scientists shared a strong desire for an opensource, Unix-like operating system. AT&T, however, had commercialized UNIX up the wazoo. BSD was available, but it was encumbered by the AT&T license. Minix was okay, but far from perfect. Although the source code was readily available, it still required a license and it couldn’t be shared freely.

Unix in 1991: Waiting for...

33614_02_009_036.indd 23

1/9/2008 12:23:38 PM

Chapter 2 GNU, on the other hand, was distributed under the auspices of the GPL copyleft, which was great. However, the kernel – the most important part of the operating system – was a long way from being ﬁnished. As fate would have it, a young student in Finland named Linus Torvalds had just started a project: just for fun, he decided to write his own operating system kernel. Little did Torvalds know that, within a decade, his personal operating system would grow into the most important open source undertaking in history, an innovative project that, in the fullness of time, would engage hundreds of thousands of programmers and, literally, change the world. WHAT’S IN A NAME? Hurd In our discussion of kernels, I explained that most operating systems use either a monolithic kernel (a single large program) or a microkernel combined with a number of smaller programs called servers. For GNU, Richard Stallman chose to use a microkernel called Mach, combined with a group of servers which he called “the Hurd”. (The name was coined by Thomas Bushnell, the main kernel programmer.) Strictly speaking, the GNU kernel should be described as the Hurd (servers) running on top of Mach. However, in common usage, the entire kernel is usually referred to as Hurd. You already know that when Stallman chose a name for the Free Software Foundation’s version of Unix, he decided to call it GNU: a recursive acronym for “GNU’s not Unix”. When it came to naming the kernel, Stallman one-upped himself. The name Hurd stands for “HIRD of Unix-Replacing Daemons”. (In Unix, a “daemon” is a program that runs by itself in the background.) The name HIRD stands for “HURD of Interfaces Representing Depth”. Thus, Hurd is an indirectly recursive acronym. (Hopefully, the only one you will ever meet in your life).

...MR. RIGHT, LINUS TORVALDS Linus Torvalds is the quintessential right guy with the right idea in the right place at the right time. In 1991, Linus (pronounced “Lee’-nus”) was a second-year computer science student at the University of Helsinki. Like tens of thousands of other programming students who loved to tinker, Linus had read Andrew Tanenbaum’s book Operating Systems: Design and Implementation, which explained the principles behind the design of Minix. As an appendix to the book, Tanenbaum had included the 12,000 lines of source code that comprised the operating system, and Linus spent hours studying the details. (At this point, you may want to take a moment to think about what it would take to read through 12,000 lines of code, and compare that to the type of information you ﬁnd in the appendices of this book.) Like many other programmers, Linus wanted a free (open source) version of Unix. Unlike the other programmers, Linus didn’t want to wait.

Harley Hahn’s Guide to Unix and Linux

Openmirrors.com 33614_02_009_036.indd 24

1/9/2008 12:23:38 PM

What Is Unix? What Is Linux? LINUS TORVALDS ANNOUNCING HIS NEW PROJECT... From: [emailprotected] (Linus Benedict Torvalds)

Newsgroups: comp.os.minix Subject: What would you like to see most in minix? Summary: small poll for my new operating system Date: 25 Aug 91 20:57:08 GMT Organization: University of Helsinki Hello everybody out there using minix I'm doing a (free) operating system (just a hobby, won't be big and professional like gnu) for 386(486) AT clones. This has been brewing since april, and is starting to get ready. I'd like any feedback on things people like/dislike in minix, as my OS resembles it somewhat (same physical layout of the file-system (due to practical reasons) among other things). I've currently ported bash(1.08) and gcc(1.40), and things seem to work. This implies that I'll get something practical within a few months, and I'd like to know what features most people would want. Any suggestions are welcome, but I won't promise I'll implement them :-) Linus ([emailprotected]) PS. Yes - it's free of any minix code, and it has a multi-threaded fs [file system]. It is NOT protable (uses 386 task switching etc), and it probably never will support anything other than AT-harddisks, as that's all I have :-(. On August 25, 1991, Linus sent a message to the Usenet discussion group that was used as a Minix forum (comp.os.minix). In retrospect, this short message is considered to be one of the historical documents in the world of Unix, so I will show it to you in its entirety. When you read it, remember that English was Linus’s second language, and that he was writing informally. Clearly, Linus was just trying to have fun by building his own operating system. He recognized that his main job would be to write a kernel because, for the most part, the utilities were already available from the Free Software Foundation in a form that he could adapt for his own use when the time came.

...Mr. Right, Linus Torvalds

33614_02_009_036.indd 25

1/9/2008 12:23:38 PM

Chapter 2 In September 1991, Linus released the ﬁrst version of his kernel, which he called LINUX. Linux was distributed over the Internet, and within a short time, Linus began to release one new version after another. Programmers around the world began to join Linus: ﬁrst by the tens, then by the hundreds, and, eventually, by the hundreds of thousands. Interesting note: Linus chose to design Linux using a monolithic kernel. Minix, written by Andrew Tanenbaum, used a microkernel. Soon after Linux began to attract attention, Tanenbaum (a well-known professor) publicly excoriated Linus (a moreor-less unknown student) for that design decision. Even today, after the Linux kernel has become the basis for the most successful form of Unix in history, Tanenbaum still criticizes Linus for using a monolithic kernel. (Personally, I have read Tanenbaum’s arguments, and I think he is wrong.) Eventually, Linux became so popular as to run on every type of computer: not only PCs, but everything from handheld devices with their small built-in processors, to massively parallel supercomputing clusters, the fastest, most powerful systems in the world. Indeed, by 2005, virtually every niche in the world of computing would come to be occupied by machines running some type of Linux, and Linux would be the most popular operating system in the world. (Windows is more widespread, but Linux is more popular.) Why was Linux so successful? There are four reasons, and they all have to do with Linus himself. First, Linus Torvalds was an extremely skillful and knowledgeable programmer. He was a fast worker, and he loved to program. In other words, he was exactly the type of guy whose idea of fun would be writing his own operating system kernel. Second, Linus was an endless perfectionist. He was dedicated to his work and, when a problem fell into his lap, Linus would not rest until he had come up with a solution. In the early days of Linux, it was not unusual for Linus to make modiﬁcations so quickly that he would sometimes release a new kernel more than once a day! Third, Linus had a pleasing personality. People describe him as a low-key, unassuming fellow: a genuinely nice guy (see Figure 2-2). In person or online, Linus pretty much gets along with everyone because he is so easygoing. As he once observed in an interview, “Unlike Richard Stallman, I really don’t have a message.” Fourth – and most important – Linus had a genius for using the Internet to tap into a wellspring of programming talent: the thousands and thousands of people who would volunteer their efforts to modify and extend the Linux kernel. How large is Linux? Today, the Linux kernel in its entirely consists of well over 17,000 ﬁles and millions of lines of code. Every day, volunteer programmers submit hundreds of patches (changes or corrections). (It is interesting to compare this to the very ﬁrst kernel, version 0.01, which consisted of 83 ﬁles. In all, there were a bit fewer than 9,000 lines of code.) Having so many people on the job also gave Linus unusual opportunities because he had the brain power to handle any new problem that came along. When a new piece of hardware came out, for example, there was always somebody somewhere who was knowledgeable enough to write the code to make the new device work under Linux.

Harley Hahn’s Guide to Unix and Linux

Openmirrors.com 33614_02_009_036.indd 26

1/9/2008 12:23:38 PM

What Is Unix? What Is Linux?

FIGURE 2-2: Linus Torvalds

In 1991, Linus founded a project to create a new operating system kernel, which we now call the Linux kernel. The Linux kernel is the basis for the various Linux operating systems, making Linus’ undertaking one of the most important software projects in history. As you can see from the photo, Linus does not take himself too seriously.

Early on, Linus made a strategic decision to release the Linux kernel under the auspices of the GNU GPL. This decision proved to be crucial, as it encouraged programmers to volunteer their time. They knew that everything they worked on would be shared freely with the rest of the world. Moreover, the GPL ensured that any operating system that ultimately used the Linux kernel would itself fall under the auspices of the GPL and, hence, would be freely distributable. From the start, Linus released new versions of the kernel as often as he could. Normally, programmers like to hold onto new software, so they can test it thoroughly and ﬁx as many bugs as possible before the program is released to the public. Where Linus’ genius came in is in realizing that, since he was releasing software to a vast audience, he would have so many people testing and reading the new code, that no bug could hide for long. This idea is embodied in what is now called LINUS’S LAW: “Given enough eyeballs, all bugs are shallow.” Linus released new versions of the kernel far more often than anyone had ever done before, and bugs were identiﬁed and ﬁxed quickly, often within hours. The result was that

...Mr. Right, Linus Torvalds

33614_02_009_036.indd 27

1/9/2008 12:23:39 PM

Chapter 2 work on Linux progressed faster and better than any major software project in the history of the world. WHAT’S IN A NAME? Linux The proper way to pronounce Linux is to rhyme with “Bin’-ex”. From the beginning, Linus Torvalds used the name Linux informally, “Linux” being a contraction of “Linus’ Minix”. However, he actually planned to use the name Freax (“free Unix”) when he released the ﬁrst public version of the kernel. As it happened, another programmer, Ari Lemmke, had convinced Linus to upload the ﬁles to a server Lemmke ran, so they could be accessed easily via a system called “anonymous FTP”. Lemmke was not happy with the name Freax, so when he set up a directory to hold the ﬁles, he called the directory linux, and the name stuck.

LINUX DISTRIBUTIONS Strictly speaking, what Linus Torvalds and the Linux project created was a kernel, not a complete operating system (and this is still the case). When the kernel was ﬁrst released, you needed to be a Unix expert in order to use it because you had to ﬁnd all the necessary bits and pieces and put them together to form an operating system. Within 5 months of Linus’ ﬁrst kernel release, however, other people were offering free operating systems that were based on the Linux kernel. Such an offering is called a Linux DISTRIBUTION (sometimes shortened to DISTRO). As you can imagine, the ﬁrst few distributions were welcomed by the Linux community but, unfortunately, they were not well-maintained. In July 1993, however, a programmer named Patrick Volkerding announced a new distribution he called SLACKWARE. The name comes from the word “slack”, referring to the feeling of exhilaration and satisfaction that comes from achieving your personal goals. (For more information, look up the Church of the SubGenius.) Slackware was a success because, over the years, Volkerding has always managed to maintain it. Today, it is the oldest Linux distribution still being actively developed. In fact, Slackware is so popular that it has spawned an entire family of distributions and supporting groups of its own. Today’s modern Linux distributions offer a complete product: the kernel, the utilities, programming tools, and at least one GUI. As I write this, there are literally hundreds of such distributions. (If you want, once you become proﬁcient at Unix, you can make a Linux distribution of our own.) Before we leave this section, I want to make sure you understand that the word “Linux” has two meanings: First, “Linux” refers to a kernel, the product of the countless programmers who work on the Linux project. Second, “Linux” is the name of any operating system that is based upon the Linux kernel. This is the way most people use the word when they talk, and that is how we will use it in this book.

Harley Hahn’s Guide to Unix and Linux

Openmirrors.com 33614_02_009_036.indd 28

1/9/2008 12:23:39 PM

What Is Unix? What Is Linux? Most Linux distributions use the GNU UTILITIES from the Free Software Foundation. For this reason, the FSF insists that Linux-based operating systems should actually be called GNU/Linux. I don’t know anyone who does this, but you might remember it in case you are hanging out at a Unix bar and you happen to run into Richard Stallman.

BSD DISTRIBUTIONS As we discussed earlier, BSD was one of the two main streams of Unix during the 1980s. (The other was System V.) BSD was developed within the Computer Science Department at U.C. Berkeley and, since 1980, the development had been managed by the CSRG (Computer Science Research Group). By the end of the 1980s, BSD aﬁcionados had become disgruntled over AT&T’s commercialization of Unix. Because BSD contained certain proprietary elements from AT&T UNIX, and because the CSRG released source code with the operating system, every BSD user was forced to buy an expensive source code license from AT&T. For this reason, the CSRG set a goal for themselves: to completely rewrite all the AT&T-based parts of BSD. Doing so would free BSD from the clutches of the AT&T lawyers (and the AT&T marketing department) once and for all. In 1989, the CSRG offered the ﬁrst totally open source BSD distribution called Networking Release 1 (more formally, 4.3BSD NET/1). NET/1, however, consisted mostly of networking tools – which were independent of AT&T UNIX – and not the full BSD operating system. To create a truly independent version of BSD would require rewriting hundreds of AT&T utilities and programming tools, but it had to be done. Using the Internet, the CSRG solicited help from outside programmers and, within a year and a half, all the AT&T utilities had been replaced with brand new programs. A careful check of the code showed that everything in BSD was ﬁne except for six kernel ﬁles. Rewriting these six ﬁles would have been a big job so, in 1991, the CSRG released a new version of BSD that was almost UNIX-free. They called it NET/2. In 1992, a programmer named Bill Jolitz rewrote the last six problematic ﬁles and used them to create a version of BSD for PCs. He called this operating system 386/BSD and began to distribute it over the Internet. This was a huge achievement. After all these years, the holy grail of Berkeley – an operating system that was independent of AT&T UNIX – existed. Finally, BSD could be distributed freely as open source software to anyone in the world. This was important because BSD contained some of the best Unix software ever written. To this day, there are very few Unixes in the world that don’t contain BSD code. Indeed, most Linux distributions use a large number of BSD utilities. For this reason, the creation of 386/BSD is one of the milestones in the history of Unix. Within a short time, 386/BSD became popular and an ever-increasing number of people began to maintain it. Jolitz, however, had a full-time job, and he was not able to keep up with all the incoming patches and enhancements. For this reason, a group of volunteers was formed to take over the job. The ﬁrst thing the group did was to rename the operating system FreeBSD.

BSD Distributions

33614_02_009_036.indd 29

1/9/2008 12:23:39 PM

Chapter 2 Initially, FreeBSD ran only on PC hardware, which was ﬁne with most of the users. Some, however, wanted to run BSD on other types of machines, so a new group was formed, with the goal of porting FreeBSD to as many other types of computers as possible. The version offered by the new group was called NetBSD. In the mid-1990s, the NetBSD group spawned yet another group that wanted to focus on security and cryptography. Their operating system was called OpenBSD. As you can see, the world of BSD, which has only three main distributions (FreeBSD, NetBSD and OpenBSD), is a lot different from the world of Linux, where there are literally hundreds of different distributions. By now, you are probably wondering, if FreeBSD is completely open source and is distributed for free over the Internet, how is it that Linux was the version of Unix that grew so popular? There are two reasons. First, Linux was distributed under the auspices of the GNU GPL, which encourages sharing. Since the GPL prohibits anyone from using Linux to create and distribute a proprietary system, anything done with Linux belongs to the world at large. The BSD license is far less restrictive than the GPL. Under the BSD license, it is allowable to use parts of BSD to create a new product without having to share it. When this happens, the world at large does not get the beneﬁt of being able to use and modify the new product. For this reason, many programmers prefer to work with Linux. (On the other hand, because the BSD license is so ﬂexible, FreeBSD has been used widely in a large variety of machines and devices, as well as a great many Web servers around the world.) The second reason Linux was more successful than FreeBSD had to do with timing. In retrospect, it was clear that, by the end of the 1980s, programmers around the world had a strong need for a completely open source version of Unix. Since Linus Torvalds released the Linux kernel in 1991, and 386/BSD was not released until 1992, Linux got a head start. As a result, Linus was able to attract a large number of programmers, who were just waiting to work on the world’s ﬁrst free version of Unix. Now you know why this book is called Harley Hahn’s Guide to Unix and Linux, not Harley Hahn’s Guide to Unix and FreeBSD. (And now you also know why I described Linus Torvalds as the right guy with the right idea in the right place at the right time.)

WHAT TYPE OF UNIX SHOULD YOU USE? With all the different types of Unix, the questions arises, what type of Unix should you use? The answers (there are several) are actually rather simple. However, before I proceed, let me tell you that, for practical purposes, basic Unix is basic Unix: if you know how to use one type of Unix, you know how to use them all. In particular, as you read this book, it really doesn’t matter what type of Unix you are using (as long as you don’t read with your mouth full). I ﬁrst used Unix in 1977, and everything I learned then still works the same way. If you had used Unix in 1977 and you fell into a time warp, emerging in 2006 in front of a Linux or FreeBSD computer, how would you feel? You would have to learn how to use

Harley Hahn’s Guide to Unix and Linux

Openmirrors.com 33614_02_009_036.indd 30

1/9/2008 12:23:39 PM

What Is Unix? What Is Linux? some new tools, including a new, sophisticated GUI (graphical user interface). However, you wouldn’t feel like you had woken up in a brand new world. Unix is Unix. That said, here is my advice. In many cases, you don’t have a choice as to what type of Unix you must use. For example, you may work for a company that uses a commercial Unix such as AIX (from IBM) or Solaris (from Sun); or you may be taking a class where everyone has to use the same type of Linux; or you might be hacking with your grandmother, who insists on using FreeBSD because it reminds her of all the fun she had in Berkeley in the ‘70s. If this is the case, just go with the ﬂow and don’t worry. Unix is Unix. If you are choosing your own Unix, here is how to decide: 1. If you are using Unix because you want to learn how it works, how to customize your environment, how to program, or you just want to have fun, use Linux. To help you with your choices, Figure 2-3 shows the most important Linux distributions. All of these are readily available on the Internet. Most of the distributions can be downloaded at no cost. If you are not sure which one to use, use Ubuntu. 2. FreeBSD is very stable and reliable, and tends to work just out of the box. So, if you are working at a company, and you are looking for a system that needs as little attention as possible, use FreeBSD. Similarly, if you are working at home and want to run a server (such as a Web server), use FreeBSD. If your system is not supported by FreeBSD, use NetBSD. If you are very interested in security, use OpenBSD. For completeness, Figure 2-4 shows the most important BSD distributions. 3. If you want to run Unix under Microsoft Windows, you can use a free product called Cygwin. Once Cygwin is installed, all you have to do is open a Cygwin window and, within that window, everything looks like Linux. I love Cygwin. Even though I have several Unix computers, I use Windows a lot and, with Cygwin, I can access my favorite Unix programs whenever I want. NAME Debian Fedora Core Gentoo Knoppix Mandriva (used to be Mandrake) MEPIS Red Hat Slackware SuSE Ubuntu Xandros FIGURE 2-3: The most important Linux distributions

What Type of Unix Should You Use?

33614_02_009_036.indd 31

1/9/2008 12:23:40 PM

Chapter 2 NAME FreeBSD OpenBSD NetBSD FIGURE 2-4: The most important BSD distributions

4. Finally, if you want Unix, but you want it to look like Windows, get a Macintosh and run OS X. OS X is the Macintosh operating system. Although it has a Mac-like look and feel, it is actually Unix under the hood. Speciﬁcally, OS X uses a microkernel based on Mach, the FreeBSD utilities, and a proprietary GUI named Aqua. To access Unix directly under OS X, just open a Terminal window. (You will ﬁnd Terminal in your Applications/Utilities folder.) WHAT’S IN A NAME? OS X OS X (the Macintosh operating system) is pronounced “O-S-ten”. The name is a pun. The previous Mac operating system — which was not based on Unix — was called OS 9. Thus, the “X” stands for the Roman numeral 10, but it also makes you think of Unix.

HOW DO YOU GET LINUX OR FREEBSD? Most of the time, Unix is installed from one or more CDs. To install Linux or FreeBSD, all you need to do is ﬁnd a free downloading site (which is easy), download the ﬁles, burn the CDs, and then use the CDs to install the operating system. Better yet, if you ask around, you may ﬁnd someone who already has the CDs. Since both Linux and FreeBSD are free software, there is no problem borrowing CDs or even making your own copies to share with other people. When you install Linux or FreeBSD in this way, the operating system resides on your hard disk. If your Unix system is the only operating system on your computer, the situation is simple. Whenever you turn on the computer, Unix will boot automatically. Alternatively, you can install more than one operating system on your computer. You might, for example, want both Windows and Unix, which allows you to switch back and forth as you wish. However, you can only run one operating system at a time and, to switch from one to the other, you need to reboot. Such a setup is called a DUAL BOOT system. When you use a dual boot system, you make use of a special program called a BOOT LOADER. The boot loader takes control every time you start or restart your computer. Its job is to show you a list of available operating systems, so you can choose the one you want. The boot manager then transfers control to the appropriate kernel, and the kernel starts the rest of the operating system. (As a general rule, if your PC takes a minute to

Harley Hahn’s Guide to Unix and Linux

Openmirrors.com 33614_02_009_036.indd 32

1/9/2008 12:23:40 PM

What Is Unix? What Is Linux? boot, about 15 seconds is used by the boot loader, and about 45 seconds is used to load the operating system.) The most common Linux boot loaders are GRUB (Grand Uniﬁed Bootloader) and LILO (Linux Loader). LILO was developed a long time ago as part of the Linux distribution, and it has been well maintained over the years. GRUB is newer and was developed as a part of GNU. Both boot loaders are ﬁne but if you have a choice, use GRUB because it is more powerful and more ﬂexible. To set up your machine as a dual boot system, you must divide your hard disk into parts called PARTITIONS. To do so, you use what is called a PARTITION MANAGER. Each operating system must be installed using its own partition (or partitions, if more than one is required). The most common type of dual boot system is one in which Windows coexists with some type of Unix. However, there is no reason why you can’t partition your disk to use more than two operating systems on the same computer. For example, you might have Windows, Fedora Core Linux and FreeBSD. The boot loader will then offer you three choices, rather than two. In such cases, we would say that you have a MULTI-BOOT system. A second way to run Unix is not to install it onto your hard disk. Instead, you can boot from a special CD called a LIVE CD. A live CD is a CD-ROM that has been made bootable, and that contains everything necessary to run a full operating system: the kernel, the utilities, and so on. When you boot from a live CD, you bypass the hard disk. This allows you to use a second (or third or fourth) operating system whenever you want, without having to install anything on your disk. For example, if you are a Windows user, you can experiment with Linux or FreeBSD without having to repartition and install a whole new operating system. Similarly, if you are a Linux user, you can try out, say, FreeBSD, without having to make changes to your system. Some people who don’t know what type of Linux they want to use permanently will use live CDs to try out different distributions. Live CDs are also useful when you are using a PC that does not belong to you. For example, if you work for a company that forces you to use Windows, you can use a live Linux CD to run Unix when no one is watching. Or, if you are ﬁxing a friend’s computer, you can boot your own operating system with your favorite tools without changing your friend’s computer in any way. To create a live CD, just pick the one you want, download the ﬁles, and burn the CD. You might even make several live CDs to experiment with different systems. There are many different live CDs available on the Internet for free. To help you, Figure 2-5 lists the most important ones. If you are not sure which one to use, use Knoppix (pronounced “Nop’-pix”). HINT If your computer boots from the hard disk even when you have a CD in the drive, check the BIOS settings. You may have to modify a setting to tell the computer to look for a bootable CD before looking for a bootable hard disk.

How Do You Get Linux or FreeBSD?

33614_02_009_036.indd 33

1/9/2008 12:23:40 PM

Chapter 2 How do you choose between a full hard disk installation and a live CD? A full installation is a commitment as it requires you to make lasting changes to your hard disk. (Of course, you can undo these changes if you want.) The advantage of a full installation is that the operating system is on your disk permanently. Not only is this convenient, but you can customize your system and store ﬁles permanently. With a live CD, there is less of a commitment. However, unless you set aside a special disk partition for the live CD to use for its data, you can’t make permanent modiﬁcations or save Unix data ﬁles. Moreover, running Unix from a live CD will decrease the performance of your computer a bit. Not only is it slower to boot off a CD than a hard disk, a live CD system will have to create a RAM disk to hold the ﬁles that would normally be on your hard disk. This will use up some of your memory. (A RAM disk is a part of memory that is used to simulate a real disk.) A nice compromise is to install Unix to some type of removable storage device, such as a USB key drive (sometimes called a USB ﬂash drive). This is a very small, portable device that plugs into a USB port and acts like a tiny hard disk. Once you have installed Unix to a key drive, all you have to do is insert it into a USB port, turn on (or restart) the computer, and Unix will boot automatically. (You may need to change your computer’s BIOS settings to tell it to boot from the key drive before the hard disk.) By installing to a key drive, you get most of the beneﬁts of a permanent Unix system without having to modify your hard disk. For example, when you boot from a key drive, you can modify your system, you can save ﬁles, and so on. Unlike a CD-ROM, which is only readable, a key drive is read/writeable. And whenever you want to use the operating system on your hard disk, all you have to do is remove the key drive and reboot. Still, once you decide to use an operating system permanently, you will ﬁnd it faster and more practical to install it on your hard disk. Flexibility is nice but, in the long run, as with most areas of life, commitment works better. NAME Damn Small Linux FreeBSD Kanoﬁx Knoppix MEPIS PCLinuxOS SLAX SuSE Ubuntu FIGURE 2-5: The most important Linux live CDs

Harley Hahn’s Guide to Unix and Linux

Openmirrors.com 33614_02_009_036.indd 34

1/9/2008 12:23:40 PM

What Is Unix? What Is Linux? HINT Downloading and installing any type of Unix will take longer than you anticipate, so don’t do it when you are in a hurry. For example, if it is your honeymoon night, and you have a half hour to kill while your wife changes her clothes and freshens up, starting a brand new Linux installation would be a bad idea.

WHAT IS UNIX? WHAT IS LINUX? Before we leave this chapter, I’d like to summarize our discussion by, once again, answering the questions: What is Unix? and What is Linux? Unix is a multiuser, multitasking operating system that consists of a Unix-like kernel, Unix-like utilities and a Unix-like shell. Linux is the name given to any Unix that uses the Linux kernel. (As we discussed, there is no good deﬁnition of the term “Unix-like”. You just have to know it when you see it.) However, there is another way to look at Unix. Over the years, a great many very, very smart people have worked on Unix to build themselves the very best tools they could. As Unix users, you and I get to reap the beneﬁts of that work. To me, Unix is an an abstract idea: an actual applied philosophy that dictates a particular approach to problem solving. In using Unix, you will learn to approach and solve problems the Unix way. For example, you will learn how to combine simple tools, like building blocks, into elegant structures to solve complex problems; you will learn how to be self-reliant, teaching yourself most of what you need to know; and you will learn how to organize your thoughts and actions in a logical way that makes the best use of your time and effort. As you do so, you will be following in the footsteps of giants, and your mind will be changed for the better. This may be hard to understand right now, but don’t worry. Just stick with Unix long enough, and you will see for yourself. For this reason, let me give you another deﬁnition of Unix. This is the best deﬁnition of Unix I know, and it is the one I would like you to remember as you read this book. I’ll give it to you in the form of a hint: HINT Unix is a set of tools for smart people.

What Is Unix? What Is Linux?

33614_02_009_036.indd 35

1/9/2008 12:23:41 PM

Chapter 2

C H A P T E R

E X E R C I S E S

REVIEW QUESTIONS 1. What is an operating system? 2. What is the kernel? Name four tasks performed by the kernel. 3. What is open source software? Why is it important? 4. What is Linux? When was the ﬁrst version of Linux released? By whom?

FOR FURTHER THOUGHT 1. Would the Internet have been possible without Unix? Would Linux have been possible without the Internet? 2. When Richard Stallman founded the Free Software Foundation in 1985, he was able to tap into the energy of many programmers around the world who wanted to work on a free operating system. Later, in the early 1990s, Linux Torvalds was also able to ﬁnd programmers to help him, in this case, to develop the Linux kernel. Why are so many programmers willing to work a long time on a difﬁcult project and then give away the fruits of their labor for free? Is this more common with young programmers than old programmers? If so, why? 3. Traditionally, lawyers are expected to offer a certain amount of work free as a public service, such work being described as pro bono publico (Latin: “for the public good”). Should the programming profession have a similar expectation of its members? If so, why? If not, why not?

Harley Hahn’s Guide to Unix and Linux

Openmirrors.com 33614_02_009_036.indd 36

1/9/2008 12:23:41 PM

The Unix Connection

Being able to connect to different types of computers has always been an integral part of Unix. Indeed, the fact that Unix has this capability is one of the main reasons there are so many computer networks in the world. (The Internet, for example, has always depended on Unix connections.) In this chapter, we’ll discuss the basic concepts that make it possible to connect to other computers: on the Internet and on local networks. Along the way, you will learn how these concepts underlie the most fundamental Unix connection of all: the one between you and your computer.

HUMANS, MACHINES AND ALIENS I’d like to start by talking about an idea that is rarely discussed explicitly. And yet, it is such an important idea that, if you don’t understand it, a lot of Unix is going to seem mysterious and confusing. The idea concerns human beings and how we use machines. Think about the most common machines in your life: telephones, cars, TVs, radios, and so on. Because you and the machine are separate entities, there must be a way for you to interact with it when you use it. We call this facility an INTERFACE. Consider, for example, a mobile phone. The interface consists of a set of buttons, a speaker or ear plug, a small video screen, and a microphone. Consider a car: the interface consists of a key, the steering wheel, the accelerator pedal, the brake pedal, a variety of dials and displays, and a gaggle of levers, knobs and buttons. The point is every machine that is used by a human being can be thought of as having two components: the interface and everything else. For example, with a desktop computer, the interface consists of the monitor, the keyboard, the mouse, speakers and (possibly) a microphone. “Everything else” consists of the contents of the box: the hard disk, the CD drive, the processors, the memory, the video card, the network adapter, and so on. In Unix terminology, we call the interface the TERMINAL (I’ll explain why later), and we call everything else the HOST. Understanding these concepts is crucial, so I am going to talk about them in detail.

Humans, Machines and Aliens

33614_03_037_054.indd 37

1/9/2008 12:24:35 PM

Chapter 3 Since the terminal provides the interface, it has two main jobs: to accept input and to generate output. With a desktop computer, the input facility consists of the keyboard, mouse and microphone. The output facility consists of the monitor and the speakers. To capture these ideas, we can describe any computer system by the following two simple equations: COMPUTER = TERMINAL + HOST TERMINAL = INPUT FACILITY + OUTPUT FACILITY Has it ever occurred to you that, as a human being, you also consist of a terminal and a host? In other words, these same two equations describe you and me and everyone else. Your “terminal” (that is, your interface to the rest of the world) provides your input facility and your output facility. Your input facility consists of your sense organs (eyes, ears, mouth, nose, and skin). Your output facility consists of the parts of your body that can make sounds (your mouth) and can create change in your environment (your hands and arms, your legs, and the muscles of facial expression). What is your “host”? Everything else: your brain, your organs, your muscles and bones, your blood, your hormones, and so on. It might seem artiﬁcial and a bit ludicrous to separate your “host” from your “terminal” because you are a single, self-contained unit. But think about a laptop computer. Even though all the components are built-in, we can still talk about the terminal (the screen, the keyboard, the touch pad, the speakers, and the microphone) and the host (everything else). Imagine two aliens from another planet watching you use a laptop computer. One alien turns to the other and says, “Look, there is a human being who is using his interface to interact with the interface of a computer.” To the aliens, it doesn’t matter that your interface is built-in because the laptop’s interface is also built-in. The aliens, being from another planet, see what you and I don’t normally see: as you use a computer, your interface communicates with the computer’s interface. Indeed, this is the only way in which you can use a computer (or any other machine, for that matter). However, what if the aliens happened to come from a Unix planet? After the ﬁrst alien made his comment, the second alien would respond, “I see what you mean. Isn’t it interesting how the human’s terminal interacts with the computer’s terminal?”

IN THE OLDEN DAYS, COMPUTERS WERE EXPENSIVE In Chapter 1, I mentioned that the very ﬁrst version of Unix was developed in 1969 by Ken Thompson, a researcher at Bell Labs, New Jersey. (At the time, Bell Labs was part of AT&T.) Thompson had been working on a large, complex project called Multics, which was centered at MIT. When Bell Labs decided to end their support of Multics, Thompson returned full-time to New Jersey, where he and several others were determined to create their own, small operating system. In particular, Thompson had a game called Space Travel that he had developed on Multics, and he wanted to be able to run the program on a system of his own.

Harley Hahn’s Guide to Unix and Linux

Openmirrors.com 33614_03_037_054.indd 38

1/9/2008 12:24:36 PM

The Unix Connection At the time, there were no personal computers. Most computers were large, expensive, temperamental machines that required their own staff of programmers and administrators. (We now call such machines MAINFRAME COMPUTERS.) Mainframe computers required their own special rooms, referred to whimsically as “glass houses”. There were three reasons for glass houses. First, the machines were somewhat fragile, and they needed to be in an environment where the temperature and humidity could be controlled. Second, computers were very expensive, often costing millions of dollars. Such machines were far too valuable to allow just anyone to wander in and out of the computer room. A glass house could be kept locked and closed to everyone but the computer operators. Finally, computers were not only complex; they were relatively rare. Putting such important machines in glass houses allowed companies (or universities) to show off their computers, especially to visitors. I have a memory as a young college student: standing in front of a huge computer room at the University of Waterloo, looking through the glass with awe, intimidated by the large number of mysterious boxes that comprised three separate IBM mainframe computers. Multics ran on a similar computer, a GE-645. The GE-645, like most mainframes, was expensive to buy, expensive to lease, and expensive to run. In those days, computer users were given budgets based on real money, and every time someone ran a program, he was charged for processing time and disk storage. For example, each time Thompson ran Space Travel on the GE-645, it cost about $75 just for the processing time ($445 in 2008 money). Once Bell Labs moved Thompson and the others back to New Jersey, the researchers knew there was no way they would be able to get their hands on another large computer. Instead, they began looking around for something smaller and more accessible. In those days, most computers cost well over $100,000, and coming up with a machine for personal research was not easy. However, in 1969, Thompson was looking around Bell Labs and he found an unused PDP-7. (The PDP-7, made by DEC, the Digital Equipment Corporation, was a so-called MINICOMPUTER. It was smaller, cheaper and much more accessible than a mainframe. In 1965 dollars, the PDP-7 cost about $72,000; the GE-45 mainframe cost about $10 million. The name PDP was an abbreviation for “Programmed Data Processor”.) This particular PDP-7 had been ordered for a project that had ﬂoundered, so Thompson was able to commandeer it. He wrote a lot of software and was able to get Space Travel running. However, the PDP-7 was woefully inadequate, and Thompson and several others lobbied to get another computer. Eventually, they were able to acquire a newer PDP-11, which was delivered in the summer of 1970. The main advantage of the PDP-11 was that its base cost was only(!) $10,800 ($64,300 in 2008 money). Thompson and a few others began to work with the PDP-11 and, within months, they had ported Unix to the new computer. (You can see Thompson and Dennis Ritchie, his Unix partner, hard at work in Figure 3-1.) Why am I telling you all of this? Because I want you to appreciate that, in the late 1960s and early 1970s, computers cost a lot and were difﬁcult to use. (The PDP-11 was expensive and inadequate. The PDP-7 was very expensive and inadequate. And the GE-645

In the Olden Days, Computers Were Expensive

33614_03_037_054.indd 39

1/9/2008 12:24:36 PM

Chapter 3 was very, very expensive and inadequate.) As a result, there was an enormous need to make computing, not only easier, but cheaper. One of the biggest bottlenecks was that, using the current software, the PDP-11 could only run one program at a time. This meant, of course, that only one person could use the machine at a time. The solution was to change Unix so that it would allow more than one program to run at a time. This was not easy, but by 1973 the goal had been achieved and Unix became a full-ﬂedged multitasking system. (The old name for multitasking is MULTIPROGRAMMING.) From there, it was but a step to enhance Unix to support more than one user at a time, turning it into a true multiuser system. (The old name is a TIME-SHARING SYSTEM.) Indeed, in 1974, when Thompson and Ritchie published the ﬁrst paper that described Unix (see Chapter 2), they called it “The UNIX Time-Sharing System”. However, in order to make such a change, the Unix developers had to come to terms with a very important concept, the one you and I discussed earlier in the chapter: human beings could only use a machine if the machine had a suitable interface. Moreover, if

FIGURE 3-1: Ken Thompson, Dennis Ritchie, and the PDP-11

Ken Thompson (sitting) and Dennis Ritchie (standing) and the Bell Labs’ PDP-11 minicomputer. Thompson and Ritchie are using two Teletype 33-ASR terminals to port Unix to the PDP-11.

Harley Hahn’s Guide to Unix and Linux

Openmirrors.com 33614_03_037_054.indd 40

1/9/2008 12:24:36 PM

The Unix Connection more than one person were to use a computer at the same time, each person would need a separate interface. This only makes sense. For example, if two people wanted to type commands at the same time, the computer would have to be connected to two different keyboards. However, in the early days of Unix, computer equipment was expensive and hard to come by. Where could Thompson and Ritchie come up with the equipment they needed to run a true multiuser system? The answer to this question proved to be crucial, as it affected the basic design of Unix, not only for the very early Unix systems, but for every Unix system that ever existed (including System V, BSD, Linux, FreeBSD and OS X).

HOST AND TERMINALS It was the early 1970s, and Ken Thompson and Dennis Ritchie had a problem. They wanted to turn Unix into a true multitasking, multiuser operating system. However, this meant that each user would need his own interface. Today, high quality color video monitors, keyboards and mice are cheap. In those days, however, everything was expensive. There was no such thing as a separate keyboard; there were no mice; and the cost of a separate computer monitor for each user was prohibitive. As a solution, Thompson and Ritchie decided to use a machine that was inexpensive and available, even though it had been designed for a completely different purpose. This machine was the Teletype ASR33 (ASR stood for Automatic Send-Receive). Teletype machines were originally developed to send and receive messages over telegraph lines. As such, the machines were called teletypewriters (“Teletype” was a brand name). The original experimental teletypewriters were invented in the early 1900s. Throughout the ﬁrst half of the twentieth century, teletypewriter technology became more and more sophisticated, to the point where Teletype machines were used around the world. AT&T (Bell Lab’s parent company) was heavily involved in such services. In 1930, AT&T bought the Teletype company and, indeed, the name AT&T stands for American Telephone and Telegraph Company. Thus, it came to pass that, in the early 1970s, Thompson and Ritchie were able to use Teletype machines as the interfaces to their new PDP-11 Unix system. You can see the actual machines in Figure 3-1 above, and a close-up view in Figures 3-2 and 3-3. As an interface, all the Teletype had was a keyboard for input and a wide roll of paper for printed output. To store programs and data, there was a paper tape punch that could make holes in a long, narrow band of paper, and a paper tape reader that could read the holes and convert them back to data. Compared to today’s equipment, the Teletype was primitive. Except for the power supply, everything was mechanical, not electronic. There was no video screen, no mouse and no sound. Moreover, the keyboard was uncomfortable and difﬁcult to use: you had to depress a key about half an inch to generate a character. (Imagine what typing was like.) What made the Teletype so valuable was that it was economical and it was available.

Host and Terminals

33614_03_037_054.indd 41

1/9/2008 12:24:36 PM

Chapter 3 Here’s where it all comes together. Thompson and Ritchie wanted to create a true multiuser system. Computing equipment was expensive. All they had were some Teletypes for the interfaces, and a single PDP-11 minicomputer to do the processing. Like the aliens I mentioned above, Thompson and Ritchie realized that they could, conceptually, separate the interface from the rest of the system, and this is the way they designed Unix. There would be a single processing element, which they called the host, along with multiple interface units, which they called terminals. At ﬁrst, the host was the PDP-11 and the terminals were Teletypes. However, that was merely for convenience. In principle, Unix could be made to work with any host and any type of terminal. (It would take work, but not too much work.) This design decision proved to be prescient. From the beginning, the connection that Unix forged between a user and the computer was dependent upon a speciﬁc design principle, not upon speciﬁc hardware. This meant that, year after year, no matter what new equipment happened to come along, the basic way in which Unix was organized would never have to change. As terminals became more sophisticated, an old one could be thrown away and a new one swapped in to take its place. As computers became more complex and more powerful, Unix could be ported to a new host and everything would work as expected.

FIGURE 3-2: Teletype 33-ASR

A Teletype 33-ASR, similar to the ones used by Ken Thompson and Dennis Ritchie with the very early Unix systems.

Harley Hahn’s Guide to Unix and Linux

Openmirrors.com 33614_03_037_054.indd 42

1/9/2008 12:24:37 PM

The Unix Connection

FIGURE 3-3: Close-up of a Teletype 33-ASR

A close-up view of a Teletype 33-ASR. Notice the tall, cylindrical keys. A key had to be depressed about half an inch to generate a character. To the left, you can see the paper tape punch/reader. The tape is 1-inch wide.

Compare this to Microsoft Windows. Because Windows was created speciﬁcally for single-user PCs, Microsoft never completely separated the terminal from the host. As a result, Windows is inelegant, inﬂexible, and is wedded permanently to the PC architecture. Unix is elegant, ﬂexible, and can be made to work with any computer architecture. After all these years, the Unix terminal/host paradigm still works marvelously.

TERMINAL ROOMS AND TERMINAL SERVERS As I explained, Unix was designed as a multiuser system. This meant that more than one person could use a computer at the same time, as long as (1) each person had his own terminal, and (2) that terminal was connected to a host. So, imagine a room full of terminals. They are not computers. In fact, they are not much more than a keyboard, a monitor, and some basic circuitry. At the back of each terminal is a cable that snakes down into a hole in the ﬂoor and, from there, makes its way to an unseen host computer.

Terminal Rooms and Terminal Servers

33614_03_037_054.indd 43

1/9/2008 12:24:37 PM

Chapter 3 The room is occupied by a number of people. Some of them are sitting in front of a terminal, typing away or looking at their screens and thinking. These people are using the same host computer at the same time. Other people are patiently waiting for their turn. It happens to be a busy time, and there are not enough terminals for everyone. The picture I just described is what it was like to use Unix in the late 1970s. At the time, computers – even minicomputers – were still expensive, and there weren’t enough to go around. Terminals, however, were relatively inexpensive. COMPUTER ROOM

HOST COMPUTER

CONSOLE

TERMINAL ROOM

FIGURE 3-4: Terminals in a terminal room

In the late 1970s, when computers were still expensive and terminals weren’t, it was common to see terminal rooms, in which multiple terminals were connected to the same host. COMPUTER ROOM

COMPUTER ROOM

TERMINAL SERVER

TERMINAL ROOM

FIGURE 3-5: Terminals connected to a terminal server

In the late 1970s, some organizations could afford to have more than one computer available for their users. It was common to have all the terminals in the organization connect to a terminal server which would act as a switch, allowing a user to access any of the host computers from any terminal.

Harley Hahn’s Guide to Unix and Linux

Openmirrors.com 33614_03_037_054.indd 44

1/9/2008 12:24:37 PM

The Unix Connection Since Unix was designed to support multiple users, it was common to see TERMINAL ROOMS ﬁlled with terminals, each of which was connected to a host. When you wanted to use the computer, you would go to the terminal room, and wait for a free terminal. Once you found one, you would log in by typing your user name and password. (We’ll talk about this process in detail in Chapter 4.) This setup is conceptualized in Figure 3-4. Some organizations, such as university departments or companies, could afford more than one host computer. In this case, it only made sense to allow people to use any host from any terminal. To do so required a TERMINAL SERVER, a device that acted as a switch, connecting any terminal to any host. To use a terminal server, you entered a command to tell it which computer you wanted to use. The terminal server would then connect you to that host. You would then enter your user name and password, and log in in the regular manner. You can see such a system in Figure 3-5. In this drawing, I have shown only six terminals and three hosts. This is a bit unrealistic. In large organizations, it was common to have many tens of terminals, all over the building, connected to terminal servers that allowed access to a number of different hosts.

THE CONSOLE Out of all the terminals that might be connected to a host, there is one terminal that is special. It is the terminal that is considered to be part of the computer itself, and it is used to administer the system. This special terminal is called the CONSOLE. To give you an example, I’d like to return, for a moment, to the late 1970s. We are being given a tour of a university department, and we see a locked room. Inside the room there is a PDP-11 minicomputer with a terminal next to it. The terminal is connected directly to the computer. This is the console, a special terminal that is used only by the system administrator. (You can see the console in Figure 3-4 above.) Down the hall, there is a terminal room with other terminals. These terminals are for the users, who access the computer remotely. Now, let’s jump forward in time to the present day. You are using a laptop computer on which you have installed Linux. Although Linux can support multiple users at the same time, you are the only person who ever uses the computer. Do you have a console? Yes, you do. Because you are using Unix, you must have a terminal. In this case, your terminal is built-in: the keyboard, the touch pad, the screen, and the speakers. That is also your console. Typically, the console is used by the system administrator to manage the system. In the ﬁrst example, when the system administrator wanted to use the console of the PDP-11, he would need to go into the computer room and sit down in front of the actual console. With your laptop, you are the administrator, and there is only one (built-in) terminal. Thus, any time you use your Linux laptop, whether or not you are actually managing the system or just doing work, you are using the console. Why do you need to know about consoles and regular terminals? There are three reasons. First, Unix systems have always distinguished between consoles and regular

The Console

33614_03_037_054.indd 45

1/9/2008 12:24:37 PM

Chapter 3 terminals and, when you are learning about Unix and come across a reference to the “console”, I want you to know what it means. Second, if you are a system administrator (which is the case when you have your own Unix system), there are certain things that can only be done at the console, not from a remote terminal. (Here is an example. If your system has a problem that arises during the boot process, you can only ﬁx the problem from the console. This is because, until the system boots, you can’t access it via a remote terminal.) Finally, from time to time, a Unix system may need to display a very serious error message. Such messages are displayed on the console to ensure that the system administrator will see them. Having said so much about consoles and why they are important, I’d like to pose the question: Are there computers that don’t have consoles? You betcha. There are lots of them. However, before I explain how a system can work without a console, I need to take a moment to talk about Unix and networks.

THE UNIX CONNECTION As we have discussed, Unix is designed so that the terminal (that is, the interface) is separate from the host (the processing unit). This means that more than one person can use the same Unix system at the same time, as long as each person has his or her own terminal. Once you understand this idea, it makes sense to ask, how far apart can a terminal be from the host? The answer is as far as you want, as long as there is a connection between the terminal and the host. When you run Unix on a laptop computer, the terminal and the host are connected directly. When you run Unix on a desktop computer, the terminal is connected to the host by cables. (Remember, the terminal consists of the keyboard, monitor, mouse, speakers, and microphone.) What about a larger distance? Is it possible to connect a terminal to a host over a local area network (LAN)? Yes, it is. For example, let’s say you use a PC that is connected to a LAN on which there are many computers, three of which are Unix hosts. It is possible to use your PC as a terminal to access any one of the three Unix hosts. (Of course, before you can use any Unix host, you must have authorization to use that computer.) When you use your computer to connect to a remote Unix host, you run a program that uses your hardware to EMULATE (act like) a terminal. This program then connects over the network to the remote host. You can do this from any type of computer system: a Windows computer, a Macintosh, or another Unix computer. Typically, the terminal program runs in its own window, and you can have as many separate windows as you want. For example, you might have three windows open at the same time, each running a terminal program. Each “terminal” can be connected to a different Unix host over the network. In this case, you would be working on four computers simultaneously: your own computer, and the three Unix hosts.

Harley Hahn’s Guide to Unix and Linux

Openmirrors.com 33614_03_037_054.indd 46

1/9/2008 12:24:38 PM

The Unix Connection You can see this illustrated in Figure 3-6. In Figure 3-6, the network connections between the PC and the three Unix hosts are via cables, as in a traditional network. However, any type of network connection will do. In particular, you can use a wireless connection. Here is an example. Let’s say you have three geeky friends, Manny, Moe and Jack. Each of you has a laptop computer that runs Unix. You use Debian Linux; Manny uses Fedora Core Linux; Moe uses Gentoo Linux; and Jack uses FreeBSD. (Jack always was a bit odd.) You get together for a Unix party (that is, computers, caffeinated drinks, and junk food), and you decide that each person should have access to the other three computers. First, each of you creates user accounts on your own computer for the other three people. (I won’t go into the details here, but it’s not hard.) Then, you all use either the iwconfig command or the wiconfig command to conﬁgure your computers in such a way as to allow them to connect, wirelessly, into a small network. Once the network is established, you each open three terminal windows on your own computer. Within each window, you connect to one of the three other computers. You now have four people in the same room, each running a different type of Unix on his laptop computer, each of which also has access to the other three computers. Could anything be cooler? So, we have talked about a terminal directly connected to a host (a laptop computer), a terminal connected to a host by cables (a desktop computer), a terminal connected to

PROGRAM RUNNING ON REMOTE COMPUTER 1

PROGRAM RUNNING ON REMOTE COMPUTER 2

PROGRAM RUNNING ON LOCAL COMPUTER

PROGRAM RUNNING ON REMOTE COMPUTER 3

FIGURE 3-6: Unix/Linux computer on a local area network

A computer on a local area network, running four terminal programs, each in its own window. Three of the “terminals” are connected, via the network, to different remote hosts. The fourth “terminal” is running a program on the local computer.

The Unix Connection

33614_03_037_054.indd 47

1/9/2008 12:24:38 PM

Chapter 3 a host over a regular LAN, and a terminal connected to a host over a wireless LAN. Can we go further? Yes. By using the Internet to connect a terminal to a host, we can have a connection that can reach anywhere in the world. Indeed, I regularly use the Net to connect to remote Unix hosts from my PC. To do so, I open a terminal window and connect to the remote host. And, as long as I have a good connection, it feels as if I were working on a computer down the hall.

HOSTS WITHOUT CONSOLES I mentioned earlier that there are many Unix host computers in the world that are not connected to terminals. This is because, if a computer can run on its own, without direct input from a human being, there is no need for a terminal. Such computers are referred to as HEADLESS SYSTEMS. On the Internet, there are many Unix hosts that run just ﬁne without terminals. For instance, there are millions of headless systems acting as Web servers and mail servers, silently doing their job without any human intervention. Many of these servers are running Unix, and most of them are not connected to a terminal. (A WEB SERVER responds to requests for Web pages and sends out the appropriate data. A MAIL SERVER sends and receives email.) If the need arises to directly control such a host computer – say, to conﬁgure the machine or to solve a problem – the system administrator will simply connect to the host over a network. When the system administrator is done, he simply disconnects from the host and leaves it to run on its own. On the Internet, there are two very common types of hosts that run automatically without terminals. First, there are the servers, such as the Web servers and mail servers I mentioned above. We’ll talk about them in a moment. Second, there are the ROUTERS: special-purpose computers that relay data from one network to another. On the Internet, routers provide the connectivity that actually creates the network itself. For example, when you send an email message, the data will pass through a series of routers on its way to the destination computer. This will happen automatically, without any human intervention whatsoever. There are millions of routers, all around the world, working automatically, 24 hours a day, and many of them are Unix hosts without a console. What if there is a problem? In such cases, it is the work of a moment for a system administrator to open a terminal window on his PC, connect to the router, ﬁx the problem, and then disconnect. Some large companies with many Unix servers use a different approach. They will connect the console of every host computer to a special terminal server. That way, when there is a problem, a system administrator can use the terminal server to log in directly to the computer that has the problem. I have a friend who once worked at a company where 95 different Unix hosts were connected to a set of terminal servers that were used only for system administration.

Harley Hahn’s Guide to Unix and Linux

Openmirrors.com 33614_03_037_054.indd 48

1/9/2008 12:24:38 PM

The Unix Connection THE CLIENT/SERVER RELATIONSHIP In computer terminology, a program that offers a service of some type is called a SERVER; a program that uses a service is called a CLIENT. These terms, of course, are taken from the business world. If you go to see a lawyer or an accountant, you are the client and they serve you. The client/server relationship is a fundamental concept, used in both networks and operating systems. Not only are clients and servers used extensively on the Internet, they are an important part of Unix (and Microsoft Windows, for that matter). Consider the following example. As I am sure you know, to access the Web you use a program called a BROWSER. (The two most important browsers are Internet Explorer and Firefox. Internet Explorer is used more widely, but Firefox is much better.) Let’s say you decide to take a look at my Web site (http://www.harley.com). To start, you type the address into the address bar of your browser and press the Enter key. Your browser then sends a message to my Web server. (I’m skipping a few details here.) Upon receiving the request, the Web server responds by sending data back to your browser. The browser then displays the data for you in the form of a Web page (in this case, my home page). What I have just described is a client/server relationship. A client (your browser) contacts a server on your behalf. The server sends back data. The client then processes the data. Let’s take another example. There are two ways to use email. You can use a Web-based service (such as Gmail), or you can run a program on your computer that sends and receives mail on your behalf. I’d like to talk about the second type of service. When you run your own email program, it uses different systems for sending and receiving. To send mail, it uses SMTP (Simple Mail Transport Protocol). To receive mail, it uses either POP (Post Ofﬁce Protocol) or IMAP (Internet Message Access Protocol). Let’s say you have just ﬁnished composing an email message, and your email program is ready to send it. To do so, it temporarily becomes an SMTP client and connects to an SMTP server. Your SMTP client then calls upon the SMTP server to accept the message and send it on its way. Similarly, when you check for incoming mail, your email program temporarily becomes a POP (or IMAP) client, and connects to your POP (or IMAP) server. It then asks the server if there is any incoming mail. If so, the server sends the mail to your client, which processes the messages appropriately. My guess is that, even if you have sent and received email for years, you may have never heard of SMTP, POP and IMAP clients and servers. Similarly, you can use the Web for years without knowing that your browser is actually a Web client. This is because client/server systems generally work so well that the clients and servers are able to do their jobs without bothering the user (you) with the details. Once you get to know Unix and the Internet, you will ﬁnd that there are clients and servers all over the place. Let me leave you with three such examples. First, to connect to a remote host, you use a client/server system called SSH. (The name stands for “secure shell”. ) To use SSH, you run an SSH client on your terminal, and

The Client/Server Relationship

33614_03_037_054.indd 49

1/9/2008 12:24:38 PM

Chapter 3 your SSH client connects to an SSH server running on the host. Second, to upload and download ﬁles to a remote computer, you use a system called FTP (File Transfer Protocol). To use FTP, you run an FTP client on your computer. Your FTP client connects to an FTP server. The client and the server then work together to transfer the data according to your wishes. As you become an experienced Unix or Linux user, you will ﬁnd yourself working with both these systems. As you do, you will come to appreciate the beauty and power of the client/server model. Finally, you may have heard of Usenet, the worldwide system of discussion groups. (If you haven’t, go to http://www.harley.com/usenet.) To access Usenet, you run a Usenet client, called a newsreader. Your newsreader connects to a Usenet server called a news server. (I’ll explain the names in a minute.) All of these examples are different, but one thing is the same. In each case, the client requests a service from the server. Strictly speaking, clients and servers are programs, not machines. However, informally, the term “server” sometimes refers to the computer on which the server program is running. For example, suppose you are taking a tour of a company and you are shown a room with two computers in it. Your guide points to the computer on the left and says, “That is our Web server.” He then points to the other computer and says, “And that is our mail server.” WHAT’S IN A NAME? Newsreader, News server The Usenet system of worldwide discussion groups was started in 1979 by two graduate students at Duke University, Jim Ellis and Tom Truscott. Ellis and Truscott conceived of Usenet as a way to send news and announcements between two universities in North Carolina (University of North Carolina and Duke University). Within a short time, Usenet spread to other schools and, within a few years, it had blossomed into a large system of discussion groups. Because of its origin, Usenet is still referred to as the NEWS (or sometimes NETNEWS), even though it is not a news service. Similarly, the discussion groups are referred to as NEWSGROUPS, the clients are called NEWSREADERS, and the servers are called NEWS SERVERS.

WHAT HAPPENS WHEN YOU PRESS A KEY? As you now understand, Unix is based on the idea of terminals and hosts. Your terminal acts as your interface; the host does the processing. The terminal and the host can be part of the same computer, such as when you use a laptop or a desktop computer. Or the terminal and host can be completely separate from one another, as when you access a Unix host over a LAN or via the Internet. Regardless, the terminal/host relationship is deeply embedded into the fabric of Unix. Having said that, I want to pose what seems like a simple question: “What happens when you press a key?” The answer is more complex than you might expect. Let’s say you are using a Unix computer and you want to ﬁnd out what time it is. The Unix command

Harley Hahn’s Guide to Unix and Linux

Openmirrors.com 33614_03_037_054.indd 50

1/9/2008 12:24:38 PM

The Unix Connection to display the time and date is date. So, you press the four keys: , , , <e>, followed by the <Enter> key. As you press each letter, it appears on your screen, so it’s natural to assume that your terminal is displaying the letters as you type them. Actually, this is not the case. It is the host, not the terminal, that is in charge of displaying what you have typed. Each time you press a key, the terminal sends a signal to the host. It is up to the host to respond in such a way that the appropriate character is displayed on your screen. For example, when you press the key, the terminal sends a signal to the host that means “the user has just sent a d character”. The host then sends back a signal that means “display the letter d on the screen of the terminal”. When this happens, we say that the host ECHOES the character to your screen. The same thing happens when you use a mouse. Moving the mouse or clicking a button sends signals to the host. The host interprets these signals and sends instructions back to your terminal. Your terminal then makes the appropriate changes to your screen: move the pointer, resize a window, display a menu, and so on. In most cases, it all happens so fast that it looks as if your keyboard and mouse are connected directly to your screen. However, if you are using a long-distance connection, say over the Internet, you may occasionally notice a delay between the time you press the key and the time you see the character appear on your screen. You may also see a delay when you move your mouse or press a mouse button and the screen is not updated right away. We refer to this delay as LAG. You might ask, why was Unix designed so that the host echoes each character? Why not have the host silently accept whatever it receives and have the terminal do the echoing? Doing so would be faster, which would avoid lag. The answer is that when the host does the echoing, you can see that what you are typing is being received successfully, and that the connection between your terminal and the host is intact. If the terminal did the echoing and you had a problem, you would have no way of knowing whether or not your connection to the host was working. This, of course, is most important when you are using a host that is physically separate from your terminal. Aside from dependability, there is another reason why the Unix designers chose to have the host do the echoing. As I will discuss in Chapter 7, there are certain keys (such as or ), that you can press to make corrections as you type. Unix was designed to work with a wide variety of terminals, and it made sense for the operating system itself to handle the processing of these keypresses in a uniform way, rather than expect each different type of terminal to be able to do the job on its own. HINT When you use Unix, the characters you type are echoed to your screen by the host, not by the terminal. Most of the time, the lag is so small that you won’t even notice it. However, if you are using a distant host over a slow connection, there may be times when there will be a delay before the characters you type are displayed on your screen. Unix allows you to type ahead many characters, so don’t worry. Just keep typing, and eventually, the host will catch up. In almost all cases, no matter how long the lag, nothing will be lost.

What Happens When You Press a Key?

33614_03_037_054.indd 51

1/9/2008 12:24:38 PM

Chapter 3 CHARACTER TERMINALS AND GRAPHICS TERMINALS Broadly speaking, there are two classes of terminals you can use with Unix. How you interact with Unix will depend on which type of terminal you are using. Take a moment to look back at Figures 3-2 and 3-3, the photos of the Teletype ASR33. As we discussed, this machine was the very ﬁrst Unix terminal. If you look at it carefully, you will see that the only input device was a rudimentary keyboard, and the only output device was a roll of paper upon which characters were printed. Over the years, as hardware developed, Unix terminals became more advanced. The keyboard became more sophisticated and a lot easier to use, and the roll of paper was replaced by a monitor with a video screen. Still, for a long time, one basic characteristic of Unix terminals did not change: the only form of input and output was characters (also called TEXT). In other words, there were letters, numbers, punctuation, and a few special keys to control things, but no pictures. A terminal that only works with text is called a CHARACTER TERMINAL or a TEXT-BASED TERMINAL. As PC technology developed, a new type of terminal became available, the GRAPHICS TERMINAL. Graphics terminals had a keyboard and mouse for input and, for output, they took full advantage of the video hardware. Not only could they handle text; they could display just about anything that could be drawn on a screen using small dots: pictures, geometric shapes, shading, lines, colors, and so on. Obviously, graphics terminals are more powerful than character terminals. When you use a character terminal, you are restricted to typing characters and reading characters. When you use a graphics terminal, you can use a full-ﬂedged GUI (graphical user interface), with icons, windows, colors, pictures, and so on. For this reason, you might think that graphics terminals are always better than character terminals. After all, isn’t a GUI always better than plain text? This is certainly true for PCs using Microsoft Windows and for Macintoshes. From the beginning, both Windows and the Macintosh operating systems were designed to use a GUI; in fact, they depend upon a GUI. Unix is different. Because Unix was developed in an era of character terminals, virtually all the power and function of the operating system is available with plain text. Although there are Unix GUIs (which we will discuss in Chapter 5) and you do need to learn how to use them, a great deal of what you do with Unix – including everything I teach you in this book – requires only plain text. With Unix, graphics are nice, but not necessary. What does this mean in practical terms? When you use Unix on your own computer, you will be working within a GUI (using a mouse, manipulating windows, and so on). This means your computer will be emulating a graphics terminal. However, much of the time, you will ﬁnd yourself working within a window that acts as a character terminal. Within that window, all you will type is text, and all you will see is text. In other words, you will be using a character terminal. In the same way, when you connect to a remote host, you usually do so by opening a window to act as a character terminal. When you ﬁnd yourself working in this way, I want you to take a moment to think about this: you are using Unix in the same way that the original users used Unix back in

Harley Hahn’s Guide to Unix and Linux

Openmirrors.com 33614_03_037_054.indd 52

1/9/2008 12:24:38 PM

The Unix Connection the 1970s. What’s interesting is that, over thirty years later, the system still works well. Most of the time, text is all you need.

THE MOST COMMON TYPES OF TERMINALS Over the years, Unix was developed to work with literally hundreds of different types of terminals. Today, of course, we don’t use actual standalone hardware terminals: we use computers to emulate a terminal. I have mentioned the idea of opening a window to emulate a character terminal. In most cases, the emulation is based on the characteristics of a very old terminal, called the VT100, which dates from 1978. (The VT100 was made by the Digital Equipment Corporation, the same company that made the PDP-11 computers we discussed at the beginning of the chapter.) Although actual VT100s haven’t been used for years, they were so well-designed and so popular, they set a permanent standard for character terminals. (You can see an actual VT100 in Figure 3-7 below.) Graphics terminals, of course, have a different standard. As you will see (in Chapter 5), Unix GUIs are all based on a system called X Window, and the basic support for X Window

FIGURE 3-7: VT100 Terminal

The most popular Unix terminal of all time, the VT100, was introduced in 1978 by the Digital Equipment Corporation. The VT100 was so popular that it set a permanent standard. Even today, most terminal emulation programs use speciﬁcations based on the VT100.

The Most Common Types of Terminals

33614_03_037_054.indd 53

1/9/2008 12:24:38 PM

Chapter 3 is provided by a graphics terminal called the X TERMINAL. Today, the X terminal is the basis of graphics terminal emulation, the same way that the VT100 is the basis of character terminal emulation. Thus, when you connect to a remote host, you have two choices. You can use a character terminal (the most common choice), in which case you will be emulating a VT100 or something like it. Or, if you want to use GUI, you can use a graphics terminal, in which case you will be emulating an X terminal. Although I won’t go into the details now, I’ll show you the two commands you will use. To connect to a remote host and emulate a character terminal, you use the ssh (secure shell) command. To emulate an X Window graphics terminal, you use the ssh -X command.

C H A P T E R

E X E R C I S E S

REVIEW QUESTIONS 1. What type of machine was used as the very ﬁrst Unix terminal and why was it chosen? 2. What are terminal rooms? Why were they necessary? 3. What is a headless system? Give two examples of headless systems that are used on the Internet to provide very important services. 4. What is a server? What is a client?

FOR FURTHER THOUGHT 1. In 1969, Ken Thompson of AT&T Bell Labs was looking for a computer to create what, eventually, became the ﬁrst Unix system. He found an unused PDP-7 minicomputer, which he was able to use. Suppose Thompson had not found the PDP-7. Would we have Unix today? 2. In the 1970s, computers (even minicomputers) were very expensive. Since no one had their own computer, people had to share. Compared to today, computers in the 1970s were slow, with limited memory and storage facilities. Today, every programmer has his own computer with very fast processors, lots of memory, lots of disk space, sophisticated tools, and Internet access. Who had more fun, programmers in the 1970s or programmers today? How about programmers 20 years from now?

Harley Hahn’s Guide to Unix and Linux

Openmirrors.com hah33614_c03_037_054.indd 54

5/20/2009 2:13:42 PM

Starting to Use Unix

When you take your very ﬁrst lesson on how to use Unix, what you need to learn depends on how you will be accessing Unix. Will you be using Unix as part of a shared, multiuser system, say, at school or on the job? Or do you have a Unix computer of your own, in which case you will control the computer and be the only user? In this chapter, we’ll discuss the ﬁrst situation: what it’s like to use a Unix system that is maintained by someone else. I’ll show you how to start and stop a work session, and I’ll explain the basic concepts, such as system administrators, passwords, userids, users and superusers. As I explain ideas to you, I will assume that you are using a straightforward text-based interface. In Chapter 5, we’ll talk about the more complex situation, in which you are using your own Unix system installed on your own computer. In that chapter, we’ll talk about the ideas underlying a graphical interface. What if you are never planning to use Unix as part of a shared system? What if you will only be using your own computer and a graphical interface? Do you still need to read this chapter? The answer is yes. No matter how you use Unix, the skills and ideas we are going to cover in this chapter are basic to Unix and important for everyone. (Besides, you don’t want to miss the story about the Hotdog-bun Boy.)

THE SYSTEM ADMINISTRATOR In the broadest sense, there are two ways in which you can access a Unix system. First, you might have your own Unix computer, in which case you are the only user and you are in charge of managing the system. Alternatively, you might use a shared multiuser system – at school or at work – in which case you will be only one of the users. If this is the case, someone else will be in charge, so you don’t have to worry about maintaining the system. However, you will have to follow some rules and work within certain limitations. Of course, you may be able to access Unix in both ways. For example, you might use a shared system at school and your own PC at home.

The System Administrator

33614_04_055_072.indd 55

1/9/2008 12:25:34 PM

Chapter 4 Although having your own Unix computer sounds simpler, it actually isn’t. The truth is it’s easier to use a shared system. Because you don’t own the system, someone else manages it for you, which is a very big deal. All Unix systems require administration and maintenance. The person who performs these duties is called the SYSTEM ADMINISTRATOR, often abbreviated as SYSADMIN or ADMIN. (The old term, not used much anymore, is SYSTEM MANAGER.) If the computer you use is owned by an organization – such as a university or a company – the admin will probably be a paid employee. Indeed, within organizations that have networks of Unix computers, system administration is a full-time job that calls for a great deal of specialized knowledge. There may be many admins with a staff of assistants. Before the mid 1990s, it was very unusual for someone to have his or her own Unix computer. Most everyone who used Unix did so by accessing a shared system. Unix systems were maintained by an organization (usually a school or a company), and there were rules that all the users were required to follow. Most of the time, people accessed Unix remotely, either by using a terminal or by using a PC to emulate a terminal (see Chapter 3). As such, the most common way to use Unix was with a text-based interface, using only a keyboard and a monitor (as we will be doing in this chapter). It was only a minority of users who used Unix with a graphical interface. When you have your own personal Unix computer, you are, for better or for worse, your own admin. At best, administering your system is a highly fulﬁlling activity, which will build your skills and conﬁdence, making you feel that you are truly in control of your computing destiny. At worst, you will, at times, feel frustrated and confused. To use Unix well, you need to understand a number of basic concepts: the ﬁle system, text editors, the shell, and so on, all of which I will teach you in this book. To be an effective administrator of a large system or a network, you need a lot more. You will have to master a great many esoteric skills, many of which are, alas, beyond the scope of this book. To manage your own personal system is a lot easier. All you will need is basic Unix skills and a thoughtful attitude. Regardless, no matter how long it takes to learn to manage your own Unix computer well, I can assure you that system administration is always a learning experience. (If nothing else, you will, at least, learn patience.) In the meantime, let’s move ahead and see what life is like when someone else is managing the system for you.

USERIDS AND PASSWORDS Before you can use a Unix computer, the system administrator must give you a name that you will use to identify yourself to the system. This name is called your USERID. The word userid is a contraction of “user identiﬁcation”, and is pronounced “user-eye-dee”. Along with the userid, you will also get a PASSWORD, which you will have to type in each time you start a work session. Once you have permission to use a Unix system, we say that you have an ACCOUNT on that computer. Even though you aren’t paying real money for your account, Unix will keep track of how much you use the system. (Unix comes with a lot of built-in accounting,

Harley Hahn’s Guide to Unix and Linux

Openmirrors.com 33614_04_055_072.indd 56

1/9/2008 12:25:34 PM

Starting to Use Unix which your system administrator can use to keep records of who is doing what.) In addition, your account will probably come with certain predeﬁned limits, such as how much disk space you are allowed for your ﬁles, or how many pages you can print. If you are a student, one limit you are likely to encounter is an expiration date on your account. For example, your account may terminate automatically at the end of the semester, which only makes sense. What will your userid be? In most cases, your system administrator will choose a userid for you. One common method is to base the userid on the person’s real name. For example, for the name Harley Q. Hahn, the userid might be harley, hahn,hhahn, harleyh or hqh. Alternatively, your userid may reﬂect some completely objective criteria. For example, if you are a student and you are the 25th person in the CS110 class to ask for a Unix account, you might be assigned the userid cs110-25. Each time you start a Unix session, you must enter your userid. From then on, this name is used by Unix to identify you. For example, when you create ﬁles, they will not belong to you; they will be “owned” by your userid. (We’ll talk about this distinction later in the chapter.) It is important to understand that userids are not secret. For example, if you use Unix for email, your userid will be part of your address. Moreover, at any time, it is easy for anyone to display all the userids that are currently using the system and – if you know what you are doing – you can even display a list of all the userids that are registered with the system. Security, of course, is important, but it does not require that userids be secret. Rather, security is maintained by making sure that passwords are secret. In this way, everyone can ﬁnd out who else uses the computer, but access to the system is controlled. Your password will probably be a meaningless group of characters, such as H!lg%12, something which is difﬁcult for someone else to guess. Later in the chapter, I’ll explain how to change your password if you don’t like it, and what types of passwords are good to use.

LOGGING IN (STARTING WORK WITH UNIX) When you sit down in front of your terminal, the process you go through to start work is called LOGGING IN. Although the idea is simple, the terminology is a bit tricky. When we talk about the idea as a verb, we write two words, “log in”. When we express the same idea as a noun or an adjective, we use a single word LOGIN. For example, “In order to log in, you need to learn the login procedure.” Or, “We need a larger computer. Our current one gets over 500 logins a day; there are too many people trying to log in at the same time.” The actual login process is straightforward. All you need to do is type your userid and your password. Here is how it works. When a Unix program wants you to type something, it displays a PROMPT, a short message indicating that you must enter input from the keyboard. When Unix wants to show that it is waiting for you to log in, it displays the following prompt: login:

Logging In (Starting Work With Unix)

33614_04_055_072.indd 57

1/9/2008 12:25:34 PM

Chapter 4 Unix is saying, “Type your userid and press the key.” Although this seems straightforward, I would like to pause for a moment to answer an important question: What, exactly, is the key? In Chapter 7, you will learn that Unix uses a set of special keys that do not necessarily correspond to the exact same physical keys on every keyboard. We’ll talk about the details then. For now, all I want you to know is that Unix has a special key that you press to indicate you have ﬁnished typing a line of input. This key is called the key. When you press the key, it sends Unix a signal called a newline. If your keyboard has an actual key, pressing it will send the newline signal. (This is the case with a Macintosh.) Otherwise, you send the newline by pressing the <Enter> key (which is the case with PCs). Thus, throughout this book, when I tell you to press , use either the key or the <Enter> key, whichever you have on your particular keyboard. Once you have typed your userid and pressed , Unix asks for your password by displaying the following prompt: Password: As you type, you will notice that your password is not echoed. This is to prevent other people from seeing your password if they happen to be looking over your shoulder. (Remember, in Unix, userids are not secret but passwords are, which is why, when you log in, userids are echoed but passwords are not.) Notice also that, unlike Windows, when you type a password, the system does not display an asterisk for each character. This means that if someone is watching you, he not only can’t see your password, but also he doesn’t even know how many characters you typed. After you have typed your password, press once again. Unix will check to conﬁrm the password is valid. If it is, Unix will complete the login process and you will be ready to start work. If either your userid or password was incorrect, Unix will display: Login incorrect and let you try again. If you are connecting remotely, some systems will disconnect you if you log in incorrectly too many times. This is to make it difﬁcult for someone who is trying to break into the system to keep guessing passwords indeﬁnitely. (Typically, you get 3-5 tries. The exact number is controlled by the system administrator.) As you type your userid and your password, there are three important things I would like you to remember. • Be sure not to mix up small letters and capital letters. For example, if your userid is “harley”, type harley, not Harley. •

Be careful not to confuse the number 0 (zero), with the capital letter O (oh).

• Be careful not to confuse the number 1 (one), with the small letter l (el).

Harley Hahn’s Guide to Unix and Linux

Openmirrors.com 33614_04_055_072.indd 58

1/9/2008 12:25:35 PM

Starting to Use Unix Before we ﬁnish this section, I want to point out a curious thing that very few people notice. On virtually all Unix systems, the login program displays login: with a small “l” and Password: with a capital “P”. No one knows why. HINT Whenever you type a userid, Unix always asks for a password, even if that particular userid is invalid. This makes it more difﬁcult for evil-minded people to guess userids. For example, if someone enters the userid harley, he or she will always be asked for a password, even if there is no such userid registered with the system. Of course, this also means that if you mistype your userid or your password, you won’t know which one was wrong. You will just be told that your login was incorrect.

WHAT’S IN A NAME? The key Today, most keyboards have an <Enter> key, not a key. Why, then, does Unix use the name ? The use is traditional. As I explained in Chapter 3, for many years Unix was accessed from terminals, not from PCs, and it happened that all terminals had a key. Even though there are now countless PC keyboards with <Enter> keys, the Unix terminology has not changed. The name “Return” comes from typewriters. In the olden days, the part of a mechanical typewriter that held the paper was called the “carriage”. Each time you put in a new piece of paper, the carriage would start at the far right. As you typed, the carriage would move to the left one character at a time. When you came to the end of a line, you would use your left hand to push a lever that would move the carriage back to the right. At the same time, the lever would also move the paper up one line. In this way, the paper would be positioned at the start of a new line. The ﬁrst Unix terminals were Teletype ASR33 machines (see Chapter 3). Unlike typewriters, they did not have a movable carriage. However, while printing text, changing from the end of one line to the beginning of the next did involve two separate motions. These motions were analogous to what happened when you pushed the lever on a typewriter, so they were described using typewriter terminology and referred to as CARRIAGE RETURN and LINEFEED. (These two terms are still important in the world of Unix, and you will meet them time and again.) Figure 4-1 shows a photo of the Teletype ASR33 keyboard. Notice that there is both a key and a key. This is why, to this day, Unix still refers to the key that terminates a line of text as the key.

WHAT HAPPENS AFTER YOU LOG IN? After you log in successfully, Unix will display some informative messages, followed by an invitation to enter a command. You can then start your work session by entering one command after another. The informative messages you see will vary, depending on how the system administrator has conﬁgured your system. Figure 4-2, for example, has a typical example from a FreeBSD system:

What Happens After You Log In?

33614_04_055_072.indd 59

1/9/2008 12:25:35 PM

Chapter 4 The ﬁrst line shows us the last time we logged into this computer using the current userid. At the end of the line, we see ttyp1, which is the name of the terminal that was used at the time. Whenever you log in, take a minute to check this line; it is here for security reasons. If the time you see is more recent than you remember, someone else may have been using your account without your permission. If so, change your password immediately. (I’ll explain how to do so later in the chapter.) The next three lines contain copyright information. As you will recall from Chapter 3, FreeBSD is based on work done at U.C. Berkeley, so it is understandable that the University of California is named as the copyright holder. The second to last line shows we are using FreeBSD version 4.9. The date and time show that the kernel was “built” – that is, generated – on March 8, 2006 at 4:26 PM. (Unix uses a 24-hour clock.) Finally, the last line is a welcome greeting put there by a friendly FreeBSD programmer. What happens after the login message is displayed depends, in part, on how your system was set up. As part of the login process, Unix executes a list of predeﬁned commands that are kept in special initialization ﬁles. Some of these ﬁles are controlled by the system administrator, so he can ensure that speciﬁc tasks are carried out each time someone logs in. For example, he may want to display a speciﬁc message to all the users whenever they log in.

FIGURE 4-1: Keyboard of the Teletype ASR33

The keyboard of the Teletype ASR33, the very ﬁrst device used as a Unix terminal. Notice the and keys (on the far right of the second row).

Harley Hahn’s Guide to Unix and Linux

Openmirrors.com 33614_04_055_072.indd 60

Harley Hahn's Guide to Unix and Linux - PDF Free Download (2024)

References