THE EXPERT’S VOICE®IN LINUX Covering Bash 4.0 Pro Bash Programming Scripting the GNU/Linux Shell Programming bash from one-liners to professional programs Chris F.A. Johnson BOOKS FOR PROFESSIONALS BY PROFESSIONALS® Pro Bash Programming: Scripting the GNU/Linux Shell Dear Reader, The shell is a programming language! A shell script is as much a program as anything written in C, Python, or any other programming language. Just because shell scripts are easier to write doesn’t mean they should take a back- seat to compiled programs or other scripting languages. Author of I wrote Pro Bash Programming to present a tutorial that introduces the shell, Shell Scripting Recipes: and bash specifically, as a programming language. If you need a program to A Problem-Solution accomplish a task on GNU/Linux or any other Unix-type system, I want you Approach to consider the shell before any other language. If you write a prototype for a program using the shell, I want you to realize that you don’t need to translate it to another language. This book will give you a grounding in programming techniques used in writing shell programs no matter what your past programming experience. If you’ve never written a computer program before, Pro Bash Programming will get you started and help you become a proficient shell programmer. If you have written a few shell programs, this book will take you to the next level and beyond. It will enable you to do things with the shell you never thought it could do. If you are already an expert shell programmer, this book will provide insight into the advanced shell programming, helping you write more, and more effi- cient, scripts. Bash is the shell of the Free Software Foundation’s GNU project and is the standard shell on almost all Linux distributions. It is the shell you probably use at the command line. It offers programmers many enhancements over the standard Unix shell. You will find bash on many versions of Unix. It may not be the default shell, but it is usually available for interpreting your scripts. I hope this book will help you become a more productive programmer and Companion eBook that your programs will be written using the Bash shell. Chris F.A. Johnson THEAPRESS ROADMAP See last page for details Expert Shell on $10 eBook version Beginning Perl 2nd Edition Scripting Beginning the Beginning Ubuntu LTS Server Pro Bash Linux Command Line Administration 2nd Edition Programming SOURCE CODE ONLINE Linux System Beginning Portable Shell Scripting Administration Recipes www.apress.com ISBN 978-1-4302-1997-2 US $34.99 53499 Shelve in Linux User level: Beginner–Intermediate Intermediate–Advanced Advanced 9 781430 219972 ■ CONTENTS Pro Bash Programming Scripting the GNU/Linux Shell ■ ■ ■ Chris F.A. Johnson i ■ CONENTS Pro Bash Programming: Scripting the GNU/Linux Shell Copyright © 2009 by Chris F.A. Johnson All rights reserved. No part of this work may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or by any information storage or retrieval system, without the prior written permission of the copyright owner and the publisher. ISBN-13 (pbk): 978-1-4302-1997-2 ISBN-13 (electronic): 978-1-4302-1998-9 Printed and bound in the United States of America 9 8 7 6 5 4 3 2 1 Trademarked names may appear in this book. Rather than use a trademark symbol with every occurrence of a trademarked name, we use the names only in an editorial fashion and to the benefit of the trademark owner, with no intention of infringement of the trademark. Lead Editor: Frank Pohlmann Technical Reviewer: Ed Schaefer Editorial Board: Clay Andres, Steve Anglin, Mark Beckner, Ewan Buckingham, Tony Campbell, Gary Cornell, Jonathan Gennick, Jonathan Hassell, Michelle Lowman, Matthew Moodie, Jeffrey Pepper, Frank Pohlmann, Douglas Pundick, Ben Renow-Clarke, Dominic Shakeshaft, Matt Wade, Tom Welsh Project Manager: Kylie Johnston Copy Editor: Kim Wimpsett Compositor: ContentWorks, Inc. Indexer: Julie Grady Distributed to the book trade worldwide by Springer-Verlag New York, Inc., 233 Spring Street, 6th Floor, New York, NY 10013. Phone 1-800-SPRINGER, fax 201-348-4505, e-mail [email protected], or visit http://www.springeronline.com. For information on translations, please contact Apress directly at 233 Spring Street, New York, NY 10013. E-mail [email protected], or visit http://www.apress.com. Apress and friends of ED books may be purchased in bulk for academic, corporate, or promotional use. eBook versions and licenses are also available for most titles. For more information, reference our Special Bulk Sales–eBook Licensing web page at http://www.apress.com/info/bulksales. The information in this book is distributed on an “as is” basis, without warranty. Although every precaution has been taken in the preparation of this work, neither the author(s) nor Apress shall have any liability to any person or entity with respect to any loss or damage caused or alleged to be caused directly or indirectly by the information contained in this work. . The source code for this book is available to readers athttp://www.apress.com ii ■ CONTENTS Contents at a Glance ■About the Author...................................................................................................xvi ■About the Technical Reviewer.............................................................................xvii ■Introduction..........................................................................................................xix ■Chapter 1: Hello, World! Your First Shell Program..................................................1 ■Chapter 2: Input, Output, and Throughput...............................................................7 ■Chapter 3: Looping and Branching........................................................................19 ■Chapter 4: Command-Line Parsing and Expansion...............................................29 ■Chapter 5: Parameters and Variables....................................................................43 ■Chapter 6: Shell Functions.....................................................................................59 ■Chapter 7: String Manipulation..............................................................................67 ■Chapter 8: File Operations and Commands...........................................................79 ■Chapter 9: Reserved Words and Builtin Commands.............................................97 ■Chapter 10: Writing Bug-Free Scripts and Debugging the Rest..........................113 ■Chapter 11: Programming for the Command Line..............................................125 ■Chapter 12: Runtime Configuration.....................................................................141 ■Chapter 13: Data Processing...............................................................................157 ■Chapter 14: Scripting the Screen.........................................................................179 ■Chapter 15: Entry-Level Programming................................................................191 ■Appendix: Shell Variables....................................................................................205 ■Index....................................................................................................................221 iii ■ CONTENTS Contents ■About the Author...................................................................................................xvi ■About the Technical Reviewer.............................................................................xvii ■Introduction..........................................................................................................xix ■Chapter 1: Hello, World! Your First Shell Program..................................................1 The Code............................................................................................................................1 The File..............................................................................................................................2 The Naming of Scripts...................................................................................................2 Selecting a Directory for the Script...............................................................................2 Creating the File and Running the Script.......................................................................3 Choosing and Using a Text Editor......................................................................................3 Building a Better “Hello, World!”.......................................................................................5 Summary...........................................................................................................................5 Commands.....................................................................................................................5 Concepts........................................................................................................................6 Variables........................................................................................................................6 Exercises...........................................................................................................................6 ■Chapter 2: Input, Output, and Throughput...............................................................7 Parameter and Variables...................................................................................................7 Positional Parameters...................................................................................................7 Special*@#0$?_!-Parameters.....................................................................................8 Variables........................................................................................................................8 Arguments and Options.....................................................................................................8 echo, and Why You Should Avoid It...................................................................................9 printf: Formatting and Printing Data..................................................................................9 Escape Sequences......................................................................................................10 Format Specifiers........................................................................................................10 v ■ CONENTS Width Specification......................................................................................................11 Printing to a Variable...................................................................................................13 Line Continuation.............................................................................................................13 Standard Input/Output Streams and Redirection.............................................................13 Redirection: >, >>, and <...........................................................................................13 Reading Input...................................................................................................................15 Pipelines..........................................................................................................................15 Command Substitution....................................................................................................16 Summary.........................................................................................................................16 Commands...................................................................................................................16 Concepts......................................................................................................................16 Exercises.........................................................................................................................17 ■Chapter 3: Looping and Branching........................................................................19 Exit Status........................................................................................................................19 Testing an Expression......................................................................................................19 test, aka [ ... ]..............................................................................................................20 [[ ... ]]: Evaluate an Expression....................................................................................21 (( ... )): Evaluate an Arithmetic Expression...................................................................22 Lists.................................................................................................................................22 Conditional execution......................................................................................................22 if..................................................................................................................................22 Conditional Operators, && and ||.................................................................................23 case.............................................................................................................................24 Looping............................................................................................................................25 while............................................................................................................................25 until.............................................................................................................................26 for................................................................................................................................26 break...........................................................................................................................26 continue.......................................................................................................................27 Summary.........................................................................................................................27 Commands...................................................................................................................27 Concepts......................................................................................................................28 Exercises.........................................................................................................................28 vi ■ CONTENTS ■Chapter 4: Command-Line Parsing and Expansion...............................................29 Quoting............................................................................................................................30 Brace Expansion..............................................................................................................31 Tilde Expansion................................................................................................................32 Parameter and Variable Expansion..................................................................................33 Arithmetic Expansion.......................................................................................................33 Command Substitution....................................................................................................35 Word Splitting..................................................................................................................36 Pathname Expansion.......................................................................................................37 Process Substitution........................................................................................................37 Parsing Options................................................................................................................38 Summary.........................................................................................................................41 Commands...................................................................................................................41 Exercises.........................................................................................................................41 ■Chapter 5: Parameters and Variables....................................................................43 The Scope of a Variable: Can You See It from Here?.......................................................43 Shell Variables.................................................................................................................44 The Naming of Variables..................................................................................................46 Parameter Expansion.......................................................................................................46 Bourne Shell................................................................................................................46 POSIX Shell..................................................................................................................49 Bash.............................................................................................................................51 Bash-4.0......................................................................................................................52 Positional Parameters......................................................................................................53 Arrays..............................................................................................................................54 Integer-Indexed Arrays................................................................................................54 Associative Arrays.......................................................................................................56 Summary.........................................................................................................................56 Commands...................................................................................................................56 Concepts......................................................................................................................57 Exercises.........................................................................................................................57 ■Chapter 6: Shell Functions.....................................................................................59 Definition Syntax..............................................................................................................59 vii