ebook img

The Art of Unix Programming PDF

549 Pages·2003·2.2 MB·English
Save to my drive
Quick download
Download
Most books are stored in the elastic cloud where traffic is expensive. For this reason, we have a limit on daily download.

Preview The Art of Unix Programming

The Art of Unix Programming by Eric Steven Raymond The Art of Unix Programming byEricStevenRaymond Copyright©2003EricS.Raymond Thisbookanditson-lineversionaredistributedunderthetermsoftheCreativeCommonsAttribution-NoDerivs1.0license, withtheadditionalprovisothattherighttopublishitonpaperforsaleorotherfor-profituseisreservedtoPearsonEducation, Inc. Areferencecopyofthislicensemaybefoundathttp://creativecommons.org/licenses/by-nd/1.0/legalcode. AIX,AS/400,DB/2,OS/2,System/360,MVS,VM/CMS,andIBMPCaretrademarksofIBM.Alpha,DEC,VAX,HP-UX, PDP,TOPS-10,TOPS-20,VMS,andVT-100aretrademarksofCompaq. AmigaandAmigaOSaretrademarksofAmiga, Inc. Apple, Macintosh, MacOS,Newton, OpenDoc, andOpenSteparetrademarksofAppleComputers, Inc. ClearCase isatrademarkofRationalSoftware, Inc. Ethernetisatrademarkof3COM,Inc. Excel, MS-DOS,MicrosoftWindows and PowerPoint are trademarks of Microsoft, Inc. Java. J2EE, JavaScript, NeWS, and Solaris are trademarks of Sun Microsystems. SPARCisatrademarkofSPARCinternational. InformixisatrademarkofInformixsoftware. Itanium isatrademarkofIntel. LinuxisatrademarkofLinusTorvalds. NetscapeisatrademarkofAOL.PDFandPostScriptare trademarksofAdobe,Inc.UNIXisatrademarkofTheOpenGroup. ThephotographofKenandDennisinChapter2appearscourtesyofBellLabs/LucentTechnologies. TheepigraphonthePortabilitychapterisfromtheBellSystemTechnicalJournal, v57#6part2(July-Aug. 1978)pp. 2021-2048andisreproducedwiththepermissionofBellLabs/LucentTechnologies. Dedication ToKenThompsonandDennisRitchie,becauseyouinspiredme. i Table of Contents Preface ........................................................................ xvi WhoShouldReadThisBook .............................................. xvii HowtoUseThisBook .................................................... xviii RelatedReferences ........................................................ xix ConventionsUsedinThisBook ............................................. xx OurCaseStudies .......................................................... xxi Author’sAcknowledgements ............................................... xxii I.Context ....................................................................... 24 1. Philosophy .............................................................. 25 Culture? WhatCulture? ............................................... 25 TheDurabilityofUnix ................................................ 25 TheCaseagainstLearningUnixCulture ................................ 27 WhatUnixGetsWrong ............................................... 28 WhatUnixGetsRight ................................................. 29 Open-SourceSoftware ........................................... 29 Cross-PlatformPortabilityandOpenStandards ..................... 29 TheInternetandtheWorldWideWeb ............................. 30 TheOpen-SourceCommunity .................................... 30 FlexibilityAlltheWayDown ..................................... 31 UnixIsFuntoHack ............................................. 32 TheLessonsofUnixCanBeAppliedElsewhere .................... 32 BasicsoftheUnixPhilosophy ......................................... 33 RuleofModularity: Writesimplepartsconnectedbycleaninterfaces. 36 RuleofClarity: Clarityisbetterthancleverness. ................... 36 Rule of Composition: Design programs to be connected with other pro- grams. ......................................................... 37 Rule of Separation: Separate policy from mechanism; separate interfaces fromengines. ................................................... 38 RuleofSimplicity: Designforsimplicity; addcomplexityonlywhereyou must. .......................................................... 39 Rule of Parsimony: Write a big program only when it is clear by demon- strationthatnothingelsewilldo. ................................. 40 RuleofTransparency: Designforvisibilitytomakeinspectionanddebug- gingeasier. ..................................................... 40 ii The Art of Unix Program- ming RuleofRobustness: Robustnessisthechildoftransparencyandsimplicity. 41 RuleofRepresentation: Foldknowledgeintodata,soprogramlogiccanbe stupidandrobust. ............................................... 42 Rule of Least Surprise: In interface design, always do the least surprising thing. .......................................................... 42 Rule of Silence: When a program has nothing surprising to say, it should saynothing. .................................................... 43 RuleofRepair: Repairwhatyoucan—butwhenyoumustfail,failnoisily andassoonaspossible. .......................................... 44 RuleofEconomy: Programmertimeisexpensive;conserveitinpreference tomachinetime. ................................................ 45 RuleofGeneration:Avoidhand-hacking;writeprogramstowriteprograms whenyoucan. .................................................. 45 Rule of Optimization: Prototype before polishing. Get it working before youoptimizeit. ................................................. 46 RuleofDiversity: Distrustallclaimsforonetrueway. .............. 47 RuleofExtensibility: Designforthefuture,becauseitwillbeheresooner thanyouthink. ................................................. 48 TheUnixPhilosophyinOneLesson .................................... 48 ApplyingtheUnixPhilosophy ......................................... 49 AttitudeMattersToo .................................................. 50 2. History ................................................................. 52 OriginsandHistoryofUnix,1969-1995 ................................. 52 Genesis: 1969–1971 ............................................. 52 Exodus: 1971–1980 ............................................. 55 TCP/IPandtheUnixWars: 1980-1990 ............................ 57 BlowsagainsttheEmpire: 1991-1995 ............................. 64 OriginsandHistoryoftheHackers,1961-1995 .......................... 66 AtPlayintheGrovesofAcademe: 1961-1980 ..................... 67 InternetFusionandtheFreeSoftwareMovement: 1981-1991 ........ 68 LinuxandthePragmatistReaction: 1991-1998 ..................... 71 TheOpen-SourceMovement: 1998andOnward ......................... 73 TheLessonsofUnixHistory ........................................... 75 3. Contrasts ............................................................... 76 TheElementsofOperating-SystemStyle ................................ 76 WhatIstheOperatingSystem’sUnifyingIdea? .................... 76 iii The Art of Unix Program- ming MultitaskingCapability .......................................... 77 CooperatingProcesses ........................................... 78 InternalBoundaries .............................................. 79 FileAttributesandRecordStructures .............................. 80 BinaryFileFormats ............................................. 81 PreferredUserInterfaceStyle .................................... 81 IntendedAudience .............................................. 82 EntryBarrierstoDevelopment .................................... 83 Operating-SystemComparisons ........................................ 83 VMS ........................................................... 87 MacOS ......................................................... 88 OS/2 ........................................................... 90 WindowsNT ................................................... 92 BeOS .......................................................... 96 MVS ........................................................... 97 VM/CMS ..................................................... 100 Linux ......................................................... 101 WhatGoesAround,ComesAround ................................... 103 II.Design ..................................................................... 106 4. Modularity ............................................................. 107 EncapsulationandOptimalModuleSize ............................... 108 CompactnessandOrthogonality ....................................... 111 Compactness .................................................. 111 Orthogonality .................................................. 113 TheSPOTRule ................................................ 115 CompactnessandtheStrongSingleCenter ........................ 116 TheValueofDetachment ....................................... 118 SoftwareIsaMany-LayeredThing .................................... 119 Top-DownversusBottom-Up .................................... 119 GlueLayers ................................................... 121 CaseStudy: CConsideredasThinGlue .......................... 122 Libraries ............................................................ 124 CaseStudy: GIMPPlugins ...................................... 125 UnixandObject-OrientedLanguages .................................. 126 CodingforModularity ............................................... 128 5. Textuality .............................................................. 130 TheImportanceofBeingTextual ...................................... 131 iv TheArtofUnixProgramming CaseStudy: UnixPasswordFileFormat .......................... 133 CaseStudy: .newsrcFormat ..................................... 135 CaseStudy: ThePNGGraphicsFileFormat ...................... 136 DataFileMetaformats ............................................... 137 DSVStyle ..................................................... 137 RFC822Format ............................................... 139 Cookie-JarFormat ............................................. 140 Record-JarFormat ............................................. 141 XML ......................................................... 142 WindowsINIFormat ........................................... 144 UnixTextualFileFormatConventions ............................ 145 TheProsandConsofFileCompression .......................... 147 ApplicationProtocolDesign .......................................... 148 CaseStudy: SMTP,aSimpleSocketProtocol ..................... 149 CaseStudy: POP3,thePostOfficeProtocol ....................... 150 CaseStudy: IMAP,theInternetMessageAccessProtocol .......... 151 ApplicationProtocolMetaformats ..................................... 153 TheClassicalInternetApplicationMetaprotocol ................... 153 HTTPasaUniversalApplicationProtocol ........................ 154 BEEP:BlocksExtensibleExchangeProtocol ...................... 156 XML-RPC,SOAP,andJabber ................................... 157 6. Transparency ........................................................... 159 StudyingCases ...................................................... 160 CaseStudy: audacity ........................................... 161 CaseStudy: fetchmail’s-voption ................................ 164 CaseStudy: GCC .............................................. 167 CaseStudy: kmail .............................................. 167 CaseStudy: SNG .............................................. 171 CaseStudy: TheTerminfoDatabase .............................. 173 CaseStudy: FreecivDataFiles .................................. 176 DesigningforTransparencyandDiscoverability ........................ 179 TheZenofTransparency ........................................ 180 CodingforTransparencyandDiscoverability ...................... 181 TransparencyandAvoidingOverprotectiveness .................... 182 TransparencyandEditableRepresentations ....................... 183 Transparency,FaultDiagnosis,andFaultRecovery ................ 185 DesigningforMaintainability ......................................... 186 v The Art of Unix Program- ming 7. Multiprogramming ..................................................... 188 SeparatingComplexityControlfromPerformanceTuning ............... 189 TaxonomyofUnixIPCMethods ...................................... 190 HandingoffTaskstoSpecialistPrograms ......................... 191 Pipes,Redirection,andFilters ................................... 192 Wrappers ...................................................... 197 SecurityWrappersandBernsteinChaining ........................ 198 SlaveProcesses ................................................ 199 Peer-to-PeerInter-ProcessCommunication ........................ 200 ProblemsandMethodstoAvoid ....................................... 208 ObsolescentUnixIPCMethods .................................. 208 RemoteProcedureCalls ........................................ 210 Threads—ThreatorMenace? .................................. 211 ProcessPartitioningattheDesignLevel ................................ 213 8. Minilanguages ......................................................... 215 UnderstandingtheTaxonomyofLanguages ............................ 217 ApplyingMinilanguages ............................................. 219 CaseStudy: sng ................................................ 219 CaseStudy: RegularExpressions ................................ 220 CaseStudy: Glade ............................................. 225 CaseStudy: m4 ................................................ 227 CaseStudy: XSLT ............................................. 228 CaseStudy: TheDocumenter’sWorkbenchTools .................. 229 CaseStudy: fetchmailRun-ControlSyntax ........................ 234 CaseStudy: awk ............................................... 235 CaseStudy: PostScript .......................................... 237 CaseStudy: bcanddc .......................................... 238 CaseStudy: EmacsLisp ........................................ 240 CaseStudy: JavaScript ......................................... 240 DesigningMinilanguages ............................................ 241 ChoosingtheRightComplexityLevel ............................ 242 ExtendingandEmbeddingLanguages ............................ 244 WritingaCustomGrammar ..................................... 245 Macros—Beware! ............................................ 245 LanguageorApplicationProtocol? .............................. 247 9. Generation ............................................................. 249 Data-DrivenProgramming ............................................ 250 vi The Art of Unix Program- ming CaseStudy: ascii ............................................... 250 CaseStudy: StatisticalSpamFiltering ............................ 252 CaseStudy: MetaclassHackinginfetchmailconf .................. 253 Ad-hocCodeGeneration ............................................. 259 CaseStudy: GeneratingCodefortheasciiDisplays ................ 259 CaseStudy: GeneratingHTMLCodeforaTabularList ............ 262 10. Configuration ......................................................... 265 WhatShouldBeConfigurable? ....................................... 265 WhereConfigurationsLive ........................................... 267 Run-ControlFiles ................................................... 268 CaseStudy: The.netrcFile ...................................... 270 PortabilitytoOtherOperatingSystems ........................... 272 EnvironmentVariables ............................................... 272 SystemEnvironmentVariables .................................. 272 UserEnvironmentVariables ..................................... 273 WhentoUseEnvironmentVariables ............................. 274 PortabilitytoOtherOperatingSystems ........................... 275 Command-LineOptions .............................................. 276 The-ato-zofCommand-LineOptions ........................... 277 PortabilitytoOtherOperatingSystems ........................... 283 HowtoChooseamongtheMethods ................................... 283 CaseStudy: fetchmail .......................................... 284 CaseStudy: TheXFree86Server ................................ 286 OnBreakingTheseRules ............................................ 287 11. Interfaces ............................................................. 289 ApplyingtheRuleofLeastSurprise ................................... 290 HistoryofInterfaceDesignonUnix ................................... 291 EvaluatingInterfaceDesigns .......................................... 293 TradeoffsbetweenCLIandVisualInterfaces ........................... 295 CaseStudy: TwoWaystoWriteaCalculatorProgram .............. 298 Transparency,Expressiveness,andConfigurability ...................... 300 UnixInterfaceDesignPatterns ........................................ 302 TheFilterPattern .............................................. 302 TheCantripPattern ............................................. 304 TheSourcePattern ............................................. 305 TheSinkPattern ............................................... 305 TheCompilerPattern ........................................... 306 vii TheArtofUnix Programming Theedpattern ................................................. 306 TheRoguelikePattern .......................................... 307 The‘SeparatedEngineandInterface’Pattern ...................... 310 TheCLIServerPattern ......................................... 318 Language-BasedInterfacePatterns ............................... 319 ApplyingUnixInterface-DesignPatterns ............................... 320 ThePolyvalent-ProgramPattern ................................. 320 TheWebBrowserasaUniversalFrontEnd ............................. 322 SilenceIsGolden .................................................... 325 12. Optimization .......................................................... 327 Don’tJustDoSomething,StandThere! ................................ 327 MeasurebeforeOptimizing ........................................... 328 NonlocalityConsideredHarmful ...................................... 330 Throughputvs. Latency .............................................. 331 BatchingOperations ............................................ 332 OverlappingOperations ......................................... 332 CachingOperationResults ...................................... 333 13. Complexity ........................................................... 335 SpeakingofComplexity .............................................. 335 TheThreeSourcesofComplexity ................................ 335 TradeoffsbetweenInterfaceandImplementationComplexity ....... 337 Essential,Optional,andAccidentalComplexity ................... 339 MappingComplexity ........................................... 340 WhenSimplicityIsNotEnough ................................. 342 ATaleofFiveEditors ................................................ 343 ed ............................................................ 344 vi ............................................................. 345 Sam .......................................................... 346 Emacs ........................................................ 347 Wily .......................................................... 349 TheRightSizeforanEditor .......................................... 350 IdentifyingtheComplexityProblems ............................. 350 CompromiseDoesn’tWork ..................................... 353 IsEmacsanArgumentagainsttheUnixTradition? ................ 355 TheRightSizeofSoftware ........................................... 357 III.Implementation ............................................................. 359 14. Languages ............................................................ 360 viii

See more

The list of books you might like

Most books are stored in the elastic cloud where traffic is expensive. For this reason, we have a limit on daily download.