Production Haskell Succeeding in Industry with Haskell Matt Parsons Thisbookisforsaleathttp://leanpub.com/production-haskell Thisversionwaspublishedon2023-02-01 ThisisaLeanpubbook.Leanpubempowersauthorsandpublishers withtheLeanPublishingprocess.LeanPublishingistheactof publishinganin-progressebookusinglightweighttoolsandmany iterationstogetreaderfeedback,pivotuntilyouhavetherightbook andbuildtractiononceyoudo. ©2020-2023MattParsons Contents Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . i AnOpinionatedTourGuide . . . . . . . . . . . . . . . . . . . . . . i AbouttheAuthor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ii Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . v Principles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vi Complexity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vi Novelty . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix Cohesion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . x Empathy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvi I Building Haskell Teams . . . . . . . . . . . 1 1. SellingHaskell . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.1 AssessingReceptiveness . . . . . . . . . . . . . . . . . . . . . 2 1.2 SoftwareProductivity . . . . . . . . . . . . . . . . . . . . . . . 3 1.3 StatisticsofProductivity . . . . . . . . . . . . . . . . . . . . . 4 1.4 KnowYourCompetition . . . . . . . . . . . . . . . . . . . . . 6 2. LearningandTeachingHaskell . . . . . . . . . . . . . . . . . . . 7 2.1 ThePhilologyofHaskell . . . . . . . . . . . . . . . . . . . . . 7 2.2 ProgrammingIsHardToLearn . . . . . . . . . . . . . . . . . 8 2.3 PickLearningMaterials. . . . . . . . . . . . . . . . . . . . . . 9 2.4 WriteLotsofCode . . . . . . . . . . . . . . . . . . . . . . . . . 10 2.5 Don’tFeartheGHC . . . . . . . . . . . . . . . . . . . . . . . . 10 2.6 StartSimple . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 2.7 SolveRealProblems . . . . . . . . . . . . . . . . . . . . . . . . 13 CONTENTS 2.8 PairProgramming . . . . . . . . . . . . . . . . . . . . . . . . . 14 2.9 ADialogue . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 2.10 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 3. HiringHaskellers . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 3.1 TheDouble-edgedSword . . . . . . . . . . . . . . . . . . . . . 23 3.2 JuniorsandSeniors . . . . . . . . . . . . . . . . . . . . . . . . 23 3.3 HiringSeniors . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 3.4 HiringJuniors . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 4. EvaluatingConsultancies . . . . . . . . . . . . . . . . . . . . . . . 32 4.1 IdentifyingtheTarget . . . . . . . . . . . . . . . . . . . . . . . 32 4.2 Well-Typed . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 4.3 FPComplete . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 II Application Structure . . . . . . . . . . . . . 37 5. TheBeginning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 5.1 AbstractionforMocking . . . . . . . . . . . . . . . . . . . . . 42 5.2 ForwardCompatibility . . . . . . . . . . . . . . . . . . . . . . 42 5.3 AppEnvironment . . . . . . . . . . . . . . . . . . . . . . . . . 44 5.4 TheReaderTPattern . . . . . . . . . . . . . . . . . . . . . . . 49 5.5 Embed,don’tStack . . . . . . . . . . . . . . . . . . . . . . . . 51 6. ThreeLayerHaskellCake . . . . . . . . . . . . . . . . . . . . . . . 54 6.1 Layer1:ImperativeProgramming . . . . . . . . . . . . . . . 54 6.2 Layer2:ObjectOrientedProgramming . . . . . . . . . . . . 56 6.3 Layer3:FunctionalProgramming . . . . . . . . . . . . . . . 62 6.4 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63 7. InvertYourMocks! . . . . . . . . . . . . . . . . . . . . . . . . . . . 64 7.1 DecomposingEffects . . . . . . . . . . . . . . . . . . . . . . . 65 7.2 StreamingDecomposition . . . . . . . . . . . . . . . . . . . . 68 7.3 Plainol’abstraction . . . . . . . . . . . . . . . . . . . . . . . . 73 7.4 Decompose!!! . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75 7.5 WhatifIneedto? . . . . . . . . . . . . . . . . . . . . . . . . . . 75 8. ProjectPreludes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 8.1 PreludeProblems . . . . . . . . . . . . . . . . . . . . . . . . . 78 8.2 CustomBenefits . . . . . . . . . . . . . . . . . . . . . . . . . . 84 CONTENTS 8.3 Off-The-ShelfPreludes . . . . . . . . . . . . . . . . . . . . . . 89 8.4 NestingPreludes . . . . . . . . . . . . . . . . . . . . . . . . . . 97 8.5 Downsides. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98 8.6 UsingaCustomPrelude . . . . . . . . . . . . . . . . . . . . . 99 9. OptimizingGHCCompileTimes . . . . . . . . . . . . . . . . . . . 100 9.1 TheProject.TypesMegamodule . . . . . . . . . . . . . . . 100 9.2 PackageSplitting. . . . . . . . . . . . . . . . . . . . . . . . . . 102 9.3 BigOl’InstancesModule . . . . . . . . . . . . . . . . . . . . . 104 9.4 TemplateHaskell . . . . . . . . . . . . . . . . . . . . . . . . . 107 9.5 Somerandompartingthoughts . . . . . . . . . . . . . . . . . 117 III Domain Modeling . . . . . . . . . . . . . . . .118 10.TypeSafetyBackandForth . . . . . . . . . . . . . . . . . . . . . . 119 10.1 TheRippleEffect . . . . . . . . . . . . . . . . . . . . . . . . . . 123 10.2 AskOnlyWhatYouNeed . . . . . . . . . . . . . . . . . . . . . 124 11.KeepYourTypesSmall… . . . . . . . . . . . . . . . . . . . . . . . 126 11.1 ExpansionandRestriction . . . . . . . . . . . . . . . . . . . . 127 11.2 ConstraintsLiberate. . . . . . . . . . . . . . . . . . . . . . . . 129 11.3 RestricttheRange . . . . . . . . . . . . . . . . . . . . . . . . . 130 11.4 Aperfectfit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131 12.TheTroublewithTypedErrors . . . . . . . . . . . . . . . . . . . 132 12.1 Monolithicerrortypesarebad . . . . . . . . . . . . . . . . . 137 12.2 Boilerplatebegone! . . . . . . . . . . . . . . . . . . . . . . . . 138 12.3 TypeClassesToTheRescue! . . . . . . . . . . . . . . . . . . . 140 12.4 Thevirtueofuntypederrors . . . . . . . . . . . . . . . . . . . 141 13.Exceptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142 13.1 ExceptionsInFiveMinutes. . . . . . . . . . . . . . . . . . . . 142 13.2 BestPractices . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144 13.3 Hierarchies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149 13.4 Reinventing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150 13.5 AsynchronousExceptions . . . . . . . . . . . . . . . . . . . . 162 13.6 TheTheory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165 13.7 HasCallStack . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172 14.EDSLDesign . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174 CONTENTS 14.1 Trickswithdo . . . . . . . . . . . . . . . . . . . . . . . . . . . 175 14.2 OverloadedLiterals . . . . . . . . . . . . . . . . . . . . . . . . 182 14.3 TypeInferenceTrick . . . . . . . . . . . . . . . . . . . . . . . 187 14.4 FluentInterfaces . . . . . . . . . . . . . . . . . . . . . . . . . . 191 14.5 CaseStudy:WeightliftingLogging . . . . . . . . . . . . . . . 194 14.6 CaseStudy:rowdy . . . . . . . . . . . . . . . . . . . . . . . . . 200 14.7 CaseStudy:hspec . . . . . . . . . . . . . . . . . . . . . . . . . 205 15.GrowingPains . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209 15.1 ATaxonomyofBreakingChanges . . . . . . . . . . . . . . . 209 15.2 AvoidingBreakingChanges . . . . . . . . . . . . . . . . . . . 214 15.3 CommunicatingToUsers. . . . . . . . . . . . . . . . . . . . . 232 IV Interfacing the Real . . . . . . . . . . . . . .238 16.Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239 16.1 LibrariesandTools . . . . . . . . . . . . . . . . . . . . . . . . 239 16.2 DesigningCodeforTesting . . . . . . . . . . . . . . . . . . . . 245 17.LoggingandObservability . . . . . . . . . . . . . . . . . . . . . . 259 17.1 OnDebug.Trace . . . . . . . . . . . . . . . . . . . . . . . . . . 259 17.2 PreferdoNotation . . . . . . . . . . . . . . . . . . . . . . . . . 263 17.3 LoggingContexts . . . . . . . . . . . . . . . . . . . . . . . . . . 265 17.4 LibrariesinBrief . . . . . . . . . . . . . . . . . . . . . . . . . . 269 18.Databases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 272 18.1 SeparateDatabaseTypes . . . . . . . . . . . . . . . . . . . . . 273 18.2 Migrations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 284 18.3 AccessPatterns. . . . . . . . . . . . . . . . . . . . . . . . . . . 285 18.4 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 302 V Advanced Haskell . . . . . . . . . . . . . . . . .303 19.TemplateHaskellIsNotScary . . . . . . . . . . . . . . . . . . . . 304 19.1 ABeginnerTutorial . . . . . . . . . . . . . . . . . . . . . . . . 304 19.2 waitthisisn’thaskellwhatamidoinghere . . . . . . . . . . 306 19.3 ConstructinganAST . . . . . . . . . . . . . . . . . . . . . . . 306 19.4 BoilerplateBeGone! . . . . . . . . . . . . . . . . . . . . . . . . 312 CONTENTS 20.BasicTypeLevelProgramming . . . . . . . . . . . . . . . . . . . 323 20.1 TheBasicTypes . . . . . . . . . . . . . . . . . . . . . . . . . . 323 20.2 TheHigherKinds . . . . . . . . . . . . . . . . . . . . . . . . . 324 20.3 DynamicallyKindedProgramming . . . . . . . . . . . . . . . 328 20.4 DataKinds. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 329 20.5 GADTs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 330 20.6 Vectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 331 20.7 TypeFamilies. . . . . . . . . . . . . . . . . . . . . . . . . . . . 334 20.8 ThisSucks. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 340 20.9 HeterogeneousLists . . . . . . . . . . . . . . . . . . . . . . . . 340 20.10InductiveTypeClassInstances . . . . . . . . . . . . . . . . . 342 20.11ExtensibleRecords . . . . . . . . . . . . . . . . . . . . . . . . 343 20.12Likewhatyouread? . . . . . . . . . . . . . . . . . . . . . . . . 347 21.FamilyValues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 348 21.1 TypeFamilies. . . . . . . . . . . . . . . . . . . . . . . . . . . . 348 21.2 OpenorClosedTypeFamilies? . . . . . . . . . . . . . . . . . 354 21.3 TheBridgeBetweenWorlds . . . . . . . . . . . . . . . . . . . 355 21.4 DataFamilies . . . . . . . . . . . . . . . . . . . . . . . . . . . . 357 21.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 360 22.Trade-offsinTypeProgramming . . . . . . . . . . . . . . . . . . 361 22.1 MPTCs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 361 22.2 MPTCs+Fundeps . . . . . . . . . . . . . . . . . . . . . . . . . 364 22.3 AssociatedTypes . . . . . . . . . . . . . . . . . . . . . . . . . 370 22.4 Comparisons . . . . . . . . . . . . . . . . . . . . . . . . . . . . 371 23.CaseStudy:Prairie . . . . . . . . . . . . . . . . . . . . . . . . . . . 379 23.1 ProblemStatement: . . . . . . . . . . . . . . . . . . . . . . . . 380 23.2 PriorArt . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 382 23.3 TheGADTApproach. . . . . . . . . . . . . . . . . . . . . . . . 384 23.4 Improvements? . . . . . . . . . . . . . . . . . . . . . . . . . . 407 23.5 Symbols . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 409 23.6 CompareandConstrast . . . . . . . . . . . . . . . . . . . . . . 423 23.7 IdentifytheIssue. . . . . . . . . . . . . . . . . . . . . . . . . . 425 23.8 GeneralizeaGADT . . . . . . . . . . . . . . . . . . . . . . . . . 425 23.9 Fundeps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 426 23.10MoreClassPlease . . . . . . . . . . . . . . . . . . . . . . . . . 430 23.11RefiningtheClass . . . . . . . . . . . . . . . . . . . . . . . . . 431 23.12TemplateHaskell . . . . . . . . . . . . . . . . . . . . . . . . . 433 CONTENTS 23.13Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 442 Introduction An Opinionated Tour Guide So you’ve learned Haskell. You’ve taught your friends about monads, you’ve worked through some beginner textbooks, and maybe you’ve played around with some open source projects. Now that you’ve had a taste,youwantmore:youwanttowriteanapplicationinHaskellforfun! MaybeyouwanttouseHaskellatwork! Yousitdownatyourcomputer,andyou’restuck. How does anyone actually get anything done with this lan- guage? Thisisacommonthingtowonder. Haskellhasalwaysenjoyedawidevarietyofhighqualitylearningmate- rialforadvancedpartsofthelanguage,ifyou’renotafraidofacademic papers. Many people have created fantastic resources for beginners in thelastfiveyears.However,thelanguagedoesnothavemanyresources for using it in production. The Haskell ecosystems can be difficult to navigate.Therearemanyresourcesofvaryingqualitywithambiguous goalsandvalues.Identifyingtherightadviceisnearlyaschallengingas findingitinthefirstplace. Haskellisahugelydiverselandscape.Therearemanyregionalgroups: UnitedKingdom,Scandinavia,mainlandEurope,Russia,theUSA,Japan, China,andIndiaallhavethrivingHaskellecosystemsthathaveinterest- ingdialectsanddifferencesincustomandpractice. People come to Haskell with many backgrounds. Some people learned Haskellwellintotheircareers,andhadalongcareerwritingJava,Scala, orC#beforehand.SomepeoplecametoHaskellfromdynamicallytyped languages,likeLISPorRuby.SomepeoplestartedlearningHaskellearly i Introduction ii on in their programming career, and use it as the basis of comparison. SomepeopleprimarilyuseHaskellinacademicresearch,whileothers primarilyuseHaskellinindustrialapplications.Somepeoplearehobby- istsandjustliketowriteHaskellforfun! ThisbookisintendedforpeoplethatwanttowriteHaskellinindustry. The trade-offs and constraints that industrial programmers face are differentfromacademicorhobbyistprogrammers.Thisbookwillcover not only technical aspects of the Haskell language, but also social and engineeringconcernsthataren’t“really”aboutHaskell. Part of this book will be objective. I will teach you how to use some interestingtechniquesandideastomakedevelopingwithHaskellmore productive.We’lllearnaboutTemplateHaskell,type-levelprogramming, andotherfuntopics. However,forthemostpart,thisbookisinherentlysubjective.Because Haskell serves so many ecosystems, it is imperative to discern what ecosystem a something is intended for. More than just giving out pre- scriptions-“Thislibraryisproductionready!Thisisatoy!”-Ihopeto show my thought process and allow you to make your own judgment calls. Ultimately,thisisabookaboutthesocialrealityofsoftwareengineering inanichelanguage. Afterreadingthisbook,youshouldfeelcomfortable: • WritinglargesoftwareprojectsinHaskell • Evaluatingcompetinglibrariesandtechniques • ProductivelyreadingmaterialfromavarietyofHaskellusers About the Author I’mMattParsons. IstartedlearningprogramminginJanuary2014withComputerScience 101 at the University of Georgia. At the time, I was working for the IT department,installingWindowsandtroubleshootingprinters.Myman- agerdislikedmeandmadeitclearthathe’dthrowmeunderthebusat