HEEE - Handler for Exceptionally Exceptional Exceptions Author(s) Vegard Solheim Olafur Johan M. Trollebø Lars Walter Westby Aleksander Steen Bachelor in Software Engineering 20 ECTS Department of Computer Science and Media Technology Norwegian University of Science and Technology, 18.05.2016 Supervisor(s) Ivar Farup HEEE-HandlerforExceptionallyExceptionalExceptions Sammendrag av Bacheloroppgaven Tittel: HEEE - Handler for Exceptionally Exceptional Excep- tions Dato: 18.05.2016 Deltakere: VegardSolheim OlafurJohanM.Trollebø LarsWalterWestby AleksanderSteen Veiledere: IvarFarup Oppdragsgiver: InnitAS Kontaktperson: JoakimJøreng,[email protected] Nøkkelord: Feilhåndtering,Javascript,Laravel,PHP,AngularJS Antallsider: 189 Antallvedlegg: 8 Tilgjengelighet: Åpen Sammendrag: All programvare har feil, hvorav mange programmer- ingsspråk uttrykker de som unntak. Innit AS drifter pro- gramvare for deres kunder, og foreløpig håndterer alle feilrapporter fra deres systemer manuelt. De ønsker der- for å få utviklet et system for å motta, prosesserer og or- ganiseredissefeilrapportene.Vårtsystem,H3E,vilutføre de oppgavene, og fra dette datasettet tilby en utvikler- orientertgrafiskoversiktoverallefeilrapporter,sådekan fokuserepååretteoppfeil. i HEEE-HandlerforExceptionallyExceptionalExceptions Summary of Graduate Project Title: HEEE - Handler for Exceptionally Exceptional Excep- tions Date: 18.05.2016 Authors: VegardSolheim OlafurJohanM.Trollebø LarsWalterWestby AleksanderSteen Supervisor: IvarFarup Employer: InnitAS ContactPerson: JoakimJøreng,[email protected] Keywords: Exceptions,Javascript,Laravel,PHP,AngularJS Pages: 189 Attachments: 8 Availability: Open Abstract: Allsoftwarehaserrors,whichinmanyprogramminglan- guages are expressed as exceptions. Innit AS is hosting softwareforitscustomers,andcurrentlymanuallyhasto handleexceptionreportsfromitssystems.Theytherefore wished to have developed a system to receive, processes and organise those exception reports from a multitude of externally hosted systems. Our system, H3E, will han- dle those tasks, and from that data provide a developer- centric graphical overview of all reports received for de- veloperstofocusonfixingissues i HEEE-HandlerforExceptionallyExceptionalExceptions Preface Wewouldliketothankallthepeoplethathashelpedusdevelopthissoftware.IvarFarup for providing excellent critique in the writing of this thesis report plus assistance in any developmentrelatedmatterwehad.JoakimJørengforbeingavailabletoanswerallour questions and thoughts at any time of the day and giving us the opportunity to develop this software for his company. Eivind Johansen for providing extremely helpful feed- back in regard to our graphical presentation. Øivind Kolloen and Mariusz Nowostawski providing valuable design pointers, guidelines and best practices in API creation plus technical assistance whenever we had issues with AngularJS. Rune Hjelsvold for giving us in-depth knowledge of database design and critical help during the development of ourdatabasetriggersandevents.Lastbutnotleast,alltheexceptionalandhighquality teacherswe’vehadoverthelastthreeyears. ii HEEE-HandlerforExceptionallyExceptionalExceptions Contents Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ii Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iii ListofFigures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.1 ProjectInception . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.2 ProjectDescription . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.2.1 Purpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.2.2 ProductObjectives . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.3 LearningObjectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.4 Audience . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 1.5 FieldofStudy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 1.6 ProjectOrganisation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 1.7 AcademicBackground . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 1.8 Glossary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 1.9 DocumentStructure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 2 ProjectManagement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 2.1 DevelopmentMethodology . . . . . . . . . . . . . . . . . . . . . . . . . . 8 2.2 TaskBoard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 2.3 Meetings. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 2.3.1 MeetingswithEmployer . . . . . . . . . . . . . . . . . . . . . . . . 11 2.3.2 MeetingswithSupervisor . . . . . . . . . . . . . . . . . . . . . . . 11 2.3.3 GroupMeetings. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 3 Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 3.1 Use-caseoverview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 3.2 DesignRequirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 3.3 DataRequirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 3.4 GeneralRequirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 3.5 UserTypes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 3.6 ArchitecturalDraft . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 4 TechnicalDesign . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 4.1 LanguagesandFrameworks . . . . . . . . . . . . . . . . . . . . . . . . . . 21 4.2 CodingConventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 4.3 Back-endArchitecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 4.4 Front-endArchitecturalOverview . . . . . . . . . . . . . . . . . . . . . . . 26 4.5 Syslog . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 iii HEEE-HandlerforExceptionallyExceptionalExceptions 4.6 DatabaseDesign . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 5 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 5.1 GraphicalDesignConsiderations . . . . . . . . . . . . . . . . . . . . . . . 33 5.2 ExceptionReportFormat . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 5.3 API-specification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 5.4 SystemAdministrationandConfiguration . . . . . . . . . . . . . . . . . . 39 5.5 Heat . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 5.5.1 HeatDegradation. . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 5.5.2 HeatDegradationRecalculation . . . . . . . . . . . . . . . . . . . . 47 5.5.3 HeatMapDataTransfer . . . . . . . . . . . . . . . . . . . . . . . . 48 5.5.4 HeatMapDataConversion . . . . . . . . . . . . . . . . . . . . . . 49 6 DevelopmentProcess . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50 6.1 Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50 6.2 DataModelDevelopment . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 6.3 IssuesandBugsEncountered . . . . . . . . . . . . . . . . . . . . . . . . . 55 6.3.1 Front-end . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 6.3.2 Back-end . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58 6.4 WorkingwithLiveData . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 7 TestingandQualityAssurance . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 7.1 CodeReview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 7.2 UnitTestingTools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 7.2.1 Front-end . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 7.2.2 Back-end . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64 7.3 StaticAnalysis. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64 7.4 IntegrationTesting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 7.5 Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 7.6 Demonstrations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68 7.7 GUIFeedbackDemonstration . . . . . . . . . . . . . . . . . . . . . . . . . 69 8 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70 8.1 TargetAchievements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70 8.1.1 AchievedLearningObjectives . . . . . . . . . . . . . . . . . . . . . 70 8.1.2 AchievedTaskObjectives . . . . . . . . . . . . . . . . . . . . . . . 71 8.1.3 AchievedProductObjectives . . . . . . . . . . . . . . . . . . . . . . 71 8.2 TeamandProcessDiscussion . . . . . . . . . . . . . . . . . . . . . . . . . 72 8.2.1 DivisionofWorkload . . . . . . . . . . . . . . . . . . . . . . . . . . 72 8.2.2 TimeScheduleAssessment . . . . . . . . . . . . . . . . . . . . . . 72 8.3 CritiqueandAlternativeApproaches . . . . . . . . . . . . . . . . . . . . . 75 8.3.1 Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75 8.3.2 Product . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77 8.4 FurtherWorkandExtendableFeatures . . . . . . . . . . . . . . . . . . . . 79 iv HEEE-HandlerforExceptionallyExceptionalExceptions 9 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81 Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82 A ProjectPlan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84 B ProjectContract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102 C MeetingSummaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104 C.1 Desember2015-Pre-MeetingwithInnit . . . . . . . . . . . . . . . . . . . 104 C.1.1 Thingstoread/learn . . . . . . . . . . . . . . . . . . . . . . . . . . 104 C.1.2 Whatsystemshoulddo . . . . . . . . . . . . . . . . . . . . . . . . 104 C.1.3 Questionsandanswers . . . . . . . . . . . . . . . . . . . . . . . . . 104 C.2 016.01.13-FirstMeetingwithInnit . . . . . . . . . . . . . . . . . . . . . 105 C.3 2016.01.19-SecondMeetingwithInnit . . . . . . . . . . . . . . . . . . . 108 C.4 2016.01.27-ThirdMeetingwithInnit . . . . . . . . . . . . . . . . . . . . 111 C.5 2016.03.07-FirstDemoMeetingwithInnit . . . . . . . . . . . . . . . . . 113 C.6 2016.04.26-SecondDemoMeetingwithInnit. . . . . . . . . . . . . . . . 114 C.7 2016.05.13-FinalDemoMeetingwithInnit . . . . . . . . . . . . . . . . . 116 C.8 2016.04.15-GUIFeedbackMeeting . . . . . . . . . . . . . . . . . . . . . 117 D APIStructure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118 D.1 Exceptionsroute . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118 D.2 ExceptionTypesroute . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119 D.3 ExceptionContextsroute . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119 D.4 SeverityRatingroute . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119 D.5 Applicationsroute . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119 D.6 Installationroute . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119 D.7 Reportsroute . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120 D.8 Statisticsroute . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120 D.9 Graphroute . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121 D.10Syslogroute . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121 D.11Supportedglobalargumentsacrossallroutes . . . . . . . . . . . . . . . . 122 E RefactoredCode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123 E.1 Front-end . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123 E.1.1 Grapher.Controller . . . . . . . . . . . . . . . . . . . . . . . . . . . 123 E.2 Back-end . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124 E.2.1 GraphController . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124 E.2.2 Refactoringtoadheretocodestandard . . . . . . . . . . . . . . . . 124 F Images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126 G Readme . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131 G.1 Front-end . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131 G.2 Back-end . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132 H TimeLog . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135 H.1 TimeLogVegard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135 v HEEE-HandlerforExceptionallyExceptionalExceptions H.2 TimeLogAleksander . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147 H.3 TimeLogLars . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155 H.4 TimeLogOlafur . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167 vi HEEE-HandlerforExceptionallyExceptionalExceptions List of Figures 1 Overviewofthesystem. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 2 ScreenshotoftheJIRAtaskboard,takenFebruary15th. . . . . . . . . . . 10 3 Generalusecasediagramoftherequiredfunctionality. . . . . . . . . . . . 13 4 Usecaseoverviewofthesystem’sexception-orientedfunctionality. . . . . 14 5 Usecaseoverviewofthesystem’ssyslog-relatedfunctionality. . . . . . . . 15 6 Anearlysketchoftheoverviewscreen.Heatmapistheshadedareainthe topright. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 7 Anearlysketchoftheoverviewscreenwithinformationonsitetraversal. . 17 8 Initialdatamodelrequirements.. . . . . . . . . . . . . . . . . . . . . . . . 18 9 Modulediagramwithback-endontheleftandfront-endontheright.. . . 20 10 Samplecodefordisplayingcodingconventionsoffront-end . . . . . . . . 24 11 Samplecodefordisplayingcodingconventionsofback-end . . . . . . . . 25 12 High-levelback-endarchitecturaloverview. . . . . . . . . . . . . . . . . . 27 13 Front-endarchitecturaloverview. . . . . . . . . . . . . . . . . . . . . . . . 28 14 Illustrationofsyslog. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 15 Modelofthemaindatabase. . . . . . . . . . . . . . . . . . . . . . . . . . . 31 16 Modeloftheread-onlysyslogdatabase. . . . . . . . . . . . . . . . . . . . 32 17 Front-end heat map design. The graph shows the heat over time for the currently selected applications. The bottom green bar is the heat map showing current heat for this application, with a number for the heat in absolutevalue. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 18 Front-end application overview design. The green bar is a heat map for this application. The graph shows number of exceptions of each type for this application. The right side shows all installation URLs for this appli- cation,andthebottom–whichcontinuesfurtherdownthepage–shows all the single exceptions for this application in descending order by time anddate. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 19 Formatforaddinganewexceptionreport . . . . . . . . . . . . . . . . . . 35 20 InitialdraftofAPIstructure. . . . . . . . . . . . . . . . . . . . . . . . . . . 37 21 FinalAPIstructure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 22 Exampleofcontentinfront-endconfigurationfile . . . . . . . . . . . . . . 40 23 Exampledatafrom.env-file . . . . . . . . . . . . . . . . . . . . . . . . . . 40 24 Codeforgeneratinglineardegradationgraphdata . . . . . . . . . . . . . 42 25 Usinglinearinputandlineardegradation,degradationvalueinthelegend 43 26 Codeforgeneratingexponentialinputanddegradationgraphdata . . . . 44 vii HEEE-HandlerforExceptionallyExceptionalExceptions 27 Using exponential input and exponential degradation. Top and bottom randomvaluesforinputmultiplierinthelegend. . . . . . . . . . . . . . . 44 28 Codeforgeneratinglinearinputandexponentialdegradationgraphdata . 45 29 Useslinearinputandexponentialdegradation.Multiplierinthelegend . . 45 30 Codeforgeneratinglinearinputandexponentialdegradationgraphdata . 46 31 Useslinearinputandexponentialdegradation.Multiplierinthelegend . . 46 32 Formatthe/graphroutereturns. . . . . . . . . . . . . . . . . . . . . . . . 58 33 Formatthe/graphroutereturnsforapplications. . . . . . . . . . . . . . . 59 34 Screenshot of the Karma coverage dashboard from March, mid-way in development.Eachrowisamoduleinthefront-end. . . . . . . . . . . . . 62 35 Samplecodeformockingaservice . . . . . . . . . . . . . . . . . . . . . . 64 36 ScreenshotoftheSonarresultsafterananalysis . . . . . . . . . . . . . . . 65 37 ScreenshotofSonar’sissueoverview . . . . . . . . . . . . . . . . . . . . . 66 38 OriginalGanttdiagramfromtheplanningstage . . . . . . . . . . . . . . . 73 39 ResultingGanttdiagramoftheactualtimeschedule . . . . . . . . . . . . 74 1 Concept-codeforheatrecalculation. . . . . . . . . . . . . . . . . . . . . . 126 2 Front-endoverview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127 3 Front-endgraphsettings . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127 4 Front-endsingleexceptionreport . . . . . . . . . . . . . . . . . . . . . . . 128 5 Front-endapplicationoverview . . . . . . . . . . . . . . . . . . . . . . . . 128 6 Front-endsyslogdisplay . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129 7 Front-endreportdisplay . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129 8 Front-endreportdisplay . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130 viii