TUUKKA KATAJA, JUHO TEPERI IMPROVING AND EVALUATING CLOJURE WEB APPLICATION ARCHITECTURE Master of Science Thesis Examiner: Prof. Hannu-Matti Järvinen Examiner and topic approved by the Faculty Council of the Faculty of Computing and Electrical Engineering on 3rd June, 2015 I ABSTRACT TUUKKA KATAJA, JUHO TEPERI:ImprovingandEvaluatingClojureWebApplica- tionArchitecture Tampere University of Technology Master of Science Thesis, 78 pages, 10 Appendix pages June 2016 Master’s Degree Programme in Information Technology Major: Software Engineering, Pervasive Systems Examiner: Prof. Hannu-Matti Järvinen Keywords: software architecture, web applications, Decision-Centric Architecture Review, Clojure In this study, the software architecture of Clojure-based web applications was improved, based on findings in previous projects where the authors have been involved. These projects are briefly introduced to provide context for the proposed improvements. Then, potential solutions were evaluated and a web application was built based on these solu- tions. In addition, some new fundamental architecture-related ideas were studied to gain more understanding of how to implement forthcoming projects. As a result, a reference architecturewascreatedbasedontheimplementedexampleapplication. Decisions related to the reference architecture were then evaluated using the Decision- Centric Architecture Review method. This method produced structured documentation about the feasibility of architectural choices. Based on the results, some decisions were considered beneficial and should be taken into account in future projects whereas some wereconsideredtohaveminorimpactorevenmajornegativeimpact. II TUUKKAKATAJA,JUHOTEPERI:Clojureweb-sovellustenarkkitehtuurinkehitysja arviointi Tampereen teknillinen yliopisto Diplomityö, 78 sivua, 10 liitesivua Kesäkuu 2016 Tietotekniikan koulutusohjelma Pääaine: Ohjelmistotuotanto, Pervasive Systems Tarkastajat: Prof. Hannu-Matti Järvinen Avainsanat: ohjelmistoarkkitehtuuri, web-sovellukset, Decision-Centric Architec- ture Review, Clojure Tässä työssä kehitettiin Clojure-pohjaisten web-sovellusten ohjelmistoarkkitehtuuria pe- rustuen havaintoihin projekteissa, joissa työn tekijät ovat olleet mukana. Projektit esitel- lään lyhyesti taustatietona tarvittaville parannuksille. Työssä käydään läpi erilaisia mah- dollisia ratkaisuja ja työn tuloksena syntyi esimerkkisovellus. Kokemusten perusteella tehtyjen parannusten lisäksi työssä käsitellään uusia perustavanlaatuisia arkkitehtuuri- ratkaisuita, joita voidaan mahdollisesti hyödyntää tulevissa projekteissa. Työn tuloksena syntyiviitearkkitehtuuri,jokapohjautuutoteutettuunesimerkkisovellukseen. Työssä tehdyt arkkitehtuuripäätökset arvioidaan DCAR-menetelmällä. Kyseinen mene- telmän avulla päätöksistä saatiin jäsennettyä dokumentaatiota arkkitehtuuriparannusten kelvollisuudesta. Tulosten perusteella joitakin hyödyllisiä arkkitehtuuripäätöksiä kannat- taisi ottaa huomioon tulevissa projekteissa. Jotkin päätöksistä olivat vähäpätöisempiä tai niilläolijopamerkittävänegatiivinenvaikutus. III TABLE OF CONTENTS 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 2. Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 2.1 AboutMetosinLtd . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 2.2 AboutClojure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 2.3 AboutClojureScript . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 2.4 CaseProjects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 2.4.1 CaseX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 2.4.2 CaseY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 2.4.3 CaseZ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 2.5 FocalPointsforArchitecturalImprovements . . . . . . . . . . . . . . . . 11 2.6 SpecificationofanExampleProject . . . . . . . . . . . . . . . . . . . . 13 3. Decision-CentricArchitectureReview . . . . . . . . . . . . . . . . . . . . . . 15 4. EvaluatingDataPersistenceandAPISolutions . . . . . . . . . . . . . . . . . 19 4.1 DataPersistence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 4.1.1 Document-model(MongoDB) . . . . . . . . . . . . . . . . . . . . . 19 4.1.2 RelationalMixedwithDocumentModel(PostgreSQL) . . . . . . . 21 4.1.3 Fact-BasedTemporalDatabase(Datomic) . . . . . . . . . . . . . . 22 4.1.4 StoringEventsandEventSourcing . . . . . . . . . . . . . . . . . . 24 4.1.5 CapturingDataChangeswithPostgreSQLandBottledWater . . . . 26 4.2 DatabaseChoicesfortheExampleApplication . . . . . . . . . . . . . . 28 4.3 ApplicationProgrammingInterface(API) . . . . . . . . . . . . . . . . . 29 4.3.1 HTTP,RepresentationalStateTransfer,RESTHTTPAPIs . . . . . . 30 4.3.2 Compojure-apiLibraryforBuildingWebAPIs . . . . . . . . . . . . 32 4.3.3 RemoteProcedureCalls(RPC)overHTTP . . . . . . . . . . . . . . 34 4.3.4 KekkonenLibrary . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 IV 4.3.5 SideEffectsofCommandsandReal-TimeEvents . . . . . . . . . . 38 4.4 BackendTechnologyChoicesforExampleApplication . . . . . . . . . . 40 5. EvaluatingFrontendTechnologies . . . . . . . . . . . . . . . . . . . . . . . . 41 5.1 WebDevelopmentBackground . . . . . . . . . . . . . . . . . . . . . . . 41 5.1.1 Model-View-Controller . . . . . . . . . . . . . . . . . . . . . . . . 42 5.2 RenderingandStateManagement . . . . . . . . . . . . . . . . . . . . . 43 5.2.1 React . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 5.2.2 Om . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 5.2.3 Reagent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 5.2.4 Re-frame . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 5.3 FetchingData . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 5.3.1 PreviouslyUsedApproaches . . . . . . . . . . . . . . . . . . . . . 54 5.3.2 Relay . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 5.3.3 Om.next . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 5.4 FrontendTechnologyChoicesforExampleApplication . . . . . . . . . . 59 6. Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 6.1 Backend . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 6.1.1 DataPersistence . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 6.1.2 API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 6.1.3 BackendImplementationConclusion . . . . . . . . . . . . . . . . . 68 6.2 Frontend . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69 6.2.1 DataFetchingImplementation . . . . . . . . . . . . . . . . . . . . . 69 6.2.2 LiveUpdates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72 6.2.3 ComponentSchemaValidation . . . . . . . . . . . . . . . . . . . . 73 6.2.4 FrontendImplementationConclusion . . . . . . . . . . . . . . . . . 74 7. ArchitectureEvaluationResults . . . . . . . . . . . . . . . . . . . . . . . . . 75 7.1 EvaluationSessionandResults . . . . . . . . . . . . . . . . . . . . . . . 75 V 8. Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79 A. Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85 B. ArchitectureDecisions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86 VI LIST OF FIGURES 2.1 High-leveloverviewofthesystemX. . . . . . . . . . . . . . . . . . . . 6 3.1 SummarizationofDCARreviewstepsandartifactsproducedduringeach step. Adaptedfrom[18,p.72]. . . . . . . . . . . . . . . . . . . . . . . . 18 4.1 Overview of the interaction between PostgreSQL, Bottled Water exten- sion,Kafkaandtheapplicationbackend. . . . . . . . . . . . . . . . . . . 28 4.2 High-levelconceptualviewofKekkonenlibrary . . . . . . . . . . . . . . 36 4.3 DiagramofinteractionbetweenthebackendandthefrontendoverHTTP andWebSocket. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 5.1 MessageflowbetweenpartsoftheMVCmodel. Adaptedfrom[62,p.5]. . 42 5.2 DiagramofhowOmcursorscanbeusedtoacccesssubtreesofapplication state. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47 5.3 VisualizationofdataflowinRe-framearchitecture. Adaptedfrom[66]. . . 52 6.1 Sequence diagram depicting how command triggered from user interface isprocessed. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71 VII LIST OF TABLES 2.1 Architecture-related patterns and their presence in each case project and thereferencearchitecture . . . . . . . . . . . . . . . . . . . . . . . . . . 13 4.1 JSONdatatypeimprovementsinPostgreSQLdatabase . . . . . . . . . . 21 6.1 Exampleoftokendata . . . . . . . . . . . . . . . . . . . . . . . . . . . 62 7.1 SummaryoftheresultsoftheDecision-CentricArchitectureReview . . . 76 VIII LIST OF PROGRAMS 4.1 BasicCompojure-apiexampleapplication . . . . . . . . . . . . . . . . . 33 4.2 BasicKekkonenexampleusingtheCQRSmodel . . . . . . . . . . . . . 37 5.1 SimpleReactcomponent . . . . . . . . . . . . . . . . . . . . . . . . . . 43 5.2 BasicOmexample . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 5.3 BasicReagentexample . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 5.4 Reagentlocalstateexample . . . . . . . . . . . . . . . . . . . . . . . . . 50 5.5 Reagentreactionexample . . . . . . . . . . . . . . . . . . . . . . . . . . 51 5.6 Codedemonstratingdatainatreeformatwithduplicatedentities. . . . . 58 5.7 Codedemonstratingnormalizeddatainagraphformat. . . . . . . . . . . 58 6.1 ExampleSQLtodemonstratechangedatacapturing. . . . . . . . . . . . 64 6.2 ExampleofaKekkonenhandlerforaddingnewaccount. . . . . . . . . . 64 6.3 Example of a Kekkonen handler for adding new account with explicit WebSocketbroadcast. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 6.4 ExampleofKekkonenCQRSAPIandinterceptors . . . . . . . . . . . . 67 6.5 Log output produced by interceptors when invoking Kekkonen handler definedinthelisting6.4 . . . . . . . . . . . . . . . . . . . . . . . . . . . 68 6.6 Exampleofdatafetchingfrontendcode . . . . . . . . . . . . . . . . . . 70 6.7 Examplefrontendcodeforremotemutationimplementation . . . . . . . 71 6.8 Example of re-frame subscription used built data for components from applicationstate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72 6.9 ExampleofSchemaannotatedReagentComponentfunction . . . . . . . 73 IX LIST OF ABBREVIATIONS AND TERMS ACID Atomicity,Consistency,IsolationandDurabilitypropertiesofadatabase API Application Programming Interface provides means to interact with the softwarecomponent. AST AbstractSyntaxTreeisaninternaldatarepresentationofparsedcode. ATAM ArchitectureTradeoffAnalysisMethodisanarchitecturereviewmethod. CQRS Command-QueryResponsibilitySegregationisanAPIdesignpattern. DCAR Decision-CentricArchitectureReviewisanarchitecturereviewmethod. DOM Document Object Model is an interface to manipulate browser docu- mentcontents. EDN ExtensibleDataNotationisanextensibledataformat. FRP FunctionalReactiveProgrammingisafunctionalprogrammingpattern. JSON JavaScriptObjectNotationisadataformat. JSONB PostgreSQL’sinternalformatofJSONthatallowe.g. efficientindexing. JSX JavascriptsyntaxextensionthatenablesinlinedmarkupforReact. JVM JavaVirtualMachine MVC Model-View-Controller is an architecture pattern designed for UI pro- gramming. REPL Read-Eval-Print-Loopallowsinteractionwithcompiler/interpreter. REST Representational State Transfer is an architectural style for distributed (hypermedia)applications RPC RemoteProcedureCallallowsinvokingproceduresovernetwork. SPA Single-Page Application is a browser application written in JavaScript andrequiringonlyinitialpageload. VDOM VirtualpresentationofDOMthate.g. Reactuses. WS WebSocketsallowTCP/IPlikecommunicationinwebapplication. Avro Dataserializationsystem Backend ServerapplicationthatusuallyprovidesAPIforclient(s). Frontend User-facingclientapplicationthatoftencommunicateswithbackend. AuditLogging Keepingtrackofeventstoe.g. identifywhodidwhatandwhen. React Javascriptrenderinglibrary S-expression Notationforexpressingnestedlists Lisp One of the oldest high-level programming languages where code and dataareexpressedsimilarlyusingS-expressions.