Tradeoffs when choosing an Application Performance Management (APM) solutions CMG Boston Region - April 2015 Meeting Rob Jahn is a senior performance architect and Joe Sicree is a technical architect at Collaborative Consulting, a business and technology consulting firm based in Burlington, MA. They have experience assessing, designing, building, and using a variety of APM tools on many enterprise applications. E: [email protected] T: @RobJahnBoston E: [email protected] T: @JoeSicree Abstract: – Having an APM solution is no longer an option for most enterprises given the growing SOA complexity, use of cloud services and hosting provides, and push for DevOps culture. – Fortunately, the APM space has matured in depth and sophistication to monitor production environments. – However like most IT efforts, APM solutions are also budget and time constrained so trade-offs must be made. – This presentation will provide an overview of APM, trade-offs faced when deciding on which solution to pick, and examples of trade-offs made from several customer implementations. Agenda • What is APM? • Some APM Vendors • What comprise APM? • Benefits • APM tradeoffs area • Summary • Appendix: Reference URLs What is APM? Application Performance Management (APM) is the monitoring and management of performance and availability of software applications. APM strives to detect and diagnose application performance problems to maintain an expected level of service. APM is "the translation of IT metrics into business meaning ([i.e.] value).“ http://en.wikipedia.org/wiki/Application_performance_management Green = Vendors in Gartner APM Some APM Vendors Magic Quadrant – AccelOps – Hagrid – Riverbed – AppNeta – HP – SAS – Appnomic – IBM – Savision – Apptio – INETCO – Sightline Systems – App Dynamics – Interlink Software – SmartBear – Aternity – KeyNote – SolarWinds – Bay Dynamics – Loggly – Splunk – BlueStripe – ManageEngine – Sumerian – BMC Software – Metafor Software – Sumo – Boundary – Microsoft – CA Technologies – Moogsoft – TeamQuest – Circonus – Nastel Technologies – Teleran – Correlsense – NetScout Systems – Terma – Corvil – Netuitive – VMTurbo – Dynatrace – New Relic – VMWare (Compuware) – Nexthink – XpoLog – Evolven – Prelert – Zenoss – ExtraHop – RISC Networks What comprises APM? • User Experience (UEM) End-user experience • Passive (Span Network Port) monitoring • Active (Synthetic Script) • Mapping Business transaction • Grouping • SLAs • SLA Management IT operations • capacity/trending/profiling analytics/reporting • standard reports Runtime application • Top down/bottom up from/to topology from/to architecture discovery transactions modeling and display • ADDM Component deep-dive • Agents on JVM/.NET monitoring in application • Middleware context https://www.gartner.com/doc/2889421/magic-quadrant-application-performance- monitoring?docdisp=share&srcId=1-4398736771 APM Observations • APM implementation takes real effort and there are pit falls. • No one-sized fits all • APM Benefits are often not understood or quantified • Often a tool is just added after production go-live without it trades off being thought through • New requirements may challenge tool choice • One APM tool can’t do it all. Finding hot spot is great help, but it may not tell you not how to fix it. • APM should start with business/end user view • You really don’t have APM if you only have logs, system metrics, synthetic monitoring scripts Benefits • Business Transaction and SLA Management – Business can immediately relate and benefit from metrics – Way to hold a SaaS vendor, Content, or client-side provider accountable • End User Experience – Real user profiling of browser side code execution. Deeper analytics, JavaScript error defection, Clicks stream – Cloud and In-Premise solutions – Generally very quick to setup – More value when integrated to component Deep Dive Benefits , Continued • Analytics – Pre-built real-time analytics engines for dash boarding – Allows Top down & bottom up analysis of transactions and infrastructure in one tool – Tops tools some with good out of box application & business process centric dashboards • Component Deep Dive & Run-Time Topology view – Proven to run in production. Often Big Win – Export traces for off line analysis and comparison – Faster root sensors that instrument code and external calls – Quickly see cause analysis – Error, Volume, & Time spent by tier, SQL, Code, external calls – Auto and custom time spent, errors, volume by tier and component – Same too to see tier and component health What is your priority? Do you want to know that site is healthy and operating in normal limits? Do you just care when volume is above 1000 TPS? Do you care about click stream of user coming in from Philadelphia? Key Stakeholders Application Application Application IT Focus Area Owner or Line of Support Development Operations Business Business transaction and SLA Management End-user experience monitoring Problem Isolation and Remediation Performance improvement Production readiness & Test IT operations analytics/reporting