Securing Advanced Metering Infrastructure (AMI) in Smart Grid using Intrusion Detection System (IDS) By Mustafa Amir Faisal A Thesis Presented to the Masdar Institute of Science and Technology in Partial Fulfillment of the Requirements for the Degree of Master of Science In Materials Science and Engineering © 2012 Masdar Institute of Science and Technology All rights reserved Abstract Advancedmeteringinfrastructure(AMI)isanimperativecomponentofsmartgrid, as it is responsible for collecting, measuring, analyzing energy usage data, and transmittingthesedatatothedataconcentratorandthentoacentralsysteminthe utility side. Therefore, the security of AMI is one of the most concerning issues in smart grid implementation. In this research, we propose an intrusion detection system (IDS) architecture for AMI which will act as a complimentary with other security measures. This IDS architecture consists of three local IDSs placed in smartmeters,dataconcentrators,andcentralsystem(AMIheadend). For detecting anomaly, we use stream data mining approach on public KD- DCUP 1999 data set for each three components in AMI. Seven stream mining classifiersarestudiedandtheirfeasibilityisanalyzedforthesecomponents. From our result and analysis, stream data mining techniques show promising potential for solving security issues in AMI. Moreover, possible required characteristics of algorithmsareidentifiedforthesecomponentsforfutureresearch. ThisresearchwassupportedbytheGovernmentofAbuDhabitohelpfulfillthe visionofthelatePresidentSheikhZayedBinSultanAlNayhanforsustainable developmentandempowermentoftheUAEandhumankind. Acknowledgments Firstofall, IthankthealmightyAllahtogivethestrengthtofulfilthistask. Then my gratitude goes to my family members without their help and support, it is im- possible. I would like to take this opportunity to thank those who actively guided and helpedmeinthisresearch. Foremost,Iwouldliketoexpressmydeepappreciation tomyadvisorDr. ZeyarAungforhiscontinuoussupportformyM.Sc. studyand research. His guidance, patience, motivation, and support helped me to develop a deep understanding of the subject. Besides my advisor, I would like to thank my thesissupervisorycommitteemembers: Dr. WeiLeeWoonandDr. JacobCrandall fortheirvaluabletime,comments,andadvices. Finally, and most importantly, once again my special thanks go to my family membersfortheirsupportandlove. Idedicatethisthesistomymother. MustafaAmirFaisal, MasdarInstitute,May15,2012. Contents 1 Introduction 1 1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.2 Thesisstatement–Objectives. . . . . . . . . . . . . . . . . . . . 3 1.3 ResearchContributions . . . . . . . . . . . . . . . . . . . . . . . 3 1.4 RelevancetoMasdar . . . . . . . . . . . . . . . . . . . . . . . . 4 1.5 ThesisOrganization . . . . . . . . . . . . . . . . . . . . . . . . . 4 2 AdvancedMeteringInfrastructure 5 2.1 FeaturesandBenefits . . . . . . . . . . . . . . . . . . . . . . . . 5 2.2 AMIComponents . . . . . . . . . . . . . . . . . . . . . . . . . . 6 3 LiteratureReview 10 3.1 AMIsecurityIssues . . . . . . . . . . . . . . . . . . . . . . . . . 10 3.2 StreamDataMininginIntrusionDetectionSystem . . . . . . . . 12 3.3 IntrusionDetectionSysteminAMI . . . . . . . . . . . . . . . . . 13 4 ProposedIDSArchitectureforAMI 15 6 4.1 Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 4.2 ProposedArchitecture . . . . . . . . . . . . . . . . . . . . . . . 16 5 ExperimentalSetUp 20 5.1 DataSet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 5.2 UsedTools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 5.2.1 WaikatoEnvironmentforKnowledgeAnalysis(WEKA) . 23 5.2.2 MassiveOnlineAnalysis(MOA). . . . . . . . . . . . . . 23 5.3 EvaluationProcedures . . . . . . . . . . . . . . . . . . . . . . . 25 5.3.1 EvaluatePrequential . . . . . . . . . . . . . . . . . . . . 25 5.3.2 EvaluateInterleavedTestThenTrain . . . . . . . . . . . . . 26 5.4 EvaluationMetrics . . . . . . . . . . . . . . . . . . . . . . . . . 26 5.4.1 Accuracy . . . . . . . . . . . . . . . . . . . . . . . . . . 26 5.4.2 KappaStatistic . . . . . . . . . . . . . . . . . . . . . . . 27 5.4.3 FalsePositiveRate(FPR)andFalseNegativeRate(FNR) 27 5.4.4 Evaluationtime(seconds) . . . . . . . . . . . . . . . . . 28 5.4.5 Modelcost(RAM-Hours) . . . . . . . . . . . . . . . . . 28 5.4.6 Modelsize(KiloBytes) . . . . . . . . . . . . . . . . . . 28 5.5 AlgorithmsExplored . . . . . . . . . . . . . . . . . . . . . . . . 28 5.5.1 HoeffdingTree . . . . . . . . . . . . . . . . . . . . . . . 29 5.5.2 AdaptiveWindowing(ADWIN) . . . . . . . . . . . . . . 30 5.5.3 AccuracyUpdatedEnsemble(AUE) . . . . . . . . . . . . 31 5.5.4 ActiveClassifier . . . . . . . . . . . . . . . . . . . . . . 33 5.5.5 LeveragingBagging . . . . . . . . . . . . . . . . . . . . 35 5.5.6 LimitedAttributeClassifier(LimAttClassifier) . . . . . . 36 5.5.7 OzaBaggingwithADWIN(OzaBagAdwin) . . . . . . . 38 5.5.8 Oza Bagging with Adaptive Size Hoffding Tree (OzaBa- gASHT) . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 5.5.9 SingleClassifierDrift . . . . . . . . . . . . . . . . . . . 40 5.6 ExperimentalFlow . . . . . . . . . . . . . . . . . . . . . . . . . 42 5.6.1 DataPreprocessing . . . . . . . . . . . . . . . . . . . . . 42 5.6.2 ExperimentalEnvironment . . . . . . . . . . . . . . . . . 42 5.6.3 Tuningparametervalues . . . . . . . . . . . . . . . . . . 43 6 ResultsandAnalysis 45 6.1 FullVersionDataSet . . . . . . . . . . . . . . . . . . . . . . . . 45 6.2 ImprovedVersionDataSet . . . . . . . . . . . . . . . . . . . . . 53 7 Discussion 59 7.1 Identifyingsuitablealgorithms . . . . . . . . . . . . . . . . . . . 59 7.2 Comparisonwithexistingworks . . . . . . . . . . . . . . . . . . 62 7.3 EvaluationofOurproposedArchitecture . . . . . . . . . . . . . . 62 8 FutureWork 64 9 Conclusion 66 A TunedParameterValues 68 A.1 TunedparametervaluesforFullVersionDataSet . . . . . . . . . 68 A.2 TunedparametervaluesforImprovedDataSet . . . . . . . . . . 69 B Abbreviations 72 List of Tables 4.1 Characteristicsofsmartmeter,dataconcentrator,andAMIheadend. 16 5.1 Attacksintrainandtestdatasets . . . . . . . . . . . . . . . . . . 21 5.2 BasicfeaturesofindividualTCPconnections . . . . . . . . . . . 21 5.3 Contentfeatureswithinaconnectionsuggestedbydomainknowl- edge . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 5.4 Trafficfeaturescomputedusingatwo-secondtimewindow . . . . 22 5.5 StatisticsforKDDCup99DataSet. . . . . . . . . . . . . . . . . 23 5.6 Variousattacks’samplesizesforimprovedversiondataset. . . . . 23 5.7 ConfusionMatrix . . . . . . . . . . . . . . . . . . . . . . . . . . 27 5.8 StepsforLimitedAttributeClassifier . . . . . . . . . . . . . . . . 37 5.9 ExperimentalEnvironment . . . . . . . . . . . . . . . . . . . . . 42 6.1 Performancecomparisonforclassifiersforfullversion. . . . . . . 46 6.2 PerformancecomparisonforclassifiersforTestdataset. . . . . . 47 6.3 Performance comparison for classifiers for improved version of trainingdataset. . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 9 6.4 Performancecomparisonforclassifiersforimprovedversion. . . . 53 7.1 Orderofclassifiersaccordingtotheirperformanceforfullversion testdata. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 7.2 Order of classifiers according to their performance for improved versiontestdataset. . . . . . . . . . . . . . . . . . . . . . . . . . 60 A.1 TunedparametervaluesforAccuracyUpdatedEnsemble. . . . . . 68 A.2 TunedparametervaluesforActiveClassifier . . . . . . . . . . . . 69 A.3 TunedparametervaluesforOzaASHT. . . . . . . . . . . . . . . . 69 A.4 TunedparametervaluesforLimAttClassifier. . . . . . . . . . . . 69 A.5 TunedparametervaluesforOzaBagAdwin. . . . . . . . . . . . . 69 A.6 TunedparametervaluesforSingleClassifierDrift. . . . . . . . . . 69 A.7 TunedparametervaluesforLeveragingBag. . . . . . . . . . . . . 69 A.8 TunedparametervaluesforAccuracyUpdatedEnsemble. . . . . . 70 A.9 TunedparametervaluesforActiveClassifier. . . . . . . . . . . . 70 A.10 TunedparametervaluesforLeveragingBag. . . . . . . . . . . . . 70 A.11 TunedparametervaluesforLimAttrClassifier. . . . . . . . . . . . 70 A.12 TunedparametervaluesforLimAttrClassifier(numAtts). . . . . . 70 A.13 TunedparametervaluesforOzaBagAdwin. . . . . . . . . . . . . 70 A.14 TunedparametervaluesforOzaBagASHT. . . . . . . . . . . . . 71 A.15 TunedparametervaluesforSingleClassifierDrift. . . . . . . . . . 71 List of Figures 2.1 OverviewofAMIcomponentsandnetworks. . . . . . . . . . . . 9 4.1 (a)SmartmeterwithIDS.(b)IDSforasmartmeter. . . . . . . . . 17 4.2 ArchitectureofwholeIDSinAMI. . . . . . . . . . . . . . . . . . 18 4.3 IntrusiondetectionprocedurefromsmartmetertoAMIheadend. . 18 5.1 Classificationcycleofdatastream[40] . . . . . . . . . . . . . . . 25 5.2 Pseudo-codeforADWIN[39]. . . . . . . . . . . . . . . . . . . . 32 5.3 Pseudo-codeforAUE[20]. . . . . . . . . . . . . . . . . . . . . . 34 5.4 StrategyFrameworkActiveLearning[20]. . . . . . . . . . . . . 34 5.5 LeveragingBagging[17]. . . . . . . . . . . . . . . . . . . . . . . 36 5.6 ExperimentalFlow(lefttoright). . . . . . . . . . . . . . . . . . . 43 6.1 Accuracyforfullversiontestdatasetwithtunedparametervalues. 48 6.2 Correctdetectionsofvariousattacksfortestdataset. . . . . . . . 48 6.3 Memoryconsumedbyvariousclassifiers(TestDatasetforfullver- sion). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 11