Event and Alarm handling Application(EVA) version 2.0 TypesetinLATEXfromSGMLsourceusingtheDOCBUILDER3.2.2DocumentSystem. Contents 1 EVAUser’sGuide 1 1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.1.1 Architecture. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.2 Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.2.1 BasicEventandAlarmService. . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.2.2 LogControlService . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 1.2.3 EVALogService . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 1.3 EVAAdaptations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 1.3.1 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 1.4 EVASNMPIinterface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 1.4.1 EVASNMPAdaptation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 1.4.2 LOGSNMPInterface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 1.4.3 EVA-LOGSNMPInterface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 1.4.4 SNMPEALOGSNMPInterface . . . . . . . . . . . . . . . . . . . . . . . . . . 22 1.5 AppendixA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 1.5.1 Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 1.6 EVAReleaseNotes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 1.6.1 EVA-EventandAlarmHandlerv2.0.2.1 . . . . . . . . . . . . . . . . . . . . . 23 1.6.2 EVA-EventandAlarmHandlerv2.0.2. . . . . . . . . . . . . . . . . . . . . . . 23 1.6.3 EVA2.0.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 1.6.4 EVA2.0.0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 1.6.5 EVA1.0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 2 EVAReferenceManual 27 2.1 eva . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 2.2 eva . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 2.3 eva log . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 2.4 eva server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 2.5 eva sup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 2.6 log . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 2.7 log server. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47 EventandAlarmhandlingApplication(EVA) iii 3 EVAReferenceManual-SNMPadaptation 47 3.1 eva log snmp . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50 3.2 eva snmp adaptation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52 3.3 log snmp . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 3.4 log snmpea . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 Glossary 59 iv EventandAlarmhandlingApplication(EVA) Chapter 1 EVA User’s Guide TheEventandAlarmHandlingapplication,EVAisanoperationandmaintenance applicationinthe faultmanagementarea. Itcontainssupportforapplicationstosendeventsandalarmsandcreatelogs. It providesmanagersfunctionsforcontrollingtheeventsandalarmsandcontrollingthelogs. EVAisamanagementprotocolindependentapplicationthatneedsprotocoladaptationsto communicatewitharemotemanager. Currently,onesuchadaptationforSNMPisincludedinEVA. 1.1 Introduction Theoperationandmaintenance supportinOTPconsistsofagenericmodelformanagement subsystemsinOTP,andsomecomponentstobeusedinthesesubsystems. Themodelthatthissupport isbaseduponisdescribedin“OAMPrinciples”. Thisdocumentdescribesoneofthesecomponents,theFaultManagementapplicationEVA.EVA consistsofsupportforEventandAlarmHandlingandsupportforgenericLogControl. EVAisamodularapplicationthatconsistsoftwomanagementprotocolindependentservices. It containsalsorulesandfunctionsfordefiningprotocoladaptationsforEVA.Onesuchadaptationis includedforSNMP.ItconsistsofSNMPMIBsandimplementationoftheseMIBs. EVAusestheapplicationsMnesiaandSASL. 1.1.1 Architecture EVAisasubapplicationthatcanbeincludedintoanotherapplication. Itisdesignedtoworkasa distributedapplication,whichmeansthatitalwaysexecutesononenode,withothernodesasstandby nodes. EVAshouldrunonthesamenodeasotheroperationandmaintenance applications,specifically themanagementprotocolterminationapplication,tominimizeinternalnetworktraffic. EVAisdesignedtobeprotocolindependent,andmaybeusedwithdifferentmanagementprotocols. Foreachsuchprotocol,anEVAadaptationmustbewritten. Forexample,wehavedefinedanSNMP adaptation,andmoreadaptationsmaybedefinedinthefuture,e.g. forHTTP,CORBAorany proprietaryprotocol(e.g. plainErlang). Theeventandalarmsupportcanrunintwodifferentmodes,inserver modeandinclient mode. In clientmode,noprocessesarerunning,butthecodeimplementingtheAPIisloaded. Theremustalways beoneserverrunningononenodeinthenetworkofErlangnodesthatthesystemconsistsof. ForEVA,allinvolvednodesareseenasone(distributed)system. Thismeansforexamplethatthereis oneactivealarmlistfortheentiresystem. EventandAlarmhandlingApplication(EVA) 1 Chapter1: EVAUser’sGuide 1.2 Services TherearetwomanagementprotocolindependentEVAservicesprovided,thebasicEventandAlarm serviceandtheLogControlservice. ThebasicEVAserviceprovidesclientswithanAPIforregistering andsendingeventsandalarms. TheLogcontrolserviceprovidesamechanismforcontrolofgeneric logs. Alsoincludedisaspecializationofthegenericlogfunctionforloggingofeventsandalarms. Eachserviceprovidesclientfunctionsthatcanbeusedfromapplicationsinthesystemto,forexample, sendalarms. ThereisalsoanAPIthatmanagementapplicationscanusetomonitorandcontrolthe system. ThisAPIcanbeextendedforspecificmanagementprotocols,suchasSNMPorCORBA. 1.2.1 Basic Event and Alarm Service ThisservicecontainsfunctionsfortheclientAPItoEVA.EVAisadistributedglobalapplication,which meansthatclientscanaccesstheEVAfunctionalityfromanynode. Clientscanregisterandsendeventsandalarms. Managementapplicationscansubscribetoeventand alarms,andcontrolthetreatmentofthem. AneventisanotificationsentfromtheNEtoamanagementapplication. Aneventisuniquely identifiedbyitsname. Aspecialformofaneventisanalarm. Analarmrepresentsafaultinthesystem thatneedstobereportedtothemanager. Anexampleofanalarmcouldbeequipment on fire. When analarmissent,itbecomesactive,andisstoredinanactivealarmlist. Whentheapplicationthatsent thealarmnoticesthatthefaultthatcausedthealarmisnotvalidanymore,itclearsthealarm. Whenan alarmiscleared,thealarmisdeletedfromtheactivealarmlist,andanclear alarmeventisgenerated byEVA.Eachfaultmaygiverisetoseveralalarms,maybewithdifferentseverities. Therecanhowever onlybeoneactivealarmforeachfaultatthesametime. Forexample,associatedwithdiskspaceusage maybetwoalarms,disk 80 percent filledanddisk 90 percent filled. Thesetwoalarms representsthesamefault,butonlyoneofthemcanbeactiveatthesametime. Anactivealarmis identifiedbyitsfault id. Incontrasttoalarms,ordinaryeventsdonotrepresentfaults,andarenotstored asthealarmsintheactivealarmlist. ThebasicEVAserverisaglobalservertowhichalleventsandalarmsaresent. Theserverupdatesits tables(e.g. theactivealarmlist),andsendstheeventoralarmtothealarm handlerprocessthatruns onthesamenodeastheglobalserver. alarm handlerisagen eventprocessdefinedintheSASL application. Beforeaclientcansendaneventoralarm,thenameoftheeventmustberegisteredinEVA.Toregister anevent,aclientcallsregister event/2. Theparametersofthisfunctionarethenameoftheevent andwhethertheeventshouldbeloggedbydefaultornot. Amanagercandecidetochangethisvalue later. Toregisteranalarm,aclientcallsregister alarm/4. Theparametersofthisfunctionarethe nameandloggingparametersasforevents,andtheclassanddefaultseverityofthealarm. EVAstoresthedefinitionsofeventsandalarmsintheMnesiatableseventTableandalarmTable respectively. Sinceanalarmisaspecialformofanevent,eachalarmispresentinbothofthesetables. TheactivealarmlistisstoredintheMnesiatablealarm. Therecordsforallthesetablesaredefinedin theheaderfileeva.hrl,availableintheincludedirectoryinthedistribution. 2 EventandAlarmhandlingApplication(EVA) 1.2: Services EventDefinitionTable AllregisteredeventsarestoredintheeventTable.Ithasthefollowingattributes: name (cid:15) log (cid:15) generated (cid:15) Theeventisuniquelyidentifiedbyitsname,whichisanatom. Thelogattributeisabooleanflagthattellswhetherthiseventshouldbestoredinsomelogwhenitis generatedornot. Thisattributeiswritable. Thegeneratedattributeisacounterthatcountshowmanytimestheeventhasbeengenerated. AlarmDefinitionTable ThealarmTableextendstheeventTable,andhasthefollowingattributes: name (cid:15) class (cid:15) severity (cid:15) Thealarmisuniquelyidentifiedbyitsname,whichisanatom. Notethateachalarmispresentinthe eventTableaswell. Theclassattributecategorizesthealarm,andisdefinedwhenthealarmisregistered. Itisasdefinedin X.733,ITUAlarmReportingFunction: communications.Analarmofthisclassisprincipallyassociatedwiththeproceduresorprocesses (cid:15) requiredtoconveyinformationfromonepointtoanother. qos. Analarmofthisclassisprincipallyassociatedwithadegradationinthequalityofservice. (cid:15) processing.Analarmofthisclassisprincipallyassociatedwithasoftwareorprocessingfault. (cid:15) equipment.Analarmofthisclassisprincipallyassociatedwithanequipmentfault. (cid:15) environmental.Analarmofthisclassisprincipallyassociatedwithaconditionrelatingtoan (cid:15) enclosureinwithequipmentresides. Theseverityparameterdefinesfiveseveritylevels,whichprovideanindicationofhowitisperceived thatthecapability ofthemanagedobjecthasbeenaffected. Thoseseveritylevelswhichrepresent serviceaffectingconditionsorderedfrommostseveretoleastseverearecritical,major,minorand warning. ThelevelsusedareasdefinedinX.733,ITUAlarmReportingFunction: indeterminate.TheIndeterminateseveritylevelindicatesthattheseveritylevelcannotbe (cid:15) determined. critical. TheCriticalseveritylevelindicates thataserviceaffectingconditionhasoccurredand (cid:15) animmediatecorrectiveactionisrequired. Suchaseveritycanbereported,forexample,whena managedobjectbecomestotallyoutofserviceanditscapabilitymustberestored. major. TheMajorseveritylevelindicatesthataserviceaffectingconditionhasdevelopedandan (cid:15) urgentcorrectiveactionisrequired. Suchaseveritycanbereported,forexample,whenthereisa severedegradationinthecapabilityofthemanagedobjectanditsfullcapabilitymustberestored. EventandAlarmhandlingApplication(EVA) 3 Chapter1: EVAUser’sGuide minor. TheMinorseveritylevelindicatestheexistenceofanon-serviceaffectingfaultcondition (cid:15) andthatcorrectiveactionshouldbetakeninordertopreventamoreserious(forexample,service affecting)fault. Suchaseveritycanbereported,forexample,whenthedetectedalarmcondition isnotcurrentlydegradingthecapacity ofthemanagedobject. warning. TheWarningseveritylevelindicates thedetectionofapotentialorimpendingservice (cid:15) affectingfault,beforeanysignificanteffectshavebeenfelt. Actionshouldbetakentofurther diagnose(ifnecessary)andcorrecttheprobleminordertopreventitfrombecomingamore seriousserviceaffectingfault. Whenanalarmiscleared,aclear alarmeventisgenerated. Thiseventclearsthealarmwiththe fault idcontainedintheevent. Itisnotrequiredthattheclearingofpreviouslyreportedalarmsare reported. Therefore,amanagingsystemcannotassumethattheabsenceofanclear alarmeventfora faultmeansthattheconditionthatcausedthegenerationofpreviousalarmsisstillpresent. Managed objectdefinersshallstateif,andunderwhichconditions,theclear alarmeventisused. ActiveAlarmList TheactivealarmlistisstoredintheorderedMnesiatablealarm. Thecorrespondingrecordissentto thealarm handlerwhenanalarmissent. Ithasthefollowingread-onlyattributes: index (cid:15) fault id (cid:15) name (cid:15) sender (cid:15) cause (cid:15) severity (cid:15) time (cid:15) extra (cid:15) Arowintheactivealarmlistisuniquelyidentifiedbyitsfault id. However,tomakethetable ordered,thealarmsusestheintegerindexasakeyintothetable. Foreachnewalarm,EVAallocatesa newindexthatisgreaterthantheindexofallotheractivealarms. Thenameisthenameofthecorrespondingalarmtype,definedinalarmTable. senderisatermthatuniquelyidentifiestheresourcethatgeneratedthealarm. causedescribestheprobablecauseofthealarm. severityistheperceivedseverityofthealarm. timeistheUTCtimethealarmwasgenerated. extraisanyextrainformationdescribingthealarm. 4 EventandAlarmhandlingApplication(EVA) 1.2: Services Event Whenaneventisgenerated,theeventrecordissenttoalarm handler. Ithasthefollowingattributes: name (cid:15) sender (cid:15) time (cid:15) extra (cid:15) Thenameisthenameofthecorrespondingeventtype,definedineventTable. senderisatermthatuniquelyidentifiestheresourcethatgeneratedtheevent. timeistheUTCtimetheeventwasgenerated. extraisanyextrainformationdescribingtheevent. Example Asanexampleofhowtoregisterandsendeventsandalarms,considerthefollowingcode: %%%----------------------------------------------------------------- %%% Resource code %%%----------------------------------------------------------------- reg() -> eva:register_event(boardRemoved, true), eva:register_event(boardInserted, false), eva:register_alarm(boardFailure, true, equipment, minor). remove_board(No) -> eva:send_event(boardRemoved, {board, No}, []). insert_board(No, BoardName, BoardType) -> eva:send_event(boardInserted, {board, No}, {BoardName, BoardType}). board_on_fire(No) -> FaultId = eva:get_fault_id(), %% Cause = fire, ExtraParams = [] eva:send_alarm(boardFailure, FaultId, {board, No}, fire, []), FaultId. Twoeventsandonealarmisdefined. Boardremovalisaneventthatisloggedbydefault,andboard insertionisaneventthatisnotloggedbydefault. ThealarmequipmentFailureisaminoralarmthatis loggedbydefault. WhentheapplicationdetectsthatboardNisonfire,board on fire(N)iscalled. Thisfunctionis responsibleforsendingthealarm. Itgetsanewfaultidentifierforthefault,andcalls eva:send alarm/5,pointingoutthefaultyboard(N),andsuggeststhattheprobablecauseforthe equipmenttroubleisfire. Theboard on firefunctionreturnsthefaultidentifierforthenewalarm. Thisfaultidentifiercanbe usedatalatertimeinacalltoeva:clear alarm(FaultId)toclearthealarm. EventandAlarmhandlingApplication(EVA) 5 Chapter1: EVAUser’sGuide 1.2.2 Log Control Service TheLogControlservicecontainsfunctionsformonitoringlogs,andfunctionsfortransferringlogsto remotehosts,e.g. managementstations. ThemainpurposeoftheLogControlserviceistoprovideone entitythroughwhichalllogsinthesystemcanbecontrolledbyamanagementstation. Regardlessof thetypelog,alllogsarecontrolledinasimiliarfashion. Clientscanregistertheirlogsinthelogserver. Managementapplicationscancontrolthelogs,and transferthelogstoaremotehost. LogMonitoring Thisserviceusesalogserver thatmonitorsalllogsinthesystem. Eachlogusesthestandardmodule disk logfortheactuallogging. Eachloghasanadministrativeandanoperationalstatus,thatbothcanbeeitherupordown. Ifthe operationalstatusisup,thelogisworking,andifitisdown,thelogdoesnotwork. Theadministrative statusiswritable,andreflectsthedesiredoperationalstatus. Normallytheyareboththesame. Ifthe administrativestatusissettoup,theoperationalstatuswillbeupaswell. However,ifthelogforsome reasondoesnotwork,e.g. ifthediskpartitionisfull,theoperationalstatuswillbedown. Whenthe operationalstatusisdown,noeventsareloggedinthelog. Alarms TheTlogservicedefinestwoEVAalarms;log file errorandlog wrap too often. log file error. Thisalarmisgeneratedifafileerroroccurswhenanitemislogged. Default (cid:15) severityiscritical. ThecauseforthisalarmcanbeanyReasonasreturnedfromfile:writein caseoferror. Thealarmisclearedifthefilesystemstartsworkingagain. Forexample,thealarm canbegeneratedifthepartitionisfull,andclearedwhenspaceisavailable. log wrap too often. Thisalarmisgeneratedwhenthelogwrapsmoreoftenthanthewraptime. (cid:15) Defaultseverityismajor. Thecauseforthisalarmisundefined. Thealarmisclearedifthelog wrapswithinthewraptime,thenexttimeitwraps. Example ThefollowingisanexampleofcodethatcreatesalogtobecontrolledbythegenericLog Controlfunction: start() -> disk_log:open([{name, "ex_log"}, {file, "ex_log/ex_log.LOG"}, {type, wrap}, {size, {10000, 4}}]), log:open("ex_log", ex_log_type, 3600). test() -> %% Log an item disk_log:log("ex_log", {1, "log this"}), %% Set the administrative status of the log to ’down’ log:set_admin_status("ex_log", down), %% Try to log - this one won’t be logged disk_log:log("ex_log", {2, "won’t be logged"}), Logs1 = log:get_logs(), 6 EventandAlarmhandlingApplication(EVA)