PROBABILITY, RANDOM VARIABLES, AND RANDOM PROCESSES PROBABILITY, RANDOM VARIABLES, AND RANDOM PROCESSES Theory and Signal Processing Applications JOHN J. SHYNK DepartmentofElectricalandComputerEngineering UniversityofCalifornia,SantaBarbara,CA,USA AJOHNWILEY&SONS,INC.,PUBLICATION Coverimage:© loops7/iStockphoto Copyright© 2013byJohnWiley&Sons,Inc.Allrightsreserved. PublishedbyJohnWiley&Sons,Inc.,Hoboken,NewJersey. PublishedsimultaneouslyinCanada. Nopartofthispublicationmaybereproduced,storedinaretrievalsystem,ortransmittedinanyformorbyanymeans, electronic,mechanical,photocopying,recording,scanning,orotherwise,exceptaspermittedunderSection107or108of the1976UnitedStatesCopyrightAct,withouteitherthepriorwrittenpermissionofthePublisher,orauthorizationthrough paymentoftheappropriateper-copyfeetotheCopyrightClearanceCenter,Inc.,222RosewoodDrive,Danvers,MA 01923,(978)750-8400,fax(978)750-4470,oronthewebatwww.copyright.com.RequeststothePublisherforpermission shouldbeaddressedtothePermissionsDepartment,JohnWiley&Sons,Inc.,111RiverStreet,Hoboken,NJ07030,(201) 748-6011,fax(201)748-6008,oronlineathttp://www.wiley.com/go/permission. LimitofLiability/DisclaimerofWarranty:Whilethepublisherandauthorhaveusedtheirbesteffortsinpreparingthis book,theymakenorepresentationsorwarrantieswithrespecttotheaccuracyorcompletenessofthecontentsofthisbook andspecificallydisclaimanyimpliedwarrantiesofmerchantabilityorfitnessforaparticularpurpose.Nowarrantymaybe createdorextendedbysalesrepresentativesorwrittensalesmaterials.Theadviceandstrategiescontainedhereinmaynot besuitableforyoursituation.Youshouldconsultwithaprofessionalwhereappropriate.Neitherthepublishernorauthor shallbeliableforanylossofprofitoranyothercommercialdamages,includingbutnotlimitedtospecial,incidental, consequential,orotherdamages. Forgeneralinformationonourotherproductsandservicesorfortechnicalsupport,pleasecontactourCustomerCare DepartmentwithintheUnitedStatesat(800)762-2974,outsidetheUnitedStatesat(317)572-3993orfax(317)572-4002. Wileyalsopublishesitsbooksinavarietyofelectronicformats.Somecontentthatappearsinprintmaynotbeavailablein electronicformats.FormoreinformationaboutWileyproducts,visitourwebsiteatwww.wiley.com. LibraryofCongressCataloging-in-PublicationData: Shynk,JohnJoseph. Probability,randomvariables,andrandomprocesses:theoryandsignalprocessingapplications/JohnJ.Shynk. p.cm. Includesindex. ISBN978-0-470-24209-4(hardback) 1.Probabilities–Textbooks. 2.Stochasticprocesses–Textbooks. 3.Engineering–Statisticalmethods–Textbooks. I.Title. QA274.S532012 519.2–dc23 2012008186 PrintedintheUnitedStatesofAmerica ISBN:9780470242094 10 9 8 7 6 5 4 3 2 1 ToTokie InmemoryofA.V.andE.F. CONTENTS PREFACE xxi NOTATION xxv 1 OverviewandBackground 1 1.1 Introduction 1 1.1.1 Signals,SignalProcessing,andCommunications 3 1.1.2 Probability,RandomVariables,andRandomVectors 9 1.1.3 RandomSequencesandRandomProcesses 11 1.1.4 DeltaFunctions 16 1.2 DeterministicSignalsandSystems 19 1.2.1 ContinuousTime 20 1.2.2 DiscreteTime 25 1.2.3 Discrete-TimeFilters 29 1.2.4 State-SpaceRealizations 32 1.3 StatisticalSignalProcessingwithMATLAB® 35 1.3.1 RandomNumberGeneration 35 1.3.2 Filtering 38 Problems 39 FurtherReading 45 PARTI Probability,RandomVariables,andExpectation 2 ProbabilityTheory 49 2.1 Introduction 49 2.2 SetsandSampleSpaces 50 2.3 SetOperations 54 2.4 EventsandFields 58 2.5 SummaryofaRandomExperiment 64 2.6 MeasureTheory 64 2.7 AxiomsofProbability 68 2.8 BasicProbabilityResults 69 2.9 ConditionalProbability 71 2.10 Independence 73 vii viii CONTENTS 2.11 Bayes’Formula 74 2.12 TotalProbability 76 2.13 DiscreteSampleSpaces 79 2.14 ContinuousSampleSpaces 83 2.15 NonmeasurableSubsetsofR 84 Problems 87 FurtherReading 90 3 RandomVariables 91 3.1 Introduction 91 3.2 FunctionsandMappings 91 3.3 DistributionFunction 96 3.4 ProbabilityMassFunction 101 3.5 ProbabilityDensityFunction 103 3.6 MixedDistributions 104 3.7 ParametricModelsforRandomVariables 107 3.8 ContinuousRandomVariables 109 3.8.1 GaussianRandomVariable(Normal) 110 3.8.2 Log-NormalRandomVariable 113 3.8.3 InverseGaussianRandomVariable(Wald) 114 3.8.4 ExponentialRandomVariable(One-Sided) 116 3.8.5 LaplaceRandomVariable(Double-SidedExponential) 119 3.8.6 CauchyRandomVariable 122 3.8.7 ContinuousUniformRandomVariable 124 3.8.8 TriangularRandomVariable 125 3.8.9 RayleighRandomVariable 127 3.8.10 RiceRandomVariable 129 3.8.11 GammaRandomVariable(Erlangforr ∈N) 131 3.8.12 BetaRandomVariable(Arcsineforα =β =1/2,PowerFunctionfor β =1) 133 3.8.13 ParetoRandomVariable 136 3.8.14 WeibullRandomVariable 137 3.8.15 LogisticRandomVariable(Sigmoidfor{μ=0,α =1}) 139 3.8.16 ChiRandomVariable(Maxwell–Boltzmann,Half-Normal) 141 3.8.17 Chi-SquareRandomVariable 144 3.8.18 F-Distribution 147 3.8.19 Student’stDistribution 149 3.8.20 ExtremeValueDistribution(TypeI:Gumbel) 150 3.9 DiscreteRandomVariables 151 3.9.1 BernoulliRandomVariable 152 3.9.2 BinomialRandomVariable 154 3.9.3 GeometricRandomVariable(withSupportZ+orN) 157