Table Of Content

Computer Science/Computer Engineering PROBABILITY AND THERRIEN TUMMALA RANDOM PROCESSES PROBABILITY RANDOM PROCESSES AND FOR F P ELECTRICAL COMPUTER ENGINEERS O AND R ELECTRICAL R O FOR AND WWiitthh uuppddaatteess aanndd eennhhaanncceemmeennttss ttoo tthhee iinnccrreeddiibbllyy ssuucccceessssffuull ffiirrsstt eeddiittiioonn,, PPrroobbaabbiilliittyy EB L aanndd RRaannddoomm PPrroocceesssseess ffoorr EElleeccttrriiccaall aanndd CCoommppuutteerr EEnnggiinneeeerrss,, SSeeccoonndd EEddiittiioonn A COMPUTER ENGINEERS E rreettaaiinnss tthhee bbeesstt aassppeeccttss ooff tthhee oorriiggiinnaall bbuutt ooffffeerrss aann eevveenn mmoorree ppootteenntt iinnttrroodduuccttiioonn ttoo CB pprroobbaabbiilliittyy aanndd rraannddoomm vvaarriiaabblleess aanndd pprroocceesssseess.. WWrriitttteenn iinn aa cclleeaarr,, ccoonncciissee ssttyyllee tthhaatt TI L iilllluussttrraatteess tthhee ssuubbjjeecctt’’ss rreelleevvaannccee ttoo aa wwiiddee rraannggee ooff aarreeaass iinn eennggiinneeeerriinngg aanndd pphhyyssiiccaall RI T aanndd ccoommppuutteerr sscciieenncceess,, tthhiiss tteexxtt iiss oorrggaanniizzeedd iinnttoo ttwwoo ppaarrttss.. TThhee ffiirrsstt ffooccuusseess oonn tthhee I CY S E C O N D E D I T I O N pprroobbaabbiilliittyy mmooddeell,, rraannddoomm vvaarriiaabblleess aanndd ttrraannssffoorrmmaattiioonnss,, aanndd iinneeqquuaalliittiieess aanndd lliimmiitt A tthheeoorreemmss.. TThhee sseeccoonndd ddeeaallss wwiitthh sseevveerraall ttyyppeess ooff rraannddoomm pprroocceesssseess aanndd A L N qquueeuuiinngg tthheeoorryy.. A D N NNeeww oorr UUppddaatteedd ffoorr tthhee SSeeccoonndd EEddiittiioonn R D A •• AA sshhoorrtt nneeww cchhaapptteerr oonn rraannddoomm vveeccttoorrss tthhaatt aaddddss ssoommee aaddvvaanncceedd nneeww C N mmaatteerriiaall aanndd ssuuppppoorrttss ttooppiiccss aassssoocciiaatteedd wwiitthh ddiissccrreettee rraannddoomm pprroocceesssseess O D •• RReeoorrggaanniizzeedd cchhaapptteerrss tthhaatt ffuurrtthheerr ccllaarriiffyy ttooppiiccss iinncclluuddiinngg rraannddoomm M O pprroocceesssseess ((iinncclluuddiinngg MMaarrkkoovv aanndd PPooiissssoonn)) aanndd aannaallyyssiiss iinn tthhee aanndd ttiimmee PM ffrreeqquueennccyy ddoommaaiinn U •• AA llaarrggee ccoolllleeccttiioonn ooff nneeww MMAATTLLAABB®®--bbaasseedd pprroobblleemmss aanndd ccoommppuutteerr TP pprroojjeeccttss//aassssiiggnnmmeennttss tthhaatt aarree aavvaaiillaabbllee ffrroomm tthhee CCRRCC PPrreessss wweebbssiittee ER R O EEaacchh CChhaapptteerr CCoonnttaaiinnss aatt LLeeaasstt TTwwoo CCoommppuutteerr AAssssiiggnnmmeennttss EC N E MMaaiinnttaaiinniinngg tthhee ssiimmpplliiffiieedd,, iinnttuuiittiivvee ssttyyllee tthhaatt pprroovveedd eeffffeeccttiivvee tthhee ffiirrsstt ttiimmee,, tthhiiss eeddiittiioonn GS iinntteeggrraatteess ccoorrrreeccttiioonnss aanndd iimmpprroovveemmeennttss bbaasseedd oonn ffeeeeddbbaacckk ffrroomm ssttuuddeennttss aanndd S tteeaacchheerrss.. FFooccuusseedd oonn ssttrreennggtthheenniinngg tthhee rreeaaddeerr’’ss ggrraasspp ooff uunnddeerrllyyiinngg mmaatthheemmaattiiccaall I NE ccoonncceeppttss,, tthhee bbooookk ccoommbbiinneess aann aabbuunnddaannccee ooff pprraaccttiiccaall aapppplliiccaattiioonnss,, eexxaammpplleess,, aanndd S E ootthheerr ttoooollss ttoo ssiimmpplliiffyy uunnnneecceessssaarriillyy ddiiffffiiccuulltt ssoolluuttiioonnss ttoo vvaarryyiinngg eennggiinneeeerriinngg pprroobblleemmss E iinn ccoommmmuunniiccaattiioonnss,, ssiiggnnaall pprroocceessssiinngg,, nneettwwoorrkkss,, aanndd aassssoocciiaatteedd ffiieellddss.. R S CHARLES W. THERRIEN MURALI TUMMALA SECOND K11329 EDITION ISBN: 978-1-4398-2698-0 90000 9 781439 826980 PROBABILITY AND RANDOM PROCESSES ELECTRICAL FOR AND COMPUTER ENGINEERS S E C O N D E D I T I O N K11329_FM.indd 1 8/26/11 10:29 AM K11329_FM.indd 2 8/26/11 10:29 AM PROBABILITY AND RANDOM PROCESSES ELECTRICAL FOR AND COMPUTER ENGINEERS S E C O N D E D I T I O N CHARLES W. THERRIEN MURALI TUMMALA Boca Raton London New York CRC Press is an imprint of the Taylor & Francis Group, an informa business K11329_FM.indd 3 8/26/11 10:29 AM MATLAB® is a trademark of The MathWorks, Inc. and is used with permission. The MathWorks does not warrant the accuracy of the text or exercises in this book. This book’s use or discussion of MATLAB® software or related products does not consti- tute endorsement or sponsorship by The MathWorks of a particular pedagogical approach or particular use of the MATLAB® software. CRC Press Taylor & Francis Group 6000 Broken Sound Parkway NW, Suite 300 Boca Raton, FL 33487-2742 © 2012 by Taylor & Francis Group, LLC CRC Press is an imprint of Taylor & Francis Group, an Informa business No claim to original U.S. Government works Version Date: 20111103 International Standard Book Number-13: 978-1-4398-9749-2 (eBook - PDF) This book contains information obtained from authentic and highly regarded sources. Reasonable efforts have been made to publish reliable data and information, but the author and publisher cannot assume responsibility for the validity of all materials or the consequences of their use. The authors and publishers have attempted to trace the copyright holders of all material repro- duced in this publication and apologize to copyright holders if permission to publish in this form has not been obtained. If any copyright material has not been acknowledged please write and let us know so we may rectify in any future reprint. Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced, transmitted, or utilized in any form by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying, microfilming, and recording, or in any information storage or retrieval system, without written permission from the publishers. For permission to photocopy or use material electronically from this work, please access www.copyright.com (http://www.copy- right.com/) or contact the Copyright Clearance Center, Inc. (CCC), 222 Rosewood Drive, Danvers, MA 01923, 978-750-8400. CCC is a not-for-profit organization that provides licenses and registration for a variety of users. For organizations that have been granted a photocopy license by the CCC, a separate system of payment has been arranged. Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used only for identifica- tion and explanation without intent to infringe. Visit the Taylor & Francis Web site at http://www.taylorandfrancis.com and the CRC Press Web site at http://www.crcpress.com To all the teachers of probability – especially Alvin Drake, who made it fun to learn and Athanasios Papoulis, who made it just rigorous enough for engineers. Also, to our families for their enduring support. Contents Preface to the Second Edition xiii Preface xv Part I Probability and Random Variables 1 1 Introduction 3 1.1 The Analysis of RandomExperiments . . . . . . . . . . . . . . . . . 3 1.2 Probabilityin Electrical and Computer Engineering . . . . . . . . . 5 1.2.1 Signal detection and classification . . . . . . . . . . . . . . . . 5 1.2.2 Speech modeling and recognition . . . . . . . . . . . . . . . . 6 1.2.3 Coding and data transmission . . . . . . . . . . . . . . . . . . 7 1.2.4 Computer networks . . . . . . . . . . . . . . . . . . . . . . . . 8 1.3 Outline of the Book . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 2 The Probability Model 11 2.1 The Algebra of Events . . . . . . . . . . . . . . . . . . . . . . . . . . 11 2.1.1 Basic operations . . . . . . . . . . . . . . . . . . . . . . . . . . 12 2.1.2 Representation of the sample space . . . . . . . . . . . . . . . 13 2.2 Probabilityof Events . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 2.2.1 Defining probability. . . . . . . . . . . . . . . . . . . . . . . . 16 2.2.2 Statistical independence . . . . . . . . . . . . . . . . . . . . . 20 2.3 Some Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 2.3.1 Repeated independent trials . . . . . . . . . . . . . . . . . . . 21 2.3.2 Problems involvingcounting . . . . . . . . . . . . . . . . . . . 22 2.3.3 Network reliability . . . . . . . . . . . . . . . . . . . . . . . . 24 2.4 ConditionalProbabilityand Bayes’ Rule . . . . . . . . . . . . . . . . 26 2.4.1 Conditionalprobability . . . . . . . . . . . . . . . . . . . . . . 26 2.4.2 Event trees . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 2.4.3 Bayes’ rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 2.5 More Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 2.5.1 The binary communicationchannel . . . . . . . . . . . . . . . 31 2.5.2 Measuring informationand coding . . . . . . . . . . . . . . . 32 2.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 3 Random Variables and Transformations 51 3.1 Discrete RandomVariables . . . . . . . . . . . . . . . . . . . . . . . 51 3.2 Some Common Discrete ProbabilityDistributions . . . . . . . . . . . 53 3.2.1 Bernoulli random variable . . . . . . . . . . . . . . . . . . . . 54 3.2.2 Binomialrandom variable . . . . . . . . . . . . . . . . . . . . 54 3.2.3 Geometric random variable . . . . . . . . . . . . . . . . . . . 55 3.2.4 Poisson random variable . . . . . . . . . . . . . . . . . . . . . 57 3.2.5 Discrete uniform random variable . . . . . . . . . . . . . . . . 58 3.2.6 Other types of discrete random variables . . . . . . . . . . . . 58 3.3 Continuous RandomVariables . . . . . . . . . . . . . . . . . . . . . . 59 vii viii CONTENTS 3.3.1 Probabilistic description . . . . . . . . . . . . . . . . . . . . . 59 3.3.2 More about the PDF . . . . . . . . . . . . . . . . . . . . . . . 60 3.3.3 A relation to discrete random variables . . . . . . . . . . . . . 61 3.3.4 Solving problems . . . . . . . . . . . . . . . . . . . . . . . . . 62 3.4 Some Common Continuous Probability Density Functions . . . . . . 64 3.4.1 Uniform random variable . . . . . . . . . . . . . . . . . . . . . 64 3.4.2 Exponential random variable. . . . . . . . . . . . . . . . . . . 65 3.4.3 Gaussian random variable . . . . . . . . . . . . . . . . . . . . 66 3.4.4 Other types of continuous random variables . . . . . . . . . . 69 3.5 CDF and PDF for Discrete and Mixed Random Variables . . . . . . 69 3.5.1 Discrete random variables . . . . . . . . . . . . . . . . . . . . 69 3.5.2 Mixed random variables . . . . . . . . . . . . . . . . . . . . . 71 3.6 Transformationof Random Variables . . . . . . . . . . . . . . . . . . 72 3.6.1 When the transformation is invertible. . . . . . . . . . . . . . 72 3.6.2 When the transformation is not invertible . . . . . . . . . . . 75 3.6.3 When the transformation has flat regions or discontinuities. . 77 3.7 Distributions Conditioned on an Event . . . . . . . . . . . . . . . . . 81 3.8 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83 3.8.1 Digitalcommunication . . . . . . . . . . . . . . . . . . . . . . 84 3.8.2 Radar/sonar target detection . . . . . . . . . . . . . . . . . . 90 3.8.3 Object classification . . . . . . . . . . . . . . . . . . . . . . . 93 3.9 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 4 Expectation, Moments, and Generating Functions 113 4.1 Expectation of a Random Variable . . . . . . . . . . . . . . . . . . . 113 4.1.1 Discrete random variables . . . . . . . . . . . . . . . . . . . . 114 4.1.2 Continuous random variables . . . . . . . . . . . . . . . . . . 115 4.1.3 Invariance of expectation . . . . . . . . . . . . . . . . . . . . . 115 4.1.4 Properties of expectation . . . . . . . . . . . . . . . . . . . . . 117 4.1.5 Expectation conditioned on an event . . . . . . . . . . . . . . 118 4.2 Moments of a Distribution . . . . . . . . . . . . . . . . . . . . . . . . 118 4.2.1 Central moments . . . . . . . . . . . . . . . . . . . . . . . . . 119 4.2.2 Properties of variance. . . . . . . . . . . . . . . . . . . . . . . 120 4.2.3 Some higher-order moments . . . . . . . . . . . . . . . . . . . 121 4.3 Generating Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . 122 4.3.1 The moment-generating function . . . . . . . . . . . . . . . . 122 4.3.2 The probability-generating function . . . . . . . . . . . . . . . 124 4.4 Application:Entropy and Source Coding . . . . . . . . . . . . . . . . 126 4.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127 5 Two and More Random Variables 141 5.1 Two Discrete Random Variables . . . . . . . . . . . . . . . . . . . . 141 5.1.1 The joint PMF . . . . . . . . . . . . . . . . . . . . . . . . . . 142 5.1.2 Independent random variables . . . . . . . . . . . . . . . . . . 143 5.1.3 ConditionalPMFs for discrete random variables . . . . . . . . 144 5.1.4 Bayes’ rule for discrete random variables . . . . . . . . . . . . 145 5.2 Two Continuous Random Variables . . . . . . . . . . . . . . . . . . . 147 5.2.1 Joint distributions . . . . . . . . . . . . . . . . . . . . . . . . 147 5.2.2 Marginal PDFs: Projections of the joint density . . . . . . . . 150 5.2.3 ConditionalPDFs: Slices of the joint density . . . . . . . . . . 152 CONTENTS ix 5.2.4 Bayes’ rule for continuous random variables . . . . . . . . . . 155 5.3 Expectation and Correlation. . . . . . . . . . . . . . . . . . . . . . . 157 5.3.1 Correlation and covariance . . . . . . . . . . . . . . . . . . . . 158 5.3.2 Conditionalexpectation . . . . . . . . . . . . . . . . . . . . . 161 5.4 Gaussian RandomVariables . . . . . . . . . . . . . . . . . . . . . . . 163 5.5 Multiple RandomVariables . . . . . . . . . . . . . . . . . . . . . . . 165 5.5.1 PDFs for multiplerandom variables. . . . . . . . . . . . . . . 165 5.5.2 Sums of random variables . . . . . . . . . . . . . . . . . . . . 167 5.6 Sums of some common random variables . . . . . . . . . . . . . . . . 170 5.6.1 Bernoulli random variables . . . . . . . . . . . . . . . . . . . . 170 5.6.2 Geometric random variables . . . . . . . . . . . . . . . . . . . 171 5.6.3 Exponential random variables . . . . . . . . . . . . . . . . . . 171 5.6.4 Gaussian random variables . . . . . . . . . . . . . . . . . . . . 173 5.6.5 Squared Gaussian random variables . . . . . . . . . . . . . . . 173 5.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173 6 Inequalities, Limit Theorems, and Parameter Estimation 187 6.1 Inequalities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187 6.1.1 Markov inequality . . . . . . . . . . . . . . . . . . . . . . . . . 188 6.1.2 Chebyshev inequality . . . . . . . . . . . . . . . . . . . . . . . 189 6.1.3 One-sided Chebyshev inequality . . . . . . . . . . . . . . . . . 190 6.1.4 Other inequalities . . . . . . . . . . . . . . . . . . . . . . . . . 190 6.2 Convergence and Limit Theorems . . . . . . . . . . . . . . . . . . . . 191 6.2.1 Laws of large numbers . . . . . . . . . . . . . . . . . . . . . . 193 6.2.2 Central limittheorem . . . . . . . . . . . . . . . . . . . . . . . 194 6.3 Estimation of Parameters . . . . . . . . . . . . . . . . . . . . . . . . 196 6.3.1 Estimates and properties . . . . . . . . . . . . . . . . . . . . . 196 6.3.2 Sample mean and variance . . . . . . . . . . . . . . . . . . . . 197 6.4 MaximumLikelihoodEstimation . . . . . . . . . . . . . . . . . . . . 199 6.5 Point Estimates and Confidence Intervals . . . . . . . . . . . . . . . 203 6.6 Applicationto Signal Estimation . . . . . . . . . . . . . . . . . . . . 205 6.6.1 Estimating a signalin noise . . . . . . . . . . . . . . . . . . . 206 6.6.2 Choosing the number of samples . . . . . . . . . . . . . . . . 207 6.6.3 Applying confidence intervals . . . . . . . . . . . . . . . . . . 210 6.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211 7 Random Vectors 219 7.1 RandomVectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219 7.1.1 Cumulative distribution and density functions . . . . . . . . . 219 7.1.2 Random vectors with independent components . . . . . . . . 221 7.2 Analysis of Random Vectors . . . . . . . . . . . . . . . . . . . . . . . 221 7.2.1 Expectation and moments . . . . . . . . . . . . . . . . . . . . 222 7.2.2 Estimating moments from data . . . . . . . . . . . . . . . . . 223 7.2.3 The multivariateGaussian density . . . . . . . . . . . . . . . 225 7.2.4 Random vectors with uncorrelated components . . . . . . . . 226 7.3 Transformations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226 7.3.1 Transformationof moments . . . . . . . . . . . . . . . . . . . 226 7.3.2 Transformationof density functions . . . . . . . . . . . . . . . 227 7.4 Cross Correlation and Covariance . . . . . . . . . . . . . . . . . . . . 231 7.5 Applications to SignalProcessing . . . . . . . . . . . . . . . . . . . . 232

ebook img

Probability and Random Processes for Electrical and Computer Engineers PDF

433 Pages·2011·7.227 MB·English
by  Therrien, Charles, Tummala, Murali
#Mathematics #Wavelets and signal processing
Save to my drive
Quick download
Download
Upgrade Premium
Most books are stored in the elastic cloud where traffic is expensive. For this reason, we have a limit on daily download.

Preview Probability and Random Processes for Electrical and Computer Engineers

Computer Science/Computer Engineering PROBABILITY AND THERRIEN TUMMALA RANDOM PROCESSES PROBABILITY RANDOM PROCESSES AND FOR F P ELECTRICAL COMPUTER ENGINEERS O AND R ELECTRICAL R O FOR AND WWiitthh uuppddaatteess aanndd eennhhaanncceemmeennttss ttoo tthhee iinnccrreeddiibbllyy ssuucccceessssffuull ffiirrsstt eeddiittiioonn,, PPrroobbaabbiilliittyy EB L aanndd RRaannddoomm PPrroocceesssseess ffoorr EElleeccttrriiccaall aanndd CCoommppuutteerr EEnnggiinneeeerrss,, SSeeccoonndd EEddiittiioonn A COMPUTER ENGINEERS E rreettaaiinnss tthhee bbeesstt aassppeeccttss ooff tthhee oorriiggiinnaall bbuutt ooffffeerrss aann eevveenn mmoorree ppootteenntt iinnttrroodduuccttiioonn ttoo CB pprroobbaabbiilliittyy aanndd rraannddoomm vvaarriiaabblleess aanndd pprroocceesssseess.. WWrriitttteenn iinn aa cclleeaarr,, ccoonncciissee ssttyyllee tthhaatt TI L iilllluussttrraatteess tthhee ssuubbjjeecctt’’ss rreelleevvaannccee ttoo aa wwiiddee rraannggee ooff aarreeaass iinn eennggiinneeeerriinngg aanndd pphhyyssiiccaall RI T aanndd ccoommppuutteerr sscciieenncceess,, tthhiiss tteexxtt iiss oorrggaanniizzeedd iinnttoo ttwwoo ppaarrttss.. TThhee ffiirrsstt ffooccuusseess oonn tthhee I CY S E C O N D E D I T I O N pprroobbaabbiilliittyy mmooddeell,, rraannddoomm vvaarriiaabblleess aanndd ttrraannssffoorrmmaattiioonnss,, aanndd iinneeqquuaalliittiieess aanndd lliimmiitt A tthheeoorreemmss.. TThhee sseeccoonndd ddeeaallss wwiitthh sseevveerraall ttyyppeess ooff rraannddoomm pprroocceesssseess aanndd A L N qquueeuuiinngg tthheeoorryy.. A D N NNeeww oorr UUppddaatteedd ffoorr tthhee SSeeccoonndd EEddiittiioonn R D A •• AA sshhoorrtt nneeww cchhaapptteerr oonn rraannddoomm vveeccttoorrss tthhaatt aaddddss ssoommee aaddvvaanncceedd nneeww C N mmaatteerriiaall aanndd ssuuppppoorrttss ttooppiiccss aassssoocciiaatteedd wwiitthh ddiissccrreettee rraannddoomm pprroocceesssseess O D •• RReeoorrggaanniizzeedd cchhaapptteerrss tthhaatt ffuurrtthheerr ccllaarriiffyy ttooppiiccss iinncclluuddiinngg rraannddoomm M O pprroocceesssseess ((iinncclluuddiinngg MMaarrkkoovv aanndd PPooiissssoonn)) aanndd aannaallyyssiiss iinn tthhee aanndd ttiimmee PM ffrreeqquueennccyy ddoommaaiinn U •• AA llaarrggee ccoolllleeccttiioonn ooff nneeww MMAATTLLAABB®®--bbaasseedd pprroobblleemmss aanndd ccoommppuutteerr TP pprroojjeeccttss//aassssiiggnnmmeennttss tthhaatt aarree aavvaaiillaabbllee ffrroomm tthhee CCRRCC PPrreessss wweebbssiittee ER R O EEaacchh CChhaapptteerr CCoonnttaaiinnss aatt LLeeaasstt TTwwoo CCoommppuutteerr AAssssiiggnnmmeennttss EC N E MMaaiinnttaaiinniinngg tthhee ssiimmpplliiffiieedd,, iinnttuuiittiivvee ssttyyllee tthhaatt pprroovveedd eeffffeeccttiivvee tthhee ffiirrsstt ttiimmee,, tthhiiss eeddiittiioonn GS iinntteeggrraatteess ccoorrrreeccttiioonnss aanndd iimmpprroovveemmeennttss bbaasseedd oonn ffeeeeddbbaacckk ffrroomm ssttuuddeennttss aanndd S tteeaacchheerrss.. FFooccuusseedd oonn ssttrreennggtthheenniinngg tthhee rreeaaddeerr’’ss ggrraasspp ooff uunnddeerrllyyiinngg mmaatthheemmaattiiccaall I NE ccoonncceeppttss,, tthhee bbooookk ccoommbbiinneess aann aabbuunnddaannccee ooff pprraaccttiiccaall aapppplliiccaattiioonnss,, eexxaammpplleess,, aanndd S E ootthheerr ttoooollss ttoo ssiimmpplliiffyy uunnnneecceessssaarriillyy ddiiffffiiccuulltt ssoolluuttiioonnss ttoo vvaarryyiinngg eennggiinneeeerriinngg pprroobblleemmss E iinn ccoommmmuunniiccaattiioonnss,, ssiiggnnaall pprroocceessssiinngg,, nneettwwoorrkkss,, aanndd aassssoocciiaatteedd ffiieellddss.. R S CHARLES W. THERRIEN MURALI TUMMALA SECOND K11329 EDITION ISBN: 978-1-4398-2698-0 90000 9 781439 826980 PROBABILITY AND RANDOM PROCESSES ELECTRICAL FOR AND COMPUTER ENGINEERS S E C O N D E D I T I O N K11329_FM.indd 1 8/26/11 10:29 AM K11329_FM.indd 2 8/26/11 10:29 AM PROBABILITY AND RANDOM PROCESSES ELECTRICAL FOR AND COMPUTER ENGINEERS S E C O N D E D I T I O N CHARLES W. THERRIEN MURALI TUMMALA Boca Raton London New York CRC Press is an imprint of the Taylor & Francis Group, an informa business K11329_FM.indd 3 8/26/11 10:29 AM MATLAB® is a trademark of The MathWorks, Inc. and is used with permission. The MathWorks does not warrant the accuracy of the text or exercises in this book. This book’s use or discussion of MATLAB® software or related products does not consti- tute endorsement or sponsorship by The MathWorks of a particular pedagogical approach or particular use of the MATLAB® software. CRC Press Taylor & Francis Group 6000 Broken Sound Parkway NW, Suite 300 Boca Raton, FL 33487-2742 © 2012 by Taylor & Francis Group, LLC CRC Press is an imprint of Taylor & Francis Group, an Informa business No claim to original U.S. Government works Version Date: 20111103 International Standard Book Number-13: 978-1-4398-9749-2 (eBook - PDF) This book contains information obtained from authentic and highly regarded sources. Reasonable efforts have been made to publish reliable data and information, but the author and publisher cannot assume responsibility for the validity of all materials or the consequences of their use. The authors and publishers have attempted to trace the copyright holders of all material repro- duced in this publication and apologize to copyright holders if permission to publish in this form has not been obtained. If any copyright material has not been acknowledged please write and let us know so we may rectify in any future reprint. Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced, transmitted, or utilized in any form by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying, microfilming, and recording, or in any information storage or retrieval system, without written permission from the publishers. For permission to photocopy or use material electronically from this work, please access www.copyright.com (http://www.copy- right.com/) or contact the Copyright Clearance Center, Inc. (CCC), 222 Rosewood Drive, Danvers, MA 01923, 978-750-8400. CCC is a not-for-profit organization that provides licenses and registration for a variety of users. For organizations that have been granted a photocopy license by the CCC, a separate system of payment has been arranged. Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used only for identifica- tion and explanation without intent to infringe. Visit the Taylor & Francis Web site at http://www.taylorandfrancis.com and the CRC Press Web site at http://www.crcpress.com To all the teachers of probability – especially Alvin Drake, who made it fun to learn and Athanasios Papoulis, who made it just rigorous enough for engineers. Also, to our families for their enduring support. Contents Preface to the Second Edition xiii Preface xv Part I Probability and Random Variables 1 1 Introduction 3 1.1 The Analysis of RandomExperiments . . . . . . . . . . . . . . . . . 3 1.2 Probabilityin Electrical and Computer Engineering . . . . . . . . . 5 1.2.1 Signal detection and classification . . . . . . . . . . . . . . . . 5 1.2.2 Speech modeling and recognition . . . . . . . . . . . . . . . . 6 1.2.3 Coding and data transmission . . . . . . . . . . . . . . . . . . 7 1.2.4 Computer networks . . . . . . . . . . . . . . . . . . . . . . . . 8 1.3 Outline of the Book . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 2 The Probability Model 11 2.1 The Algebra of Events . . . . . . . . . . . . . . . . . . . . . . . . . . 11 2.1.1 Basic operations . . . . . . . . . . . . . . . . . . . . . . . . . . 12 2.1.2 Representation of the sample space . . . . . . . . . . . . . . . 13 2.2 Probabilityof Events . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 2.2.1 Defining probability. . . . . . . . . . . . . . . . . . . . . . . . 16 2.2.2 Statistical independence . . . . . . . . . . . . . . . . . . . . . 20 2.3 Some Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 2.3.1 Repeated independent trials . . . . . . . . . . . . . . . . . . . 21 2.3.2 Problems involvingcounting . . . . . . . . . . . . . . . . . . . 22 2.3.3 Network reliability . . . . . . . . . . . . . . . . . . . . . . . . 24 2.4 ConditionalProbabilityand Bayes’ Rule . . . . . . . . . . . . . . . . 26 2.4.1 Conditionalprobability . . . . . . . . . . . . . . . . . . . . . . 26 2.4.2 Event trees . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 2.4.3 Bayes’ rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 2.5 More Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 2.5.1 The binary communicationchannel . . . . . . . . . . . . . . . 31 2.5.2 Measuring informationand coding . . . . . . . . . . . . . . . 32 2.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 3 Random Variables and Transformations 51 3.1 Discrete RandomVariables . . . . . . . . . . . . . . . . . . . . . . . 51 3.2 Some Common Discrete ProbabilityDistributions . . . . . . . . . . . 53 3.2.1 Bernoulli random variable . . . . . . . . . . . . . . . . . . . . 54 3.2.2 Binomialrandom variable . . . . . . . . . . . . . . . . . . . . 54 3.2.3 Geometric random variable . . . . . . . . . . . . . . . . . . . 55 3.2.4 Poisson random variable . . . . . . . . . . . . . . . . . . . . . 57 3.2.5 Discrete uniform random variable . . . . . . . . . . . . . . . . 58 3.2.6 Other types of discrete random variables . . . . . . . . . . . . 58 3.3 Continuous RandomVariables . . . . . . . . . . . . . . . . . . . . . . 59 vii viii CONTENTS 3.3.1 Probabilistic description . . . . . . . . . . . . . . . . . . . . . 59 3.3.2 More about the PDF . . . . . . . . . . . . . . . . . . . . . . . 60 3.3.3 A relation to discrete random variables . . . . . . . . . . . . . 61 3.3.4 Solving problems . . . . . . . . . . . . . . . . . . . . . . . . . 62 3.4 Some Common Continuous Probability Density Functions . . . . . . 64 3.4.1 Uniform random variable . . . . . . . . . . . . . . . . . . . . . 64 3.4.2 Exponential random variable. . . . . . . . . . . . . . . . . . . 65 3.4.3 Gaussian random variable . . . . . . . . . . . . . . . . . . . . 66 3.4.4 Other types of continuous random variables . . . . . . . . . . 69 3.5 CDF and PDF for Discrete and Mixed Random Variables . . . . . . 69 3.5.1 Discrete random variables . . . . . . . . . . . . . . . . . . . . 69 3.5.2 Mixed random variables . . . . . . . . . . . . . . . . . . . . . 71 3.6 Transformationof Random Variables . . . . . . . . . . . . . . . . . . 72 3.6.1 When the transformation is invertible. . . . . . . . . . . . . . 72 3.6.2 When the transformation is not invertible . . . . . . . . . . . 75 3.6.3 When the transformation has flat regions or discontinuities. . 77 3.7 Distributions Conditioned on an Event . . . . . . . . . . . . . . . . . 81 3.8 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83 3.8.1 Digitalcommunication . . . . . . . . . . . . . . . . . . . . . . 84 3.8.2 Radar/sonar target detection . . . . . . . . . . . . . . . . . . 90 3.8.3 Object classification . . . . . . . . . . . . . . . . . . . . . . . 93 3.9 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 4 Expectation, Moments, and Generating Functions 113 4.1 Expectation of a Random Variable . . . . . . . . . . . . . . . . . . . 113 4.1.1 Discrete random variables . . . . . . . . . . . . . . . . . . . . 114 4.1.2 Continuous random variables . . . . . . . . . . . . . . . . . . 115 4.1.3 Invariance of expectation . . . . . . . . . . . . . . . . . . . . . 115 4.1.4 Properties of expectation . . . . . . . . . . . . . . . . . . . . . 117 4.1.5 Expectation conditioned on an event . . . . . . . . . . . . . . 118 4.2 Moments of a Distribution . . . . . . . . . . . . . . . . . . . . . . . . 118 4.2.1 Central moments . . . . . . . . . . . . . . . . . . . . . . . . . 119 4.2.2 Properties of variance. . . . . . . . . . . . . . . . . . . . . . . 120 4.2.3 Some higher-order moments . . . . . . . . . . . . . . . . . . . 121 4.3 Generating Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . 122 4.3.1 The moment-generating function . . . . . . . . . . . . . . . . 122 4.3.2 The probability-generating function . . . . . . . . . . . . . . . 124 4.4 Application:Entropy and Source Coding . . . . . . . . . . . . . . . . 126 4.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127 5 Two and More Random Variables 141 5.1 Two Discrete Random Variables . . . . . . . . . . . . . . . . . . . . 141 5.1.1 The joint PMF . . . . . . . . . . . . . . . . . . . . . . . . . . 142 5.1.2 Independent random variables . . . . . . . . . . . . . . . . . . 143 5.1.3 ConditionalPMFs for discrete random variables . . . . . . . . 144 5.1.4 Bayes’ rule for discrete random variables . . . . . . . . . . . . 145 5.2 Two Continuous Random Variables . . . . . . . . . . . . . . . . . . . 147 5.2.1 Joint distributions . . . . . . . . . . . . . . . . . . . . . . . . 147 5.2.2 Marginal PDFs: Projections of the joint density . . . . . . . . 150 5.2.3 ConditionalPDFs: Slices of the joint density . . . . . . . . . . 152 CONTENTS ix 5.2.4 Bayes’ rule for continuous random variables . . . . . . . . . . 155 5.3 Expectation and Correlation. . . . . . . . . . . . . . . . . . . . . . . 157 5.3.1 Correlation and covariance . . . . . . . . . . . . . . . . . . . . 158 5.3.2 Conditionalexpectation . . . . . . . . . . . . . . . . . . . . . 161 5.4 Gaussian RandomVariables . . . . . . . . . . . . . . . . . . . . . . . 163 5.5 Multiple RandomVariables . . . . . . . . . . . . . . . . . . . . . . . 165 5.5.1 PDFs for multiplerandom variables. . . . . . . . . . . . . . . 165 5.5.2 Sums of random variables . . . . . . . . . . . . . . . . . . . . 167 5.6 Sums of some common random variables . . . . . . . . . . . . . . . . 170 5.6.1 Bernoulli random variables . . . . . . . . . . . . . . . . . . . . 170 5.6.2 Geometric random variables . . . . . . . . . . . . . . . . . . . 171 5.6.3 Exponential random variables . . . . . . . . . . . . . . . . . . 171 5.6.4 Gaussian random variables . . . . . . . . . . . . . . . . . . . . 173 5.6.5 Squared Gaussian random variables . . . . . . . . . . . . . . . 173 5.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173 6 Inequalities, Limit Theorems, and Parameter Estimation 187 6.1 Inequalities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187 6.1.1 Markov inequality . . . . . . . . . . . . . . . . . . . . . . . . . 188 6.1.2 Chebyshev inequality . . . . . . . . . . . . . . . . . . . . . . . 189 6.1.3 One-sided Chebyshev inequality . . . . . . . . . . . . . . . . . 190 6.1.4 Other inequalities . . . . . . . . . . . . . . . . . . . . . . . . . 190 6.2 Convergence and Limit Theorems . . . . . . . . . . . . . . . . . . . . 191 6.2.1 Laws of large numbers . . . . . . . . . . . . . . . . . . . . . . 193 6.2.2 Central limittheorem . . . . . . . . . . . . . . . . . . . . . . . 194 6.3 Estimation of Parameters . . . . . . . . . . . . . . . . . . . . . . . . 196 6.3.1 Estimates and properties . . . . . . . . . . . . . . . . . . . . . 196 6.3.2 Sample mean and variance . . . . . . . . . . . . . . . . . . . . 197 6.4 MaximumLikelihoodEstimation . . . . . . . . . . . . . . . . . . . . 199 6.5 Point Estimates and Confidence Intervals . . . . . . . . . . . . . . . 203 6.6 Applicationto Signal Estimation . . . . . . . . . . . . . . . . . . . . 205 6.6.1 Estimating a signalin noise . . . . . . . . . . . . . . . . . . . 206 6.6.2 Choosing the number of samples . . . . . . . . . . . . . . . . 207 6.6.3 Applying confidence intervals . . . . . . . . . . . . . . . . . . 210 6.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211 7 Random Vectors 219 7.1 RandomVectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219 7.1.1 Cumulative distribution and density functions . . . . . . . . . 219 7.1.2 Random vectors with independent components . . . . . . . . 221 7.2 Analysis of Random Vectors . . . . . . . . . . . . . . . . . . . . . . . 221 7.2.1 Expectation and moments . . . . . . . . . . . . . . . . . . . . 222 7.2.2 Estimating moments from data . . . . . . . . . . . . . . . . . 223 7.2.3 The multivariateGaussian density . . . . . . . . . . . . . . . 225 7.2.4 Random vectors with uncorrelated components . . . . . . . . 226 7.3 Transformations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226 7.3.1 Transformationof moments . . . . . . . . . . . . . . . . . . . 226 7.3.2 Transformationof density functions . . . . . . . . . . . . . . . 227 7.4 Cross Correlation and Covariance . . . . . . . . . . . . . . . . . . . . 231 7.5 Applications to SignalProcessing . . . . . . . . . . . . . . . . . . . . 232

See more

The list of books you might like

Upgrade Premium
Most books are stored in the elastic cloud where traffic is expensive. For this reason, we have a limit on daily download.
DMCA & Copyright: Dear all, most of the website is community built, users are uploading hundred of books everyday, which makes really hard for us to identify copyrighted material, please contact us if you want any material removed.
Copyright @ PDFdrive.to, 2022 | We love our users