Stochastic Processes: An Introduction Solutions Manual Third Edition (cid:13)c Peter W Jones and Peter Smith School of Computing and Mathematics, Keele University, UK Preface The website includes answers and solutions of all the end-of-chapter problems in the textbook Stochastic Processes: An Introduction, third edition. We hope that they will prove helpful to lecturersindesigningcourses,andtostudentsasasourceofmodelexamples. Theoriginalproblems as numbered in the text are also included. There are obviously references to results and examples from the textbook, and the manual shouldbe viewedas a supplement to the book. To help identify the sections andchapters,the full contents of Stochastic Processes follow this preface. Every effort has been made to eliminate misprints or errors (or worse), and the authors, who were responsible for the LaTeX code, apologise in advance for any which occur. Peter W. Jones Peter Smith Keele, 2017 1 Contents of Stochastic Processes Chapter 1: Some Background in Probability 1.1 Introduction 1.2 Probability 1.3 Conditional probability and independence 1.4 Discrete random variables 1.5 Continuous random variables 1.6 Mean and variance 1.7 Some standard discrete probability distributions 1.8 Some standard continuous probability distributions 1.9 Generating functions 1.10 Conditional expectation Problems Chapter 2: Some Gambling Problems 2.1 Gambler’s ruin 2.2 Probability of ruin 2.3 Some numerical simulations 2.4 Expected duration of the game 2.5 Some variations of gambler’s ruin 2.5.1 The infinitely rich opponent 2.5.2 The generous gambler 2.5.3 Changing the stakes Problems Chapter 3: Random Walks 3.1 Introduction 3.2 Unrestricted random walks 3.3 Probability distribution after n steps 3.4 First returns of the symmetric random walk Problems Chapter 4: Markov Chains 4.1 States and transitions 4.2 Transition probabilities 4.3 General two-state Markov chain 4.4 Powers of the transition matrix for the m-state chain 4.5 Gambler’s ruin as a Markov chain 4.6 Classification of states 4.7 Classification of chains 4.8 A wildlife Markov chain model Problems Chapter 5: Poisson Processes 5.1 Introduction 5.2 The Poisson process 5.3 Partition theorem approach 5.4 Iterative method 5.5 The generating function 5.6 Variance for the Poissonprocess 2 5.7 Arrival times 5.8 Summary of the Poisson process Problems Chapter 6: Birth and Death Processes 6.1 Introduction 6.2 The birth process 6.3 Birth process: generating function equation 6.4 The death process 6.5 The combined birth and death process 6.6 General population processes Problems Chapter 7: Queues 7.1 Introduction 7.2 The single server queue 7.3 The stationary process 7.4 Queues with multiple servers 7.5 Queues with fixed service times 7.6 Classification of queues Problems Chapter 8: Reliability and Renewal 8.1 Introduction 8.2 The reliability function 8.3 The exponential distribution and reliability 8.4 Mean time to failure 8.5 Reliability of series and parallel systems 8.6 Renewal processes 8.7 Expected number of renewals Problems Chapter 9: Branching and Other Random Processes 9.1 Introduction 9.2 Generational growth 9.3 Mean and variance 9.4 Probability of extinction 9.5 Branching processes and martingales 9.6 Stopping rules 9.7 A continuous time epidemic 9.8 A discrete time epidemic model 9.9 Deterministic epidemic models 9.10 An iterative scheme for the simple epidemic Problems Chapter 10: Brownian Motion: Wiener Process 10.1 Introduction 10.2 Brownian motion 10.3 Wiener process as a limit of a random walk 10.4 Brownian motion with drift 10.5 Scaling 10.6 First visit times 3 10.7 Other Brownian motions in one dimension 10.8 Brownian motion in more than one dimension Problems Chapter 11: Computer Simulations and Projects 4 Contents of the Solutions Manual Chapter 1: Some Background in Probability 6 Chapter 2: Some Gambling Problems 20 Chapter 3: Random Walks 34 Chapter 4: Markov Chains 49 Chapter 5: Poisson Processes 74 Chapter 6: Birth and Death Processes 80 Chapter 7: Queues 105 Chapter 8: Reliability and Renewal 121 Chapter 9: Branching and Other Random Processes 130 Chapter 10: Brownian Motion: Wiener Process 150 5 Chapter 1 Some background in probability 1.1. The Venn diagram of three events is shown in Figure 1.5(in the text). Indicate on the diagram the following events: (a) A B; (b) A (B C); (c) A (B C); (d) (A C)c; (e) (A B) Cc. ∪ ∪ ∪ ∩ ∪ ∩ ∩ ∪ S S A B A B C C (a) (b) S S A B A B C C (c) (d) S A B C (e) Figure 1.1: The eventsare shaded in Figure 1.1. 1.2. In a random experiment, A, B, C are three events. In set notation write down expressions for the events: (a) only A occurs; (b) all three events A, B, C occur; (c) A and B occur but C does not; (d) at least one of the events A, B, C occurs; (e) exactly one of the events A, B, C occurs; (f) not more than two of the events occur. (a) A (B C)c; (b) A (B C)=A B C; (c) (A B) Cc; (d) A B C; ∩ ∪ ∩ ∩ ∩ ∩ ∩ ∩ ∪ ∪ 6 (e) A (B C)c representsan event in A butnot in either B norC: therefore theanswer is ∩ ∪ (A (B C)c) (B (A C)c) (C (A B)c). ∩ ∪ ∪ ∩ ∪ ∪ ∩ ∪ (f) A B C ∩ ∩ 1.3. For two events A and B, P(A)=0.4, P(B)=0.5 and P(A B)=0.3. Calculate ∩ (a) P(A B); (b) P(A Bc); (c) P(Ac Bc). ∪ ∩ ∪ (a) From (1.1) P(A B)=P(A)+P(B) P(A B), it follows that ∪ − ∩ P(A B)=0.4+0.5 0.3=0.6. ∪ − (b) SinceA=(A Bc) (A B) and A Bc, and A B are mutually exclusive, then, ∩ ∪ ∩ ∩ ∩ P(A)=P[(A Bc) (A B)]=P(A Bc)+P(A B), ∩ ∪ ∩ ∩ ∩ so that P(A Bc)=P(A) P(A B)=0.4 0.3=0.1. ∩ − ∩ − (c) Since Ac Bc =(A B)c, then ∪ ∩ P(Ac Bc)=P[(A B)c]=1 P(A B)=1 0.3=0.7. ∪ ∩ − ∩ − 1.4. Two distinguishable fair dice a and b are rolled. What are the elements of the sample space? What is the probability that the sum of the face values of the two dice is9? What isthe probability that at least one 5 or at least one 3 appears? The 36 elements of the sample space are listed in Example 1.1. The event A , that the sum is 9, is 1 given by A = (3,6),(4,5),(5,4),(6,3) . 1 { } HenceP= 4 = 1. 36 9 Let A be the event that at least one 5 or at least one 3 appears. Then by counting the elements in 2 thesample space in Example 1.1, P(A )= 20 = 5. 2 36 9 1.5. Two distinguishable fair dice a and b are rolled. What is the probability that the sum of the faces is not more than 6? LettherandomvariableX bethesumofthefaces. BycountingeventsinthesamplespaceinExample 1.1, P(X)= 15 = 5. 36 12 1.6. For the probability generating function 1 G(s)=(2 s)−2 − find p and its mean. n { } Note that G(1)=1. Usingthe binomial theorem (see the Appendix) 1 1 1 1 ∞ 1 s n 1 G(s)= (1 s)−2 = −2 where −2 =1. √2 − 2 √2 n 2 0 Xn=0(cid:18) (cid:19)(cid:16) (cid:17) (cid:18) (cid:19) Hence 1 1 1 1 p0 = √2, pn= √22n −n2 , (n=1,2,...). (cid:18) (cid:19) The mean is µ=G′(1)= 21(2−s)−32 = 12. s=1 (cid:12) (cid:12) (cid:12) 7 1.7. Findtheprobabilitygenerating functionG(s)ofthePoissondistribution(see Section1.7)withparam- eter α given by e−ααn p = , n=0,1,2,.... n n! Determine the mean and variance of p from the generating function. n { } Given pn =e−ααn/n!, thegenerating function is given by G(s)= ∞ pnsn = ∞ e−αnα!nsn =e−α ∞ (αns!)n =eα(s−1). n=0 n=0 n=0 X X X The mean and variance are given by d µ=G′(1)= eα(s−1) =α, ds s=1 (cid:2) (cid:3) σ2 =G′′(1)+G′(1) [G′(1)]2 =[α2eα(s−1)+αeα(s−1) α2e2α(s−1)]s=1 =α. − − 1.8. A panel contains n warning lights. The times to failure of the lights are the independent random variables T ,T ,...,T which have exponential distributions with parameters α ,α ,...,α respectively. 1 2 n 1 2 n Let T be the random variable of the time to first failure, that is T =min T ,T ,...,T . 1 2 n { } Show that T has an exponential distribution with parameter n α . j=1 j P The probability that no warning light has failed by time t is P(T t) = P(T t T t T t) 1 2 n ≥ ≥ ∩ ≥ ∩···∩ ≥ = P(T t)P(T t) P(T t) 1 2 n ≥ ≥ ··· ≥ = e−α1te−α2t e−αnt =e−(α1+α2+···+αn)t. ··· 1.9. The geometric distribution with parameter p is given by p(x)=qx−1p, x=1,2,... whereq=1 p(seeSection1.7). Finditsprobabilitygeneratingfunction. Calculatethemeanandvariance − of the geometric distribution from its pgf. The generating function is given by ∞ p ∞ p qs ps G(s)= qx−1psx = (qs)x = = , q q1 qs 1 qs x=1 x=1 − − X X usingthe formula for thesum of a geometric series. The mean is given by d ps p pqs 1 µ=G′(1)= = + = . ds 1 qs 1 qs (1 qs)2 p (cid:20) − (cid:21)s=1 (cid:20) − − (cid:21)s=1 Forthe variance, d p 2pq G′′(s)= = . ds (1 qs)2 (1 qs)3 (cid:20) − (cid:21) − is required. Hence 2q 1 1 q σ2 =G′′(1)+G′(1) [G′(1)]2 = + = . − p2 p − p2 p2 8 1.10. Two distinguishable fair dice a and b are rolled. What are the probabilities that: (a) at least one 4 appears; (b) only one 4 appears; (c) the sum of the face values is 6; (d) the sum of the face values is 5 and one 3 is shown; (e) the sum of the face values is 5 or only one 3 is shown? From theTable in Example 1.1: (a) If A is theevent that at least one 4 appears, then P(A )= 11. 1 1 36 (b) IfA is the eventthat only one 4 appears, then P(A )= 10 = 5. 2 2 36 18 (c) If A is theevent that thesum of the faces is 6, then P(A )= 5 . 3 3 36 (d) IfA is the eventthat theface valuesis 5 and one 3 is shown, then P(A )= 2 = 1 . 4 4 36 18 (e) If A is theevent that thesum of thefaces is 5 or only one3 is shown, then P(A )= 10 = 5 . 5 5 36 18 1.11. Two distinguishable fair dice a and b are rolled. What is the expected sum of the face values? What is the variance of the sum of the face values? Let N be the random variable representing the sum x+y, where x and y are face values of the two dice. Then 6 6 6 6 1 1 E(N)= (x+y)= 6 x+6 y =7. 36 36 " # x=1y=1 x=1 y=1 XX X X and 6 6 1 V(N) = E(N2) E(N)2 = (x+y)2 72 − 36 − Xx=1Xy=1 6 6 2 1 = 12 x2+2 x 49 36 − " ! # x=1 x=1 X X 329 35 = 49= =5.833.... 6 − 6 1.12. Three distinguishable fair dice a, b and c are rolled. How many possible outcomes are there for the faces shown? When the dice are rolled, what is the probability that just two dice show the same face values and the third one is different? The sample space contains 63 =216 elements of theform, (in theorder a,b,c), S = (i,j,k) , (i=1,...,6;j =1,...,6;k=1,...,6). { } Let A be therequired event. Supposethat a and b have thesame face values, which can occur in 6 ways, andthatchasadifferentfacevaluewhichcanoccursin5ways. Hencethetotalnumberofwaysinwhich a and b are the same but c is different is 6 5= 30 ways. The faces b and c, and c and a could also be × the same so that the total number of ways for the possible outcome is 3 30 = 90 ways. Therefore the × required probability is 90 5 P(A)= = . 216 12 1.13. In a sample space S, the events B and C are mutually exclusive, but A and B, are not. Show that P(A (B C))=P(A)+P(B)+P(C) P(A (B C)). ∪ ∪ − ∩ ∪ From awell-shuffled packof52playingcards asinglecard israndomlydrawn. Findtheprobabilitythat it is a club or an ace or the king of hearts. From (1.1) (in thebook) P(A (B C))=P(A)+P(B C) P(A (B C)) (i). ∪ ∪ ∪ − ∩ ∪ 9