Preface xv Introduction 1 Why Study Probability? 1 Software Use in Probability 2 Modern Application of Classic Probability Problems 2 Applications to Business 3 Applications to the Life Sciences 4 Applications to Engineering and Operations Research 4 Applications to Finance 6 Probability in Everyday Life 7 1 Introduction to Probability 13 Introduction 13 1.1 Sample Spaces and Events 13 The Sample Space of an Experiment 13 Events 15 Some Relations from Set Theory 16 Exercises Section 1.1 (1–12) 18 1.2 Axioms Interpretations and Properties of Probability 19 Interpreting Probability 21 More Probability Properties 23 Contingency Tables 25 Determining Probabilities Systematically 26 Equally Likely Outcomes 27 Exercises Section 1.2 (13–30) 28 1.3 Counting Methods 30 The Fundamental Counting Principle 31 Tree Diagrams 32 Permutations 33 Combinations 34 Partitions 38 Exercises Section 1.3 (31–50) 39 Supplementary Exercises (51–62) 42 2 Conditional Probability and Independence 45 Introduction 45 2.1 Conditional Probability 45 The Definition of Conditional Probability 46 The Multiplication Rule for P(A ∩ B) 49 2.2 The Law of Total Probability and Bayes’ Theorem 52 The Law of Total Probability 52 Bayes’ Theorem 55 Exercises Section 2.2 (17–32) 59 2.3 Independence 61 The Multiplication Rule for Independent Events 63 Independence of More Than Two Events 65 Exercises Section 2.3 (33–54) 66 2.4 Simulation of Random Events 69 The Backbone of Simulation: Random Number Generators 70 Precision of Simulation 73 Exercises Section 2.4 (55–74) 74 Supplementary Exercises (75–100) 77 3 Discrete Probability Distributions:general Properties 82 Introduction 82 3.1 Random Variables 82 Two Types of Random Variables 84 Exercises Section 3.1 (1–10) 85 3.2 Probability Distributions for Discrete Random Variables 86 Another View of Probability Mass Functions 89 Exercises Section 3.2 (11–21) 90 3.3 The Cumulative Distribution Function 91 Exercises Section 3.3 (22–30) 95 3.4 Expected Value and Standard Deviation 96 The Expected Value of X 97 The Expected Value of a Function 99 The Variance and Standard Deviation of X 102 Properties of Variance 104 Exercises Section 3.4 (31–50) 105 3.5 Moments and Moment Generating Functions 108 The Moment Generating Function 109 Obtaining Moments from the MGF 111 Exercises Section 3.5 (51–64) 113 3.6 Simulation of Discrete Random Variables 114 Simulations Implemented in R and Matlab 117 Simulation Mean Standard Deviation and Precision 117 Exercises Section 3.6 (65–74) 119 Supplementary Exercises (75–84) 120 4 Families of Discrete Distributions 122 Introduction 122 4.1 Parameters and Families of Distributions 122 Exercises Section 4.1 (1–6) 124 4.2 The Binomial Distribution 125 The Binomial Random Variable and Distribution 127 Computing Binomial Probabilities 129 The Mean Variance and Moment Generating Function 130 Binomial Calculations with Software 132 Exercises Section 4.2 (7–34) 132 4.3 The Poisson Distribution 136 The Poisson Distribution as a Limit 137 The Mean Variance and Moment Generating Function 139 The Poisson Process 140 Poisson Calculations with Software 141 Exercises Section 4.3 (35–54) 142 4.4 The Hypergeometric Distribution 145 Mean and Variance 148 Hypergeometric Calculations with Software 149 Exercises Section 4.4 (55–64) 149 4.5 The Negative Binomial and Geometric Distributions 151 The Geometric Distribution 152 Mean Variance and Moment Generating Function 152 Alternative Definitions of the Negative Binomial Distribution 153 Negative Binomial Calculations with Software 154 Exercises Section 4.5 (65–78) 154 Supplementary Exercises (79–100) 156 5 Continuous Probability Distributions:general Properties 160 Introduction 160 5.1 Continuous Random Variables and Probability Density Functions 160 Probability Distributions for Continuous Variables 161 Exercises Section 5.1 (1–8) 165 5.2 The Cumulative Distribution Function and Percentiles 166 Using F(x) to Compute Probabilities 168 Obtaining f(x) fromF(x) 169 Percentiles of a Continuous Distribution 169 Exercises Section 5.2 (9–18) 171 5.3 Expected Values Variance and Moment Generating Functions 173 Expected Values 173 Variance and Standard Deviation 175 Properties of Expectation and Variance 176 Moment Generating Functions 177 Exercises Section 5.3 (19–38) 179 5.4 Transformation of a Random Variable 181 Exercises Section 5.4 (39–54) 185 5.5 Simulation of Continuous Random Variables 186 The Inverse CDF Method 186 The Accept–Reject Method 189 Precision of Simulation Results 191 Exercises Section 5.5 (55–63) 191 Supplementary Exercises (64–76) 193 6 Families of Continuous Distributions 196 Introduction 196 6.1 The Normal (Gaussian) Distribution 196 The Standard Normal Distribution 197 Arbitrary Normal Distributions 199 The Moment Generating Function 203 Normal Distribution Calculations with Software 204 Exercises Section 6.1 (1–27) 205 6.2 Normal Approximation of Discrete Distributions 208 Approximating the Binomial Distribution 209 Exercises Section 6.2 (28–36) 211 6.3 The Exponential and Gamma Distributions 212 The Exponential Distribution 212 The Gamma Distribution 214 The Gamma and Exponential MGFs 217 Gamma and Exponential Calculations with Software 218 Exercises Section 6.3 (37–50) 218 6.4 Other Continuous Distributions 220 The Weibull Distribution 220 The Lognormal Distribution 222 The Beta Distribution 224 Exercises Section 6.4 (51–66) 226 6.5 Probability Plots 228 Sample Percentiles 228 A Probability Plot 229 Departures from Normality 232 Beyond Normality 234 Probability Plots in Matlab and R 236 Exercises Section 6.5 (67–76) 237 Supplementary Exercises (77–96) 238 7 Joint Probability Distributions 242 Introduction 242 7.1 Joint Distributions for Discrete Random Variables 242 The Joint Probability Mass Function for Two Discrete Random Variables 242 Marginal Probability Mass Functions 244 Independent Random Variables 245 More Than Two Random Variables 246 Exercises Section 7.1 (1–12) 248 7.2 Joint Distributions for Continuous Random Variables 250 The Joint Probability Density Function for Two Continuous Random Variables 250 Marginal Probability Density Functions 252 Independence of Continuous Random Variables 254 More Than Two Random Variables 255 Exercises Section 7.2 (13–22) 257 7.3 Expected Values Covariance and Correlation 258 Properties of Expected Value 260 Covariance 261 Correlation 263 Correlation Versus Causation 265 Exercises Section 7.3 (23–42) 266 7.4 Properties of Linear Combinations 267 Expected Value and Variance of a Linear Combination 268 The PDF of a Sum 271 Moment Generating Functions of Linear Combinations 273 Exercises Section 7.4 (43–65) 275 7.5 The Central Limit Theorem and the Law of Large Numbers 278 Random Samples 278 The Central Limit Theorem 282 A More General Central Limit Theorem 286 Other Applications of the Central Limit Theorem 287 The Law of Large Numbers 288 Proof of the Central Limit Theorem 290 Exercises Section 7.5 (66–82) 290 7.6 Simulation of Joint Probability Distributions 293 Simulating Values from a Joint PMF 293 Simulating Values from a Joint PDF 295 Exercises Section 7.6 (83–90) 297 Supplementary Exercises (91–124) 298 8 Joint Probability Distributions:additional Topics 304 Introduction 304 8.1 Conditional Distributions and Expectation 304 Conditional Distributions and Independence 306 Conditional Expectation and Variance 307 The Laws of Total Expectation and Variance 308 Exercises Section 8.1 (1–18) 313 8.2 The Bivariate Normal Distribution 315 Conditional Distributions of X and Y 317 Regression to the Mean 318 The Multivariate Normal Distribution 319 Bivariate Normal Calculations with Software 319 Exercises Section 8.2 (19–30) 320 8.3 Transformations of Jointly Distributed Random Variables 321 The Joint Distribution of Two New Random Variables 322 The Distribution of a Single New RV 323 The Joint Distribution of More Than Two New Variables 325 Exercises Section 8.3 (31–38) 326 8.4 Reliability 327 The Reliability Function 327 Series and Parallel System Designs 329 Mean Time to Failure 331 The Hazard Function 332 Exercises Section 8.4 (39–50) 335 8.5 Order Statistics 337 The Distributions of Yn and Y1  337 The Distribution of the ith Order Statistic 339 The Joint Distribution of All n Order Statistics 340 Exercises Section 8.5 (51–60) 342 8.6 Further Simulation Tools for Jointly Distributed Random Variables 343 The Conditional Distribution Method of Simulation 343 Simulating a Bivariate Normal Distribution 344 Simulation Methods for Reliability 346 Exercises Section 8.6 (61–68) 347 Supplementary Exercises (69–82) 348 9 the Basics of Statistical Inference 351 Introduction 351 9.1 Point Estimation 351 Estimates and Estimators 352 Assessing Estimators: Accuracy and Precision 354 Exercises Section 9.1 (1–18) 357 9.2 Maximum Likelihood Estimation 360 Some Properties of MLEs 366 Exercises Section 9.2 (19–30) 367 9.3 Statistical Intervals 368 Constructing a Confidence Interval 369 Confidence Intervals for a Population Proportion 369 Confidence Intervals for a Population Mean 371 Further Comments on Statistical Intervals 375 Confidence Intervals with Software 375 Exercises Section 9.3 (31–48) 376 9.4 Hypothesis Tests 379 Hypotheses and Test Procedures 380 Hypothesis Testing for a Population Mean 381 Errors in Hypothesis Testing and the Power of a Test 385 Hypothesis Testing for a Population Proportion 388 Software for Hypothesis Test Calculations 389 Exercises Section 9.4 (49–71) 391 9.5 Bayesian Estimation 393 The Posterior Distribution of a Parameter 394 Inferences from the Posterior Distribution 397 Further Comments on Bayesian Inference 398 Exercises Section 9.5 (72–80) 399 9.6 Simulation-Based Inference 400 The Bootstrap Method 400 Interval Estimation Using the Bootstrap 402 Hypothesis Tests Using the Bootstrap 404 More on Simulation-Based Inference 405 Exercises Section 9.6 (81–90) 405 Supplementary Exercises (91–116) 407 10 Markov Chains 411 Introduction 411 10.1 Terminology and Basic Properties 411 The Markov Property 413 Exercises Section 10.1 (1–10) 416 10.2 The Transition Matrix and the Chapman–Kolmogorov Equations 418 The Transition Matrix 418 Computation of Multistep Transition Probabilities 419 Exercises Section 10.2 (11–22) 423 10.3 Specifying an Initial Distribution 426 A Fixed Initial State 428 Exercises Section 10.3 (23–30) 429 10.4 Regular Markov Chains and the Steady-State Theorem 430 Regular Chains 431 The Steady-State Theorem 432 Interpreting the Steady-State Distribution 433 Efficient Computation of Steady-State Probabilities 435 Irreducible and Periodic Chains 437 Exercises Section 10.4 (31–43) 438 10.5 Markov Chains with Absorbing States 440 Time to Absorption 441 Mean Time to Absorption 444 Mean First Passage Times 448 Probabilities of Eventual Absorption 449 Exercises Section 10.5 (44–58) 451 10.6 Simulation of Markov Chains 453 Exercises Section 10.6 (59–66) 459 Supplementary Exercises (67–82) 461 11 Random Processes 465 Introduction 465 11.1 Types of Random Processes 465 Classification of Processes 468 Random Processes and Their Associated Random Variables 469 Exercises Section 11.1 (1–10) 470 11.2 Properties of the Ensemble: Mean and Autocorrelation Functions 471 Mean and Variance Functions 471 Autocovariance and Autocorrelation Functions 475 The Joint Distribution of Two Random Processes 477 Exercises Section 11.2 (11–24) 478 11.3 Stationary and Wide-Sense Stationary Processes 479 Properties of WSS Processes 483 Ergodic Processes 486 Exercises Section 11.3 (25–40) 488 11.4 Discrete-Time Random Processes 489 Special Discrete Sequences 491 Exercises Section 11.4 (41–52) 493 Supplementary Exercises (53–64) 494 12 Families of Random Processes 497 Introduction 497 12.1 Poisson Processes 497 Relation to Exponential and Gamma Distributions 499 Combining and Decomposing Poisson Processes 502 Alternative Definition of a Poisson Process 504 Nonhomogeneous Poisson Processes 505 The Poisson Telegraphic Process 506 Exercises Section 12.1 (1–18) 507 12.2 Gaussian Processes 509 Brownian Motion 510 Brownian Motion as a Limit 512 Further Properties of Brownian Motion 512 Variations on Brownian Motion 514 Exercises Section 12.2 (19–28) 515 12.3 Continuous-Time Markov Chains 516 Infinitesimal Parameters and Instantaneous Transition Rates 518 Sojourn Times and Transitions 520 Long-Run Behavior of Continuous-Time Markov Chains 523 Explicit Form of the Transition Matrix 526 Exercises Section 12.3 (29–40) 527 Supplementary Exercises (41–51) 529 13 Introduction to Signal Processing 532 Introduction 532 13.1 Power Spectral Density 532 Expected Power and the Power Spectral Density 532 Properties of the Power Spectral Density 535 Power in a Frequency Band 538 White Noise Processes 539 Cross-Power Spectral Density for Two Processes 541 Exercises Section 13.1 (1–21) 542 13.2 Random Processes and LTI Systems 544 Properties of the LTI System Output 545 Ideal Filters 548 Signal Plus Noise 551 Exercises Section 13.2 (22–38) 554 13.3 Discrete-Time Signal Processing 556 Random Sequences and LTI Systems 558 Sampling Random Sequences 560 Exercises Section 13.3 (39–50) 562 A Statistical Tables A- 1 A. 1 Binomial CDF A- 1 A. 2 Poisson CDF A- 4 A. 3 Standard Normal CDF A- 5 A. 4 Incomplete Gamma Function A- 7 A. 5 Critical Values for t Distributions A- 7 A. 6 Tail Areas of t Distributions A- 9 B Background Mathematics A- 13 B. 1 Trigonometric Identities A- 13 B. 2 Special Engineering Functions A- 13 B. 3 o(h) Notation A- 14 B. 4 The Delta Function A- 14 B. 5 Fourier Transforms A- 15 B. 6 Discrete-Time Fourier Transforms A- 16 C Important Probability Distributions A- 18 C. 1 Discrete Distributions A- 18 C. 2 Continuous Distributions A- 20 C. 3 Matlab and R Commands A- 23 Bibliography B- 1 Answers to Odd-numbered Exercises S- 1 Index I- 1