by Konstadinos G. Politis, Markos V. Koutras, and N. Balakrishnan
CONTENTS
1 The Concept of Probability 1
1.1 Chance Experiments – Sample Spaces 2
1.2 Operations Between Events 11
1.3 Probability as Relative Frequency 27
1.4 Axiomatic Definition of Probability 38
1.5 Properties of Probability 45
1.6 The Continuity Property of Probability 54
1.7 Basic Concepts and Formulas 60
1.8 Computational Exercises 61
1.9 Self-assessment Exercises 63
1.9.1 True–False Questions 63
1.9.2 Multiple Choice Questions 64
1.10 Review Problems 67
1.11 Applications 71
1.11.1 System Reliability 71
Key Terms 77
2 Finite Sample Spaces – Combinatorial Methods 79
2.1 Finite Sample Spaces with Events of Equal Probability 80
2.2 Main Principles of Counting 89
2.3 Permutations 96
2.4 Combinations 105
2.5 The Binomial Theorem 123
2.6 Basic Concepts and Formulas 132
2.7 Computational Exercises 133
2.8 Self-Assessment Exercises 139
2.8.1 True–False Questions 139
2.8.2 Multiple Choice Questions 140
2.9 Review Problems 143
2.10 Applications 150
2.10.1 Estimation of Population Size: Capture–Recapture
Method 150
Key Terms 152
3 Conditional Probability – Independent Events 153
3.1 Conditional Probability 154
3.2 The Multiplicative Law of Probability 166
3.3 The Law of Total Probability 174
3.4 Bayes’ Formula 183
3.5 Independent Events 189
3.6 Basic Concepts and Formulas 206
3.7 Computational Exercises 207
3.8 Self-assessment Exercises 210
3.8.1 True–False Questions 210
3.8.2 Multiple Choice Questions 211
3.9 Review Problems 214
3.10 Applications 220
3.10.1 Diagnostic and Screening Tests 220
Key Terms 223
4 Discrete Random Variables and Distributions 225
4.1 Random Variables 226
4.2 Distribution Functions 232
4.3 Discrete Random Variables 247
4.4 Expectation of a Discrete Random Variable 261
4.5 Variance of a Discrete Random Variable 281
4.6 Some Results for Expectation and Variance 293
4.7 Basic Concepts and Formulas 302
4.8 Computational Exercises 303
4.9 Self-Assessment Exercises 309
4.9.1 True–False Questions 309
4.9.2 Multiple Choice Questions 310
4.10 Review Problems 313
4.11 Applications 317
4.11.1 Decision Making Under Uncertainty 317
Key Terms 320
5 Some Important Discrete Distributions 321
5.1 Bernoulli Trials and Binomial Distribution 322
5.2 Geometric and Negative Binomial Distributions 337
5.3 The Hypergeometric Distribution 358
5.4 The Poisson Distribution 371
5.5 The Poisson Process 385
5.6 Basic Concepts and Formulas 394
5.7 Computational Exercises 395
5.8 Self-Assessment Exercises 399
5.8.1 True–False Questions 399
5.8.2 Multiple Choice Questions 401
5.9 Review Problems 403
5.10 Applications 411
5.10.1 Overbooking 411
Key Terms 414
6 Continuous Random Variables 415
6.1 Density Functions 416
6.2 Distribution for a Function of a Random Variable 431
6.3 Expectation and Variance 442
6.4 Additional Useful Results for the Expectation 451
6.5 Mixed Distributions 459
6.6 Basic Concepts and Formulas 468
6.7 Computational Exercises 469
6.8 Self-Assessment Exercises 474
6.8.1 True–False Questions 474
6.8.2 Multiple Choice Questions 476
6.9 Review Problems 479
6.10 Applications 486
6.10.1 Profit Maximization 486
Key Terms 490
7 Some Important Continuous Distributions 491
7.1 The Uniform Distribution 492
7.2 The Normal Distribution 501
7.3 The Exponential Distribution 531
7.4 Other Continuous Distributions 542
7.4.1 The Gamma Distribution 543
7.4.2 The Beta Distribution 548
7.5 Basic Concepts and Formulas 555
7.6 Computational Exercises 557
7.7 Self-Assessment Exercises 561
7.7.1 True–False Questions 561
7.7.2 Multiple Choice Questions 562
7.8 Review Problems 565
7.9 Applications 573
7.9.1 Transforming Data: The Lognormal Distribution 573
Key Terms 578
Appendix A Sums and Products 579
Appendix B Distribution Function of the Standard Normal Distribution 593
Appendix C Simulation 595
Appendix D Discrete and Continuous Distributions 599
Bibliography 603
Index 605
PREFACE
Probability theory deals with phenomena whose outcome is affected by random events, and therefore they cannot be predicted with certainty. For example, the result of throwing a coin or a dice, the time of occurrence of a natural phenomenon or disaster (e.g. snowfall, earthquake, tsunami etc.) are some of the cases where “randomness” plays an important role and the use of probability theory is almost inevitable.
It is more than five centuries ago, when the Italians Luca Pacioli, Niccolo Tartaglia, Galileo Galilei and the French Pierre de Fermat and Blaise Pascal started setting the foundations of probability theory. Nowadays this area has been fully developed as an independent research area and offers valuable tools for almost all applied sciences. As a consequence, introductory concepts of Probability Theory are taught in the first years of most University and College programs.
This book is an introductory textbook in probability and can be used by majors in Mathematics, Statistics, Physics, Computer Science, Actuarial Science, Operations Research, Engineering etc. No prior knowledge of probability theory is required. In most Universities and Colleges where an introductory Probability course, such as one that may be based on this textbook, is offered, it would normally follow a rigorous Calculus course. Consequently, the Probability course can make use of differential and integral calculus, and formal proofs for theorems and propositions may be presented to the students, thereof offering them a mathematically sound understanding of the field.
For this reason, we have taken a calculus-based approach in this textbook for teaching an introductory course on Probability. In doing so, we have also introduced some novelties hoping that these will be of benefit to both students and instructors.
In each chapter,we have included a sectionwith a series of examples/problems for which the use of a computer is required. We demonstrate, through ample examples, how one can make effective use of computers for understanding probability concepts and carrying out various probability calculations. For these examples it is suggested to use a computer algebra software such as Mathematica, Maple, Derive, etc. Such programs provide excellent tools for creating graphs in an easy way as well as for performing mathematical operations such as derivation, summation, integration, etc; most importantly, one can handle symbols and variables without having to replace them with specific numerical values. In order to facilitate the reader, an example set of Mathematica commands is given each time (analogous commands can be assembled for the other programs mentioned above). These commands may be used to perform a specific task and then various similar tasks are requested in the form of exercises. No effort is made to present the most effective Mathematica program for tackling the suggested problem and no detailed description of the Mathematica syntax is provided; the interested reader is referred to the Mathematica Instruction Manual (Wolfram Research) to check the, virtually unlimited, commands available in this software (or alternative computer algebra software) and use them for creating several alternative instruction sets for the suggested exercises.
Moreover, a novel feature of the book is that, at the end of each chapter, we have included a section detailing a case study through which we demonstrate the usefulness of the results and concepts discussed in that chapter for a real-life problem; we also carry out the required computations through the use of Mathematica.
At the beginning of each chapter we provide a brief historical account of some pioneers in Probability who made exemplary contributions to the topic of discussion within that chapter. This is done so as to provide students with a sense of history and appreciation of the vital contributions made by some renowned probabilists. Apart from the books on the history of probability and statistics that can be found in the bibliography, we have used Wikipedia as a source for biographical details.
In most sections, the exercises have been classified into two groups, A and B. Group A exercises are usually routine extensions of the theory or involve simple calculations based on theoretical tools developed in the section and should be the vehicle for a self-control of the knowledge gained so far by the reader. Group B exercises are more advanced, require substantial critical thinking and quite often include fascinating applications of the corresponding theory.
In addition to regular exercises within each chapter, we have also provided a long list of True/False questions and another list of multiple choice questions. In our opinion, these will not only be useful for students to practice with (and assess their progress), but can also be helpful for instructors to give regular in-class quizzes.
Particular effort has been made to give the theoretical results in their simplest form, so that they can be understood easily by the reader. In an effort to offer the book user an additional means of understanding the concepts presented, intuitive approaches and illustrative graphical representations/figures are provided in several places. The material of this book emerged from a similar book (Introduction to Probability: Theory and Applications, Stamoulis Publications) written by one of us (MVK) in Greek, which is being used as a textbook for many years in several Greek Universities. Of course, we have expanded and transformed this material to reach an international audience.
This is the first volume in a set of two for teaching probability theory. In this volume, we have detailed the basic rules and concepts of probability, combinatorial methods for probabilistic computations, discrete random variables, continuous random variables, and well-known discrete and continuous distributions. These form the core topics for an introduction to probability. More advanced topics such as joint distributions, measures of dependence, multivariate random variables, well-known multivariate discrete and continuous distributions, generating functions, Laws of Large Numbers and the Central Limit Theorem should come out as core topics for a second course on probability. The second volume of our set will expand on all these advanced topics and hence it can be used effectively as a textbook for a second course on probability; the form and structure of each chapter will be similar to those in the present volume.
We wish to thank our colleagues G. Psarrakos and V. Dermitzakis who read parts of the book and to our students who attended our classes and made several insightful remarks and suggestions through the years.
In a book of this size and content, it is inevitable that there are some typographical errors and mistakes (that have clearly escaped several pairs of eyes). If you do notice any of them, please inform us about them so that we can do suitable corrections in future editions of this book.
It is our sincere hope that instructors find this textbook to be easy-to-use for teaching an introductory course on probability, while the students find the book to be user-friendly with easy and logical explanations, plethora of examples, and numerous exercises (including computational ones) that they could practice with! Finally, we would like to thank the Wiley production team for their help and patience during the preparation of this book!