Data Analysis Using the Method of Least Squares

Contents
Chapter 1 INTRODUCTION.............................................................................1
1.1 Quantitative Experiments..................................................................1
1.2 Dealing with Uncertainty..................................................................5
1.3 Statistical Distributions.....................................................................6
The normal distribution...............................................................8
The binomial distribution ..........................................................10
The Poisson distribution............................................................11
The χ 2 χχ distribution.....................................................................13
The t distribution .......................................................................15 t
The F distribution......................................................................16 F
1.4 Parametric Models .........................................................................17
1.5 Basic Assumptions .........................................................................19
1.6 Systematic Errors............................................................................22
1.7 Nonparametric Models ...................................................................24
1.8 Statistical Learning .........................................................................27
Chapter 2 THE METHOD OF LEAST SQUARES ........................................31
2.1 Introduction.....................................................................................31
2.2 The Objective Function...................................................................34
2.3 Data Weighting ...............................................................................38
2.4 Obtaining the Least Squares Solution............................................. 44
2.5 Uncertainty in the Model Parameters.............................................. 50
2.6 Uncertainty in the Model Predictions ............................................. 54
2.7 Treatment of Prior Estimates .......................................................... 60
2.8 Applying Least Squares to Classification Problems ....................... 64
Chapter 3 MODEL EVALUATION................................................................. 73
3.1 Introduction..................................................................................... 73
3.2 Goodness-of-Fit .............................................................................. 74
3.3 Selecting the Best Model ................................................................ 79
3.4 Variance Reduction......................................................................... 85
3.5 Linear Correlation........................................................................... 88
3.6 Outliers ........................................................................................... 93
3.7 Using the Model for Extrapolation ................................................. 96
3.8 Out-of-Sample Testing ................................................................... 99
3.9 Analyzing the Residuals ............................................................... 105
Chapter 4 CANDIDATE PREDICTORS....................................................... 115
4.1 Introduction................................................................................... 115
4.2 Using the F Distribution ............................................................... 116 F
4.3 Nonlinear Correlation ................................................................... 122
4.4 Rank Correlation........................................................................... 131
Chapter 5 DESIGNING QUANTITATIVE EXPERIMENTS..................... 137
5.1 Introduction................................................................................... 137
5.2 The Expected Value of the Sum-of-Squares................................. 139
5.3 The Method of Prediction Analysis .............................................. 140
5.4 A Simple Example: A Straight Line Experiment.......................... 143
5.5 Designing for Interpolation........................................................... 147
5.6 Design Using Computer Simulations............................................ 150
5.7 Designs for Some Classical Experiments ..................................... 155
5.8 Choosing the Values of the Independent Variables ...................... 162
5.9 Some Comments about Accuracy .................................................167
Chapter 6 SOFTWARE ...................................................................................169
6.1 Introduction...................................................................................169
6.2 General Purpose Nonlinear Regression Programs ........................170
6.3 The NIST Statistical Reference Datasets ......................................173
6.4 Nonlinear Regression Convergence Problems..............................178
6.5 Linear Regression: a Lurking Pitfall .............................................184
6.6 Multi-Dimensional Models...........................................................191
6.7 Software Performance...................................................................196
6.8 The REGRESS Program ...............................................................198
Chapter 7 KERNEL REGRESSION..............................................................203
7.1 Introduction...................................................................................203
7.2 Kernel Regression Order Zero ......................................................205
7.3 Kernel Regression Order One .......................................................208
7.4 Kernel Regression Order Two ......................................................212
7.5 Nearest Neighbor Searching .........................................................215 r
7.6 Kernel Regression Performance Studies.......................................223
7.7 A Scientific Application ...............................................................225
7.8 Applying Kernel Regression to Classification ..............................232
7.9 Group Separation: An Alternative to Classification .....................236
Appendix A: Generating Random Noise ........................................................239
Appendix B: Approximating the Standard Normal Distribution ................243
References ......................................................................................................245
Index ......................................................................................................249

Other Layman Books
Download