xviii Contents
11.4 The Support Vector Machine ................................................................. 260
11.4.1 The Hinge Loss ..................................................................... 261
11.4.2 Regularization ...................................................................... 262
11.4.3 Finding a Classifier with Stochastic Gradient Descent ..................................... 262
11.4.4 Searching for ..................................................................... 264
11.4.5 Example: Training an SVM with Stochastic Gradient Descent .............................. 266
11.4.6 Multi-Class Classification with SVMs .................................................. 268
11.5 Classifying with Random Forests ............................................................. 268
11.5.1 Building a Decision Tree: General Algorithm ............................................ 270
11.5.2 Building a Decision Tree: Choosing a Split.............................................. 270
11.5.3 Forests ............................................................................ 272
11.6 You Should ............................................................................... 274
11.6.1 Remember These Definitions ......................................................... 274
11.6.2 Remember These Terms ............................................................. 274
11.6.3 Remember These Facts .............................................................. 275
11.6.4 Use These Procedures ............................................................... 275
11.6.5 Be Able to ......................................................................... 276
12 Clustering: Models of High Dimensional Data ...................................................... 281
12.1 The Curse of Dimension ..................................................................... 281
12.1.1 Minor Banes of Dimension ........................................................... 281
12.1.2 The Curse: Data Isn’t Where You Think It Is ............................................ 282
12.2 Clustering Data ............................................................................ 283
12.2.1 Agglomerative and Divisive Clustering ................................................. 283
12.2.2 Clustering and Distance .............................................................. 285
12.3 The K-Means Algorithm and Variants ......................................................... 287
12.3.1 How to Choose K ................................................................... 288
12.3.2 Soft Assignment .................................................................... 290
12.3.3 Efficient Clustering and Hierarchical K Means ........................................... 291
12.3.4 K-Mediods ........................................................................ 292
12.3.5 Example: Groceries in Portugal ....................................................... 292
12.3.6 General Comments on K-Means ....................................................... 293
12.4 Describing Repetition with Vector Quantization ................................................. 294
12.4.1 Vector Quantization ................................................................. 296
12.4.2 Example: Activity from Accelerometer Data ............................................ 298
12.5 The Multivariate Normal Distribution .......................................................... 300
12.5.1 Affine Transformations and Gaussians .................................................. 301
12.5.2 Plotting a 2D Gaussian: Covariance Ellipses ............................................. 301
12.6 You Should ............................................................................... 302
12.6.1 Remember These Definitions ......................................................... 302
12.6.2 Remember These Terms ............................................................. 302
12.6.3 Remember These Facts .............................................................. 303
12.6.4 Use These Procedures ............................................................... 303
13 Regression ..................................................................................... 305
13.1 Regression to Make Predictions .............................................................. 305
13.2 Regression to Spot Trends ................................................................... 306
13.3 Linear Regression and Least Squares .......................................................... 308
13.3.1 Linear Regression ................................................................... 308
13.3.2 Choosing ˇ ........................................................................ 309
13.3.3 Solving the Least Squares Problem .................................................... 309
13.3.4 Residuals
.......................................................................... 310
13.3.5 R-Squared ......................................................................... 310