Contents Preface xv Prologue: A machine learning sampler 1 1 The ingredients of machine learning 13 1.1 Tasks: the problems that can be solved with machine learning . . . . . . . 14 Looking for structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 Evaluating performance on a task . . . . . . . . . . . . . . . . . . . . . . . . 18 1.2 Models: the output of machine learning . . . . . . . . . . . . . . . . . . . . 20 Geometric models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 Probabilistic models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 Logical models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 Grouping and grading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 1.3 Features: the workhorses of machine learning . . . . . . . . . . . . . . . . 38 Two uses of features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 Feature construction and transformation . . . . . . . . . . . . . . . . . . . 41 Interaction between features . . . . . . . . . . . . . . . . . . . . . . . . . . 44 1.4 Summary and outlook . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 What you’ll ?nd in the rest of the book . . . . . . . . . . . . . . . . . . . . . 48 2 Binary classi?cation and related tasks 49 2.1 Classi?cation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52 ixx Contents Assessing classi?cation performance . . . . . . . . . . . . . . . . . . . . . . 53 Visualising classi?cation performance . . . . . . . . . . . . . . . . . . . . . 58 2.2 Scoring and ranking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 Assessing and visualising ranking performance . . . . . . . . . . . . . . . . 63 Turning rankers into classi?ers . . . . . . . . . . . . . . . . . . . . . . . . . 69 2.3 Class probability estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . 72 Assessing class probability estimates . .