iv [Ch. 0
7 Methods for Comparison 107
7.1 ESTIMATION OF ERROR RATES IN CLASSIFICATION RULES
107
7.1.1 Train-and-Test
108
7.1.2 Cross-validation
108
7.1.3 Bootstrap
108
7.1.4 Optimisation of parameters
109
7.2 ORGANISATION OF COMPARATIVE TRIALS
110
7.2.1 Cross-validation
111
7.2.2 Bootstrap
111
7.2.3 Evaluation Assistant
111
7.3 CHARACTERISATION OF DATASETS
112
7.3.1 Simple measures
112
7.3.2 Statistical measures
112
7.3.3 Information theoretic measures
116
7.4 PRE-PROCESSING
120
7.4.1 Missing values
120
7.4.2 Feature selection and extraction
120
7.4.3 Large number of categories
121
7.4.4 Bias in class proportions
122
7.4.5 Hierarchical attributes
123
7.4.6 Collection of datasets
124
7.4.7 Preprocessing strategy in StatLog
124
8 Review of Previous Empirical Comparisons 125
8.1 INTRODUCTION
125
8.2 BASIC TOOLBOX OF ALGORITHMS
125
8.3 DIFFICULTIES IN PREVIOUS STUDIES
126
8.4 PREVIOUS EMPIRICAL COMPARISONS
127
8.5 INDIVIDUAL RESULTS
127
8.6 MACHINE LEARNING vs. NEURAL NETWORK
127
8.7 STUDIES INVOLVING ML, k-NN AND STATISTICS
129
8.8 SOME EMPIRICAL STUDIES RELATING TO CREDIT RISK
129
8.8.1 Traditional and statistical approaches
129
8.8.2 Machine Learning and Neural Networks
130
9 Dataset Descriptions and Results 131
9.1 INTRODUCTION
131
9.2 CREDIT DATASETS
132
9.2.1 Credit management (Cred.Man)
132
9.2.2 Australian credit (Cr.Aust)
134
9.3 IMAGE DATASETS
135
9.3.1 Handwritten digits (Dig44)
135
9.3.2 Karhunen-Loeve digits (KL)
137
9.3.3 Vehicle silhouettes (Vehicle)
138
9.3.4 Letter recognition (Letter)
140
评论0