4 Contents
5.2.3 Parsing with a Pfa 104
5.2.4 Eliminating λ-transitions in a Pfa 109
5.3 Probabilistic context-free grammars 115
5.4 Distances between two distributions 117
5.4.1 Equivalence questions 118
5.4.2 Samples as automata 119
5.4.3 Distances between automata 120
5.4.4 Some properties 122
5.5 Computing distances 123
5.5.1 Computing prefixial distances between states 123
5.5.2 Computing the KL-divergence 124
5.5.3 Co-emission 125
5.5.4 Some practical issues 126
5.5.5 Perplexity 127
5.6 Exercises 128
5.7 Conclusions of the chapter and further reading 129
5.7.1 Bibliographical background 129
5.7.2 Some alternative lines of research 131
5.7.3 Open problems and possible new lines of research 132
6 About Combinatorics 133
6.1 About Vc-dimens ions 134
6.1.1 Why Vc-dimension? 134
6.1.2 Vc-dimension of Dfa 135
6.1.3 Vc-dimension of Nfa 136
6.2 About consistency 137
6.2.1 The problem of finding the minimum consistent automaton138
6.2.2 N P-completeness of the problem of finding the minimum consistent automaton138
6.2.3 Related questions 141
6.3 The search space f or the Dfa learning p roblem 142
6.3.1 Typical approach 142
6.3.2 The partition lattice 143
6.3.3 Structural completeness 146
6.3.4 Defining the search space by structural completeness149
6.4 About the equivalence problem and its relation to characteristic sets151
6.5 Some remarkable automata 152
6.5.1 Parity automata 152
6.5.2 From Sat to Nfa 154
6.5.3 A very long string 155
6.6 Exercises 156