现在有一个文本序列(真实): The book is forthcoming on Cambridge University Press (1) 写出这句话所有的 n-gram; (2) 假设预测序列为:The book is forthcoming at Cambridge,计算所有的 pn 及 BLEU; (3) 假设预测序列为:The book is forthcoming at Cambridge University online bookstore,计算所有的 pn 及 BLEU。
时间: 2023-12-26 13:04:35 浏览: 172
(1) 所有的n-gram如下:
- unigram: The, book, is, forthcoming, on, Cambridge, University, Press
- bigram: The book, book is, is forthcoming, forthcoming on, on Cambridge, Cambridge University, University Press
- trigram: The book is, book is forthcoming, is forthcoming on, forthcoming on Cambridge, on Cambridge University, Cambridge University Press
- quadrigram: The book is forthcoming, book is forthcoming on, is forthcoming on Cambridge, forthcoming on Cambridge University, on Cambridge University Press
(2) 预测序列为:The book is forthcoming at Cambridge。所有的pn如下:
- unigram: 5/6
- bigram: 4/5
- trigram: 3/4
- quadrigram: 2/3
BLEU = (0.5 * exp(log(5/6)) + 0.25 * exp(log(4/5)) + 0.167 * exp(log(3/4)) + 0.083 * exp(log(2/3))) * 100% = 75.61%
(3) 预测序列为:The book is forthcoming at Cambridge University online bookstore。所有的pn如下:
- unigram: 5/9
- bigram: 4/8
- trigram: 3/7
- quadrigram: 2/6
BLEU = (0.5 * exp(log(5/9)) + 0.25 * exp(log(4/8)) + 0.167 * exp(log(3/7)) + 0.083 * exp(log(2/6))) * 100% = 47.65%
阅读全文