Gradient Boosting for Regression
Let’s play a game...
You are given (x
1
, y
1
), (x
2
, y
2
), ..., (x
n
, y
n
), and the task is to fit a
model F (x) to minimize square loss.
Suppose your friend wants to help you and gives you a model F .
You check his model and find the model is good but not perfect.
There are some mistakes: F (x
1
) = 0.8, while y
1
= 0.9, and
F (x
2
) = 1.4 while y
2
= 1.3... How can you improve this model?
Rule of the game:
I
You are not allowed to remove anything from F or change any
parameter in F .
I
You can add an additional model (regression tree) h to F , so
the new prediction will be F (x) + h(x).
剩余79页未读，继续阅读
