You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: notes/main.typ
+4-4Lines changed: 4 additions & 4 deletions
Original file line number
Diff line number
Diff line change
@@ -435,21 +435,21 @@ to measure the importance of variable $x$ by comparing the loss when including $
435
435
Ensemble learning, as the name suggests, combines multiple basde models to improve prediction performance. Common ensemble learning methods include bagging, boosting, and stacking.
436
436
437
437
=== Bagging
438
-
BoostrapAggregating (bagging) is proposed by Leo Breiman @breiman1996bagging. The key idea is to generate multiple boostrap samples from the original data and train the base model on each boostrap sample then aggragate the predictions from all models, such as by averaging for regression or majority voting for classification.
438
+
BoostrapAggregating (bagging) is proposed by Leo Breiman @breiman1996bagging (4 w + citations). The key idea is to generate multiple boostrap samples from the original data and train the base model on each boostrap sample then aggragate the predictions from all models, such as by averaging for regression or majority voting for classification.
439
439
440
440
==== Theory properties of bagging
441
441
442
442
- @breiman1996bagging Bagging can reduce the variance of unstable base models.
443
-
- Peter Bühlmann and Bin Yu @buhlmann2002analyzing give some convergence rate analysis for bagging.
443
+
- Peter Bühlmann and Bin Yu @buhlmann2002analyzing(1.2 k + citations) give some convergence rate analysis for bagging.
444
444
- In the 2000s, many works on the theoretical understanding of bagging, seems no more work needed now.
445
445
446
446
==== Random forest
447
447
448
-
- Leo Breiman @breiman2001random proposed random forest, an ensemble learning method that builds multiple decision trees and merges their results to improve accuracy.
448
+
- Leo Breiman @breiman2001random(17 w + citations) proposed random forest, an ensemble learning method that builds multiple decision trees and merges their results to improve accuracy.
449
449
450
450
==== Causal forest
451
451
452
-
- @athey2016recursiveproposed causal tree to estimate heterogeneous treatment effects namely the conditional average treatment effect (CATE) by extending decision tree, then @wager2018estimation using random forest to improve the estimation accuracy and provide asymptotic normality for inference.
452
+
- @athey2016recursive(2.5 k + citations) proposed causal tree to estimate heterogeneous treatment effects namely the conditional average treatment effect (CATE) by extending decision tree, then @wager2018estimation(4.3 k + citations) using random forest to improve the estimation accuracy and provide asymptotic normality for inference.
453
453
454
454
- @cattaneo2025honest establishes an inconsistency lower bound on the point wise convergence rate of causal tree, and challenges the $alpha$-regularity condition (each split leaves at least a fraction $alpha$ of available samples on each side) needed to establish the convergence rate in @wager2018estimation.
0 commit comments