ML Module Week 3 Topic 3: Decision Trees and Ensemble Learning

[EricThomson]

Discuss and apply decision tree classifier with scikit-learn.

Highlight the interpretability of a single tree and then introduce random forest as an ensemble method to improve robustness and combat overfitting. Compare performance using metrics from Topic 2 (knn lesson) on the same dataset.

[EricThomson]

For general setup for the lesson -- iris dataset, digits dataset, and classifier evaluation metrics please see issue #59.

Some resources for decision Tree/Gini Impurity Index:

• https://www.youtube.com/watch?v=JcI5E2Ng6r4
• https://www.youtube.com/watch?v=u4IxOk2ijSs
• https://www.youtube.com/watch?v=zs6yHVtxyv8
• https://www.ibm.com/think/topics/decision-trees

For random forest (fighting overfitting): https://www.youtube.com/watch?v=gkXX4h3qYm4

These are all implemented in scikit-learn. The main point is to give a nice conceptual explanation. We want to avoid bullet-point dumps, and give nice motivating explanations and prose for the students, coupled with links to good resources. Decision trees and random forests in particular can be tough to understand, so we want to give a nice intuitive explanation not just terse bullet points.

[EricThomson]

Here is an example of using decision tree with the iris dataset:
https://scikit-learn.org/stable/auto_examples/tree/plot_iris_dtc.html