Explainable AI for Understanding Decisions & Data-Driven Optimization in Deep Neural Networks

PART 1: Building Deep Neural Networks on ImageNet

Repository contains code for training different architectures of image classification (i.e. GoogleNet, ResNet-50 & 101, AlexNet etc.) on ImageNet(Orginal) dataset.

---

Features:

1. The code reads dataset information from a text or csv file and directly loads images from disk.
2. Code Utilizes the GPU parellelization during training phase.
3. Code is built for running on High-Performance Computing(Super Computing) Infrastructure.
4. Completely designed, by performing all necessary validation checks.
5. Compatible with both Python 3.5.x and Python 3.6.x (tested)
6. Code has personalization for selecting optimization algorithm, learning rate and weight decay policies\
7. Code can build various architectures in optimized manner
8. Code Supports full automation for training, validation and testing phase.

---

---

Required Modules/Packages/libraries:
Python 3.7 : os, sys, datetime, time, future, argparse, numpy, tensorflow

---

---

Required Scripts/files:
Data files : /path-to/dataset/train_imagenet and /path-to/data/val_imagenet, /path-to/train.txt and /path-to/val.txt
Main source code file: run.py
Dependent code files : data_loader.py, utils.py, common.py, model.py, googlenet.py, resnet.py

---

Usage:

To start, train.txt file is needed. it looks something like this,

train_imagenet/n01440764/n01440764_7173.JPEG,0
train_imagenet/n01440765/n01440764_3724.JPEG,0
train_imagenet/n01440766/n01440764_7719.JPEG,0
train_imagenet/n01440767/n01440764_7304.JPEG,0
train_imagenet/n01440768/n01440764_8469.JPEG,0

Use the --delimiter option to specify the delimiter character, and --path_prefix to add a constant prefix to all the paths.

Model Training:

For training GoogleNet execute run.py with given command,

python run.py train --architecture googlenet --path_prefix ${HOME}/path-to-dataset-folder --train_info train.txt --optimizer adam --num_epochs 5

For training ResNet-50 & ResNet-101 execute run.py with given command,

python run.py train --architecture resnet --path_prefix ${HOME}/path-to-dataset-folder --train_info train.txt --optimizer adam --num_epochs 5 --depth 50

python run.py train --architecture resnet --path_prefix ${HOME}/path-to-dataset-folder --train_info train.txt --optimizer adam --num_epochs 5 --depth 101

For training Alexnet(CaffeNet) execute run.py with given command,

python run.py train --architecture alexnet --path_prefix ${HOME}/path-to-dataset-folder --train_info train.txt --optimizer adam --num_epochs 5

Model Validation:

For validation, execute run.py with given command,

python run.py eval --architecture googlenet --log_dir "googlenet_Run-02-12-2018-15:40:00" --path_prefix /path/to/imagenet/train/ --val_info val.txt

Model Testing (Making Predictions):

For testing, execute run.py with given command,

python run.py inference --architecture googlenet --log_dir "googlenet_Run-02-12-2018-15:40:00" --path_prefix /path/to/imagenet/train/ --val_info val.txt --save_predictions preds.txt

---

Customization options:
1. Deep neural networks : (option)--architecture --> (possible values) googlenet and resnet
2. Execution methods : train (for training), eval (for validating), inference (for testing)
3. dataset path prefix : (option)--path_prefix --> ${HOME}/path-to-dataset-folder
4. Train/validation info : (option)--train_info train.txt and (option)--val_info val.txt
5. Optimizer for DNN : (option)--optimizer --> (momentum(default), adam, adadelta, adagrad, rmsprop, sgd)
6. learning rate policy : (option)--policy_type --> (constant, piecewise_linear(default), exponential)
7. LR Change detials : (option)--LR_details --> (19, 30, 44, 53, 0.01, 0.005, 0.001, 0.0005, 0.0001)(default)
8. GPU numbers(Training) : (option)--num_gpus --> default is 1, INTEGER(N --> 5,10,50,...)
9. epoch for training : (option)--num_epochs --> 5,10,50,10,200
10. Depth for ResNet : (option)--depth --> default is 50 (can change to 50,101)
11. Log(validation/testing): (option)--log_dir --> "googlenet_Run-02-12-2018-15:40:00"
12. Save prections : (option)--save_predictions --> "predictions.csv" (default) (can specify other file name). Save top-n predictions of the networks along with their confidence in the specified file
13. Weight decay policy : (option)--WD_policy --> (constant, piecewise_linear(default), exponential)
14. WD change details : (option)--WD_details --> (30, 0.0005, 0.0)(default)
15. Batch size : (option)--batch_size --> 128 (default) (can specify other value)
16. No of Prefectch Images : (option)--num_prefetch --> 2000 (default) (can specify other value)
17. Shuffle training data : (option)--shuffle --> TRUE (default) (can change it to false)
18. Top N accuracy : (option)--top_n --> 5(default) (specify top n accuracy number)
19. Debugging log : (option)--log_dir --> NONE (default) (can specify Path for saving debugging info & checkpoints)
20. Log runtime & mem usage: (option)--log_debug_info --> False(default) (can be TRUE)
21. Maximum snapshot : (option)--max_to_keep --> 5(default) (Specify Maximum number of snapshot files to keep)

---

---

Scripts:
1. run.py : Main python script for DNN, A program to apply different well-known deep learning architectures.Ties all scripts together & performs training, validation,& testing of DNN using all scripts/functions
2. data_loader.py : Performs data loading using given text files and prepares data for model training.
3. utils.py : Utility class for computing averages of loss and accuracies, getting batches for each epoch. Determines learning rate policy and optimization algorithm type as per arguments.
4. common.py : helper function file for each model training (Contains functions/methods for batch normalization,flatten, max pool, avg pool, fully connected, spatial Convolution etc.)
5. model.py : Helper file with necessary methods/functions for simulating model building, training & validation.
6. alexnet.py : Alexnet(Caffenet) Implementation (Deep Convolution Network architecture)
7. googlenet.py : GoogleNet Implementation (Deep Convolution Network architecture)
8. resnet.py : ResNet-50 and ResNet-101 Implementation (Deep Convolution Network architecture)

---

PART 2: Investigating various XAI methods (Future Work- Coming soon)