From 2b89381b9f7183abf786c16aedd4266f1ea007eb Mon Sep 17 00:00:00 2001 From: Naveen karthik Rajakumar <139682009+Nav3005@users.noreply.github.com> Date: Sun, 5 Jan 2025 10:53:11 +0530 Subject: [PATCH 1/8] Add files via upload --- .../black_and_white_image_colorizer.md | 188 ++++++++++++++++++ 1 file changed, 188 insertions(+) create mode 100644 docs/OpenCV/projects/black_and_white_image_colorizer.md diff --git a/docs/OpenCV/projects/black_and_white_image_colorizer.md b/docs/OpenCV/projects/black_and_white_image_colorizer.md new file mode 100644 index 00000000..b1efa59c --- /dev/null +++ b/docs/OpenCV/projects/black_and_white_image_colorizer.md @@ -0,0 +1,188 @@ +# Black and White Image Colorizer + +### AIM + +**Colorization of Black and White Images using OpenCV and pre-trained caffe models.** + + +### PRE-TRAINED MODELS + +[colorization_deploy_v2.prototxt](https://github.com/richzhang/colorization/blob/caffe/models/colorization_deploy_v2.prototxt) +[colorization_release_v2.caffemodel](wget http://eecs.berkeley.edu/~rich.zhang/projects/2016_colorization/files/demo_v2/colorization_release_v2.caffemodel -O ./models/colorization_release_v2.caffemodel) + +[pts_in_hull.npy](https://github.com/richzhang/colorization/blob/caffe/resources/pts_in_hull.npy) + +### NOTEBOOK LINK + +### [Code](lhttps://colab.research.google.com/drive/1w5GbYEIsX41Uh8i_5q7c8Nh0y5UOpBGb) + + +### LIBRARIES NEEDED + +??? quote "LIBRARIES USED" + + - numpy + - cv2 + +--- + +### DESCRIPTION + +!!! info "What is the requirement of the project?" + + - The project aims to perform colorization of black and white images. + - It involves in showcase the capabilities of OpenCV's DNN module and caffe models. + - It is done by processing given image using openCV and use Lab Color space model to hallucinate an approximation of how colorized version of the image "might look like". + +??? info "Why is it necessary?" + +- It helps preserving historical black-and-white photos. +- It can be used adding color to grayscale images for creative industries. +- It acts an advancing computer vision applications in artistic and research fields. + +??? info "How is it beneficial and used?" + +- **Personal use :** It helps in restoring old family photographs. +- **Cultural and Political :** it also enhances grayscale photographs of important historic events for modern displays. +- **Creativity and Art :** it improves AI-based creative tools for artists and designers. + +??? info "How did you start approaching this project? (Initial thoughts and planning)" + +- **Initial approach** : reading various research papers and analyze different approaches on how to deal with this project. +- Identified Richzhang research paper on the title : Colorful Image colorization. +- Did some research on pre-trained models for image colorization. +- Understood OpenCV's DNN module and its implementation. +- Experimented with sample images to test model outputs. + +??? info "Mention any additional resources used (blogs, books, chapters, articles, research papers, etc.)." + - [Richzhang's Colorful Image Colorization](https://richzhang.github.io/colorization) + - [Lab Color space](https://www.xrite.com/blog/lab-color-space) + - [openCV Documentation ](https://pypi.org/project/opencv-python/) + + +--- + +### EXPLANATION + +#### DETAILS OF THE DIFFERENT FEATURES + +\- **Pre-trained Model**: Uses the Caffe-based neural network for colorization. + +\- **LAB Color Space**: Converts input grayscale images to LAB, where the model predicts A and B channels. + +\- **Cluster Centers**: Provides color information for prediction using `pts_in_hull.npy`. + + +--- + +#### PROJECT WORKFLOW + +=== "Step 1" + + Initial data exploration and understanding: + + - Load the grayscale input image. + - Load pre-trained caffe models using openCV dnn module. + +=== "Step 2" + + Data cleaning and preprocessing: + + - Preprocess image to normalize and convert to LAB color space. + - Resize image for the network. + - Split L channel and perform mean subtraction. + - Predict ab channel from the input of L channel. + +=== "Step 3" + + Feature engineering and selection: + + - Resize predicted ab channel's volume to same dimension as our image. + - Join L and predicted ab channel. + - Convert image from Lab back to RGB. + +=== "Step 4" + + Result : + + - Resize and Show the Original and Colorized image. + +=== "Step 5" + +```code language  +Model Validation : + +- This system cannot be validated as the colorization is a result of hallucination. +``` + +--- + +#### PROJECT TRADE-OFFS AND SOLUTIONS + +=== "Trade Off 1" + +- Computational efficiency vs. color accuracy. +- **Solution : **Used optimized preprocessing pipelines to reduce runtime. + +=== "Trade Off 2" + +- Pre-trained model generalization vs. custom training. +- **Solution : **Choose the pre-trained model for faster implementation and reliable results. + +--- + +### SCREENSHOTS + +!!! success "Project workflow" + + ``` mermaid + graph LR + A[Start] --> B{Error?}; + B -->|Yes| C[Preprocess Image]; + C --> D[Generate Output]; + D --> B; + B ---->|No| E[Yay!]; + ``` + +??? tip "Visualizations of results" + + === "Original Image" + ![Original Image](https://drive.google.com/file/d/1NWzvtYUW4Q5hSOrDdhSVJmraNuyoV9zW/view?usp=sharing) + + === "Colorized Image" + ![Colorized Image](https://drive.google.com/file/d/1kuY93h7sTT-WaO7EcgeZfpf3DSTR4Q_L/view?usp=sharing) + +--- + +### CONCLUSION + +#### KEY LEARNINGS + +!!! tip "Insights gained from the data" +- **Color Space : **LAB color space facilitates colorization tasks. +- **Pre-trained Models :** Pre-trained models can generalize across various grayscale images. + +??? tip "Improvements in understanding machine learning concepts" + +- **OpenCV : **Enhanced knowledge of OpenCV's DNN module. +- **Caffe Models : **Usage of pre-trained models. +- **Image Dimensionality : **Understanding how Image can be manipulated. + +??? tip "Challenges faced and how they were overcome" + - **Color Space Conversion : **Initial difficulties with LAB to RGB conversion; resolved using OpenCV documentation. + +--- + +#### USE CASES + +=== "Application 1" + + **Image Restoration** + + - Restoring old family photographs to vivid colors. + +=== "Application 2" + + **Creative Industries** + + - Colorizing artistic grayscale sketches for concept designs. \ No newline at end of file From 2bac14d0761888c52f2c5b6395aafeaf565261ab Mon Sep 17 00:00:00 2001 From: Naveen karthik Rajakumar <139682009+Nav3005@users.noreply.github.com> Date: Sun, 5 Jan 2025 10:59:06 +0530 Subject: [PATCH 2/8] Update black_and_white_image_colorizer.md --- docs/OpenCV/projects/black_and_white_image_colorizer.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/OpenCV/projects/black_and_white_image_colorizer.md b/docs/OpenCV/projects/black_and_white_image_colorizer.md index b1efa59c..3e172d7e 100644 --- a/docs/OpenCV/projects/black_and_white_image_colorizer.md +++ b/docs/OpenCV/projects/black_and_white_image_colorizer.md @@ -8,7 +8,7 @@ ### PRE-TRAINED MODELS [colorization_deploy_v2.prototxt](https://github.com/richzhang/colorization/blob/caffe/models/colorization_deploy_v2.prototxt) -[colorization_release_v2.caffemodel](wget http://eecs.berkeley.edu/~rich.zhang/projects/2016_colorization/files/demo_v2/colorization_release_v2.caffemodel -O ./models/colorization_release_v2.caffemodel) +[colorization_release_v2.caffemodel](http://eecs.berkeley.edu/~rich.zhang/projects/2016_colorization/files/demo_v2/colorization_release_v2.caffemodel -O ./models/colorization_release_v2.caffemodel) [pts_in_hull.npy](https://github.com/richzhang/colorization/blob/caffe/resources/pts_in_hull.npy) @@ -185,4 +185,4 @@ Model Validation : **Creative Industries** - - Colorizing artistic grayscale sketches for concept designs. \ No newline at end of file + - Colorizing artistic grayscale sketches for concept designs. From 984e1f791b2bf565491efc0ca69f4f7e0da9b1c1 Mon Sep 17 00:00:00 2001 From: Naveen karthik Rajakumar <139682009+Nav3005@users.noreply.github.com> Date: Sun, 5 Jan 2025 10:59:47 +0530 Subject: [PATCH 3/8] Update black_and_white_image_colorizer.md --- docs/OpenCV/projects/black_and_white_image_colorizer.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/OpenCV/projects/black_and_white_image_colorizer.md b/docs/OpenCV/projects/black_and_white_image_colorizer.md index 3e172d7e..df12bd09 100644 --- a/docs/OpenCV/projects/black_and_white_image_colorizer.md +++ b/docs/OpenCV/projects/black_and_white_image_colorizer.md @@ -8,7 +8,7 @@ ### PRE-TRAINED MODELS [colorization_deploy_v2.prototxt](https://github.com/richzhang/colorization/blob/caffe/models/colorization_deploy_v2.prototxt) -[colorization_release_v2.caffemodel](http://eecs.berkeley.edu/~rich.zhang/projects/2016_colorization/files/demo_v2/colorization_release_v2.caffemodel -O ./models/colorization_release_v2.caffemodel) +[colorization_release_v2.caffemodel](wget http://eecs.berkeley.edu/~rich.zhang/projects/2016_colorization/files/demo_v2/colorization_release_v2.caffemodel -O ./models/colorization_release_v2.caffemodel) [pts_in_hull.npy](https://github.com/richzhang/colorization/blob/caffe/resources/pts_in_hull.npy) From ab70d7d9ebcb4fc06df9cb5a1b47872c3fd8e936 Mon Sep 17 00:00:00 2001 From: Naveen karthik Rajakumar <139682009+Nav3005@users.noreply.github.com> Date: Sun, 5 Jan 2025 11:02:54 +0530 Subject: [PATCH 4/8] Update black_and_white_image_colorizer.md --- docs/OpenCV/projects/black_and_white_image_colorizer.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/docs/OpenCV/projects/black_and_white_image_colorizer.md b/docs/OpenCV/projects/black_and_white_image_colorizer.md index df12bd09..a6160eba 100644 --- a/docs/OpenCV/projects/black_and_white_image_colorizer.md +++ b/docs/OpenCV/projects/black_and_white_image_colorizer.md @@ -8,8 +8,7 @@ ### PRE-TRAINED MODELS [colorization_deploy_v2.prototxt](https://github.com/richzhang/colorization/blob/caffe/models/colorization_deploy_v2.prototxt) -[colorization_release_v2.caffemodel](wget http://eecs.berkeley.edu/~rich.zhang/projects/2016_colorization/files/demo_v2/colorization_release_v2.caffemodel -O ./models/colorization_release_v2.caffemodel) - +[colorization_release_v2.caffemodel](https://www.dropbox.com/s/dx0qvhhp5hbcx7z/colorization_release_v2.caffemodel?dl=1)) [pts_in_hull.npy](https://github.com/richzhang/colorization/blob/caffe/resources/pts_in_hull.npy) ### NOTEBOOK LINK From 4035d6dfd1be1c393af866e7d01e0493084e460f Mon Sep 17 00:00:00 2001 From: Naveen karthik Rajakumar <139682009+Nav3005@users.noreply.github.com> Date: Sun, 5 Jan 2025 11:03:21 +0530 Subject: [PATCH 5/8] Update black_and_white_image_colorizer.md --- docs/OpenCV/projects/black_and_white_image_colorizer.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/OpenCV/projects/black_and_white_image_colorizer.md b/docs/OpenCV/projects/black_and_white_image_colorizer.md index a6160eba..838ad72d 100644 --- a/docs/OpenCV/projects/black_and_white_image_colorizer.md +++ b/docs/OpenCV/projects/black_and_white_image_colorizer.md @@ -8,7 +8,9 @@ ### PRE-TRAINED MODELS [colorization_deploy_v2.prototxt](https://github.com/richzhang/colorization/blob/caffe/models/colorization_deploy_v2.prototxt) -[colorization_release_v2.caffemodel](https://www.dropbox.com/s/dx0qvhhp5hbcx7z/colorization_release_v2.caffemodel?dl=1)) + +[colorization_release_v2.caffemodel](https://www.dropbox.com/s/dx0qvhhp5hbcx7z/colorization_release_v2.caffemodel?dl=1) + [pts_in_hull.npy](https://github.com/richzhang/colorization/blob/caffe/resources/pts_in_hull.npy) ### NOTEBOOK LINK From dbfd3a2b1f194100d998b969850946f8995f2826 Mon Sep 17 00:00:00 2001 From: Naveen karthik Rajakumar <139682009+Nav3005@users.noreply.github.com> Date: Sun, 5 Jan 2025 11:26:38 +0530 Subject: [PATCH 6/8] Update black_and_white_image_colorizer.md --- docs/OpenCV/projects/black_and_white_image_colorizer.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/OpenCV/projects/black_and_white_image_colorizer.md b/docs/OpenCV/projects/black_and_white_image_colorizer.md index 838ad72d..9defd968 100644 --- a/docs/OpenCV/projects/black_and_white_image_colorizer.md +++ b/docs/OpenCV/projects/black_and_white_image_colorizer.md @@ -148,10 +148,10 @@ Model Validation : ??? tip "Visualizations of results" === "Original Image" - ![Original Image](https://drive.google.com/file/d/1NWzvtYUW4Q5hSOrDdhSVJmraNuyoV9zW/view?usp=sharing) + ![Original Image](https://github.com/user-attachments/assets/98a68022-eb8a-4e2e-b87a-edf5b8a392fa) === "Colorized Image" - ![Colorized Image](https://drive.google.com/file/d/1kuY93h7sTT-WaO7EcgeZfpf3DSTR4Q_L/view?usp=sharing) + ![Colorized Image](https://github.com/user-attachments/assets/181f585e-a2de-4bf5-aead-1c3a56ac7f8e) --- From 1c972dbae0ecff970168bf08b39fd1ba112cf523 Mon Sep 17 00:00:00 2001 From: Naveen karthik Rajakumar <139682009+Nav3005@users.noreply.github.com> Date: Sun, 5 Jan 2025 11:27:10 +0530 Subject: [PATCH 7/8] Update black_and_white_image_colorizer.md --- docs/OpenCV/projects/black_and_white_image_colorizer.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/OpenCV/projects/black_and_white_image_colorizer.md b/docs/OpenCV/projects/black_and_white_image_colorizer.md index 9defd968..dd15958b 100644 --- a/docs/OpenCV/projects/black_and_white_image_colorizer.md +++ b/docs/OpenCV/projects/black_and_white_image_colorizer.md @@ -56,6 +56,7 @@ - Experimented with sample images to test model outputs. ??? info "Mention any additional resources used (blogs, books, chapters, articles, research papers, etc.)." + - [Richzhang's Colorful Image Colorization](https://richzhang.github.io/colorization) - [Lab Color space](https://www.xrite.com/blog/lab-color-space) - [openCV Documentation ](https://pypi.org/project/opencv-python/) From 1db28a65713caf4c76bcad762e4fd51eebe7e4be Mon Sep 17 00:00:00 2001 From: Naveen karthik Rajakumar <139682009+Nav3005@users.noreply.github.com> Date: Sun, 5 Jan 2025 11:28:39 +0530 Subject: [PATCH 8/8] Update black_and_white_image_colorizer.md --- docs/OpenCV/projects/black_and_white_image_colorizer.md | 1 - 1 file changed, 1 deletion(-) diff --git a/docs/OpenCV/projects/black_and_white_image_colorizer.md b/docs/OpenCV/projects/black_and_white_image_colorizer.md index dd15958b..9defd968 100644 --- a/docs/OpenCV/projects/black_and_white_image_colorizer.md +++ b/docs/OpenCV/projects/black_and_white_image_colorizer.md @@ -56,7 +56,6 @@ - Experimented with sample images to test model outputs. ??? info "Mention any additional resources used (blogs, books, chapters, articles, research papers, etc.)." - - [Richzhang's Colorful Image Colorization](https://richzhang.github.io/colorization) - [Lab Color space](https://www.xrite.com/blog/lab-color-space) - [openCV Documentation ](https://pypi.org/project/opencv-python/)