From b0aabcb9831576446c574c8eea6a5d6d57fc4ac9 Mon Sep 17 00:00:00 2001 From: samay2504 Date: Wed, 3 Dec 2025 21:39:22 +0530 Subject: [PATCH] docs: Add comprehensive GPU hardware requirements and quantization details - Added specific GPU examples for both NVIDIA and AMD cards - Included consumer and professional GPU recommendations - Added quantization benefits section (INT4/INT8/Float16) - Covers 2B and 7B checkpoints with detailed VRAM requirements - Makes hardware compatibility immediately clear for new users Resolves #395 --- README.md | 26 ++++++++++++++++++++++++-- 1 file changed, 24 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 4536c842..b587e0a7 100644 --- a/README.md +++ b/README.md @@ -86,8 +86,30 @@ To download the model weights. See ### System Requirements -Gemma can run on a CPU, GPU and TPU. For GPU, we recommend 8GB+ RAM on GPU for -The 2B checkpoint and 24GB+ RAM on GPU are used for the 7B checkpoint. +Gemma can run on a CPU, GPU and TPU. Below are the recommended hardware specifications: + +#### GPU Requirements + +**Gemma 2B Checkpoint:** +- **Minimum VRAM:** 8GB+ +- **Recommended GPUs:** + - NVIDIA: RTX 3060 (12GB), RTX 4060 (8GB), RTX 3070 (8GB), RTX 4060 Ti (8GB/16GB) + - AMD: RX 6600 XT (8GB), RX 6700 XT (12GB), RX 7600 (8GB) + - Professional: NVIDIA T4 (16GB), A4000 (16GB) + +**Gemma 7B Checkpoint:** +- **Minimum VRAM:** 24GB+ +- **Recommended GPUs:** + - NVIDIA: RTX 4090 (24GB), RTX 3090 (24GB), RTX 3090 Ti (24GB) + - AMD: Radeon RX 7900 XTX (24GB), Radeon RX 7900 XT (20GB) + - Professional: NVIDIA A5000 (24GB), A6000 (48GB), L40 (48GB), H100 (80GB) + +**Quantization Benefits:** +- **Full Precision (Float16/BF16):** ~19GB VRAM for 7B models +- **INT8 Quantization:** ~10GB VRAM for 7B models +- **INT4 Quantization:** ~5GB VRAM for 7B models + +> **Note:** Using quantization techniques (INT8/INT4) can significantly reduce VRAM requirements, making 7B models accessible on GPUs with 12-16GB VRAM. ### Contributing