Utility for merging RoSA adapters from saved checkpoints #5
Description
I'm using free Colab T4 GPU for fine-tuning Mistral-7B-Instruct-V2 with the synthetic dataset provided in the repo. The limited Colab runtime is just enough for fine-tuning and 2 inferences. My runtime gets disconnected a few minutes after fine-tuning and then I need to wait for 24 hours and fine-tune again. Therefore, I want to directly use the saved checkpoints (RoSA adapter weights) and merge them back into the model after loading it in 4 bits.