I want to train llama using qlora on multiple tasks in a sequential manner, ie task A-> B-> C. Would it be possible to combine, say adapter weights trained on A with B and subsequentially, A with C? How can i can go about doing this?
Has anyone tried doing so and achieving reasonable results? i am aiming for task A to be continual pretraining for domain adaptation, while B and C to be the downstream tasks.
You can merge lora A to the base model, and than to finetune B and C on the merged model.