As reported by The Decoder, The University of Washington came up with a new method, QLoRA, that allows them to tweak complex computer language models using just one GPU.
Using this, they created a chatbot named Guanaco that works almost as well as ChatGPT.
Normally, adjusting these kinds of language models to perform better requires lots of computer power, which can be difficult and expensive to manage.
QLoRA lets them make these adjustments in a way that requires less computer memory, making the process more manageable.
For example, instead of needing 780 GB of GPU memory, they can get the same results with less than 48 GB.
This is big news because it makes the fine-tuning process accessible to more people and not just big tech companies with lots of resources.
Now, even small teams with basic GPUs can adjust these language models.
One person even showed it could be done using Google’s cloud service, Colab.
The team tested QLoRA on a bunch of models and found that the quality of the data they used to train the model was more important than how much data they used.
They found the models trained on OpenAssistant’s data, which had fewer examples but were gathered from real humans, performed better than those trained on a million examples from another source.
They used QLoRA to train Guanaco, a new chatbot.
They trained different versions of Guanaco with different amounts of computer power, and even the second-best version worked nearly as well as ChatGPT.
They were able to train this version on a standard GPU in less than half a day.
They also mention that QLoRA might be useful for tweaking language models on mobile devices while preserving privacy.
They think you could use an iPhone 12 Plus to adjust 3 million words each night.
This suggests we could soon have chatbots on phones that are fine-tuned for each individual app.
There’s a demo of Guanaco-33B, one of their trained models, available online. More information and the code they used to train it are also available online.
Guanaco is based on Meta’s LLaMA models, and it’s not licensed for commercial use.