MIT's new fine-tuning method lets LLMs learn new skills without losing old ones
When enterprises fine-tune LLMs for new tasks, they risk breaking everything the models already know. This forces companies to maintain separate models for every skill.Researchers at MIT, the Improbable AI Lab and ETH Zurich have developed a new technique that enables large language models to learn new skills and knowledge without forgetting their past capabilities.Their technique, called self-distillation fine-tuning (SDFT), allows models to learn directly from demonstrations and their own experiments by leveraging the inherent in-context learning abilities of modern LLMs. Experiments show that SDFT consistently outperforms traditional supervised fine-tuning (SFT) while addressing the limitations of reinforcement learning algorithms.For enterprise applications, the method enables a single [...]
Rating
Innovation
Pricing
Technology
Usability
We have discovered similar tools to what you are looking for. Check out our suggestions for similar AI tools.
How Anthropic’s ‘Skills’ make Claude faster, cheaper, and more consistent for business workflows
Anthropic launched a new capability on Thursday that allows its Claude AI assistant to tap into specialized expertise on demand, marking the company's latest effort to make artificial intelligenc [...]
Self-improving language models are becoming reality with MIT's updated SEAL technique
Researchers at the Massachusetts Institute of Technology (MIT) are gaining renewed attention for developing and open sourcing a technique that allows large language models (LLMs) — like those underp [...]
Baseten takes on hyperscalers with new AI training platform that lets you own your model weights
Baseten, the AI infrastructure company recently valued at $2.15 billion, is making its most significant product pivot yet: a full-scale push into model training that could reshape how enterprises wean [...]
Anthropic launches enterprise ‘Agent Skills’ and opens the standard, challenging OpenAI in workplace AI
Anthropic said on Wednesday it would release its Agent Skills technology as an open standard, a strategic bet that sharing its approach to making AI assistants more capable will cement the company [...]
New framework lets AI agents rewrite their own skills without retraining the underlying model
One major challenge in deploying autonomous agents is building systems that can adapt to changes in their environments without the need to retrain the underlying large language models (LLMs).Memento-S [...]
Phi-4 proves that a 'data-first' SFT methodology is the new differentiator
AI engineers often chase performance by scaling up LLM parameters and data, but the trend toward smaller, more efficient, and better-focused models has accelerated. The Phi-4 fine-tuning methodology [...]
OpenAI report reveals a 6x productivity gap between AI power users and everyone else
The tools are available to everyone. The subscription is company-wide. The training sessions have been held. And yet, in offices from Wall Street to Silicon Valley, a stark divide is opening between w [...]
Researchers find that retraining only small parts of AI models can cut costs and prevent forgetting
Enterprises often find that when they fine-tune models, one effective approach to making a large language model (LLM) fit for purpose and grounded in data is to have the model lose some of its abiliti [...]
Skullcandy Method 360 ANC review: A cheaper version of Bose’s best earbuds, with a few key sacrifices
Skullcandy isn’t a name that usually comes to mind when you think of premium headphones. The Utah-based company has primarily made its name in the budget space, selling more on low prices and loud, [...]