Microsoft has unveiled Phi-3 Mini, marking the debut of a series of three small-scale AI models slated for release by the company. With 3.8 billion parameters, Phi-3 Mini represents the company’s move towards efficient and cost-effective artificial intelligence solutions, particularly for personal devices like smartphones and laptops.
Unlike larger models such as GPT-4, Phi-3 Mini is designed to efficiently balance the two most important targets: performance and resource utilisation.
It is trained on a relatively smaller dataset but boasts enhanced capabilities compared to its predecessor, Phi-2. As per the company, Phi-3 is at par with GPT-3.5, albeit in a more compact form factor.
One of the notable advantages of smaller AI models like the Phi-3 Mini is their affordability and efficiency in operation, especially on devices with limited computational resources. This aligns with a broader industry trend where companies increasingly prioritise models that offer optimal performance while being resource-efficient.
Microsoft has been focusing on developing small models since January 2024 and reportedly invested several million dollars to develop a ChatGPT supercomputer.
The functionality of the smaller AI is still in question. While unable to do heavy tasks, these AI models can perform small tasks such as summarisation chat assistance or coding with the same perfection as their larger counterparts.
To perform these specific tasks, Google launched Gemma 2B and 7B, Anthropic has Claude 3 Haiku and Meta possess Llama 3 8B.
The company introduced an innovative approach to train Phi-3 Mini, given the host of lawsuits OpenAI had to deal with while training GPT. There are reports that OpenAI used over a million hours of YouTube video transcriptions to train GPT-4. Furthermore, there is an impending shortage of datasets for these AI models to train.
To counter this, Microsoft drew inspiration from children’s learning methods. By leveraging a curated word list and building upon existing models, developers have enhanced Phi-3 Mini’s capabilities in coding and reasoning, albeit acknowledging its limitations compared to larger models in terms of knowledge breadth.
“There aren’t enough children’s books out there, so we took a list of more than 3,000 words and asked an LLM to make ‘children’s books’ to teach Phi,” said Eric Boyd, corporate vice president of Microsoft Azure AI platform in a conversation with The Verge.
In the News: Forest Blizzard deploys ‘GooseEgg’ via Windows Print Spooler flaw
