Elon Musk-led xAI, has announced the open-sourcing of Grok-1, its AI language model on GitHub. The release includes the AI’s base model weights and network architecture and is considered a step towards democratising AI development and research.
Grok-1, a massive 314 billion parameter Mixture-of-Experts model, was trained from scratch by xAI using a custom training stack built on top of JAX and Rust. Grok-1 is not fine-tuned for any specific application, such as dialogue systems, unlike some proprietary models.
This raw base model checkpoint represents the culmination of extensive pre-training efforts, concluding in October 2023.
Developers and researchers eager to delve into Grok-1’s capabilities can access the model under the Apache 2.0 license that facilitates both commercial and non-commercial use.
Musk’s strategic decision to open-source Grok-1 comes amidst competitive tensions in the AI industry. Notably, Musk has been vocal about the importance of transparency and accessibility in AI development, contrasting approaches taken by companies such as OpenAI, Meta, and Google.
Grok-1’s release has sparked interest among AI tool makers. Perplexity CEO Arvind Srinivas has expressed intentions to fine-tune Grok for conversational search applications. This reflects a broader trend in the industry, where open-source AI models like Grok-1 serve as foundational building blocks for diverse AI-driven solutions.
Musk is currently engaged in a legal battle with OpenAI‘s Sam Altman. He contends that OpenAI has effectively become a ‘for-profit’ organisation, deviating from its initial non-profit model.
Musk also claimed that OpenAI has become a subsidiary of Microsoft after the latter infused more than $10 billion into the company. In response to Musk’s claims, OpenAI said that developing Advanced General Intelligence (AGI), a supposedly theoretical AI model surpassing human intelligence, requires more resources, hence the for-profit model.
OpenAI also claimed that Elon Musk suggested that the company should transition to a profit model in 2017 and even suggested a merger with Tesla to compete with multi-billion dollar enterprises like Google.
In the News: Chrome gets real-time URL protection and password security
