Skip to content

Cerebras announces 13.5 million core AI supercomputer Andromeda

  • by
  • 3 min read
Photo: Michael Vi / Shutterstock.com

Photo: Michael Vi / Shutterstock.com

Cerebras Systems, previously popular in the tech space for its work on an AI chip, has unveiled its supercomputer, Andromeda. The supercomputer is built by linking up 16 Cerebras CS-2 systems built around the company’s Wafer-Scale Engine 2.

The supercomputer is available for commercial and academic research starting Monday, with companies and researchers, including those from U.S national labs can, access it remotely. The Argonne National Laboratory, AMD, JasperAI and the University of Cambridge are already running real workloads on the machine. 

Andrew Feldman, founder and CEO of Cerebras, stated that the computer is owned by the company and is built at the Colovore high-performance data centre located in Santa Clara.

The company claims that Andromeda has one Exaflop worth of AI computing power and 120 Petaflops of dense computing thanks to its 13.5 million cores. Based on a 16-bit floating point format, this comes down to at least one quintillion operations per second.  According to Feldman, the cost to build the computer was under $35 million.

Cerebras announces 13.5 million core AI supercomputer Andromeda
The WSE-2 processor forms the heart of Andromeda. | Source: Cerebras

Cerebras claims that, unlike any known GPU-based cluster, Andromeda can deliver near-perfect scaling using simple data parallelism across GPT-class large language models, including GPT-3, GPT-J and GPT-NeoX. This means that as additional CS-2s are included, training time reduces in a near-perfect proportion. 

The Cerebras WSE-2 processor also has 1000 times more memory bandwidth as compared to a GPU, allowing the computer to harvest structured and unstructured sparsity as well as static and dynamic sparsity. This means that Cerebras can now train models with more than 90% sparse to state-of-the-art accuracy.

Andromeda isn’t the first AI supercomputer to break the one Exaflop barrier either, with Frontier at the Oak Ridge National Laboratory already achieved the milestone. However, there are a couple of major differences between the two. 

Reuters reported Feldman admitting that the Frontier is a bigger machine and Andromeda won’t compete to beat them, emphasising the cost difference between the two. The Frontier cost nearly $600 million to build, while the Andromeda is nearly $35 million. Additionally, Frontier is based on 64-bit double-precision, a more computationally expensive format than Andremeda’s 16-bit floating point format. 

In the News: Turkey suspends FB, Insta, Twitter, YT, Telegram following blast

nv-author-image

Yadullah Abidi

Yadullah is a Computer Science graduate who writes/edits/shoots/codes all things cybersecurity, gaming, and tech hardware. When he's not, he streams himself racing virtual cars. He's been writing and reporting on tech and cybersecurity with websites like Candid.Technology and MakeUseOf since 2018. You can contact him here: yadullahabidi@pm.me.

>