Photo: Michael Vi / Shutterstock.com
Cerebras Systems, previously popular in the tech space for its work on an AI chip, has unveiled its supercomputer, Andromeda. The supercomputer is built by linking up 16 Cerebras CS-2 systems built around the company’s Wafer-Scale Engine 2.
The supercomputer is available for commercial and academic research starting Monday, with companies and researchers, including those from U.S national labs can, access it remotely. The Argonne National Laboratory, AMD, JasperAI and the University of Cambridge are already running real workloads on the machine.
Andrew Feldman, founder and CEO of Cerebras, stated that the computer is owned by the company and is built at the Colovore high-performance data centre located in Santa Clara.
The company claims that Andromeda has one Exaflop worth of AI computing power and 120 Petaflops of dense computing thanks to its 13.5 million cores. Based on a 16-bit floating point format, this comes down to at least one quintillion operations per second. According to Feldman, the cost to build the computer was under $35 million.
Cerebras claims that, unlike any known GPU-based cluster, Andromeda can deliver near-perfect scaling using simple data parallelism across GPT-class large language models, including GPT-3, GPT-J and GPT-NeoX. This means that as additional CS-2s are included, training time reduces in a near-perfect proportion.
The Cerebras WSE-2 processor also has 1000 times more memory bandwidth as compared to a GPU, allowing the computer to harvest structured and unstructured sparsity as well as static and dynamic sparsity. This means that Cerebras can now train models with more than 90% sparse to state-of-the-art accuracy.
Andromeda isn’t the first AI supercomputer to break the one Exaflop barrier either, with Frontier at the Oak Ridge National Laboratory already achieved the milestone. However, there are a couple of major differences between the two.
Reuters reported Feldman admitting that the Frontier is a bigger machine and Andromeda won’t compete to beat them, emphasising the cost difference between the two. The Frontier cost nearly $600 million to build, while the Andromeda is nearly $35 million. Additionally, Frontier is based on 64-bit double-precision, a more computationally expensive format than Andremeda’s 16-bit floating point format.
In the News: Turkey suspends FB, Insta, Twitter, YT, Telegram following blast
