Sep 19, 2024

Thu 19 Sep 2024 // 08:25 UTC SiFive, having designed RISC-V CPU cores for various AI chips, is now offering to license the blueprints for its own homegrown full-blown machine-learning accelerator. Announced this week, SiFive's Intelligence XM series clusters promise a scalable building block for developing AI chips large and small. The idea is that others can license the RISC-V-based designs to integrate into processors and system-on-chips – to be placed in products from edge and IoT gear to datacenter servers – and hopefully foster more competition between architectures. Fabless SiFive is no stranger to the AI arena. As we've previously reported , at least some of Google's tensor processing units are already using SiFive's X280 RISC-V CPU cores to manage the machine-learning accelerators and keep their matrix multiplication units (MXUs) fed with work and data. Likewise, John Ronco, SVP and GM of SiFive UK, told The Register that SiFive's RISC-V designs also underpin the CPU cores found in Tenstorrent's newly disclosed Blackhole accelerator, which we looked at in detail at Hot Chips last month. And in a canned statement , SiFive CEO Patrick Little claimed the US-based outfit is now supplying RISC-V-based chip designs to five of the "Magnificent 7" companies – Microsoft, Apple, Nvidia, Alphabet, Amazon, Meta and Tesla – though we suspect not all that silicon necessarily involves AI. What sets SiFive's Intelligence XM-series apart from previous engagements with the likes of Google or Tenstorrent is that rather than having its CPU cores attached to a third-party matrix math engine, all packaged up in the same chip, SiFive is instead bringing out its own complete AI accelerator design for customers to license and put into silicon. This isn't aimed at semiconductor players capable of crafting their own accelerators, such as Google and Tenstorrent – it's aimed at organizations that want to take an off-the-shelf design, customize it, and send it to the fab. "For some customers, it's still going to be right for them to do their own hardware," Ronco said. "But, for some customers, they wanted more of a one-stop shop from SiFive." In this sense, these XM clusters are a bit like Arm's Compute Subsystem (CSS) designs in that they offer customers a more comprehensive building block for designing custom silicon. But instead of general application processors, SiFive is targeting those who want to make their own AI accelerators. A closer look at the XM Cluster SiFive's base XM cluster is built around four of SiFive's Intelligence X RISC-V CPU cores which are connected to an in-house matrix math engine specifically for powering through neural network calculations in hardware. If you're not familiar, we've previously explored SiFive's X280 and newer X390 X-series core designs, the latter of which can be configured with a pair of 1,024 vector arithmetic logic units. The base XM cluster comprises four Intelligence X RISC-V CPU cores tied to a matrix engine – Click to enlarge. Source: SiFive Each of these clusters boasts support for up to 1TB/sec of memory bandwidth via a coherent hub interface, and is expected to deliver up to 16 TOPS (tera-operations per second) of INT8 or 8 teraFLOPS of BF16 performance per gigahertz. TeraFLOPS per gigahertz might seem like an odd metric, but it's important to remember this isn't a complete chip and performance is going to be determined in large part by how many clusters the customer places in their component, how it's all wired up internally, what else is on the die, what the power and cooling situation is, and how fast it ends up clocked. At face value, these XM clusters may not sound that powerful – especially when you consider SiFive expects most chips based on the design to operate at around 1GHz. However, stick a few together and its performance potential adds up quickly. Ronco expects most chips based on the design will utilize somewhere between four and eight XM clusters, which in theory would allow for between 4–8TB/sec of peak memory bandwidth and up to 32–64 teraFLOPS of BF16 performance – and that's assuming a 1GHz operating operating clock. That's still far slower than something like an Nvidia H100, which can churn out nearly a petaFLOPS of dense BF16 performance. But as we mentioned earlier, FLOPS aren't everything – especially when it comes to bandwidth constrained workloads like AI inferencing. There are considerations like price, power, process node, and everything else.