How RISC-V is changing the server market [Q&A]

Data centers have a strong emphasis on performance and delivering workloads whilst remaining power efficient.

In order to deliver on these aims the open instruction set architecture RISC-V is increasingly being used as an accelerator in data centers since it offers a lot of flexibility, which is especially important with AI. We spoke to Ian Ferguson, senior director at SiFive to discuss why this technology has become key to the server market.

BN: From your perspective how has the server market evolved?

IF: In 2008, I was one of the first two instigators of Arm's exploration and subsequent entry into the server market. At that point in time, Intel was the dominant provider with greater than 97 percent market share. AMD owned the rest. There was no GPGPU market (NVIDIA sold GPUs to client applications like performant PC, notebooks, etc.). AI didn't exist outside a few research experiments. Software was largely owned by 3rd parties.
Fast forward to today, AMD is strong. NVIDIA is even more dominant as GPGPUs have become a key technology to implement AI training algorithms. Companies like Amazon have indicated a plan to split its CPU supplier base between x86 providers (Intel AND AMD) and then Arm (internally developed and external suppliers) and RISC-V. Software (with the exception of some of the GPGPU pieces) are a combination of:

• Open source
• First-party software (code owned, maintained, advanced, and optimized by Apple, Amazon, Google, Meta, and Microsoft)

BN: Why is the server market seeing more demand for RISC-V?

IF: There are three major shifts happening in the server market right now:

• It is impossible to define the perfect ASIC since there is a diverse set of workloads and the market is changing rapidly. What is clear is that there is a need for general-purpose compute to be combined with accelerator technology. The latter can be a combination of GPGPUs and other specialist hardware.
• Cloud companies want to optimize their solutions to get a competitive advantage over rivals. For many of these companies, there is a business justification for them to create custom silicon. They know their workloads best and see their optimization as an area for differentiation over rivals. The big opportunity for vendors is to provide cloud companies with the building blocks they need for their designs.
• AI is moving to the edge. As a result, a server’s cost, power, and size requirements are forcing companies to look at different components as compared with those they have traditionally used.

Customers need an architecture that can evolve at a pace that aligns with the rate of change in the market. RISC-V provides a modern, clean slate design with unparalleled flexibility, extensibility, and scalability. Developers can customize their designs to optimize performance, power, cost and/or latency. The most recent example is adding improvements for AI workloads. This has started with the RISC-V Vector (RVV) specification, and working groups are collaboratively working on other enhancements to further improve the implementation of AI training and inference workloads. Also, with RISC-V, there is no vendor lock-in, so companies have more choice.

On the software side, cloud companies are using both open-source software and first-party software. The primary initiative that is driving the open-source work is the RISC-V Software Ecosystem (RISE) project, a collaborative effort to develop a robust software ecosystem for RISC-V processors in the datacenter, automotive, mobile, consumer electronics, and beyond. We will continue to see significant performance improvements as we transition from 'ported to RISC-V' to 'optimized for RISC-V.' This provides the foundational platform on which the cloud companies can optimize their first party software.

RISC-V interest is growing globally and China in particular is moving quickly to implement RISC-V into datacenter, especially in the sophisticated cloud companies.

BN: How is RISC-V being adopted in cloud applications today?

IF: The great thing about RISC-V is that is empowers innovation. SiFive's focus is more around augmenting existing solutions for which a software ecosystem exists as opposed to going head-to-head against incumbent processor architectures. We see immense interest from cloud companies in our Intelligence line of products that help components design efficient neural processing units, delivering great performance for a specific workload, whether that be image recognition, speech translation, medical diagnosis, etc.

Other creators of RISC-V solutions are focusing their efforts on winning the primary CPU socket in a server.

BN: What about the demand for RISC-V in the high-performance computing segment?

IF: There is a huge amount of RISC-V activity in the high-performance computing segment. One reason is that there is a lot of first-party software, which lessens the barriers to entry for new processor technology. Additionally, many workloads do not require high single thread performance, since having multiple smaller processors works well.

There is also a growing focus on sustainability as companies become more concerned with the impact of computing on the environment. It's not just about building the fastest machines, companies are also striving to build the most eco-friendly machines (what we're seeing on the Green500). RISC-V is a clean architecture -- without legacy baggage from decades of use -- which allows companies to design solutions that are extremely efficient.

Of course, if companies want to design extremely high-performance systems on RISC-V they can do that too. We're also seeing many regions -- including China, Europe, India, Japan, and the US -- developing extremely powerful RISC-V machines.

BN: How is SiFive addressing the demand for RISC-V in servers?

IF: SiFive's semiconductor partners have shipped in excess of two billion SiFive processors across a diverse set of markets. As mentioned above, SiFive is primarily focused on the cloud and high-performance computing segments where our customers are using our RISC-V solutions to augment the main processor in a system. SiFive’s Intelligence processors offer high performance scalar and vector compute capability to meet the latest AI requirements in these segments. Companies like Google are using RISC-V vector processors, combined with a systolic matrix multiplier accelerator, to offer more flexibility for compute-intensive AI/ML workloads.

We also developed the SiFive Vector Coprocessor Interface Extension (VCIX), which allows custom vector instructions to be executed on the accelerator from the vector pipeline. This helps to streamline the design process for a custom accelerator, while also making it easier to program. Additionally, this approach enables higher system performance and improved power and area efficiency.

BN: What do you think RISC-V adoption in servers will look like 10 years from now?

IF: That is a challenging time window since this area of industry is changing so quickly. AI and NVIDIA are both at the forefront of so many conversations now and were barely discussed seven years ago, let alone ten. There have been huge changes over the last few years with the architecture of AI models, from convolutional networks to transformers. This rate of change has led to harnessing accelerators that can be adjusted to support changes to the models. Over time, I expect this rate of change of models to slow down, as winning technology approaches (both hardware and software) start to emerge.

What we have seen for other technologies is that when things become mainstream and stable, there are more opportunities for integration. While you can find discrete chips delivering USB or Ethernet functionality, most mainstream semiconductor components integrate those functions. In the arena of AI, there will be less need for application specific accelerators. Instead, this functionality will become part of the standard CPU to save on footprint, power, and costs.

Additionally, over the next decade we’ll see the performance gap close between RISC-V and incumbents. With this leveling of the playing field, the winners will be the companies that are able to deliver their value and differentiation most cost-efficiently and quickly.

Image credit: monsit/depositphotos.com