Accelerating the Accelerator:
Automated Static Checking for Regular-Structure Synchronous Accelerator

SSO Symposium Case Study by Igor Arsovski of Groq

Case Study Overview

Igor Arsovski of Groq presented at Real Intent’s 2023 Static Sign-Off Symposium on how Groq uses automated static checking for their regular-structure synchronous accelerator.

Below are text and graphic highlights of what he presented. Real Intent Ascent Lint, Meridian CDC and Meridian RDC were deployed.

The video has the following chapters:

AI compute needs & Groq’s AI chip
Groq’s Lint, CDC & RDC sign-off flow
Results from Groq’s static sign-off flow
Future static sign-off needs

Groq’s Simplified Chip Architecture

Groq’s chip has a very simplified, domain-specific architecture

Slide 1: AI compute demand
Slide 2: Domain-specific architectures
Slide 3: Groq’s simplified architecture
Slide 4: Rapid AI model compilation

[Toggle the slides to view Groq’s technology background graphics and explanations.]

AI compute demand

We're living in the age of AI. Wherever you go in the Bay Area everybody's talking about large language models; everybody's playing with Chad GPT, Bard, and LLaMA. What we need to talk about is the amount of compute necessary to support these language models. On this slide in the top right corner, I put a graph from Microsoft showing the rapid growth in the size of these AI models. They are roughly doubling in size every three and a half months. On the bottom is our supply for this compute. The gap between them needs to be closed -- either by throwing more chips at the problem, or coming up with designs that are more specific for that specific application.

Domain-specific architectures

We have an explosion of these domain-specific architectures that are really focused on a set of workloads to efficiently Implement those specific workloads. Groq’s chip has one of those domain-specific architectures. But we have taken a very different approach. We started Groq as a software first company. Before we even wrote a single code of RTL, we started by figuring out the software and the software compilers that would be necessary to map these algorithms into hardware.

Groq’s simplified architecture

Here is the typical AI accelerator of today, the GPU. You see a large number of cores arranged in a two-dimensional array; each of these scores executes a portion of the workload. They're non-deterministic; they execute the fastest they possibly can, and they all fight for resources, such as I/O and DRAM resources, to solve a problem. You can also see the layout complexity. If you look at the floor plan, there is a lot of control logic in the GPU that's managing the flow of data between these cores. The Groq chip has a very regular structure. We've stripped away a lot of the dynamic control off the chip and moved it to software. What we're left with are the actual execution units -- the data paths that are executing these workloads. And we have IOs to provide the data. About three percent of the silicon area is devoted to control versus about thirty percent that you see in a typical accelerator. This has actually created a chip that's deterministic in nature. At any point in time, we know exactly where the data is located in the chip. We know exactly which functional units are activated on a clock-by-clock basis; this has enabled us to solution software much more efficiently, which is the biggest problem in today's AI accelerators.

Rapid AI model compilation

This enabled the Renaissance as to how quickly we can compile new AI models. In the first six quarters, starting last year, we went from about zero to sixty AI models that were compiled. But because our compiler is evolving so quickly, in a push button approach we were able to compile an order of magnitude more models in only six weeks.

Download Case Study PDF

Groq’s Static Sign-off Methodology

Our chip’s regular nature enables us to focus even our static sign off and simplify that as well.

The top and the bottom of the chip is where you see most of the asynchronous interfaces. This is where we focus our clock domain crossing sign-off with Meridian CDC and our reset domain crossing sign-off with Meridian RDC.

And then the rest of the core is running on a single clock, so there are no different clock domain crossings. So here we can focus on RTL linting to get the maximum benefit out of that.

We have repeated blocks, which streamline how quickly we can move through static sign-off. We engage with [Real Intent Ascent Lint] very early in the flow.

This is all automated, so we maintain our repositories to always be lint clean. As soon as someone makes a change in RTL, we automatically launch RTL linting and other checks to make sure the RTL is clean before it is checked in into our golden repository.

Results — Groq static sign off methodology

Using this approach, we’ve been able to really increase the speed of our iterations. We have really enjoyed the [Real Intent] tools from the aspect of reduction in false positives. This minimizes distractions for our logic team.

We also have really good FAE support. Let’s face it every EDA tool is going to have their bugs — it’s how fast the response is to address these bugs that makes the difference of how we pick our vendors.

As I mentioned, the automated lint really keeps our repositories always clean and ready to kind of tape out. So, we’re always kind of keeping the cleanest possible state of our blocks. So that’s our current static sign-off approach.

Conclusion – Future Static Sign-Off Vision

What we want to see in the future is an increase in the overlap between logic design verification and physical design. We would love to see lint co-pilots or autopilots that are constantly managing as we write RTL code, so we get that feedback immediately.

We would love to see suggestions of how we would fix some of these problems that are identified by lint. We’d love to get even further shift left right so basically get the feedback on the fly.

I would love the ability to stack logic design verification and physical design and compress those time-to-market cycles. For example, potentially do some light synthesis, and provide feedback on how wire-constrained we are like, what are the flop-to-flop logic stages? Can we actually get feedback on power, maybe suggest assertions that need to be coded into specific blocks, and even implement those?

These are ideas where you can argue you can use AI to help the next generation of AI accelerators.