Posted Monday, September 30, 2019
By Lauro Rizzatti, Verification Expert
When you want to improve or change a dish, the expert explained, you start with a basic, well-known recipe, and proceed to make small changes. For example, you might modify the ratios of ingredients or replace some ingredients with others that you are familiar with to find out how they contribute to a new recipe. Based on your experience and skill, you have an idea of what to expect.
At the end, you might be content with the results or unhappy. If the results are not satisfactory, you try it again and again until you get a recipe right.
Likewise, when you design a neural network for an application –– let’s say, an image classification algorithm –– you often start with a basic, well-known NN, such as a ResNet50 that includes 50 layers, and proceed to make small changes.
For instance, you may resize one or more layers, smaller layers lead to lower power consumption and faster computation. You may remove one or more layers, making it a ResNet45, to lower computational power and reduce power consumption. Or you may add one or more layers making it a ResNet55 expecting that the quality of results (QoR) will improve at the cost of additional computing power and increased power consumption. Or, you may make minor changes to one or more layers assuming that they improve the QoRs.
Similar to cooking, you may have some idea of what the results ultimately may be, but until you try them out, you do not know what the results will be. At the end, you discover that whatever you did either enhanced the identification of your images or deteriorated their recognition. In that case, you try it again, and again, until you get acceptable results.
Of course, there is a catch.
While changing a cooking recipe can be tasted in less than an hour, a modification to a NN requires retraining the network with many input samples. The more the samples, the higher the probability the algorithm will predict a correct answer or a wrong answer in case of a flawed design.
The problem is that it may take several days or weeks to see the results of a single change with traditional software-based verification tools. Evaluating a series of changes on an algorithm that you are pleased with is time consuming and excessively fatiguing.
This simple analogy justifies the adoption of a hardware emulation platform to accelerate the verification of a NN, shrinking the time to results by orders of magnitude.
Hardware Emulation: The Perfect Verification Engine for Neural Networks
Hardware emulation evolved over years. Back in the day, it started as an esoteric functional verification tool restricted to the largest semiconductor designs ever created. Today, it is a mandatory verification platform for a wide range of design types and a multitude of verification tasks.
It all started when the emulator was deployed in virtual mode instead of the traditional in-circuit emulation (ICE) mode. Suddenly, a plethora of advantages were made available to the user. Among them, simpler deployment without hardware dependencies typical of ICE, and the ability to perform several verification tasks not possible in ICE, including power estimation, low-power verification, design for test (DFT) verification and hardware/software debug. Add ease/deterministic design debug and remote access for multiple concurrent users that moved the emulator out of a lab and into a data center.
A recent class of design creations that can benefit from a hardware emulator are AI designs, specifically, DNN designs. Three reasons support its adoption:
- First, DNN designs cover a wide range of sizes. DNN designs targeting cloud computing that reside in data centers are typically large, reaching into two-to-four billion application specific integrated circuit (ASIC) equivalent gates. They boast high computing power and dissipate substantial power that requires ventilation and air conditioning. They also cover a large footprint and possess considerable weight. This is a territory where hardware emulation remains unchallenged. A modern emulation system scales in capacity from a few tens of millions of ASIC-equivalent gates to more than 10-billion gates. And unlike software-based tools, its performance drops moderately as capacity increases and requires multiple interconnected chasses.
- Second, unlike traditional electronic designs that process deterministic applications, such as video, graphics and networking, DNN designs are stochastic in nature –– they have a random probability distribution that may be analyzed statistically but may not be predicted precisely. Their deployment requires a training process to “teach” them what to predict by feeding them a large set of input data.
- Third, a DNN design includes software stacks made of drivers, OS, firmware and frameworks, typically validated by performance benchmarks, such as MLPerf and DAWNBench. Virtualization of the verification environment enables execution of a broad range of AI frameworks and performance benchmarks, not feasible in ICE mode.
Conversely, DNN designs that target the edge are comparably smaller, boast low latency and low-power consumption. Here hardware emulation may have less appeal, yet its ability to accelerate embedded software gives it an unparallel advantage.
To tame the random behavior of a DNN design, a deterministic verification environment is mandatory. Hardware emulation can provide such an environment. From design compilation to design execution to design debug, an emulator supports a consistent, repeatable verification methodology.
Conclusion
At a first glance, the process of designing deep neural networks is akin to creating a cooking recipe. The analogy drastically diverges when the complexity of developing a NN is taken into account.
While verifying the success of a cooking recipe may take minutes or a few hours, requiring just a cooktop stove and various cooking utensils, the verification of a DNN design can take days and weeks, and require a best-in-class hardware emulation platform.
The AI market is highly competitive and meeting time-to-market and QoR objectives is uppermost critical to achieve success. Only hardware emulation can help to reach challenging goals.