Brainwave, the Latest in a Surge of Deep Learning Acceleration Platforms

microsoft brainwave
Ktsdesign |

Many tech giants looking to stay ahead of the AI curve are creating different kinds of platforms to boost AI tech and the startups that create them. In comes Microsoft Brainwave.

As a result of this, deep learning acceleration platforms are gaining traction. Small tech startups have increased competition, too, now that Microsoft released its latest tool.

What’s the deal with Brainwave and how is it boosting AI programs?

New Microsoft tool #Brainwave Boosts AI ProgramsClick To Tweet

Is AI Deep Learning Acceleration Really That Popular?

image of Microsoft logo for release of Microsoft Project Brainwave for deep learning acceleration programs and software and machine learning
Microsoft Corp. Logo | Billy H.C. Kwok/Bloomberg via Getty Images

In case you missed it, Hot Chips 2017 — a “Symposium for High Performance Chips” — just happened in August in the hottest hotbed of startup tech: Silicon Valley. Forays into deep learning acceleration programs include KnuEdge. This startup, founded by former NASA chief Daniel Goldin, dedicated more than $100 million USD to research and development.

image of Mythic logo for article about Microsoft Project Brainwave for AI deep learning acceleration programs
Mythic logo | Mythic

Others, such as Graphcore based in Bristol, received a paltry $30 million USD for their machine learning accelerator.

Texas’ own startup Mythic utilized $9.5-million USD plus $2.5 million USD in grants to pursue their ultra small, power packed GPU chip.

Of course, not all of the names at the forefront of AI acceleration software and hardware are small names. NVIDIA recently launched the NVIDIA Volta-based processor called simply: Tesla.

The two models (P100 and P4 GPU accelerators) claim to have 58 times higher throughput regarding deep learning performance per watt. Beyond that, it also claims to have 33 times higher throughput in deep learning inference throughput.

image of NVIDIA Tesla chip in article about Microsoft Project Brainwave for AI deep learning acceleration program
Tesla | NVIDIA

Of course, this is all compared to Alex Net, GoogleNet, VGG-19, and ResNet-152. It also came with a price tag of $2-billion USD.

Microsoft is still using a code name for their deep learning acceleration hardware known as “Project Brainwave”.

As stated by Doug Burger, distinguished Microsoft engineer, the Brainwave is “…a major leap forward in both performance and flexibility for cloud-based serving of deep learning models”. So what does it do exactly?

So what does it do exactly?

FPGA Infrastructure & Real Time AI Support

image of Stratix 10 Intel Chip to work with Microsoft Project Brainwave for AI deep learning acceleration programs
Stratix 10 Chip | Intel

Microsoft has gone all-in on field programmable gate array (FPGA) infrastructure. Intel’s new Stratix 10 chip supports real time AI using Microsoft’s data center network. This allows Microsoft to facilitate DNNs as hardware micro services and map a collection of remote FPGAs. Then, you can call the DNN using servers and NO software in the loop.

Pro-tip: “DNN” stands for “Dynamic Neural Network”.

Ideally, this structure allows for extremely low latency, as well as high throughput. What does this mean? The CPU no longer needs to process incoming requests AND the FPGA infrastructure processes requests as soon as the network receives them.

Even in early Stratix 10 tests, Microsoft’s Brainwave achieved higher performance output on a GRU model five times larger than Resnet-50.

The test sustained 39.5 teraflops, running every individual request in less than one millisecond (and that’s without batching). This latest tool provides support for the Microsoft Cognitive Toolkit and Google’s Tensorflow. The lack of batching to boost throughput allows for better results with real-time AI programs.

image of hardware from Microsoft for Project Brainwave to act as AI deep learning acceleration programs
Project Brainwave | Microsoft

Applications for Any and All Real Time Tech

image of Netflix logo for video streaming service which could use rapid AI support from Microsoft Project Brainwave and AI deep learning acceleration programs
Netflix Logo | Netflix

What is “real time tech”? You’ve probably already rattled off a list in your head, starting with Netflix or YouTube. Doug Berger elaborated further on Project Brainwave.

“If it’s a video stream, if it’s a conversation, if it’s looking for intruders, anomaly detection, all the things where you care about interaction and quick results, you want those in real time.”

Apart from streaming services for media, cybersecurity software could benefit immensely from deep learning acceleration programs such as Brainwave.

Though no timeline for release has been made public, Microsoft claims Project Brainwave could do great things for AI and humans alike. Beyond the throughput improvements and rapid AI support, Project Brainwave could improve UX in a variety of ways.

What other applications could this hardware have in everyday life?

banner ad to seo services page