Many tech giants looking to stay ahead of the AI curve are creating different kinds of platforms to boost AI tech and the startups that create them. In comes Microsoft Brainwave.
As a result of this, deep learning acceleration platforms are gaining traction. Small tech startups have increased competition, too, now that Microsoft released its latest tool.
What’s the deal with Brainwave and how is it boosting AI programs?New Microsoft tool #Brainwave Boosts AI ProgramsClick To Tweet
Is AI Deep Learning Acceleration Really That Popular?
In case you missed it, Hot Chips 2017 — a “Symposium for High Performance Chips” — just happened in August in the hottest hotbed of startup tech: Silicon Valley. Forays into deep learning acceleration programs include KnuEdge. This startup, founded by former NASA chief Daniel Goldin, dedicated more than $100 million USD to research and development.
Others, such as Graphcore based in Bristol, received a paltry $30 million USD for their machine learning accelerator.
Of course, not all of the names at the forefront of AI acceleration software and hardware are small names. NVIDIA recently launched the NVIDIA Volta-based processor called simply: Tesla.
The two models (P100 and P4 GPU accelerators) claim to have 58 times higher throughput regarding deep learning performance per watt. Beyond that, it also claims to have 33 times higher throughput in deep learning inference throughput.
Microsoft is still using a code name for their deep learning acceleration hardware known as “Project Brainwave”.
As stated by Doug Burger, distinguished Microsoft engineer, the Brainwave is “…a major leap forward in both performance and flexibility for cloud-based serving of deep learning models”. So what does it do exactly?
So what does it do exactly?
FPGA Infrastructure & Real Time AI Support
Microsoft has gone all-in on field programmable gate array (FPGA) infrastructure. Intel’s new Stratix 10 chip supports real time AI using Microsoft’s data center network. This allows Microsoft to facilitate DNNs as hardware micro services and map a collection of remote FPGAs. Then, you can call the DNN using servers and NO software in the loop.
Pro-tip: “DNN” stands for “Dynamic Neural Network”.
Ideally, this structure allows for extremely low latency, as well as high throughput. What does this mean? The CPU no longer needs to process incoming requests AND the FPGA infrastructure processes requests as soon as the network receives them.
Even in early Stratix 10 tests, Microsoft’s Brainwave achieved higher performance output on a GRU model five times larger than Resnet-50.
The test sustained 39.5 teraflops, running every individual request in less than one millisecond (and that’s without batching). This latest tool provides support for the Microsoft Cognitive Toolkit and Google’s Tensorflow. The lack of batching to boost throughput allows for better results with real-time AI programs.
Applications for Any and All Real Time Tech
What is “real time tech”? You’ve probably already rattled off a list in your head, starting with Netflix or YouTube. Doug Berger elaborated further on Project Brainwave.
“If it’s a video stream, if it’s a conversation, if it’s looking for intruders, anomaly detection, all the things where you care about interaction and quick results, you want those in real time.”
Apart from streaming services for media, cybersecurity software could benefit immensely from deep learning acceleration programs such as Brainwave.
Though no timeline for release has been made public, Microsoft claims Project Brainwave could do great things for AI and humans alike. Beyond the throughput improvements and rapid AI support, Project Brainwave could improve UX in a variety of ways.