Power to the Data Report: Introduction to Neural Magic

Neural Magic is a startup company that focuses on developing technology that enables deep learning models to run on commodity CPUs rather than specialized hardware like GPUs. The company was founded in 2018 by Alexander Matveev, a former researcher at MIT, and Nir Shavit, a professor of computer science at MIT. They raised a total of $50 million in funding to date over 3 rounds, from investors such as Comcast Ventures, NEA, Andreessen Horowitz, Pillar VC, and Amdocs.

The company’s core technology is called “Neural Magic Inference Engine,” consisting of the 1.4 product release of the DeepSparse, SparseML, and SparseZoo libraries. The software optimizes deep learning models to run on CPUs with high performance and efficiency. This is achieved through a combination of algorithmic and hardware optimizations that take advantage of the unique characteristics of CPUs, such as their large cache sizes, multi-threading capabilities, and vector instructions.

The Neural Magic libraries are compatible with popular deep learning frameworks like TensorFlow and PyTorch, making it easy for developers to integrate them into their existing workflows. The software can be used for a variety of applications, including natural language processing, computer vision, and time series analysis.

One of the key advantages of using CPUs instead of specialized hardware is cost. CPUs are much cheaper and more widely available than GPUs, which can be prohibitively expensive for some organizations. This makes deep learning more accessible to a wider range of businesses and researchers.

Another advantage is flexibility. CPUs are general-purpose processors that can be used for a variety of tasks, whereas GPUs are designed specifically for parallel computing. This means that CPUs can be used for a wider range of applications, making them a more versatile option.

Neural Magic also offers improved performance compared to running deep learning models on CPUs without optimization. In benchmarks conducted by the company, the software was able to achieve up to 10 times faster performance on CPUs compared to unoptimized models.

The company has already attracted a number of high-profile customers, including Lockheed Martin. The two organizations have partnered to explore the use of Neural Magic’s technology for defense applications, including autonomous vehicles and drones. The partnership demonstrates the potential for Neural Magic’s technology to have significant impact in industries beyond traditional AI and machine learning applications. Additionally, Harvard Medical School is a customer of the company. The school’s Department of Biomedical Informatics has used Neural Magic’s technology to optimize deep learning models for processing medical images, with the aim of improving patient care and outcomes.

Neural Magic has also developed a number of pre-trained models that can be used out-of-the-box for a variety of applications. These models are optimized to run on CPUs, making them a fast and efficient option for organizations that need to get up and running quickly.

Neural Magic’s technology has the potential to transform the field of deep learning by making it more accessible and affordable for a wider range of organizations. By enabling deep learning models to run on CPUs with high performance and efficiency, the company is helping to democratize AI and unlock its potential for new applications.

Neural Magic has strong ties to the AI research community, and has written a number of compelling research papers.

In conclusion, Neural Magic is a promising startup company that is developing innovative technology to optimize deep learning models for CPUs. The company’s software libraries and tools enable high-performance and efficient deep learning on commodity hardware. Its technology has the potential to make deep learning more accessible and affordable for a wider range of organizations, unlocking its potential for new applications.

Contributed by Daniel D. Gutierrez, Editor-in-Chief and Resident Data Scientist for insideBIGDATA. In addition to being a tech journalist, Daniel also is a consultant in data scientist, author, educator and sits on a number of advisory boards for various start-up companies. 

Sign up for the free insideBIGDATA newsletter.

Join us on Twitter: https://twitter.com/InsideBigData1

Join us on LinkedIn: https://www.linkedin.com/company/insidebigdata/

Join us on Facebook: https://www.facebook.com/insideBIGDATANOW