Red Hat Announces Definitive Agreement to Acquire Neural Magic

Red Hat, Inc., the world’s leading provider of open source solutions, has announced that it has signed a definitive agreement to acquire Neural Magic, a pioneer in software and algorithms that accelerate generative AI (gen AI) inference workloads. Neural Magic’s expertise in inference performance engineering and commitment to open source aligns with Red Hat’s vision of high-performing Artificial Intelligence (AI) workloads that directly map to customer-specific use cases and data, anywhere and everywhere across the hybrid cloud.

While the promise of gen AI dominates much of today’s technology landscape, the large language models (LLMs) underpinning these systems continue to increase in size. As a result, building cost-efficient and reliable LLM services requires significant computing power, energy resources, and specialised operational skills. These challenges effectively put the benefits of customised, deployment-ready, and more security-conscious AI out of reach for most organisations.

“AI workloads need to run wherever customer data lives across the hybrid cloud; this makes flexible, standardised, and open platforms and tools a necessity, as they enable organisations to select the environments, resources, and architectures that best align with their unique operational and data needs,” said Matt Hicks, President and CEO at Red Hat. “We’re thrilled to complement our hybrid cloud-focused AI portfolio with Neural Magic’s groundbreaking AI innovation, furthering our drive to not only be the ‘Red Hat’ of open source, but the ‘Red Hat’ of AI as well.”

Red Hat intends to address these challenges by making gen AI more accessible to more organisations through the open innovation of vLLM. Developed by UC Berkeley, vLLM is a community-driven open source project for open model serving (how gen AI models infer and solve problems), with support for all key model families, advanced inference acceleration research and diverse hardware backends, including AMD GPUs, AWS Neuron, Google TPUs, Intel Gaudi, NVIDIA GPUs, and x86 CPUs. Neural Magic’s leadership in the vLLM project combined with Red Hat’s strong portfolio of hybrid cloud AI technologies will offer organisations an open pathway to building AI strategies that meet their unique needs, wherever their data lives.

Red Hat + Neural Magic: Enabling a Future of Hybrid Cloud-Ready Gen AI

Neural Magic spun out of MIT in 2018 with the goal of building performant inference software for deep learning. With Neural Magic’s technology and performance engineering expertise, Red Hat aims to accelerate its vision for AI’s future, powered by the Red Hat AI technology portfolio. Built to break through the challenges of wide-scale enterprise AI, Red Hat uses open source innovation to further democratise access to AI’s transformative power via:

• Open source-licenced models, from the 1B to 405B parameter scale, that can run anywhere and everywhere needed across the hybrid cloud—in corporate data centres, on multiple clouds. and at the edge.
• Fine-tuning capabilities that enable organisations to more easily customise LLMs to their private data and uses cases with a stronger security footprint;
• Inference performance engineering expertise, resulting in greater operational and infrastructure efficiencies; and
• A partner and open source ecosystem and support structures that enable broader customer choice, from LLMs and tooling to certified server hardware and underlying chip architectures.

vLLM Leadership to Enhance Red Hat AI

Neural Magic uses its expertise and knowledge in vLLM to build an enterprise-grade inference stack, which enables customers to optimise, deploy, and scale LLM workloads across hybrid cloud environments with full control over infrastructure choice, security policies and model lifecycle. Neural Magic also develops model optimisation research, builds LLM Compressor (a unified library for optimizing LLMs with state-of-the-art sparsity and quantisation algorithms), and maintains a repository of pre-optimised models ready to deploy with vLLM.

“Open source has proven time and again to drive innovation through the power of community collaboration. At Neural Magic, we’ve assembled some of the industry’s top talent in AI performance engineering with a singular mission of building open, cross-platform, ultra-efficient LLM serving capabilities. Joining Red Hat is not only a cultural match, but will benefit companies large and small in their AI transformation journeys,” said Brian Stevens, CEO at Neural Magic.

Red Hat AI aims to help customers lower AI’s costs and skill barriers with powerful technologies, including:

• Red Hat Enterprise Linux AI (RHEL AI), a foundation model platform to more seamlessly develop, test, and run the IBM Granite family of open source LLMs for enterprise applications on Linux server deployments;
• Red Hat OpenShift AI, an AI platform that provides tools to rapidly develop, train, serve, and monitor machine learning models across distributed Kubernetes environments on-site, in the public cloud or at the edge; and
• InstructLab, an approachable open source AI community project created by Red Hat and IBM that enables anyone to shape the future of gen AI via the collaborative improvement of open source-licenced Granite LLMs using InstructLab’s fine-tuning technology.

“As our clients look to scale AI across their hybrid environments—virtualised, cloud-native LLMs built on open foundations will become the industry standard. Red Hat’s leadership in open source combined with the choice of efficient, open source models like IBM Granite and Neural Magic’s offerings for scaling AI across platforms empower businesses with the control and flexibility that they need to deploy AI across the enterprise,” said Dario Gil, IBM Senior Vice President and Director of Research.

Neural Magic’s technology leadership in vLLM will enhance Red Hat AI’s ability to support LLM deployments anywhere and everywhere across the hybrid cloud with a ready-made, highly-optimized and open inference stack.

The transaction is subject to applicable regulatory reviews and other customary closing conditions.