Google Open-Sources ML Drift, Its On-Device GPU AI Engine

Google’s AI Edge Team has released ML Drift as an open-source project under the Apache 2.0 license, making its high-performance, on-device GPU inference engine freely available to any developer who wants to use, modify or build on it commercially. The engine runs across OpenGL ES, OpenCL, Metal and WebGPU, meaning a single codebase can target Android, iOS, the web and now desktop platforms including Windows and Linux.

This is not a brand-new technology. ML Drift has been running quietly inside Google’s own products for some time, and the formal announcement marks its transition from internal tool to public infrastructure. It now serves as the core GPU acceleration engine inside LiteRT — Google’s successor to TensorFlow Lite — and is also available as a standalone library for developers who want to plug it into their own graphics or inference runtimes.

What Actually Changed From TensorFlow Lite

The legacy TFLite GPU delegate, which ML Drift replaces, was a foundational piece of mobile AI infrastructure. But it was built for a narrower set of workloads. Modern on-device AI — think large vision models, spatiotemporal models and on-device language models — pushed consumer hardware in ways the older runtime wasn’t designed to handle.

ML Drift addresses this with several concrete architectural upgrades. The most significant is tensor virtualization, which decouples a tensor’s logical representation from its physical allocation on the GPU. In plain terms, this means developers no longer have to maintain separate, backend-specific shader code for OpenGL, OpenCL and Metal — one unified shader model covers all three. The engine also adds 5D tensor support out of the box, something the TFLite delegate never had. That unlocks models like YOLO 11n, MobileViT v2 and Swin Transformer v2 for direct execution on edge GPUs without layout workarounds.

For large language models specifically, ML Drift introduces stage-aware optimizations that dynamically switch kernel configurations depending on whether the model is in the compute-heavy prefill phase or the memory-bound decode phase — a distinction that matters a lot when you’re running a Gemma model on a phone rather than a data center GPU.

LiteRT with ML Drift averages 1.4x faster performance over the legacy TFLite GPU delegate across a broad range of models. In memory efficiency benchmarks on Gemma models, ML Drift achieves up to 12% lower memory overhead compared to other frameworks, which directly affects how many tasks a device can handle at once.

Production Numbers From Real Deployments

The scale of existing deployments gives these figures more weight than typical benchmark claims. YouTube Shorts moved its segmentation-based visual effects to ML Drift and saw up to a 40% reduction in average frame latency across Android and iOS. Google Photos reported up to a 2-second speedup for photo editing pipelines compared to the legacy GPU delegate. Adobe integrated ML Drift into Lightroom and Photoshop’s mobile AI features — including Select Subject, Select Sky and Adaptive Portrait — and achieved up to 30% faster on-device performance. Snapchat reported 30% model latency improvements for ML-powered face and style effects in Snapchat Lenses.

Desktop support is an early-stage preview, enabled by reusing ML Drift’s WebGPU backend through Dawn, the Chromium project’s WebGPU implementation. This allows the engine to run natively on Windows and Linux without requiring developers to wrangle DirectX or Vulkan directly. macOS support uses a separate, already-mature Metal backend.

Where ML Drift Fits in a Crowded Field

The on-device inference space has become genuinely competitive. Meta’s ExecuTorch offers cross-platform hardware optimization with delegates for Apple’s CoreML and Android chipsets from Qualcomm, Arm and MediaTek, but it is tightly coupled to the PyTorch ecosystem. Apple’s CoreML delivers near-zero runtime overhead — but only on Apple hardware. ONNX Runtime gives broad portability at the cost of startup latency and integration complexity, and typically runs 10–20% slower than native CoreML on iOS.

ML Drift’s specific bet is cross-platform GPU unification without locking developers into a single vendor’s ecosystem or a single framework’s conventions. Its production deployments at Adobe and Snapchat provide third-party credibility that newer entrants like ExecuTorch are still building toward.

Why This Matters If You’re a Student or Early-Career Developer

Before something like ML Drift existed as a free, open library, building a GPU-accelerated AI feature for mobile meant either developing deep expertise in platform-specific APIs — OpenCL on Android, Metal on iOS — or paying for cloud inference every time your app needed to run a model. Neither option is realistic for a student project or an early-stage product.

ML Drift abstracts all of that. A developer can prototype an on-device AI feature on a laptop and deploy it to Android or iOS without rewriting the inference stack for each platform. The engine also ships with an agentic SKILL.md guide designed to help AI coding tools like GitHub Copilot or Cursor author, register and verify custom GPU shaders — which means students can write GPU-optimized inference code even without a background in graphics programming.

For anyone targeting mobile AI or edge computing roles, familiarity with LiteRT and ML Drift is becoming a practical credential. Google, Adobe and Snapchat are all deploying it in production today, and the Apache 2.0 license means there’s no legal friction in building a portfolio project or a startup on top of it.

Source: Google AI