Android developers can use ML Kit GenAI APIs to access Gemini Nano without shipping or managing the model themselves.
2
AI Core puts one shared model in the system, handles hardware optimization and scheduling, and gives foreground apps priority over background work.
3
GenAI APIs currently target recent flagship devices, while hybrid inference and LiteRT provide ways to reach more devices or use cloud models.
Summary
Florina Muntenescu and Oli Gaymond explain the Android options for adding on-device AI. ML Kit GenAI APIs provide task-specific APIs and a Prompt API for Gemini Nano, with text and image input and text output. AI Core installs and manages the model at the system level, so apps share one roughly 3 to 4 GB model instead of shipping their own copy. It handles hardware optimization, isolation, queuing, and background scheduling. On-device inference keeps sensitive data local, works offline, and has no per-request inference cost, but Gemini Nano currently requires recent flagship hardware. Hybrid inference can fall back to Gemini Flash in the cloud when Nano is unavailable. Classic ML Kit vision and OCR models work on a much wider device range. The speakers also discuss using Prompt API for RAG, with an embedding API planned, and recommend LiteRT when developers need custom models or broader device coverage.
Android apps can choose between on-device, hybrid, and cloud inference
Muntenescu lays out three ways to build intelligent experiences on Android. On-device inference processes prompts directly on the phone, so sensitive information such as banking data does not leave the device. It also supports offline work and does not add an inference charge. Hybrid inference uses the local model when it is available and sends the request to the cloud otherwise. Developers can also choose cloud inference for access to more powerful models. The on-device path is useful for personalization, private data, translation, and other tasks that fit a shorter context window.
AI Core lets every app share one system-managed Gemini Nano model
Gemini Nano reaches the device through the AI Core system service. The model is installed once, then accessed by apps through the ML Kit GenAI APIs. Oli Gaymond compares this with using a managed cloud service: developers focus on their feature and prompt while the platform obtains the model, optimizes it for the available hardware, and runs inference. Requests and outputs are isolated between apps, and the input and output data is not stored on the device. This removes model setup and hardware-specific management from the app developer.
The Prompt API is the most flexible ML Kit GenAI interface
ML Kit GenAI includes APIs for tasks such as summarization, proofreading, and rewriting. The Prompt API accepts natural-language requests and currently supports text and images as input, with text as output. Muntenescu says this makes it suitable for image understanding, content assistance, content analysis, and entity extraction. Apps can pass files and other information they already have permission to access into the prompt. The API is a lower-level building block, so other tools can compose skills or instructions into prompts before sending them to Gemini Nano.
AI Core handles battery tradeoffs and schedules background work
Running a model continuously can drain a phone quickly, but Gaymond says the interactive use cases seen today tend to involve a user asking a question or manipulating data around 10 or 20 times per day. Larger batch workloads can run in the background, often overnight while the device is charging. AI Core handles optimization, queuing, and scheduling, so developers do not manage those details. Foreground apps receive top priority. Background requests may wait behind active use or other queued work. Developers using custom models have to profile and measure their own battery impact.
A system-shared model avoids shipping several gigabytes in every app
Gaymond says the smallest useful models are around 1 GB, while the models being shipped are close to 3 to 4 GB in total. Shipping that package with every app would require an unusually strong reason from the developer and impose the cost repeatedly on users. Putting the model in the system means it is installed once and shared by apps. AI Core can also attribute usage and manage requests across applications. Developers simply make the inference call, while the system handles the model and scheduling.
Hybrid inference expands reach beyond phones that support Gemini Nano
Gemini Nano is currently available on recent flagship devices, including Pixel 9 and Pixel 10 generation phones and devices from other manufacturers. Firebase AI Logic supports hybrid inference, which runs locally when Nano is available and falls back to the cloud otherwise. The speakers describe using Gemini Nano on the device and Gemini Flash in the cloud for a similar feature across more phones. Firebase AI Logic also provides access to Gemini API and Vertex AI providers for cloud-only use cases.
RAG can be built with Prompt API now, with embeddings planned
The speakers say developers can build a RAG-like experience with the Prompt API by passing relevant content into the model. ML Kit does not yet provide an embedding API in this talk, but Muntenescu says one is planned. The future API is expected to make it possible to use the Gemini embedding model directly from the interface. Until then, developers who need vectorized notes or similarity search need another embedding path. AI Edge Gallery demonstrates broader possibilities, while AI Core is aimed at production apps that need less setup.
Classic ML Kit models reach far more devices than GenAI APIs
The wider ML Kit offering includes conventional text, OCR, and vision models. Muntenescu says those models are much smaller and can run on more than a billion devices. The newer GenAI APIs require fairly recent flagship devices. AI Core tells the developer whether a supported device can run the model well. Developers who need wider coverage can use LiteRT and test custom models across more hardware, but they then take responsibility for that testing and optimization.
