Google DeepMind is researching AI systems beyond language models, including embedding models, weather forecasters, and interactive world models.
2
Gemini Embeddings 2 maps text, video, audio, and PDFs into a shared semantic space for retrieval, querying, and agent logic.
3
GraphCast, GenCast, and FGN use neural networks to forecast atmospheric conditions and cyclones more efficiently and accurately than established physics-based approaches.
Summary
Raia Hadsell describes Frontier AI research at Google DeepMind by focusing on systems that do more than generate language. She begins with embedding models, explaining how Gemini Embeddings 2 creates shared representations for text, video, audio, and PDFs. These representations support retrieval and comparison across modalities, with embeddings that can be queried at different dimensions. She then discusses weather forecasting. GraphCast predicts atmospheric conditions up to 15 days ahead, GenCast produces probabilistic forecasts, and FGN predicts cyclone properties directly within the model. Hadsell closes with Genie, DeepMind's work on interactive world models. Genie developed from short 2D platformer environments into 3D worlds that can respond to user actions, remember earlier states, and change when prompted during use. Her argument is that frontier research should target deep unsolved problems that can enable many later applications, while choosing problems worth solving for humanity.
Frontier research should target deep problems that enable later applications
Hadsell says DeepMind is asking which architectures may come next for Gemini, which problems only AI can address, and how intelligence may develop across artificial systems, people, and robots. Her research strategy looks for the root of a problem space rather than working on its downstream leaves. The team seeks deep unsolved problems, then works with partners to carry solutions into practical applications. She connects this to DeepMind's mission to build AI responsibly for the benefit of humanity, saying the group should work on problems that are worth solving.
Embedding models connect the same concept across different modalities
Hadsell uses the neuroscience idea of a 'Jennifer Aniston cell' to explain embeddings. A small group of neurons can respond to the same person or concept when it appears as a name, image, video, or voice. AI embedding models aim for a similar effect, supporting fast retrieval, recognition, and comparison. She describes embeddings as a companion to generative AI because some systems need to generate while others need to retrieve. The models use contrastive losses to place related concepts together and compare different forms of the same information.
Gemini Embeddings 2 puts text, video, audio, and PDFs into one semantic space
Hadsell presents Gemini Embeddings 2 as a fully omnimodal embedding model derived from Gemini. It can represent up to 8K tokens of text, 128 seconds of video, 80 seconds of audio, and a full PDF in a single vector. A unified representation avoids separate processing steps that could lose information when combining audio, visual, and text inputs. She says the embeddings can support retrieval, querying, and agentic logic. Matryoshka representation learning lets the same network produce embeddings at different dimensions, so a system can begin retrieval with 256 dimensions and expand the representation when it needs more expressiveness.
GraphCast predicts the atmosphere with a spherical graph neural network
DeepMind began its weather work after a UK Met Office scientist asked whether AI could predict rainfall better than physics-based models. Hadsell says 40 years of global weather data made the problem tractable for neural networks. GraphCast predicts atmospheric conditions up to 15 days ahead across the Earth. Its spherical graph neural network has nodes from the surface into the lower stratosphere and predicts around 100 atmospheric variables, including wind speed, temperature, and humidity. In her Hurricane Lee example, GraphCast predicted landfall location accurately nine days ahead, compared with six days for the physics-based gold-standard models.
GenCast adds probability and makes long-range forecasting practical
GenCast is a probabilistic weather model built on a mesh. Hadsell explains that weather is chaotic, so useful forecasting requires information about possible tail outcomes rather than one single prediction. In comparisons against 1,300 gold-standard benchmark forecasts, GenCast was more accurate 97% of the time. It could also produce a 15-day forecast in eight minutes on a single chip, whereas the comparison required hours on a very large supercomputer. The probabilistic output is intended to support operational use for actual weather prediction.
FGN predicts cyclone behavior directly inside the network
Hadsell describes FGN, or functional generative network, as the latest model in the weather sequence. Instead of first predicting general weather and then applying a cyclone detector, FGN predicts cyclone behavior directly. Its training includes cyclone categorization, trajectory, wind speed, and eye formation. Hadsell says the model has already been used by the US National Hurricane Center, whose staff were excited by the advantage it provided. She expects the approach to be used more widely in the coming years.
Genie develops from simple platformers into interactive 3D worlds
DeepMind's world-model work grew from games and robotics simulations, where the team studied agents and their environments. Genie 1 generated short 2D platformer environments from prompts and responded to left and right controls. Genie 2 moved to interactive 3D environments, although it was not yet real time and lacked realistic high-definition quality. Hadsell then shows a later system generating a 3D world from a prompt about walking down a muddy lane in Kent. The user can move through the scene, ski, and interact with objects. Another example brings a short artist-created video fragment to life as a navigable world.
Genie worlds can remember their state and change while the user is inside
Hadsell says the generated environments have memory, consistency, and control. In an origami world, the user can run away and return to find the scene as it was at the beginning. The model can also accept prompts during interaction. A scene of walking along London's Camden Canal changes immediately when the user asks for a different world. Hadsell imagines this supporting new kinds of games in which players can alter each other's experiences, as well as educational environments where people learn by entering and exploring a world.
"These environments are not only diverse and interactive high quality. They also have memory."18:24
Who should watch
You are building retrieval or multimodal agents and want to understand why a shared embedding space can simplify queries across text, images, video, and audio.
Your team works on forecasting, simulation, or scientific machine learning and wants concrete examples of neural models replacing parts of a physics-based workflow.
You are interested in interactive environments for games, robotics, or education and want to see what DeepMind's Genie line is designed to do.
