How Google DeepMind is researching the next Frontier of AI for Gemini
summary
Gemini Embeddings 2 claims a unified omnimodal vector space (text up to 8k tokens, 128s video, 80s audio, full PDF) for fast retrieval, using Matryoshka representation learning. Weather models move from physics sims to learned meshes: GraphCast gives 15‑day global forecasts, GenCast is probabilistic and beats benchmarks 97% of the time while producing 15‑day forecasts in ~8 minutes on one chip, and FGN directly predicts cyclones and is already used by the US National Hurricane Center. World models (Genie 1→3) build interactive photoreal 3D environments with memory and real‑time prompting.
AI-written summary · about
chapters (7)
- 0:00Introduction
- 5:05Advanced Embedding Models: Raia discusses the importance of embedding models for fast retrieval and recognition, similar to how the human brain uses 'Jennifer Aniston cells' to identify concepts across modalities. She highlights Gemini Embeddings 2, a fully omnimodal model that processes text, video, and audio into unified semantic vectors.
- 9:53AI for Weather Forecasting: The team has developed revolutionary models for atmospheric prediction, moving away from traditional physics simulations. Notable breakthroughs include:
- 11:00GraphCast: A spherical graph neural network that provides accurate 15-day weather forecasts.
- 12:47GenCast: A probabilistic model that offers higher efficiency and accuracy (97% of the time compared to gold-standard benchmarks).
- 13:51FGN: A functional generative network that directly predicts cyclone behavior, which is currently being utilized by the US National Hurricane Center.
- 14:35World Models: Hadsell introduces Genie, a project focused on creating interactive, real-time environments. Starting from Genie 1 (2D platformers) and progressing to Genie 3, these models allow users to create and interact with high-quality, 3D photorealistic worlds. These environments demonstrate capabilities like memory, consistency, and the ability to be dynamically prompted by the user to change the surroundings in real-time.
related talks
11mapproach to generative AI
51mGemini, Google's Large Language Model
54mGemini, Google's Large Language Model
19mGemini 2.0: Beyond the Basics
51mGemini, Google's Large Language Model
08mAI-Powered transformation: Driving innovation and reshaping organizations
01h 45mAI Engineering with the Google Gemini 2.5 Model Family
18mIntroduction to Gemini APIs and AI Studio