Applications by domain

Why this chapter exists

Part II described the families as ideas. This chapter describes where they are used. Each domain imposes its own constraints: a humanoid must not fall, a car must handle the rare event, a warehouse cell must pick for eight hours without a supervisor, a service robot must understand a person. Those constraints decide which family dominates, and each domain has a failure mode of its own that a general benchmark does not show.

One section per domain follows, each with the family that dominates, a named system, the failure mode to expect and, where it adds something, a figure. Deployment counts and adopter lists in this chapter come from vendors and industry surveys, and are named as claims.

After this chapter you will be able to:

• say which families dominate in each of six domains, and why the domain's constraints favour them;
• name a representative system per domain and the first failure mode to plan for;
• explain with figure 16.3 why look-ahead planning is only as good as the world model doing the looking.

Tabletop and bimanual manipulation

This is the home turf of reactive VLAs (chapter 5) and the large behavior models they grew from (chapter 11). The π0 line, GR00T, OpenVLA and their open relatives are developed and benchmarked here, and Diffusion Policy and ACT remain the workhorses of bimanual teleoperation research (Physical Intelligence et al. 2026, arXiv 2604.15483; NVIDIA 2026; Chi et al. 2023, arXiv 2303.04137; Zhao et al. 2023, arXiv 2304.13705). The frontier claims are about learning quickly: Generalist AI reports GEN-1.5 learning a task from a single 3 to 12 second demonstration, at 59 percent average success across 10 tasksvendor claim (Generalist AI 2026).

The failure mode to expect is distribution shift that looks small to a person. LIBERO-Plus showed success falling from 95 percent to below 30 percent under modest changes of camera viewpoint and initial robot state (Fei et al. 2025, arXiv 2510.13626). Figures 5.1 and 11.3 cover the mechanics of this domain; this chapter does not repeat them.

Humanoids and whole-body control

Humanoids stack three layers: something that reasons about the task, something that turns it into motion, and something that keeps a two-legged body balanced while it moves. Figure 16.2 shows three ways the layers are arranged today.

GR00T N1.7 drives a Unitree G1 through SONIC: the VLA emits compact latent action tokens and a learned whole-body controller turns them into joint commands for legs, arms and hands (NVIDIA 2026; Luo et al. 2025, arXiv 2511.07820). Helix splits a 7-billion-parameter reasoner at 7 to 9 Hz from an 80-million-parameter controller at 200 Hz (Figure AI 2025). Gemini Robotics ER 2 orchestrates a humanoid's own controllers as tools (Google DeepMind 2026). NVIDIA previewed GR00T N2 at GTC in March 2026 and named AGIBOT, Humanoid, LG Electronics, NEURA Robotics and Noble Machines as adopting GR00T models for industrial humanoidsvendor claim (NVIDIA Newsroom 2026). For research, NVIDIA's Isaac GR00T reference humanoid combines a Unitree H2 Plus with Sharpa five-fingered hands and Jetson Thor compute, with Ai2, ETH Zurich, Stanford and UC San Diego among the first users (NVIDIA Newsroom 2026).

The failure mode to expect is physical, and it collides with how industrial safety works. Every industrial robot has a safety stop that cuts power to the motors, but cutting power to a balancing robot can make it fall. Rodney Brooks recalls that at Rethink Robotics his team kept the arms of Baxter and Sawyer from dropping on power cut-off with a circuit that braked the motors without active power, and adds: "Perhaps there are similar possible solutions for humanoid robots and falling, but they need to be invented yet" (Rodney Brooks 2026).

Mobile manipulation and navigation

Navigation was one of the first places world models earned their keep, because a robot that can imagine where a path leads can compare paths before taking one. Navigation World Models, a 1-billion-parameter video diffusion model trained on egocentric video from humans and robots, plans by simulating candidate trajectories and evaluating whether they reach the goal, either from scratch or by ranking trajectories proposed by another policy (Bar et al. 2024, arXiv 2412.03572). X-Mobility learns a latent world model separately from its action policy, so it can train on data with or without expert actions (Liu et al. 2024, arXiv 2410.17491). InternVLA-N1, published as DualVLN, puts a VLM that predicts mid-term waypoints over a fast diffusion policy, the fast-slow split of chapter 5 applied to navigation (Wei et al. 2025, arXiv 2512.08186). For mobile manipulation, DreamTrajectory predicts an end-effector trajectory together with the whole-body action chunk, and checks the motion it intended against the motion it achieved (Yang et al. 2026, arXiv 2608.01381).

The failure mode to expect is the one in figure 16.3. With an accurate model the robot threads past the first pillar to the goal. If the model believes the robot turns 25 percent more sharply than it does, the chosen path is really straighter than imagined and clips the first pillar. If it believes the robot turns 40 percent less sharply, the path turns too far and hits the second. This is chapter 8's model bias in a form a reader can drive.

Autonomous driving

Driving is where world models are most established, and they are used mainly to generate data and scenarios rather than to drive. A car must handle events too rare to collect, and a world model that can produce them on demand is worth more than one more policy. Figure 16.4 shows three roles.

GAIA-1 cast driving world modelling as next-token prediction over video, text and action (Hu et al. 2023, arXiv 2309.17080). GAIA-2, a latent diffusion model, generates consistent multi-camera video conditioned on ego dynamics, other agents, environment and road semantics, across the UK, the US and Germany, so that "both common and rare driving scenarios" can be simulated (Russell et al. 2025, arXiv 2503.20523). OccWorld forecasts 3D occupancy and the ego path with a GPT-like transformer, a geometry-first model in chapter 9's sense (Zheng et al. 2023, arXiv 2311.16038). DriveDreamer builds its world model entirely from real-world driving (Wang et al. 2023, arXiv 2309.09777). Public evaluation runs on datasets and benchmarks such as nuScenes and NAVSIM (Caesar et al. 2019, arXiv 1903.11027; Dauner et al. 2024, arXiv 2406.15349).

The failure mode to expect is the one rare scenarios are meant to fix: a generated scenario looks plausible but does not behave like the real world, so a system validated on it is less safe than it appears. Chapter 8's gap between plausible and feasible applies with higher stakes.

Industrial and logistics

Warehouse picking was among the first commercial uses of robot foundation models. Covariant's RFM-1, an 8-billion-parameter any-to-any model trained on data from a fleet of warehouse robots, was announced in March 2024. Its makers described a data collection system that had gathered "tens of millions of trajectories" from robots deployed to dozens of customersvendor claim (Covariant 2024). By the first quarter of 2026, one industry survey counts at least eleven commercial deployments with VLAs as the primary policyindustry survey (SVRC (Robotics Center of Silicon Valley) 2026). Inspection is a second industrial use: Gemini Robotics-ER 1.6's instrument reading was developed with Boston Dynamics for facility inspection with Spotvendor claim (Google DeepMind 2026).

The failure mode to expect is integration, not intelligence. Chapter 15's readiness survey found even the highest-rated models fulfilling none of its criteria on precision, real-time performance or cost-effective integration (Kube et al. 2026, arXiv 2603.06749). A cell fails in production because it cannot meet the conveyor's cycle time or talk to the warehouse system, more often than because the model cannot recognise a product.

Mobile service robots

Service robots in homes, hospitals and shops are where language matters most, and where embodied reasoning models (chapter 10) lead. A systematic review published in Robotics in 2026 names the domain's core challenges: translating natural-language instructions into executable actions, perceiving reliably in human-centred environments, estimating uncertainty for safe decisions, and fitting computation on board (Lisondra et al. 2025, arXiv 2505.20503). Benchmarks for instruction-following navigation, such as R2R and VLN-CE, come from this line of work (Anderson et al. 2017, arXiv 1711.07280; Krantz et al. 2020, arXiv 2004.02857).

The failure mode to expect is misunderstanding. A service robot that executes the wrong interpretation of a request confidently is worse than one that asks. Chapter 10's success detection and chapter 14's memory are the tools; uncertainty estimation, which the review lists as a core challenge, is the missing one.

In code

The crate's navigation world is two pillars, a goal and arcs of constant curvature.

pub type P = [f64; 2];

/// A robot's pose: position and heading in radians.
#[derive(Clone, Copy, Debug, PartialEq)]
pub struct Pose {
    pub x: f64,
    pub y: f64,
    pub heading: f64,
}

/// A round obstacle on the floor.
#[derive(Clone, Copy, Debug)]
pub struct Obstacle {
    pub c: P,
    pub r: f64,
}

/// A room with two pillars: one beside the straight line to the goal, one beyond the goal.
pub fn scene() -> (Pose, P, Vec<Obstacle>) {
    (
        Pose { x: 0.0, y: 0.0, heading: 0.0 },
        [4.6, 3.1],
        vec![Obstacle { c: [2.6, -0.2], r: 0.5 }, Obstacle { c: [3.6, 4.6], r: 0.5 }],
    )
}

/// Candidate paths are arcs of constant curvature, as a sampling planner or a policy would propose.
pub const CURVATURES: [f64; 13] = [-0.2, -0.15, -0.1, -0.05, 0.0, 0.05, 0.1, 0.15, 0.2, 0.25, 0.3, 0.35, 0.4];
pub const STEPS: usize = 30;
pub const STEP: f64 = 0.2;

pub fn dist(a: P, b: P) -> f64 {
    ((a[0] - b[0]).powi(2) + (a[1] - b[1]).powi(2)).sqrt()
}

Exercises

The stubs are in rust/ch16-applications/src/exercises.rs. This chapter is equation-free; the exercises build the look-ahead planner of figure 16.3 one piece at a time.

cargo test -p ch16-applications --test exercises ex16_1

cargo test -p ch16-applications --test exercises ex16_2

cargo test -p ch16-applications --test exercises ex16_3

Further reading

• The NVIDIA reference humanoid announcement, for the hardware research teams now share (NVIDIA Newsroom 2026).
• Navigation World Models, for look-ahead planning with a video model (Bar et al. 2024, arXiv 2412.03572).
• GAIA-2, for controllable scenario generation in driving (Russell et al. 2025, arXiv 2503.20523).
• OccWorld, for occupancy forecasting (Zheng et al. 2023, arXiv 2311.16038).
• The systematic review of foundation models for mobile service robots (Lisondra et al. 2025, arXiv 2505.20503).
• The WAM survey's section on applications (Lu et al. 2026, arXiv 2609.16074).