Learning-Based Methods (AI Navigation)

8.1Where Learning Plugs In

"AI navigation" is not one idea; it is five distinct surgeries on the classical pipeline, each replacing a different organ. Sorting any paper or product claim into the right slot is half the literacy this phase teaches:

8.2Learned Front-Ends: SuperPoint, LightGlue, DROID-SLAM

Phase 2’s visual front-end had two handcrafted organs — detect features, match features — and both fell to networks first. SuperPoint detects interest points and descriptors in one forward pass, trained by self-supervision: synthetic corners bootstrap a detector, then "homographic adaptation" (aggregating detections over many random warps of real images) labels its own training data — no human annotation, and dramatically better repeatability than ORB under lighting and viewpoint change. SuperGlue replaced nearest-neighbor descriptor matching with a graph neural network doing cross-image attention — matching becomes a joint assignment problem informed by geometry-ish context — and LightGlue made it fast enough for real-time SLAM by letting easy pairs exit early. The pragmatic 2026 recipe for a robust visual front-end is literally: ORB-SLAM3’s architecture, SuperPoint + LightGlue features, classical RANSAC verification unchanged.

DROID-SLAM goes deeper: instead of replacing pieces, it makes the whole visual-SLAM loop differentiable. A recurrent GRU repeatedly predicts dense optical-flow revisions between keyframes, and each iteration ends in a dense bundle adjustment layer — a real Gauss-Newton solve over poses and per-pixel depth, implemented as a differentiable operator, so the network learns flow that makes geometry converge:

DROID-SLAM update, per iteration:
  flow revision + confidence  ←  GRU(correlation features, current estimate)
  poses, depths  ←  dense BA:  argmin Σ ‖ predicted_flow − flow(T, d) ‖²_Σ
                    (a real damped Gauss-Newton step — Phase 1.8 as a layer)

the network never outputs poses; it outputs measurements FOR the optimizer.
that division of labor is why it generalizes across cameras and datasets.

DeepMapping 2.0 applies the same philosophy to lidar: registration of large point-cloud collections as self-supervised optimization, with networks providing the data association that classical ICP guesses by proximity. The pattern across all of these — networks propose, optimizers dispose — is Phase 4’s funnel again, and it is the single most reliable design rule in learned robotics.

8.3Neural Maps: NeRF and 3D Gaussian Splatting SLAM

Phase 3 stored the world as grids and distance fields. The neural alternative stores it as a function you can render: give me a camera pose, I give you the photograph. NeRF represents the scene as an MLP mapping position and view direction to density and color, rendered by integrating along rays; 3D Gaussian Splatting represents it as millions of explicit anisotropic Gaussians, rendered by rasterizing them — orders of magnitude faster, which is what made real-time SLAM on these representations plausible:

NeRF render (per pixel, sample points along the ray):
  C = Σᵢ Tᵢ·(1 − exp(−σᵢδᵢ))·cᵢ,     Tᵢ = exp(−Σ_{j<i} σⱼδⱼ)
  → tens–hundreds of MLP queries per pixel: photoreal, slow

3DGS render: project each Gaussian, sort, alpha-composite front-to-back
  → pure rasterization: real-time, and the map is explicit and editable

neural SLAM = tracking (optimize T against the render: "photometric ICP")
            + mapping (optimize the representation from posed views)

The SLAM systems: Nice-SLAM and Vox-Fusion anchor neural implicit features in voxel grids for RGB-D dense mapping (the hierarchical-grid trick that made implicit SLAM tractable at room scale); SplaTAM and GS-SLAM do tracking-and-mapping directly on Gaussian splats, exploiting rasterization speed and per-Gaussian silhouette masks to know what the map does and doesn’t yet explain. What these buy over TSDF maps: photorealism (sim-and-rehearse anywhere, appearance-based relocalization for free) and continuous, differentiable geometry. What they still owe: mature loop-closure handling (deforming a splat field after Phase 3’s jump is an open craft), large-scale memory management, and the honest admission that for a planner’s "how far is the wall", Phase 3’s ESDF remains the cleaner answer. Watch this space per quarter, not per decade.

8.4Deep RL for Local Navigation

Phase 5’s local planners optimize a hand-written cost at runtime. Deep reinforcement learning moves the optimization offline: train a policy π(action | observation) across millions of simulated episodes, then run it as a cheap forward pass. The formulation is the standard MDP, and two algorithms carry the field — PPO, whose clipped objective keeps policy updates conservatively close to the data-collecting policy (stable, forgiving, the default), and SAC, whose entropy bonus keeps exploration alive (sample-efficient off-policy learning):

objective:  max_θ  E[ Σₜ γᵗ r(sₜ, aₜ) ]        reward: progress − collision
                                                − jerk − social margins…
PPO:  maximize  E[ min( ρₜAₜ,  clip(ρₜ, 1±ε)·Aₜ ) ],   ρₜ = π_θ/π_old
      → never trust one batch enough to leap far. boring. works.

the real enabler is the simulator: Isaac Gym/Lab runs 10,000+ environments
on one GPU — months of robot-time per wall-clock hour — and sim-to-real is
bridged by domain randomization: train across randomized physics, sensor
noise, and layouts until reality is just another sample.

Where DRL genuinely earns its complexity in navigation: dense dynamic crowds, where costmap-based planners freeze (Phase 5’s warfield) but a policy trained among simulated pedestrians learns anticipatory, socially-plausible motion; mapless goal-seeking from raw sensors when maps are unavailable; and legged locomotion (adjacent, and the field’s clearest DRL success). Where it doesn’t: anywhere a Phase 5 stack already meets spec — the classical planner is debuggable, verifiable, and parameter-changeable in the field, and a policy is none of those. Production deployments therefore wrap policies in classical clothing: a safety monitor (velocity limits, lidar-based emergency stop) that the network cannot override, and a fallback planner when the policy’s inputs go out of distribution.

8.5Neural Planners: MPNet, Neural A*, VIN

Between "hand-written planner" and "end-to-end policy" sits the most transferable idea of this phase: learn the proposal, keep the guarantee. MPNet trains a network to propose the next configuration toward the goal, plugged into a classical framework that collision-checks every proposed segment and falls back to sampling when the network stalls — so it inherits RRT-grade completeness while cutting planning time by large factors in learned domains. Neural A* learns the guidance map (effectively a heuristic field) through a differentiable A* relaxation, so search effort concentrates where paths actually go — the trained heuristic pays for itself in expansion counts, exactly the currency Setup A of Phase 5 measured. Value Iteration Networks made the founding observation: value iteration on a grid is a convolution (each cell’s value updates from its neighbors) iterated k times, so a planner can live inside a network as a differentiable module and be trained through:

value iteration (what VIN unrolls as conv layers):
  V_{k+1}(s) = min_a [ cost(s, a) + V_k(next(s, a)) ]
  → k sweeps of a 3×3 min-convolution ≈ shortest-path field
  → Setup A implements the real thing; VIN learns cost(s,a) end-to-end

// vi.cc — value iteration on a grid: the planner inside VIN, run honestly.
#include <algorithm>
#include <iostream>
#include <vector>

int main() {
  std::vector<std::string> map = {
    "........................",
    "..####......##########..",
    "..#..........#.......#..",
    "..#..######..#..####..#..",
    "..#..#....#..#..#..#..#..",
    "S.#..#.G..#.....#..#....",
    "..#..#....#######..#..#..",
    "..#..######........#..#..",
    "..#.........########..#..",
    "..####................#..",
    "........................",
  };
  const int H = map.size(), W = map[0].size();
  int gx, gy, sx, sy;
  for (int y = 0; y < H; ++y) for (int x = 0; x < W; ++x) {
    if (map[y][x] == 'G') { gx = x; gy = y; }
    if (map[y][x] == 'S') { sx = x; sy = y; }
  }
  const double INF = 1e9;
  std::vector<double> V(W * H, INF);
  V[gy * W + gx] = 0.0;
  const int DX[4] = {1, -1, 0, 0}, DY[4] = {0, 0, 1, -1};

  int sweeps = 0;
  for (bool changed = true; changed; ++sweeps) {          // V ← min(1 + V')
    changed = false;
    for (int y = 0; y < H; ++y) for (int x = 0; x < W; ++x) {
      if (map[y][x] == '#') continue;
      for (int a = 0; a < 4; ++a) {
        const int nx = x + DX[a], ny = y + DY[a];
        if (nx < 0 || ny < 0 || nx >= W || ny >= H || map[ny][nx] == '#') continue;
        if (1.0 + V[ny * W + nx] < V[y * W + x] - 1e-12) {
          V[y * W + x] = 1.0 + V[ny * W + nx];
          changed = true;
        }
      }
    }
  }
  std::cout << "converged after " << sweeps << " sweeps (VIN: k conv layers)\n\n";

  auto shade = [&](double v) {                            // render the field
    const char* ramp = "@%#*+=-:. ";
    return (v >= INF) ? '#' : ramp[std::min(9, int(v / 5))];
  };
  auto shown = map;                                        // greedy rollout
  for (int x = sx, y = sy; !(x == gx && y == gy);) {
    int ba = -1; double bv = INF;
    for (int a = 0; a < 4; ++a) {
      const int nx = x + DX[a], ny = y + DY[a];
      if (nx < 0 || ny < 0 || nx >= W || ny >= H || map[ny][nx] == '#') continue;
      if (V[ny * W + nx] < bv) { bv = V[ny * W + nx]; ba = a; }
    }
    x += DX[ba]; y += DY[ba];
    if (shown[y][x] == '.') shown[y][x] = 'o';
  }
  for (int y = 0; y < H; ++y) {
    for (int x = 0; x < W; ++x) std::cout << shade(V[y * W + x]);
    std::cout << "   " << shown[y] << "\n";
  }
}

8.6Vision-Language-Action: Navigation You Can Talk To

The newest surgery replaces the goal specification. Classical stacks navigate to coordinates; VLA-era systems navigate to meaning: "find the office coffee machine" with no coordinate, no prior detection class list, often no prior map. The enabling artifact is the pretrained vision-language embedding (CLIP and descendants): images and text in one space, so "does this camera view contain a coffee machine" becomes a dot product — zero-shot, no robot-specific training. CLIP on Wheels established the recipe: classical exploration (frontier-based, Phase 3’s unknown-space logic) + CLIP saliency deciding which frontier smells like the target. VLFM matured it: build the occupancy map as usual, score each frontier with a vision-language model’s similarity between the view toward it and the query, and go — zero-shot semantic navigation that wins ObjectNav benchmarks while remaining, structurally, a Phase 3+5 stack with a learned scoring function:

Beyond navigation-specific systems sit the generalist robot policies: OpenVLA and Octo, transformers pretrained on hundreds of thousands of robot episodes, mapping images + language directly to actions, fine-tunable to new embodiments. Today their center of gravity is manipulation; their trajectory matters to navigation because the interface — "language in, behavior out, one model across robots" — is a plausible endgame for the whole goal-specification layer. The literate position: track them, benchmark them, and hold them to the same standard as everything else in this chapter — what happens when they’re wrong, and who catches it?

8.7What Survives Contact With Reality

The deployment scorecard, module by module, as of this book’s writing:

// pg.cc — REINFORCE with a baseline on a walled gridworld. Pure C++.
#include <algorithm>
#include <cmath>
#include <iostream>
#include <random>
#include <vector>

const int N = 8;                                 // 8×8 world
bool blocked(int x, int y) {                     // a wall with one gap
  return x == 4 && y != 6 && y >= 2;
}

int main() {
  std::mt19937 rng(2);
  std::uniform_real_distribution<double> U(0.0, 1.0);
  // tabular policy: logits[state][action], actions = E,W,N,S
  std::vector<std::array<double, 4>> th(N * N, {0, 0, 0, 0});
  const int DX[4] = {1, -1, 0, 0}, DY[4] = {0, 0, 1, -1};
  const double alpha = 0.05, gamma = 0.98;
  double baseline = 0.0;

  for (int ep = 1; ep <= 4000; ++ep) {
    int x = 0, y = 0;
    std::vector<int> S, A;  std::vector<double> R;
    for (int t = 0; t < 60; ++t) {               // rollout
      const int s = y * N + x;
      double e[4], z = 0.0;                      // softmax over logits
      for (int a = 0; a < 4; ++a) z += (e[a] = std::exp(th[s][a]));
      double u = U(rng) * z;  int a = 0;
      while (a < 3 && u > e[a]) u -= e[a], ++a;
      int nx = x + DX[a], ny = y + DY[a];
      double r = -0.05;                          // small step penalty
      if (nx < 0 || ny < 0 || nx >= N || ny >= N || blocked(nx, ny)) {
        nx = x; ny = y; r = -0.25;               // bump
      }
      S.push_back(s); A.push_back(a);
      x = nx; y = ny;
      if (x == N - 1 && y == N - 1) { R.push_back(10.0); break; }
      R.push_back(r);
    }
    double G = 0.0;  std::vector<double> Gs(R.size());
    for (int t = int(R.size()) - 1; t >= 0; --t) Gs[t] = G = R[t] + gamma * G;
    baseline += 0.01 * (Gs[0] - baseline);       // running-mean baseline
    for (size_t t = 0; t < S.size(); ++t) {      // ∇ log π · (G − b)
      double e[4], z = 0.0;
      for (int a = 0; a < 4; ++a) z += (e[a] = std::exp(th[S[t]][a]));
      for (int a = 0; a < 4; ++a) {
        const double grad = (a == A[t] ? 1.0 : 0.0) - e[a] / z;
        th[S[t]][a] += alpha * grad * (Gs[t] - baseline);
      }
    }
    if (ep % 500 == 0) {                         // evaluate greedily
      int gx = 0, gy = 0, steps = 0;
      while (!(gx == N - 1 && gy == N - 1) && steps < 64) {
        const int s = gy * N + gx;
        const int a = std::max_element(th[s].begin(), th[s].end()) - th[s].begin();
        const int nx = gx + DX[a], ny = gy + DY[a];
        if (nx < 0 || ny < 0 || nx >= N || ny >= N || blocked(nx, ny)) break;
        gx = nx; gy = ny; ++steps;
      }
      std::cout << "ep " << ep << "  baseline " << baseline
                << "  greedy: " << (gx == N - 1 && gy == N - 1 ?
                   "reaches goal in " + std::to_string(steps) + " steps"
                   : "stuck") << "\n";
    }
  }
}

8.8Reading List

8.9Interview Questions