Skip to main content
Contents5 / 11 ▾
  1. ~Cover & contents
  2. ISLAM & State Estimation
  3. IINavigation Architecture
  4. IIIFirmware & RTOS
  5. IVMiddleware & Multi-Agent Systems
  6. VComputer Vision
  7. VILearning-Based Robotics
  8. VIIVideo Streaming & Edge Deployment
  9. VIIIManipulation & Kinematics
  10. IXSystems & Leadership
  11. AStory Bank
  12. BGlossary
Chapter V

Computer Vision

Computer vision pipelines for aerial robotics combine multispectral payload processing, semantic segmentation, and 6-DOF geometric pose estimation.

5.1

How would you build a segmentation pipeline for crop health monitoring?

rating — possible

Building a precision crop-health segmentation pipeline requires multispectral sensor integration, custom loss functions, and spatial reprojection onto geo-referenced field maps.

1. Multispectral Vegetation Index Features

Combine 4-channel imagery (RGB + Near-Infrared NIR) to compute Normalized Difference Vegetation Index (NDVI) and Normalized Difference Red Edge (NDRE) channels as model inputs:

$$\text{NDVI} = \frac{\text{NIR} - \text{Red}}{\text{NIR} + \text{Red}}, \quad \text{NDRE} = \frac{\text{NIR} - \text{RedEdge}}{\text{NIR} + \text{RedEdge}}$$

2. Model Architecture & Loss Function

Deploy a **DeepLabV3+** model with a ConvNeXt-Tiny encoder backbone. To handle severe class imbalance (small disease lesions vs. massive healthy canopy background), train using a combined **Focal Loss + Dice Loss**:

$$\mathcal{L}_{\text{total}} = \mathcal{L}_{\text{Focal}}(p, y) + \left(1 - \frac{2 \sum p_i y_i}{\sum p_i^2 + \sum y_i^2}\right)$$
5.2

Detection vs. segmentation vs. pose estimation — when would you use each on a drone?

rating — possible
Table 5.1 — Vision Output Task Selection
Perception OutputOutput DimensionCompute OverheadRobotics Use Case
2D Bounding Box Detection$\mathcal{O}(N \times 4)$Minimal ($<5\text{ms}$)Search and rescue human detection, obstacle presence check
Semantic / Instance Segmentation$\mathcal{O}(H \times W)$Moderate ($15-30\text{ms}$)Crop health mapping, navigable surface corridor detection
6-DOF Object Pose Estimation$\mathbf{R} \in SO(3), \mathbf{t} \in \mathbb{R}^3$High ($20-50\text{ms}$)Aerial grasping, precision pad landing, manipulator docking
5.3

Marker-based vs. markerless visual state estimation — what are the trade-offs?

rating — possible

Marker-based (AprilTag / ArUco) pose estimation extracts known 3D corner coordinates to solve the **Perspective-n-Point (PnP)** problem via Direct Linear Transform (DLT) + Levenberg-Marquardt optimization. It provides instant metric scale recovery with zero drift at $<5\text{ms}$ compute. Markerless VIO (ORB-SLAM3) tracks visual features across frame sequences, estimating scale by integrating IMU accelerations ($\Delta p = \iint R a dt^2$). Markerless generalizes to uninstrumented environments but suffers from scale drift and tracking loss under fast rotation.