Using Deep Learning for Perception in Autonomous Systems: A Perspective from Control Theory
summary
A convolutional model trained to predict short waypoints (ResNet-50 backbone) supervises planning and control (third-order spline planner + LQR) instead of end-to-end outputs, and achieves about 25% higher success on held-out Stanford 3D indoor scenes, with smoother, lower-jerk trajectories. Training uses optimal-control-generated waypoint labels from known maps; sim-to-real transfer on a TurtleBot shows ~18% failure driven by unseen chair geometries. A runtime reachable-set safety filter (Hamilton-Jacobi computations, warm-started) is added to prevent collisions.
AI-written summary · about
chapters (9)
- 0:00Using deep learning for perception in robotics: a perspective from control theory
- 2:13Safe Learning for Dynamics and Control
- 8:30Outline
- 10:12How to efficiently navigate an autonomous system with a monocular RGB camera to a goal in an a priori unknown environment?
- 26:24Success Rate
- 27:07Control Profile
- 31:16Comparison with Geometric Mapping-based Approaches
- 39:12Some lessons learned
- 47:40Summary
related talks
31mDeep learning for self-driving vehicle technology
40mTeaching a Car to Drive Itself by Imitation and Imagination
04mAI and the Robotics Revolution
01h 35mDeep Robotic Learning
08mUnified Perception: Models for Autonomous Driving
45m(Deep) Learning to Fly
48m(Deep) Learning to Fly
20mPerception Contracts for Safety of ML-Enabled Systems