This is my Master’s thesis, completed in 2023 for the MSE in Robotics at the University of Pennsylvania, advised by Prof. Vijay Kumar with M. Ani Hsieh on the committee. The full document is available here: Imitation Learning for Autonomous Quadrotor Flight (PDF).
Motivation
Learning-based methods have become increasingly popular for autonomous quadrotor flight. Most of that work leans on Reinforcement Learning (RL) — but RL depends on carefully hand-designed reward functions, is prone to reward hacking and reward sparsity, and needs an enormous number of training steps to converge. This is especially painful for quadrotors, where exploration happens in a high-dimensional state and action space, and where policies trained purely in simulation often fail to transfer to the real world.
Imitation Learning (IL) offers an appealing alternative. Because it has access to an expert that provides demonstrations, IL has far better sample efficiency and more stable inference-time behavior. The usual objection to IL — “you need an expert” — barely applies to quadrotors: decades of research have produced many mature optimization-based motion planners that can serve directly as experts. IL can also be used for knowledge distillation, learning a cheap policy that mimics an expensive planner so it can run onboard.
The gap I set out to close: most existing IL pipelines for quadrotors are built on OpenAI Gym or gym-like interfaces, use oversimplified physics, and cannot leverage the large ecosystem of ROS/C++ planners — so they never make it out of the simulator. This thesis proposes a high-fidelity, ROS-based IL pipeline and a complementary simulation environment designed for training policies that have a real path to physical deployment.
System Overview
The proposed pipeline is composed of an expert planner, data collectors, training pipelines, neural networks, an inference pipeline, a controller, the AirSim simulator, and a ROS bridge. It is built on ROS and leverages the collision simulation and advanced dynamics of Unreal Engine 4 and AirSim, and it is fully compatible with the Kumar Robotics software stack.
Overview of the proposed imitation learning pipeline.
Simulation Environment
I built the simulation environment in Unreal Editor 4.25.4 with the AirSim plugin. The scene is deliberately cluttered with blocks of varying position, scale, and orientation so the learned policy is trained and evaluated against real obstacle avoidance. During initialization, a voxel map of the environment is reconstructed from the Unreal game objects for use by the planners.
| Unreal Engine 4 environment | Reconstructed voxel map |
|---|---|
 |
 |
To reduce the sim2real gap, the drone is modeled after Falcon250V, the quadrotor platform from Kumar Lab, matching its real 0.25 m × 0.25 m × 0.25 m dimensions both visually and physically.
Visualization of the quadrotor model used in simulation.
Expert, Controller, and Network
- Expert. I use Fast Planner as the expert — a planner with a kinodynamic A* front end and a B-spline optimization back end that takes a depth image plus odometry and plans locally.
- Controller.
kr_mav_control, a PID controller with trackers from Kumar Robotics, executes the planned trajectories via a B-spline tracker. Gains were tuned with the Ziegler–Nichols method. - Network. The architecture has two backbones — a fine-tuned MobileNet V3 for the depth image and a 3-layer MLP for state (position, velocity, orientation, goal) — feeding a planning MLP that outputs a flattened B-spline trajectory.
Getting the controller to responsively track the expert’s commands was a prerequisite for collecting clean demonstrations:
PID responses when flying to goal position (3.0, 3.0, 3.0), visualized with PlotJuggler.
Data Collection: Python vs. C++
A recurring theme of the thesis is a versatility-vs-efficiency trade-off in data collection, so I built two collectors:
- A Python collector that intercepts messages from the expert planner. By overriding a callback it supports interactive IL algorithms (DAgger, HG-DAgger, EIL, MEGA-DAgger), and saves directly to
npzfor fast loading — but it suffers latency from inter-node ROS communication. - A C++ collector that grabs demonstrations the moment the expert finishes computing the B-spline. It only supports Behavioral Cloning (no way to switch between expert and learner), but its tight observation-action synchronization matters a lot at scale.
Experiments
Loss function and scheduler
Comparing MSE vs. SSE loss on an identical 5.95 GB dataset, MSE was more stable and converged faster, so I used it going forward. To fight IL’s tendency to overfit, I implemented a cosine decay scheduler with restarts, which reached a lower validation loss than the stock PyTorch scheduler.
| MSE vs. SSE loss | Custom scheduler vs. PyTorch |
|---|---|
 |
 |
Why the C++ collector mattered
Training on the large (25.6 GB) dataset gathered by the Python collector produced extremely noisy validation loss — a sign of overfitting caused by latency-induced mismatches between observations and actions. Re-collecting the same-size dataset with the C++ collector produced markedly lower training and validation loss, confirming that the culprit was data-collection latency, not the learning setup.
| Python collector | C++ collector |
|---|---|
 |
 |
Final policy
The final policy — trained with MSE loss, the custom scheduler, and the C++ collector — closely reproduces the expert’s B-splines both visually and statistically. Across 12 trials it achieved an average MSE of 2.04, cosine similarity of 0.94, and Bhattacharyya distance of 0.025.
Twelve expert-labeled B-splines (per-trial) compared against the learned policy’s inferred B-splines.
Contributions
- A ROS-based IL pipeline that, unlike gym-based approaches, can leverage existing ROS/C++ planners and has a real path to sim2real transfer. Code: learning_perception_aware.
- An extension of the AirSim ecosystem with a complementary Unreal Engine 4 simulation environment purpose-built for IL.
- Full compatibility with the Kumar Robotics stack, plus a proposed network architecture for quadrotor trajectory planning.
- Extensive experiments characterizing the effects of loss function, learning-rate scheduler, and data-collector implementation.
Limitations and Future Work
At submission the inference pipeline was still in Python (a C++/LibTorch version was in progress), and the learned policy had been validated in simulation but not yet flown on hardware. The natural next steps were a sim2real transfer procedure for a physical quadrotor, distilling a low-cost policy from a jointly-optimized perception-aware planner, and integrating RL algorithms (PPO, SAC) for direct IL-vs-RL comparison inside the same pipeline.
📄 Read the full thesis: Imitation Learning for Autonomous Quadrotor Flight