Learning to Control from Vision: Evaluating Imitation and Supervised Methods in a Competitive Simulated Environment

Abstract

This paper investigates the performance of two learning paradigms—DAgger-based imitation learning and supervised learning—for vision-based control in an autonomous agent setting. We evaluate both approaches in a simulated competitive environment, where agents must interpret visual input and make real-time decisions to achieve task-specific goals. Our experiments show that while both methods are viable, the supervised learning approach consistently outperformed the imitation-based method under time constraints. The final vision-based controller achieved low-latency inference (<100ms per frame on CPU) and demonstrated superior game performance across all difficulty levels. These findings offer valuable insights for selecting control strategies in real-time computer vision applications.

More broadly, our findings contribute to the growing body of research at the intersection of deep learning and real-time control, offering practical guidance for designing robust, efficient agents in fast-paced or safety-critical environments. The methods and insights presented here can inform future development in domains ranging from robotic automation to augmented reality, where rapid perception-action loops are essential.