Abstract: Unmanned aerial vehicles (UAVs) have been extensively employed across various fields; however, the increasing occurrence of unauthorized UAV "black flight" incidents poses a significant threat to public safety. In anti-UAV systems, infrared imaging sensors that are operational day and night are becoming increasingly prevalent in UAV detection and surveillance. This study addresses infrared image detection for UAVs and proposes a real-time infrared lightweight (IRLT)-YOLO target detection algorithm. In designing lightweight networks to reduce the depth of the backbone network, lightweight operations with a shared convolution in the header are employed, thereby minimizing redundant features. Real-time detection is achieved while preserving detection performance by introducing a tiny target detector–based on the normalized Wasserstein distance (NWD)–embedded in the loss function to replace the original Intersection over Union (IoU) metric. Experimental results indicate that the IRLT-YOLO model achieves precision, recall, mean average precision (mAP)@0.5, floating-point operations per second (FLOPs), and frames per second (FPS) of 95.4%, 85.9%, 89.5%, 4.9G, and 167.0, respectively, representing a dual enhancement in computational accuracy and speed compared to the baseline model. Simulation experiments show that the IRLT-YOLO model enhances the detection and recognition capabilities of UAV targets in infrared scenarios, offering fast and effective real-time detection when deployed on edge devices, thereby meeting the demands of real-time detection applications in anti-UAV systems.