Robots are everywhere in today’s society. They vacuum our floors, assemble vehicles, handle hazardous objects, assist in surgeries and explore deep sea environments, improving our precision, efficiency and productivity.
Most current robotic systems are programmed to do specific tasks and do not function autonomously. With the rise of artificial intelligence, however, researchers are envisioning the next generation of autonomous robotic technology, incorporating visual perception, decision-making, path planning and control, to create the robots of the future.
Robots are divided into three functional categories: service robots that clean, assist, entertain and educate; industrial robots that assemble cars and other machines in manufacturing settings; and special robots that are designed for highly specialized tasks, such as surveying drones, all-terrain four-legged robots and surgical robots.
In order for robots to become more autonomous, they must be able to accurately assess their surroundings through improved visual intelligence. In contrast to vacuum robots that bump into walls, back up, spin, and move forward in a different direction, autonomous robots must be able to perceive their surroundings to identify objects, obstacles and relevant features through cameras and other sensors to facilitate decision making.
Scientists are making significant advances in visual intelligence, bringing autonomous robots closer to reality. Recently, a group of researchers from Hunan University wrote a review article outlining the latest work in robotic visual perception, decision-making, path planning and control from the perspective of visual intelligence. The team also summarized many of the challenges facing researchers in the field and future directions of the technology.
The team published their review in Visual Intelligence on May 9 , published by Tsinghua University Press.
“With the rapid development of large models and embodied AI, robots are gradually moving from traditional, constrained settings into more complex and open environments, creating an increasing need for more intelligent, adaptive and reliable perception, decision-making, planning and control,” said Hengcan Shi, professor in the School of Artificial Intelligence and Robotics at Hunan University in Changsha, China and primary author of the review paper.
Specifically, robots use a planning module to compute whether specific paths or trajectories can safely work within geometric constraints in the environment, taking into account the robot’s physical capabilities and environmental obstacles. The robot control module operates at the lowest level to execute the planned movements by generating motor commands to accomplish the task safely in the real world. This layered decision-making structure, integrating abstract reasoning with real-time physical interaction, allows robots to perform complex tasks autonomously in the real world.
“Robotic intelligence does not come from a single powerful algorithm or module. Instead, it emerges from the systematic integration of visual perception, decision-making, path planning, and control. A robot can only behave reliably in the real world when information flows smoothly from seeing and understanding the environment to reasoning, planning, and executing appropriate actions,” said Yaonan Wang, professor in the School of Artificial Intelligence and Robotics at Hunan University and corresponding author of the review paper.
The key to the successful design of autonomous robots is enhancing the accuracy of each layer in the decision-making process. Achieving tight integration and consistency across each layer is one of the central challenges in robotics research, particularly in uncertain environments or surroundings that change quickly.
Today, researchers are focusing on enhancing the visual intelligence required for robots to identify objects in their environment and the relationships between those objects. In the future, robots will need to comprehensively reason in the real world by, for example, anticipating collisions or assessing clearances, which can be very challenging in dynamic environments.
Ideally, researchers will create an end-to-end decision-making process for AI robots that isn’t prone to the information loss and error propagation that can occur in systems with current multilevel decision-making processes. This type of system could potentially speed reaction time. Researchers would also like to further investigate collaborative intelligent systems for cooperative swarm robots and lightweight AI systems that require fewer resources to run reliably and quickly.
“The ultimate goal is to make robots have human-like intelligence, such as autonomously understanding complex environments and dealing with highly complex tasks. Ideally, robots will be able to communicate, cooperate, and learn from each other, so that they can ultimately solve complex tasks by collaboration, just like human beings,” said Min Liu, professor in the School of Artificial Intelligence and Robotics at Hunan University and also corresponding author of the review paper.
Wen Liu, Zheng Li, Xinpu Fang, Xiangfeng Meng, Weixing Peng and Hang Zhong from the School of Artificial Intelligence and Robotics at Hunan University in Changsha, China also contributed to this research.
Funding information
This research was supported by the National Natural Science Foundation or China (Nos. 62401202, 62425305, 62221002, 62293510, and 62573181) and the National Key Scientific Instrument and Equipment Development Projects of China (No.62427813).
D OI Link:
https://doi.org/10.1007/s44267-026-00116-2
A bout Visual Intelligence
Visual Intelligence is an international, peer-reviewed, open-access journal devoted to the theory and practice of visual intelligence . This journal is the official publication of the China Society of Image and Graphics (CSIG), with Article Processing Charges fully covered by the Society. It focuses on the foundations of visual computing, the methodologies employed in the field, and the applications of visual intelligence, while particularly encouraging submissions that address rapidly advancing areas of visual intelligence research.
About the Authors
Yaonan Wang received his PhD degree in electrical engineering from Hunan University, Changsha, China, in 1994. From 1994 to 1995, he was a postdoctoral research fellow with the National University of Defense Technology, Changsha. From 1998 to 2000, he was supported as a Senior Humboldt fellow by the Federal Republic of Germany with the University of Bremen, Bremen, Germany. From 2001 to 2004, he was a visiting professor with the University of Bremen. Since 1995, he has been a Professor with the College of Electrical and Information Engineering, Hunan University. He is currently a member of the Chinese Academy of Engineering. His research interests include robotics and image processing.
Min Liu (Senior Member, IEEE) received his bachelor degree from Peking University and PhD degree in electrical engineering from the University of California, Riverside, in 2012. He is currently a professor with Hunan University. His research interests include robot vision and pattern recognition. He is also an associate editor for IEEE Transactions on Neural Networks and Learning Systems.
Hengcan Shi is a Professor at the School of Artificial Intelligence and Robotics, Hunan University. Before that, he was a Research Fellow at the Department of Data Science & AI, Monash University. He received his Ph.D. degree in computer vision from University of Electronic Science and Technology of China in 2019. His research interests include Large Model, Image Segmentation/Detection, Vision+Language, and Affective Computing. He has served as an area chair for ACMMM and a program committee member for CVPR, ICCV, ECCV, etc.
Visual Intelligence
Intelligent robot systems: a survey from the perspective of visual intelligence
9-May-2026