Intelligent Following Car Based on Dual Detection Positioning Using Ultrasonic and Camera

: To achieve real-time positioning of the target object for given tasks, a smart car system is designed using STM32F407 as the core control unit, mainly relying on ultrasonic sensors for detection and supplemented by camera detection for target locating and tracking. The ultrasonic sensor module is used as the main device for target information collection and transmission, and the distance difference between the two modules and the follow-up object is used to determine the target position. Meanwhile, the camera module is used for accurate positioning of the target, and algorithms such as Kalman filtering and PID closed-loop control are used to achieve interaction between the two detection modules and intelligent following. By doing so, the system can always maintain a distance from the target person and provide timely assistance as needed.


Study Background and Meaning
With the increasing emphasis on technological innovation and rapid development of intelligence in various countries around the world in recent years, various innovative products have emerged, and all kinds of robots have sprung up.Currently, the cost of labor in China is showing a significant increasing, and intelligent follow-up cars have emerged.There are higher research needs and application challenges for intelligent follow-up robots in human-machine space interaction and understanding [2,3].Automatic tracking technology refers to the technology that can provide spatial positioning, posture, structural behavior, and performance of motion targets.It is a multi-functional and high-precision tracking and measurement method for motion targets, and automatic tracking consists of position sensors, signal processing systems, servo systems, and tracking racks.According to the different sensors used, target tracking methods include laser information tracking [4,5], visual information tracking [6,7], and multi-sensor information fusion tracking [8], etc.
In the late 1960s, the Shakey robot project at the Stanford Research Institute (SRI) opened up the field of mobile robot research [9].The robot used multiple sensors such as cameras, distance sensors, and collision sensors to detect targets.In 1999, Luoson-III, an intelligent wheelchair developed by Chung Cheng University in Taiwan, used a computer as the computing core and applied fuzzy decision-making algorithms.It also added image sensors, speech interaction systems, etc. to the mobile robot to achieve autonomous navigation [10].
On January 4, 2004, the NASA Mars Exploration Rover "Opportunity" successfully landed on Mars, attracting worldwide attention and ushering in remote control of intelligent mobile follow-up robots [11].In 2010, the intelligent wheelchair robot developed by the Institute of Automation of the Chinese Academy of Sciences was the first multimodal interactive intelligent wheelchair prototype in China [12,13].The research comprehensively used computer image recognition and processing, speech recognition, and other technologies, allowing users to control the wheelchair using voice commands.Ghorbel [13] and others applied cloud technology, which has been popular in recent years, to design an auxiliary system for intelligent wheelchairs.Family members can also remotely monitor important information such as the position and safety of wheelchair users through their smartphones.However, at present, mobile robots with single-chip microcontrollers as their core still have limitations in processing data and unstable control.There is no truly significant intelligent cargo-carrying small car with tracking ability in the domestic market [14][15][16].Although the intelligent small cars that are mass-produced and put into use on the market have high control precision and strong stability, they still suffer from problems such as target loss and high cost.In industries such as industrial and express delivery, which have high demand for such products, the high price and complex operation will inevitably affect the development of products in this field.Moreover, many existing solutions have shortcomings such as insufficient robustness, complex structure, and vulnerability to interference, and have not yet met production standards.

Major Research Focus
The main focus of this article is to study the combination of two detection methods, camera and ultrasound, to track the lower limb movements of the target so as to design an intelligent small car with autonomous following and obstacle avoidance functions.By using the camera to assist the ultrasound positioning module, the algorithm is used to recognize and track the user, achieving automatic following of the wheelchair.
The HY-SRF05 ultrasound is used to assist the multi-module combination to detect different angles, and the Camshift tracking algorithm is used to determine the position of obstacles for avoidance.The optimal path is selected for forward movement, achieving the effect of "intelligent control".The camera captures the image in front of the car, compares frame by frame to select effective ones, and transfers the image information to the main control chip through Kalman filtering to achieve target following through fusion algorithm.
This solution is not only cost-effective but can also be applied in some low-precision application scenarios on the market, such as following staff in express stations for transporting and placing express packages, following rehabilitation wheelchairs for the elderly, and intelligent following of golf bags on golf courses, etc.This solution has a certain market application prospect.

Driving Circuit
In order to make the system more stable and suitable for various application scenarios, smooth control of the motor is necessary on the basis of obtaining the target speeds of the two wheels [17].Using a dual-wheel differential motor as displayed in figure 1 can achieve low power consumption, low heat generation, and stable control, and can work for a long time.
A dynamic model of vehicle ramp starting is established, and a preloaded uphill assistance control algorithm with "feedforward + feedback" structure is constructed to ensure real-time control.The impact mechanism of control parameters on system stability, steady-state error, and dynamic performance is revealed by time-domain analysis.This enables the system to achieve smooth operation and compensation for uphill and slope when driving on a ramp, thereby achieving "intelligent" fine-tuning control functionality.

Control Algorithm
Regarding the control algorithm, PID closed-loop and motor speed control are shown in the Figure 2 and 3.The input to the system is the target speed command of the wheelchair's left and right motors, which is the output value of fuzzy control.The control of the motor speed loop typically uses the proportional adjustment P and the integral adjustment I in PID.The proportional adjustment is used to quickly stabilize the motor speed to the target speed, while the integral adjustment is used to eliminate the steady-state error of the system.After the PI adjustment, the speed value is converted into a digital signal command, which drives the motor after being converted by an analog-to-digital converter.The optical encoder installed on the outer shaft of the motor obtains the current speed, provides negative feedback, and achieves closed-loop control.The technology route involves using HY-SRF05 ultrasonic modules to assist in multi-module combination angle detection.The system emits or detects ultrasonic waves from an object and uses ultrasonic ranging technology to transmit distance information to the microcontroller.By combining with a fuzzy control algorithm, the system can effectively control obstacle avoidance.
Figure 4 illustrates the principle of ultrasonic module following.Two ultrasonic modules are installed at each end of the carrier, with one held in each hand, forming a triangle.When the robot faces the person, the distance between the two ultrasonic modules is A = B, which effectively transmits the ultrasonic waves.When the person turns, A must be shorter than B. Similarly, when the person turns right, A must be longer than B. When the person moves forward, the length between A and B must be longer than the predetermined length.By controlling these lengths, the system can turn left, turn right, and move straight on a simple path.

Open-CV Test
By using a camera to assist the ultrasonic module in transmitting and receiving signals, the wheelchair's forward view can be captured, as shown in the Figure 5.The wheelchair's initial positioning is achieved through spatial geometry algorithms and coordinate transformation principles.Then, improvements are made to the traditional trajectory extrapolation method to enhance the relative positioning accuracy of the wheelchair.The image information is processed using the Kalman filter, which uses the predicted and observed values to determine its estimated value [18].Finally, a detection and tracking fusion algorithm is used to recognize and track the user.Simulation results have demonstrated a significant improvement in positioning accuracy.
The camera module can capture external environmental images and transmit them through the serial port to the main control chip for precise control.However, due to the complexity of the external environment, the obtained information needs to be processed to better achieve the effect of the Open-CV motion target tracking algorithm, as shown in Figure 5. Based on the camera's focal length f and lens optical center model c, we can calculate the distance d between the object and the camera plane, as well as the actual height h of the object.Using this model, we can determine the distance between the target object and the camera plane.Assuming that the actual object is perpendicular to the plane where the camera is located, we can achieve the shooting.Using Open-CV library functions called by Raspberry Pi, we can implement a simple version of video recognition.The actual detected image is shown in Figure 6.

Depth Camera Projection Technology and Visual Image Processing Technology
The ranging principle of the depth camera is to project structured light with a certain structural feature onto the object being photographed through a near-infrared laser, and then collect it by a specialized infrared camera.This light in Figure 7, which has a certain structure (generally with stripe pattern, coding structured light, or speckle structured light according to different encoding patterns), will capture the information of the reflected structured light pattern due to different depth areas of the object being photographed.Then, the change of this structured pattern is converted into depth information through the calculation unit, in order to obtain the three-dimensional structure.The main parameters of the depth camera are shown in Table 1.
Using this method, we can obtain three-dimensional structure.The principle of a depth camera is shown in Figure 7 and main parameters of the depth camera are shown in Table 1.In and out door The information acquisition and transmission system we designed is shown in the following figure.We use the Raspberry Pi 4B control board to process the images transmitted by the depth camera.Figure 8 shows the two-dimensional positioning effect, and Figure 9 shows the three-dimensional positioning effect.

Product Technical Roadmap Diagram
As shown in Figure 10, this is the technical route of our research.The power supply provides power to the product chip through the voltage stabilization module.The product is based on the STM32F407 as the core control unit, which controls the autonomous obstacle avoidance system, motor drive module, and remote control module respectively.Through serial communication, it controls the Raspberry Pi 4B controller and the Esp8266 wireless communication module.The Raspberry Pi 4B controller is responsible for controlling the camera to capture and process target images, while the wireless communication module can upload real-time data of the intelligent transport vehicle to the cloud and monitor it in real-time through a mobile device APP, realizing functions such as human-computer interaction and danger alarm.Firstly, each module in the model was tested separately, and after confirming their functionality, a simple physical model was built.Then, the software was debugged, and after confirming that the wiring and soldering were correct, the self-following function of the small car was tested.To avoid the influence of other factors, the researchers first placed the small car in an indoor environment at a temperature of 20°C and with moderate light intensity of about 400lx for the experiment, and obtained the following data in Table 2: Lose track Then, the researchers placed the car in an outdoor environment with a temperature of 29°C, clear weather at noon, and a high light intensity of about 80000 lx, and obtained the following conclusions in Table 3.
Finally, the car was placed in an outdoor environment with a temperature of 10℃, cloudy weather, evening time, and low light intensity of about 100lx.The following conclusion is drawn in Table 4:

Error Analysis of the System Testing
In this experiment, both normal and extreme operating environments were simulated, and the errors in the experiment can be divided into the following three aspects:

The Influence of Airflow
Compared with the indoor environment, there is air flow in the outdoor environment, which causes the ultrasonic signal to decay too fast during the propagation process, resulting in a delay in the ultrasonic response and a significant impact on the ultrasonic detection module.

The Influence of Light Intensity
When the light is strong, the reflected light from the ground, walls, etc. enters the camera, causing the camera image to be distorted and the target object unable to be distinguished.When the light is weak, the target object in the camera image is not easily distinguished from the surrounding environment, and when the distance is too large, it is easy to lose the target.

The Influence of Temperature
The environmental temperature has a certain effect on both the ultrasonic and camera detection modules.The higher the temperature is, the faster the ultrasonic wave velocity will be.And the long-term exposure to high temperatures will cause the camera to transmit images with a delay.However, at present, the temperature range of the experiment has a small effect on the experiment's error.

Conclusion
This article introduces an intelligent following car based on a dual-modal detection system using ultrasonic and camera sensors.It mentions some related technologies such as sensors, machine vision, automatic following, and positioning technology, and how to implement them in the intelligent car system.Finally, it describes how to evaluate the performance of the intelligent car system using both simulated and real environment experiments.Based on the experimental data, it is found that although the car may lose its target and cause detection timeouts in some extreme environments, it can be used in most normal environments and has a short delay response time of 0-1.5 seconds when the target is within 0-3 meters, and can achieve synchronous following within 3 meters.The intelligent car system has the characteristics of low cost, easy operation, and convenience.Moreover, the results show that the safety and reliability of the intelligent car system are significantly improved under the dual-modal detection mode of ultrasonic and camera sensors, which provides useful references for future research on intelligent car systems.

Figure 2 :
Figure 2: Schema of PID closed-loop control

Figure 3 :
Figure 3: Flowchart of motor closed-loop speed control

Figure 4 :
Figure 4: Principles of the following system

Figure 6 :
Figure 6: The actual detected image

Figure 7 :
Figure 7: Schema of the principle of a depth camera

Table 1 :
Main parameters of the depth camera