Design control system for Pan-Tilt Camera for Visual Tracking based on ADAR method taking into account energy output

. In this paper, we propose a controller for a camera-based Pan-Tilt device. The approach uses the target image feature to steer the Pan-Tilt device to the target with random movement. Since these systems are often mounted on mobile equipment, energy efficiency should also be taken into account. In order to improve the control energy consumption efficiency of the Pan-Tilt system mounted with cammera, the authors use a dark objective function with optimal energy components to find controller parameters. Simulation results on the Pan-Tilt system show that the camera can track a moving target quickly and stably and the control energy can be changed through the controller parameter according to the purpose of use.


Introduction
In recent years, computer vision has become an important onboard sensor for autonomous mobile robots.Among the various applications of vision systems, object tracking plays an important role in vision-based monitoring and interaction with human computers and mobile robots [1][2][3].Especially when a robot enters an unknown new environment, there is a big hit in visual detection, object tracking, and visual object locking.However, most robots have a single, forward-facing camera on board.And this type of vision system only covers part of the field and requires a fast moving robot to track the target [4].Even moving targets have the potential to exit the image view and lead to unsuccessful tracking and locking of the image.But a camera-mounted pan-tilt system can remove this limitation and make the system more accessible better performance tracking system.Using a camera-mounted pan-tilt system is an option to increase the ability to track and lock objects.Furthermore, it can make the camera move freely and track the moving subject, keeping the subject in the center of the image.
So far, several research works on camera pan-tilt systems have been published.Thilo et al published work on this system, but the tilting system was relatively slow and could not track the target effectively [5].In the study [6], a PID algorithm was proposed to control the camera with a closed loop, but the tilt camera system needs an operator and cannot operate automatically.In the study [7] proposed a fuzzy logic control for tilt camera on drone.Besides, Zhang et al. presented a non-PID algorithm for tilting camera control using only image feature error, but this approach cannot guarantee the goal to stay in the center of the image [8].In the study [9], an active noise cancellation controller (ADRC) is used which is a new type of controller that can automatically detect the real-time effect of the model and the external noises and then automatically offset it.But in the above studies, only improving the accuracy of the tracking process does not take into account the energy efficiency.
In this paper, the camera pan-tilt system is controlled by nonlinear control law taking into account the energy efficiency factor.Control law is designed based on a target function containing the optimal energy component.Method used to design control laws based on a synergistic approach.The quality of the proposed control law is shown through simulation results.
The rest of this article is organized as follows.Part II presents a brief description of the camera pan-tilt system and a mathematical model of the system.In Part III, Technical description of design of composite control rule based on AKAR method for pan-tilt system with camera attached.Part IV is the simulation results performed on some typical cases.Finally, conclusions and future work are given in Part V.

Pan-tilt camera platform description 2.1 Constructing a camera-mounted pan-tilt system
The pan-tilt camera tracking system is composed of three modules, shown in Fig. 1a.+ Camera: In this paper, we use the ISD-2500HD camera with a lens with a fixed focal length of f = 2.5 cm with a resolution of 1280x720, connect to Raspberry PI computer by USB port.

Mathematical model of the camera pan-tilt system
In order to maintain the image of the target as close to the center of the image plane, we establish a relative relationship between the observed variable xc and yc which is the coordinates of the object on the coordinate system attached to the image plane ( IP), for two control variables of the pan-tilt system φ and θ.
In Fig. 1b, we can see that φ and θ are rotations around the axes OZ1 and OZ2, respectively, such that, after moving the center of the moving target, coordinates P(X, Y, Z) will be mapped onto the center of the image plane.
+ The geometrical approximation model.Let OX0Y0Z0 be the real coordinate system of the robot-camera system and OpXpYpZp the coordinate system associated with the camera.P(X0,Y0,Z0) is the coordinates of the object on the real coordinate system, the coordinates of P projected on the image plane are (xc,yc).Fig. 2 shows the geometric relationship of the rotation angle φ is the rotation angle of the robotcamera system so that the center of the object P(X0,Y0, Z0) lies on the plane XpOpYp.From Fig. 2, it can be seen that if the difference between the two planes XpOpYp and X0OZ0 is not significant, we can calculate the angles φ and φ' as follows: tan( ') where X and Y represent the position of the target center in the real coordinate system, L is the distance from the origin O to the origin Op , f is the camera focal length.

Fig 2. Image coordinate system associated with pan-tilt coordinate system
From (1) we realize that in order to calculate we must know that X and Y, which are neither measurable nor observable, when using only a still camera.Also from (2) we find that, since zp is an observable variable, and f is a measurable quantity, we can compute zp.From ( 1) and ( 2) we have: From formula (3), it can be seen that if the distance (X) of the object to the camera of the system is large or the structure of the system so that L can be considered very small, the angle φ can be approximated as follows: For the angle approximation θ is also calculated similarly.The purpose of this approximation is that we can calculate pan rotation φ and tilt angle θ from the observable variables xc , yc and the measurable f. + Kinetic model of Pan-Tilt system mounted with Camera.
In order to find a description of the relationship between xc and yc for φ and θ, we first find the homogeneous transformation matrix of the two camera rotations around O at an angle φ and θ.With the way of choosing the coordinate system in Figure 2, we have a uniform matrix of the direction that changes from the OpXpYpZp coordinate axis to the real coordinate axis of the Pan-Tilt system mounted with an OXYZ camera mounted camera is Rt : cos( ) cos( ) cos( )sin( ) sin( ) 0 sin( ) cos( ) sin( ) sin( ) cos( ) 0 sin( ) cos( ) 0 0 The identity matrix when converting from the real O coordinate system to the camera coordinate system OpXpYpZp, by definition we have: Points P(X0,Y0,Z0) will be projected on the center of the image plane Op after performing rotations φ and θ.Therefore, knowing the directional identity matrix R t , we can write the following: 0 0 0 cos( ) cos( ) sin( ) cos( ) sin( ) 0 0 sin( ) cos( ) sin( )sin( ) cos( ) 0 0 sin( ) cos( ) 0 0 where, (p, 0, 0, 1) and (X0,Y0,Z0,1) are identical coordinates of point P when viewed in the Op and O coordinate systems, respectively.

+ Equations of Motion
The dynamic model of the camera-mounted pan-tilt system is built on the Lagrange-Euler formula.Referring to Fig. 2 the kinematic relations for each mass can be written easily: The potential and kinetic energy of the system are given by the equation: By applying Lagrange's equation [11] we get the following system of equations, after some manipulation ( ) ( , ) ( ) M q q C q q q g q τ where the state variable q, τ is defined by: q=[ϕ θ] T , τ=[τ1 τ2] T and M represent the inertia matrix (symmetric positive definite) , C is the Coriolis and centrifugal matrix, gravity vector g=[g12 g21] T which is given by: Finally, τ=[τ1 τ2] T is torque exerted by actuators (servo motors) at each joint

Synthesis of control law of camera pan-tilt system by ADAR method taking into account the output energy
Camera pan-tilt systems are often mounted on mobile vehicles such as robots, drones or where power supplies are limited.One of the requirements of this equipment is to save energy when operating in order to prolong the operation time.One of the factors affecting the power problem is the control signal.The authors present an energy efficient control law synthesis method based on the ADAR method.The ADAR method is based on the idea of constructing invariant stable manifolds 1 ( ,..., ) s n x x ψ [10][11][12][13].
To synthesize the control law for the camera pan-tilt system, it is necessary to transform the system of equations ( 14) into an equation of state with the state variable as follows [ ] Let ei (i=1,3) be the difference between the actual angular position x1 = φ x3 = θ and the desired trajectory x1s=φ s, x3s=θs as follows: There, β is the scalar association parameter of the control rule.
Taking the derivative of equation ( 17) we get: ( ) ( ) ( ) To ensure stability and ensure the convergence of the state trajectories to their respective desired manifolds and remain there for the next time, the dynamic evolution of the macro variable with respect to the The manifold is defined as: ( ) ( ) 0 where Ti > 0 represents the convergence rate of the macro variable to the manifold domain ψi (e) = 0.
Solving the system of equations ( 18) we get the control law τ1,τ2, because the control law is too long, so we do not introduce it in this paper.
The ADAR objective function has the form (18).When analyzing specifically each component in this functional, it is easy to see that it contains components τ1 2 , τ2 2 , with reasonable selection of Ti and βi parameters, which will ensure control quality and control energy.( ( ) ( )) 4 Simulation results of tracking control for Pan-Tilt system with camera The parameters of the system model are determined experimentally on the real model as follows: m1=30(kg), m2=20(kg), d1=0.3(m),d2=0.4(m),L= 0.05, g=9.81(m/s2), β1=0.1, β3=0.1.Controller parameters T1, T3 are changed in a solution to observe system response and control signal value.The set values are taken in different forms to demonstrate the effectiveness of the control law.Fig. 3.4 is the angular response, control signal of pan and Tilt in the system when the set value is step function.And in Fig. 5,6 is the angular response, control signal of pan and Tilt in the system when the set value is a sine function.From the above results, it can be seen that the proposed control law always makes the system stable asymptotically, the quality of the control system depends on the accuracy of the model and the controller parameters.The control signal depends largely on the parameters T1={0.6,0.3,0.1},T3=={0.6,0.3,0.1} of the controller, with appropriate selection will ensure the system has low energy consumption while still ensuring control quality.

Conclusion
Synthetic synthesis method for ensuring the system is asymptotically stable and the objective function of the method contains the optimal energy component.Surveying and selecting controller parameters can ensure that the system has an effective control law in terms of energy consumption but still ensure the quality of system control.To ensure that the limit value of the control signal can change the change of the manifold stability guarantee parameter.The proposed control rule also works well with many different types of set values.Since the cameramounted pan tilt system is characterized by a rapidly changing unpredicted set signal, this is also an advantage of this method.However, in this study, the controller parameter selection method has not been given.Therefore, in future studies, the authors will study methods to choose the optimal parameter of the controller.

Fig. 1 .
Fig. 1. a-Pan-Tilt system with camera; b-Kinetic model of the pan-tilt system + Processing Center: The camera pan-tilt system is controlled by the central computer, the Raspberry PI-4.The computer is connected to the ISD-2500HD camera via USB and controls the pan-tilt two-axis motor drivers via variable frequency pulse I/O.+ 2-DOF Pan-Tilt Device: The 2-DOF tilt device consists of four parts, including two stepper motors, an encoder, a reducer and a motor driver from ORIENTAL MOTOR.Control the direction of rotation and speed of the motor through the driver via IO pins or through Modbus RTU (RS-485) communication.The Raspberry PI 4 central processor can communicate with the motor control driver through the direct IO port or RS_485.In this paper, the central processor communicates with the drivers via IO ports.2-DOF tilt can pan and tilt respectively.The model's physical limitations include: In the vertical direction, the pitch angle range is from -60 deg. to 60 deg.According to horizontal rotation angle range of rotation angle is from -90 deg. to 90 deg.The maximum angular acceleration of the inclined device is 167 deg./sec.