Integration of artificial intelligence and video surveillance technology to monitor construction equipment

. This article discusses the integration of artificial intelligence (AI) and video surveillance technology for construction equipment management. The use of artificial intelligence algorithms and video surveillance systems can improve equipment management by increasing the efficiency of video surveillance on construction sites, improving the safety and efficiency of construction equipment. The article discusses the potential benefits of using AI to analyze data from video feeds, including the ability to identify anomalies in equipment usage patterns, predict maintenance needs, and optimize equipment utilization. The article provides an example of the practical implementation and use of AI and video surveillance technologies in the construction industry today, highlighting their potential.


Introduction
Construction sites are inherently complex and dynamic environments, with multiple machines operating simultaneously, workers performing various tasks, and loads being moved from one location to another.With such intensive work, it's critical to ensure proper video surveillance to ensure safety, security and regulatory compliance.
As a result, the use of video surveillance cameras is becoming increasingly popular at construction sites.However, manually monitoring such objects is a complex and timeconsuming task, so this is where artificial intelligence (AI) comes in.
The use of artificial intelligence (AI) can solve these problems and increase the effectiveness of video surveillance on construction sites.AI algorithms can be taught to recognize different types of construction machines and equipment, allowing cameras to automatically identify and track these machines.This can provide a wealth of data that can be used for a variety of purposes, such as detecting malfunctions, identifying security threats, and improving productivity.This article discusses the use of AI to determine the type of construction machines for CCTV cameras, discussing the potential benefits and challenges associated with this technology.Using AI, construction sites can improve their monitoring and control capabilities, leading to a safer and more efficient work environment.

Problem statement
The article also attempts to present various artificial intelligence algorithms and video surveillance methods that can be used to monitor and analyze construction equipment, and discuss the practical implementation and deployment of such technologies on construction sites.
In addition, an important outcome of the paper is the opportunity to inform construction industry professionals and stakeholders about the potential of AI and video surveillance technology to improve the management of construction equipment, and to provide guidance on best practices for implementing this technology.By shedding light on the benefits and challenges associated with AI-based equipment management, the article aims to help inform decisions and promote wider adoption of this technology in the construction industry.

Analysis of video surveillance systems
As technology advances, surveillance systems are becoming increasingly sophisticated and can include features such as motion detection, face detection and license plate recognition.These features enable cameras to automatically detect and track objects and people, providing valuable information to site managers.
As a result of analyzing the experience of practical application of video surveillance systems, a list of video surveillance systems that can be used on construction sites was formed: Fixed CCTV cameras: are placed in strategic locations to capture a specific area or activity.
PTZ (Pan-Tilt-Zoom) cameras: controlled remotely to move and zoom in on specific areas or locations.360-degree cameras: provide a panoramic view of the construction site, capturing all activities from a single vantage point.
Thermal imaging cameras: use infrared technology to detect thermal signatures and are especially useful for identifying hot spots or detecting intruders.
Motion detection cameras: Activated by motion, they can alert security personnel or facility managers in real time.
Face recognition cameras: use artificial intelligence technology to identify and track people by their facial features.
License Plate Recognition Cameras: Use artificial intelligence technology to read and recognize license plates of vehicles entering or exiting a construction site.
Mobile video surveillance systems: mounted on vehicles or trailers and can be moved around a construction site to capture different areas and activities.
Wireless video surveillance systems: use wireless technology to transmit video footage to a central monitoring station, eliminating the need for physical cabling and allowing for more flexible installation.
Cloud-based video surveillance systems: store video footage in the cloud, allowing remote access and management.

Analysis of foreign and Russian experience in the field of AI
In recent years, the field of artificial intelligence (AI) has developed rapidly, with both foreign and Russian companies investing in AI technology.In terms of foreign experience, countries such as the United States and China are at the forefront of AI development, and companies such as Google, Amazon and Alibaba are investing heavily in AI research and development.
Foreign companies have developed a wide range of AI applications, from voice recognition to image and speech analysis, and have successfully implemented AI in various industries such as healthcare, finance and manufacturing.For example, Amazon's recommendation system uses AI algorithms to analyze customer data and provide personalized recommendations, and Google's self-driving car project uses AI to enable the car to navigate and make data-driven decisions in real time.
Russia has also seen significant investment in AI technology, with companies such as Yandex, Sberbank and Mail.ruGroup leading the way.Yandex, for example, has developed a number of AI-based products, including a virtual assistant and a search engine that uses machine learning algorithms to provide more accurate search results.
Russian companies are also working on the application of AI in industries such as healthcare, transportation and energy.For example, Sberbank has developed an AI-based risk assessment tool for small businesses, and Gazprom Neft uses AI to optimize oil production.
However, there are still problems of implementation and use of AI in the construction industry, so it is necessary to actively develop and implement AI systems.

AI technologies
The study of foreign and Russian experience in the field of technological solutions used in the development of AI [1,2,7] highlighted the main used architectures of neural networks, presented in Table 1.

Convergent Neural Network (CNN)
Uses convolution layers to extract features from images or other data with spatial relationships.

Recurrent Neural Network (RNN)
Can process sequential data with feedbacks that retain information about previous inputs.

Long Term Short Term Memory (LSTM)
A type of recurrent neural network that can selectively remember or forget information, allowing it to be used to analyze sequential data where not all events have the same weight.

Generative Adversarial Network (GAN)
A two-part network, a generator and a discriminator, that work together to create new, synthetic data similar to the input data.

Deep Probabilistic Model (DBN)
Used to model probability distributions of data, including implicit relationships between features.
According to the results of studying the works of different authors [3,4,5,6], the advantages of neural networks were also highlighted:


The ability to learn and generalize: neural networks can learn complex relationships between input and output data and generalize those relationships.
 Resistant to noise and missing data: Neural networks can still make accurate predictions even if the input data is noisy or contains missing values.
 Processing nonlinear relationships: Neural networks can model nonlinear relationships between input and output data, which is difficult for traditional linear models.
 The ability to process large amounts of data: Neural networks can efficiently process large data sets, making them well suited for big data applications.
 Versatility: Neural networks can be used for a wide range of tasks, including classification, regression, and clustering, as well as for more specialized tasks such as image and speech recognition.
 As well as the shortcomings of neural networks:  Complexity: Neural networks can be complex and difficult to interpret, making it difficult to understand how they make predictions.
 Overtraining: Neural networks can easily adjust to training data, resulting in poor performance on new data.
 Training time: Training a neural network can require significant computational resources and time, especially for large datasets or complex models.
 Lack of transparency: neural networks can be considered a black box because it can be difficult to understand how they make decisions, which can be a problem in applications such as healthcare or finance, where decisions have a significant impact.
 Vulnerability to hostile attacks: neural networks can be vulnerable to hostile examples, where small changes in input data lead to incorrect predictions.


The requirement for large amounts of marked-up data.Neural networks often require large amounts of partitioned data for effective learning, which can be difficult and time-consuming to obtain.
In addition, it should be noted that today there are several popular ready-made Python libraries for creating and developing neural networks: TensorFlow: an open-source software library for data flows and differentiated programming for a range of tasks.
PyTorch: an open-source machine learning library based on the Torch library, used for applications such as computer vision and natural language processing.
Keras: an open-source software library providing a Python interface for ANN.
Theano: an open-source numerical computing library for Python that allows you to efficiently define, optimize and evaluate mathematical expressions involving multidimensional arrays.
Caffe: a deep learning environment designed with expressiveness, speed and modularity in mind.
MATLAB MXNet: an open-source deep learning platform designed for efficiency and flexibility.Chainer: a flexible, intuitive and high-performance deep learning environment for Python.
DL4J: Deep Learning Library for Java, designed to support the creation of a wide range of deep neural networks.

Types of construction equipment and their purpose.
To develop the algorithm and software implementation of the AI from a wide range of construction equipment was selected a list of only the main types of construction equipment, presented in Table 2.These are just some of the many types of construction equipment used in the industry.Each type of equipment has a specific purpose and is designed to perform construction tasks more efficiently and effectively.
In the next stage of the work, a pool of open access datasets was formed, such as: Car Object Detection (118MB), Stanford Cars Dataset (2 GB), Construction Vehicle Images (1GB), Construction Vehicle (781 MB).Based on these datasets, our own, test construction vehicle image dataset was developed.
A dataset is a collection of data that is used for a variety of purposes, such as analyzing, exploring, and training machine learning models.In the context of AI and machine learning, having a high-quality dataset is essential for developing accurate and efficient models.
The purpose of a dataset is to provide a representative sample of data reflecting the realworld scenarios that a machine learning model is designed to analyze or predict.The dataset serves as input to the machine learning algorithm and is used to train the model to recognize patterns, make predictions, or classify new data.
Without a good dataset, a machine learning model cannot accurately learn the underlying patterns in the data, which can lead to inaccurate predictions or classifications.Moreover, having a diverse and complete dataset can help ensure that the model is robust enough to handle a wide range of scenarios and inputs.
Software development of an artificial intelligence model.
To implement a neural network-based AI model, a convolution layer architecture was used to analyze image data.
A convolutional layer is a type of layer in a neural network that works by moving a small matrix (known as a filter or kernel) over the input image and calculating element-byelement multiplications between the filter elements and the input data, creating a new feature map.This process helps the network learn local features of the image, such as edges, shapes, and textures.
The practical implementation of the model was carried out in Google Colab, a free cloud-based data processing and machine learning development platform.The platform provides access to the Jupyter notebook environment, which supports several programming languages, including Python.The platform also provides access to free GPUs and TPUs, making it easier for users to perform complex and resource-intensive computations.

Converged neural networks (CNNs) are a type of artificial neural network commonly used in computer vision applications. CNN training involves several steps:
1. Data preparation: First, a data set of labeled images is prepared.This data set is usually divided into sets for training, verification and testing.2. Initialization: then the CNN weights are initialized randomly.3. Direct propagation: CNN takes the input image and passes it through a series of convolution, activation, merging, and normalization layers to produce the output.4. Loss calculation: the CNN output is compared to the true label of the input image, and a loss function is calculated to measure the difference between the two. 5. Inverse propagation: The gradient of the loss function with respect to the CNN weights is calculated using the chain rule of computation.6. Weight update: CNN weights are updated using an optimization algorithm such as stochastic gradient descent or Adam, which adjusts the weights in a direction that reduces the loss function.7. Repeat: Steps 3-6 are repeated for multiple epochs (iterations over the entire dataset) to minimize the loss function and improve CNN accuracy in the validation and sets.8. Model evaluation: after the training process is complete, the CNN is evaluated on a separate set of tests to measure its accuracy and generalization performance.This process is iterative and involves tuning the hyperparameters of the CNN (such as the number of layers, filter size and the learning rate of the optimizer) to achieve the best performance for the task Figure 1.As practical experience has shown, the results of neural network training depend on various factors, including the quality of training data, network architecture, the optimization algorithm used, the amount of training data, the choice of hyperparameters and the amount of available computing power.
The goal was to train the network to minimize the difference between its predictions and the actual output, known as a loss or cost function.During training, the weights and offsets of the network are updated to reduce the loss of training data.If the network is over-trained, which means that it performs well with the training data but poorly with the new data, the model can be streamlined, for example, by adding dropout or weight reduction.
Ultimately, the performance of a trained neural network can be quantified using metrics such as accuracy, reproducibility, completeness.The best results depend on the specific problem to be solved and the desired performance tradeoff.

The results of the work
An AI prototype was created that allows you to determine the type of construction equipment from the video camera image Figure 2. In the first phase, the AI program was trained on a large data set of images of different types of construction equipment, such as excavators, cranes and bulldozers.
After implementing the neural network algorithm, the prototype program sees a new image from the camera, analyzes the image, and looks for certain visual characteristics specific to each type of equipment.For example, it may look for the shape of the equipment or the color of its paint.
The program then compares the visual characteristics found in the new image with what it learned from the dataset during training.By doing this, the program can determine what type of construction equipment is shown in the new image.
Finally, the software provides a response to the user with the identified equipment type.This information can be used by construction site managers to automatically track equipment usage, schedule maintenance, or verify the location of equipment.
In general, the AI program uses machine learning algorithms to recognize and classify different types of construction equipment based on visual features extracted from images.This technology can improve the efficiency and safety of construction sites, allowing managers to better track and monitor equipment use.

Conclusions:
The following problems arise when implementing AI algorithms: 1.The quality and quantity of data available to train AI algorithms can have a significant impact on their performance.Poor quality data can lead to biased models.2. Retraining occurs when the AI model is too complex and matches the training data too closely.3. Training some AI algorithms can be computationally expensive, requiring large amounts of memory and processing power.This may limit their practical use in systems with limited resources.4. AI models can sometimes produce results that are difficult to interpret or explain. 5. Safety and Reliability.This can be a problem because AI models can sometimes behave in unexpected ways.6. industries.7. Models based on neural networks (NS) are already actively used in the analysis of images, video, text, speech and other data.8. Collecting data for the NS is a complex task.If possible, it is recommended to take data not only specific to the task, but also to use public datasets.9. Learning small (up to 10M -100M teachable parameters) is possible in a browser (for example, Google Colab) or on 1 GPU.Learning the latest models is possible only on clusters with a large number of video cards.
5 Application of the system and directions for further research: 1.The system can automatically determine the type of machine in use, its current status and its location on the construction site, in order to increase the efficiency of construction work.2. Increase the efficiency of accounting and control of construction equipment at the site, which can increase work productivity and reduce maintenance costs.3. Increase the transparency of the construction process and improve resource management through more accurate recording and analysis of construction equipment usage data.
4. The ability to remotely monitor construction equipment and alert on emergency situations in real time, allowing you to quickly respond to problems at the construction site. 5. Reducing the risk of theft and theft of construction equipment through a more effective system of its accounting and control.6. Ability to improve planning of construction equipment use based on collected data on its use and efficiency.7. The AI system can track the operating time of each machine and provide real-time feedback to the operator.This allows operators to make adjustments to machine operations to reduce downtime.8.The artificial intelligence system can detect potential safety hazards, such as collisions or rollovers, and alert operators to take corrective action.This increases safety at the construction site and reduces the risk of accidents.
In conclusion, the application of artificial intelligence to identify the type of construction machinery using CCTV cameras has shown great potential.With the development of AI technology, CCTV cameras equipped with AI algorithms can accurately identify and classify different types of construction machines in real time.This information can be used to improve efficiency and reduce costs.The use of AI in video surveillance also opens up new possibilities for advanced data analysis and decision-making in construction.
Deep Learning Toolbox: provides algorithms and tools for creating, training and visualizing deep neural networks.Microsoft Cognitive Toolkit (CNTK): an open-source deep learning toolkit developed by Microsoft Research.

Fig. 2 .
Fig. 2. Results of artificial intelligence to determine the type of construction equipment.

Table 2 .
Type of construction equipment.