Smart and Sustainable Surveillance System

. Safety and security are major concerns in the modern day. People and organizations can employ security mechanisms to safeguard their property for their homes or commercial enterprises. Present security systems involve the utilization of Assorted Sensors in cameras for video surveillance. This paper aims at providing one such idea to ensure the protection and security of one's property. This technique performs Face Recognition as an authentication procedure when a new face is detected by a snapshot. We propose to present a sensible , smart and sustainable Closed - Circuit Television (CCTV) camera with intrusion detection using the LBPH-Local binary pattern histogram algorithm, SIM-Structural Similarity Index Measure, Haar Cascade Classifier, and TKinter. By utilizing intrusion detection, CCTV cameras record real-time videos and process the video at the time of recording to search out the unwanted people arriving within the surveillance area. Our GUI has different buttons supported with features. Adding DL support would create broad scope in this paper such as with DL we would be able to add up much more functionality. We can have future enhancements on this paper such as creating Portable CCTV, Deadly weapon detection, Accident Detection, Fire Detection.


Introduction
Closed-circuit television (CCTV) is a popular security technology that is used to monitor areas and detect crime.However, CCTV systems are limited in their ability to identify criminals.This is because CCTV systems typically only record video footage, which must then be manually reviewed by security personnel.This can be a time-consuming and labour intensive process, and it can also lead to missed opportunities to identify and apprehend criminals.Smart surveillance systems use advanced technology such as artificial intelligence, computer vision, and machine learning to improve the capabilities of CCTV systems.Smart surveillance systems can automatically analyze video footage and identify suspicious behavior or potential security risks.This allows security personnel to focus on critical areas and potential threats, and it can also help to identify criminals more quickly [19].
OpenCV is a well-known open-source computer vision library that can be used to develop smart surveillance systems.OpenCV provides a number of tools and methods for processing images and videos, including object detection, face recognition, and license plate recognition.These methods can be used to automate the analysis of video footage and identify suspicious behavior or potential security risks [20].For example, OpenCV can be used to develop a smart surveillance system that can automatically detect people who are carrying weapons, or to identify vehicles that have been reported stolen [16] and [17].This information could then be used to alert security personnel or law enforcement.In addition to object detection, face recognition, and license plate recognition, OpenCV can also be used for a variety of other tasks, such as activity recognition, crowd counting, and scene understanding [18].Smart surveillance systems can also be integrated with other technologies, such as artificial intelligence and machine learning, to provide more advanced capabilities.

Literature survey
The Low-Cost Smart Security Camera with Night Vision Capability Using Raspberry Pi and OpenCV is the work [1].The paper uses night vision to detect smoke and people in a warehouse and to send photographs of what is seen to the user's email.The dataset in this case consists of the movies and photos taken by the low-light-capable, portable, and compact Raspberry Pi camera, which can be simply connected to the Raspberry Pi board [2].The algorithms employed are the Background Subtraction method, the Human Detection algorithm, and the Haar Cascade Classifier.Smoke and human detection accuracy are 83.56% and 83.33%, respectively, accurate.With the help of OpenCV and the Raspberry Pi 3, Thinesh Prathaban and colleagues developed a vision-based home security system.The OpenCV implementation used the Haar-Cascade algorithm in conjunction with a background subtraction technique.With a passing rate average of 100% versus 76%, this OpenCV motion detection system was demonstrated to be more efficient than the developed PIR motion detection system [3].
A Raspberry Pi-based indoor intrusion detection and filtering system has been developed by Umi Najiah Ahmad Razim and her team.The current state of surveillance technology still has several shortcomings, particularly in terms of cost and alarm system adaptability.In this paper, a Raspberry Pi-based intelligent home surveillance system is proposed.The camera included into the Raspberry Pi board records videos and takes pictures.By performing unit testing for clip recording, the accuracy for motion detection is estimated to be 93.7%.
Face Recognition Using OpenCV Based on IoT for Smart Doors was the focus in the paper [4].The term "smart door" refers to a special smart surveillance system that is implanted to the door in order to maintain security for a specific room.Each database subdirectory now contains 45 photos of every person in it.They have used OpenCV2, the LBPH method, and the Haar Cascade classifier, all of which are open-source technologies.According to OpenCV, authorised person recognition accuracy is 89.3% [5].MVD Prasad and his team developed security systems based on video surveillance utilising OpenCV and an Arduino Uno.Security systems based on CCTV are not real-time because the owner learns about the occurrence after it has occurred.This study attempts to develop a security system based on video surveillance that can recognise unauthorised individuals in real time and send out alerts.According to the results, the detection rate for this video surveillancebased system is 86%.Smart Surveillance was proposed in [6] utilising Motion Analysis, Facial Landmark, and OpenCV.Putting the Local Binary Pattern Histogram (LBPH) technique to use when working with low-resolution photos.The goal was to use LPBH and Haar cascade to more accurately and efficiently check a person's face.The dataset was processed using the Local Binary Pattern Histogram (LBPH) feature vector and Haar cascade training methodologies.Using Local Binary Pattern Histogram (LBPH) feature vectors and a Haar cascade, the author was able to achieve accuracy rates of 85%-95% [7].
Ashok Kumar and his team members worked on the Advance Intelligent Video Surveillance System Using OpenCV.To implement an intelligent open-source tool that can work for people based on facial recognition, motion detection can be used to stop spoofing the employee [8].The objective is to implement the Intelligent Video Surveillance System which reduces both, the power consumption and the storage needed by only monitoring data when a human is detected.Methodology model, which is PCA, and SVM, and the better precision and easy understanding we are using the LBPH Algorithm.Accuracy rate of the Advanced intelligent is reported as 93% accurate with the PCA technique on the preprocessing [9].Fatima Mahmood and her team suggested employing deep learning methods to develop an intelligent exam supervision system [5].Due to human limitations in handling students in testing facilities, the Automated Invigilation System can help prevent mistakes from occurring during exam supervision.An object detection algorithm is faster RCNN.Real-time data was utilised to create the study's dataset.98.5% of the time, cheating was discovered [10].
The Smart Surveillance Vigilant Detection and Notification System is recommended in [11].The smart surveillance system's goal is to alert the user when impending threats are found.In the vicinity of the surveillance region, the dataset is gathered, and a brief video clip is captured.The intruders' mobility is the methodology, and image processing algorithms are used to find human faces when they are present [12].More than 90% of the system is found to be efficient.Smart Surveillance Camera Using Raspberry Pi 2 and OpenCV was the idea put up by Suthagar and his team.This system's goal is to be reasonably priced and equipped with night vision capabilities utilising a raspberry pi.The information is gathered from a surveillance area that includes video and still photos.Methodologies are Raspberry pi2 and pi camera using OpenCV also face detection using Harr Cascade classifier.Smart Surveillance Camera using Raspberry Pi 2 and OpenCV has an 83% accuracy rate.
The team led by Arun proposed smart surveillance.Real-time alerts, automatic forensic video retrieval, and situation awareness are the three main goals of this technology.The smart surveillance has an accuracy rate of roughly 86%.Raspberry Pi-based smart security surveillance system was proposed by Umesh and his team.There are still a number of issues with current surveillance technologies, especially in terms of cost and alarm system adaptability [13].The Raspberry Pi board's built-in camera captures images and video.PIR sensor, Pi, and Raspberry Pi.Sandeep Singh and his team members suggested Smart Surveillance use Security Camera with Night Vision Using Open-CV.They used LBPH Face recognition algorithm to predict the Person.They developed front-end platform using ReactJS and CSS.The Packages are Open-CV, NumPy, Tkinter [14].The accuracy is about 83%.Hritika and her team members suggested Smart Surveillance System.The suggested system is a Python GUI application created with the most up-to-date programming language in the "Computer Vision" field of computer science, which is rapidly expanding.In this paper, a smart surveillance system that leverages OpenCV for object and face identification is described.The accuracy is about 89% [15].

Problem statement
Current surveillance systems are often manual and error-prone.To address these challenges, we propose to develop a smart surveillance system using OpenCV with a GUI application.The system will automatically detect and track objects or events of interest in real-time, making it more efficient and accurate than traditional systems.

Objectives
1. To automatically detect and track objects of interest in real-time.2. To Create a user-friendly GUI application that allows operators to easily configure and customize the surveillance system, including setting up monitoring zones, adjusting detection parameters, and reviewing captured events 3. To recognize faces of individuals in the video stream and match them against a database of known faces.

Architecture of the proposed work
Figure 1 depicts the proposed architecture diagram.The first thing we start is to build the base structure of the proposed method.We built a graphical user interface with the help of Tkinter library, which is a standard GUI toolkit for Python.Tkinter provides a wide range of built-in widgets (such as buttons, labels, text boxes, etc.) and functions to create windows, handle user input, and manage GUI components.This allows you to create a visually appealing and cohesive user interface that aligns the overall design.Next, we use OpenCV methods to write the code for all the features present in the interface.The "cv2" module in OpenCV is the main module that provides a wide range of methods and functions for image processing, computer vision, and video analysis tasks.Some of the key methods and functions from the "cv2" module that we have used for implementing the features in our paper are specified below.Object For face detection, we made use of OpenCV which provides pre-trained models for face detection using methods like the Haar-cascade classifier and local binary search pattern histogram which can be used for identifying authorized or unauthorized persons in the surveillance footage.For video recording we made use of OpenCV video recording methods to capture called "cv2.VideoWriter".The saved video frames from the surveillance footage, which can be used for recording the footage or storing evidence.We make use of the above methods for building up the program for the features of our interface application.After all the features program is written we import them to the main.pyfile which is the interface file.Then we run the program in python idle to get the interface.

Monitor
The monitor module enables the user to check on the things which are stolen from monitored frame.We make use of two python file namely "find noise" and "spot_diff" to detect any motion and find any differences in consecutive frames captured by our webcam.Motion has been recognised if there are more contours than a predetermined threshold (5 in this case).The spot_diff() method is called to compare the frames and find differences if the motion lasts for more than 4 seconds.The function also tracks the length of motion using a timer.Using the structural_similarity() function from the skimage.metricsmodule, the spot_diff(frame1, frame2) function compares the structural similarity (SSIM) between the two input frames.Then, using OpenCV methods, its thresholds the SSIM score to create a difference image, finds contours in the difference image, and creates bounding boxes around the contours on the original frame.Beepy is also used to play a beep sound.

Identify
Using OpenCV cv2.VideoCapture function, it first takes video frames from the default camera.Using cv2.CascadeClassifier, it loads the pre-trained Haar cascade classifier for face identification via the "haarcascade_frontalface_default.xml"file.The IDs of the individuals in the "persons" directory are then mapped to their corresponding names in a dictionary created by the programme called labels list.This is accomplished by removing the IDs and names from the filenames of the photographs in the "persons" subfolder.Using the cv2.face.LBPHFaceRecognizer_create and recog.readfunctions, it loads the pretrained LBPH face recognizer from the "model.yml"file.To process video frames continually, it enters an infinite loop.It uses the cv2.cvtColor function to turn each frame into a grayscale image.Using the coordinates of the detected face, it extracts the area of interest (ROI), which is the grayscale face picture, from the frame.In order to identify the face and generate the anticipated label and confidence score, it sends the ROI to the LBPH face recognizer's predict function.This procedure is then repeated for every detected face in the frame.When the 'Esc' key is pushed, the loop is broken, and the function is ended.Utilising cap.release() to release the camera and cv2.destroyAllWindows() to close every OpenCV window, respectively.

Visitor detection
This in out visitor's detection helps in finding out the people passing through the restricted area especially where permissions for person is not permitted.Using the OpenCV cv2.VideoCapture function, the code records video frames from the default camera (camera index 0).It uses cv2.absdiff to compute the absolute difference between two successive frames after reading them from the camera.The difference image is then blurred with cv2.blur on a 5x5 kernel to lessen noise.Utilising cv2.cvtColor, the blurred image is transformed to grayscale.Using cv2.threshold with a threshold value of 40, a binary threshold is applied to the grayscale image to produce a binary image in which motion areas are represented by white pixels.Utilising the retrieval method of cv2.findContours, contours are found in the binary image, the contour approximation approach and cv2.RETR_TREE cv2.CHAIN_APPROX_SIMPLE.
Using cv2.contourArea and the max function, the largest contour-which symbolises the detected motion is found.Using cv2.rectangle, a bounding rectangle is created on the original frame surrounding the observed motion area.The x-coordinate of the bounding rectangle is used to determine whether the motion is left or right.The x-coordinate denotes movement in one of two directions: either towards the right (if it is larger than 500) or the left (if it is less than 200).The programme uses cv2.waitKey with a 1 millisecond delay to wait for a key occurrence.If the key "ESC" (key code 27) is hit, all OpenCV windows are closed using cv2.destroyAllWindows, the video capture is released, and the loop ends, ending the function's execution.

Recording
This function uses cv2.VideoCapture to record video from a camera (device index 0), and cv2.VideoWriter to record the video frames into an AVI file format.Using datetime, the AVI file's filename is produced depending on the current date and time.Now().strftime("%H-%M%S").The cv2.putText method, which adds text to the frames, annotates the recorded video frames with the current date and time.The current date and time, retrieved from datetime, are used to format the text.The text has the font type cv2.FONT_HERSHEY_COMPLEX, the font scale of 0.6, the text colour of (255, 255, 255), and a thickness of 2 pixels.It is located at the coordinates (50, 50).now().strftime("%D-%H-%M-%S").Using the out.Write(frame) function, which records each frame to the video file, the recorded video is written to the AVI format.

Noise detection
Noise or motion feature is used for detecting any motion in the monitored frame.We'll discuss about both monitoring the whole frame or a given geometric rectangle shape.The first thing we do is use the cap.read() function to grab two consecutive camera frames, which we then save as frame1 and frame2, respectively.Afterward, use cv2.absdiff(frame2, frame1) to get the absolute difference between frames 1 and 2. This generates a grayscale image that emphasises the variations between the two frames.cv2.cvtColor(diff,cv2.COLOR_BGR2GRAY) converts the difference image to grayscale.We use cv2.blur(diff, (5,5)) to apply a blur filter to the grayscale difference image to reduce noise.By using cv2.threshold(diff, 25, 255, cv2.THRESH_BINARY), you can threshold a blurred difference image to produce a binary image where pixels with intensity differences above a specific threshold (25 in this case) are set to 255 (white), and those with differences below the threshold are set to 0 (black).Then, usingcv2.findContours(thresh,cv2.RETR_TREE, and cv2.CHAIN_APPROX_SIMPLE), find the contours in the binary threshold image.Simply said, contours are the limits of the motion-detected areas in the threshold image.The function designates the contour with the largest area as the motion region if contours are identified (i.e., Len(contr) > 0).Using the functions cv2.rectangle() and cv2.putText(), the motion region is confined by a rectangle with the text "MOTION" overlaid in the topleft corner of the frame.

Interface of the smart surveillance 4 Results and discussions
The user interface for smart surveillance using OpenCV is depicted in the image below.The interface of the surveillance system comprises of a logo and buttons that control its automated functionality.Each feature has a unique icon that accurately represents its purpose.The features include rectangle, in and out, monitor, identify, record and noise.In the pages that follow, we'll go over each feature's outcome.

Monitor
When nothing is stolen at first, a frame is taken from the surveillance camera.This is used as a standard reference or starting point for comparison.The system continuously checks for motion in the video frames or images.Once the motion ends, the system captures the current frame, which may contain the stolen item, and the frame that was captured initially when no item was stolen.The system then applies a Structural Similarity (SSIM) algorithm to compare the two frames to find the stolen item.

Identify
Detect faces in frames captured from a video feed, using techniques like Haar cascades.Once a face is detected, the LBPH model is used to extract features from the face image and compare it with the learned features of known faces in the dataset.The algorithm computes the similarity between the extracted features and the stored features of known faces, and based on predefined data, predicts the identity of the detected face.

In and Out
This function concentrates on distinguishing between left and right directional motions.The system tracks motion to see if it terminates towards the opposite side when motion has been detected on either side.The system takes a picture of the area where the motion happened and saves it in the visitor's folder when the motion is identified and terminates towards the opposite side.

Recording
In order for intelligent surveillance systems to collect and retain meaningful data that may be used for a variety of purposes, including monitoring, analysis, and investigation, recording is crucial.Video captured by surveillance cameras, for instance, can be used to recognise people, track their travels, and study their behaviours.Depending on the system's architecture and requirements, recorded data in smart surveillance systems can be saved locally on a storage device or on the cloud.Authorised people, such as security or law enforcement officers, can access and analyse the data to spot suspicious activity, investigate incidents, or offer evidence in legal processes.

Noise
Smart surveillance systems frequently have motion detection, which looks any movement in the area being watched.Numerous technologies, including video cameras, motion detectors, and infrared sensors, be used to do this.The system can start recording, notify security staff, or sound an alarm when motion is detected.In locations like parking lots, office buildings, and public spaces, motion detection can assist you in keeping an eye on and spotting unusual activities.No motion detection, which finds the lack of motion in a monitored area, is the opposite of motion detection.It's especially useful in circumstances when a person or thing should be present, like a safe or a storage space.The system has the ability to send an alert or trigger an alarm to alert security personnel if no motion is seen after a predetermined amount of time.These datasets can be used to test and train computer vision models in graphical user interface and OpenCV-based smart surveillance systems.The GUI offers an interactive interface for controlling the surveillance system and can be used to view analytical data.We must collect images or videos pertaining to your smart surveillance programme.Then, if required, add the proper object classes and bounding boxes to each image or frame.Tools for annotation like Labelling, RectLabel, and VGG Image Annotator (VIA) can help with this stage.Before using the dataset with OpenCV, it might need to be pre-processed after it has been labelled.

Conclusion and future enhancements
The smart and sustainable surveillance system using OpenCV based on GUI is a powerful and flexible solution that can be enhanced in a number of ways to improve its effectiveness and reliability.The system uses OpenCV, a popular and powerful computer vision library, to perform advanced object detection and tracking tasks.The GUI provides an intuitive interface for users to interact with the system, making it user-friendly and accessible for operators.The system can be enhanced by integrating more advanced machine learning algorithms, integrating with IoT devices, being robust against environmental factors, incorporating privacy protection measures, and detecting deadly weapons or fire.
The integration of advanced machine learning algorithms, such as YOLO or SSD, can provide higher accuracy and faster processing speeds compared to traditional Haar-cascade or LBPH methods.Integration with IoT devices, such as sensors, actuators, or smart cameras, can enable more advanced surveillance capabilities.The system can also be enhanced to be robust against environmental factors such as changes in lighting conditions, weather conditions, or occlusions.