Application of computer vision technology in the development of ultrasonic repeller

. The issues that are nowadays identified during the implementation of the «Digital agriculture» project are considered. Directions of development of modern agriculture in Russia where digital technologies are being introduced are fixed. It is the Internet of things, robotics, artificial intelligence, and big data analysis. We have analyzed agricultural directions and scientific works where researches are doing and the technologies of computer vision are implementing. Scientific issues that are solved in plant growing by using computer vision are highlighted. Conclusions are made on the implementation of this technology in animal husbandry and fish farming. A device for ultrasonic repelling of synanthropic mammals with the possibility of detecting a synanthropic organism has been developed. The research on the influence of ultrasonic signals on mink behavior is conducted. Further ways of using computer vision in fish farming are defined for working with applied issues that can be solved exclusively with the help of deep learning neural networks.


Introduction
Agriculture plays a key role in providing the population with food and as a result, any country is interested in increasing the level of automation of agricultural processes to improve the efficiency of production. That is why nowadays the departmental project «Digital agriculture» is currently being implemented in Russia.
In this project, digital agriculture is agriculture where the modes of production are based on the usage of modern digital technologies (the Internet of things, robotics, artificial intelligence, and big data analysis) to increase productivity and reduce production costs.
Due to the fact that about 70% of Russia's territory is a zone of risky agriculture [1], animal husbandry, poultry farming and fish farming are actively developing. These branches of agriculture have a high level of the automation process that provides high productivity. As the increase in the animal's live weight primarily depends on controlled feeding, maintaining the required parameters of the microclimate, disinfection and filtration of air and a number of other factors, which ultimately depend on both the methods of keeping animal and the climatic characteristics of an area where it is bred.
The above processes are quite difficult to provide a person by hand that is why they are automated. The integration of robotic devices opens new horizons up as technologies that are based on computer vision, can continuously measure, analyze, calculate, and control various aspects of agrotechnological processes.

Computer vision in agriculture
Such researches are already being actively conducted, for example, the authors of the article [2] combined the possibilities of computer vision for sorting tomatoes. Having trained the resulting model, using 43 843 images and based on such tomato parameters as size and color, the authors achieved 94,6% elimination of low-quality products in the test mode.
The other research team [3] moved further and used the technology of 3D image analysis. They called their project "The Potato Operation" and referred it to agricultural robotics. The main idea of the project is to automate the detection of viral diseases of potatoes, analyzing its pulp samples. Then the team used the collected data or if being more precise the developed database was used for certification and labeling of potatoes. The result of this research is the robotic apparatus.
The next research team [4] delved more into the extended theory. In the review article, they consider agriculture as a heterogeneous domain, which means the nonhomogeneous structure from a mathematical point of view. In the article, they focus attention that despite the huge amount of works only a small number of researches draw attention to the mathematical structure of the process. Consequently, they conducted extensive theoretical research that became a library of feature descriptors, detecting defects based on images of fruits and vegetables. These actions led them to create a specific model and a method of pre-processing data and their analysis for improving the accuracy of computer vision.
The next group of authors in the article [5] engages in the phenotyping of grain crops which means evaluation immediately after collection. Tried and tested patterns images in the technology of computer vision helps to estimate the quality of the incoming grain ( fig.1.).  The group of authors of the article [6] uses the computer vision technology to map small farms and separate them from commercial ones on maps.
As you can see from the analysis of the articles above, in plant growing, this technology is used almost everywhere, from mapping an area to product control during and after assembly.
In animal husbandry, the success of computer vision is less significant. For example, in the article [7] the authors claim that such technologies will play an important role in the next generation of dairy farming processes. Also, they note that these technologies will be used in monitoring processes, feeding, grazing and milking of animals. Reasoning and analyzing data they made a conclusion that precision livestock farming will require computer vision technology that can work in more heterogeneous conditions than in plant growing. It leads to the need to create other mathematical structures for computer vision technology. Also, we should notice that there are many more articles about the usage of computer vision in plant grooving than in animal husbandry.
Unfortunately, there was found none article about the usage of computer vision in fish farming ( fig.2). Perhaps, it may be due to necessity even more heterogeneous conditions [8][9][10]. Minks systematically destroy fish in fish traps thereby causing damage to the products. An important distinctive feature is the extermination of minks is prohibited at the legislative level. That is why the solution to the problem can be the development and implementation of an automatic mink repellent system using ultrasound. Of course, the system will be implemented, based on computer vision technology [11][12][13][14].
Typical experimental image processing procedure, showing the RGB image (A) and YUV image (B), U, V intensity profile along the white line (C), U and V components intensity images (D and E), the subtraction result of (V − U + 5) (F), the histogram equalized image (G), the result image of background subtraction (H), which was used to calculate CVFAI, the binary result (I).

Computer vision and deep learning
Understanding the behavioral characteristics of commercially farmed fish plays an important role in the effective management of a fish farm and represents a major advance in aquaculture research. We need to understand that when breeding fish in captivity operators should pay attention to its welfare, behavior, feeding, and a number of other factors that result in a high-quality final product. But we are talking about the factors which are "inside" of the technological process of fish farming, while external factors are important too. The external factors include the protection of fish from diseases, parasites and dangerous synanthropic organisms (rats, minks, gulls, and the like, depending on the region). Video surveillance technologies are already widely used by many commercial fish farms to optimize the processes associated with feeding fish. Specifically, to monitor the feeding of fish, to record the reaction of the fish feeding and volume of waste. Such observation helps to optimize the amount of feed, supplied to fish during the specified feeding periods. Also, it corrects the time of feeding, the speed of fish's feeding and the amount of feed and so on. Basing on collecting data, the operators can reduce the amount of feed waste, which is economically profitable for the enterprise and the environment.
Video surveillance is also widely used, both for research purposes at aquaculture sites, and from the commercial side. However, the analysis of the material obtained during the observation always plays a key role in solving problems and unfortunately it is most often done manually. The use of video surveillance technologies for employees of commercial enterprises is popular and it has great demand since they can reduce the number of workers which are responsible for monitoring the situation at the enterprise and have high-quality recorded material about the situation at the enterprise online. However, such technologies have a number of disadvantages, namely: 1. Video images may be blurred.

2.
If the light is reflected from the water surface, the video quality may deteriorate.

3.
A video recording must be analyzed manually by a person. 4. Recording in night flames and /or muddy water is ineffective. As we can see, the integration of digital technologies such as computer vision and deep learning technologies can eliminate these shortcomings. Most of the work about the analyzing video images using digital methods in fish farming has focused on using video cameras with a stereoscopic system for determining fish biomass with the integration of artificial neural networks to correct lens distortions in stereo images. Such innovation made it possible to provide more accurate of the fish's shape and adjust the estimate of their total biomass. In addition, methods of preprocessing and additional image segmentation were used. This was done primarily to detect fish entangled in fish traps or to estimate the size of an individual. In addition, based on the collected data, predictive modeling of the plant design was carried out in order to increase its efficiency in real conditions. Therefore, no attempts have been made to automatically analyze the situation with synanthropic organisms that pose a threat to fisheries products using computer vision technology and deep learning.
In this regard there is a huge area for researching the development of a number of computer software that allows you to computerize the technological processes of fish farming. Specifically, this article is focused on the development of software that allows you to analyze video materials for the presence or absence of movement of synanthropic mammals (minks) on the territory of a fish-breeding enterprise. This system has the potential to improve farm management by alerting operators to critical behaviors online, making long-term forecasts, and making an overall assessment of fish welfare. The existence of video surveillance systems at many fish farms pieces of evidence that it is possible to minimize equipment costs for implementing an automated video analysis system However, there are still a number of difficulties. Due to the high variability and complex color distribution in fish farming processes (feeding fish, moving, providing protection at night from synanthropic pests), the analysis and General calibration of the color signal of any computer vision system used to assess the quality of color is a necessary condition for objective and consistent analysis.
There are two methods of color definition that are used to the computer vision system: 1. The CIE using the computer vision system (CVS). The method is based on the analysis of digital photos.
2. Polynomial transformation procedure proposed by the sRGB standard. The last one is used to animal husbandry to evaluate the appearance of the final product (for example, estimates of ham fat and connective tissue). The results of using this method showed high precision of definition. For example, a polynomial transfer matrix with dimensions [3 × 20] has given an average simulation accuracy of fewer than 2.2 units. Using the sRGB conversion, the recognition variability is in the range (8.8 ± 4.2). While a correctly calibrated laboratory installation with the computer vision system (CVS), the device which is implemented using an inexpensive digital camera, demonstrated reproducible color signals in a wide range of colors that can accurately determine the areas of interest and allowed extracting quantitative information from the entire surface of the analyzed object with high accuracy. The extracted color and morphological features showed the ability to correctly characterize the analyzed surface As a result, there is a proven procedure based on the sRGB standard and newer and cheaper CIE with using the CVS. After a series of logical conclusions, we have chosen the last one.
Over the past few years, we can observe an increased growth of various methods and technologies related to the field of visual analysis of food quality, which in total amount to the fact that they extract an information array about the analyzed surface and compare them with many characteristics (images) in the library. Collecting an information array consists of collecting characteristics such as color, texture, shape, and size. In rare cases, something else is analyzed. These simple appearance characteristics allow us to analyze and interpret task-specific data with high accuracy, objectivity, and speed when evaluating the quality and classification of many foods. In fact, these appearance properties correlate well with many physical, chemical, and sensory indicators of the quality of the final product. Therefore, they can also be used to automate the control of the territory where fish breeding and monitoring of synanthropic pests take place by incorporating effective image processing algorithms into computer vision systems (CVS).
Over the past decade, there has been an explosive growth in both the variety of methods and the range of applications related to food image analysis, which can extract various image characteristics from food surfaces, such as color, image texture, shape and size, among others (Zheng, Sun & Zheng, 2006). These simple appearance characteristics made it possible to analyze and interpret task-specific data with high accuracy, objectivity, and speed when evaluating the quality and classification of many foods. In fact, these appearance properties correlate well with many physical, chemical, and sensory indicators of food quality (Hutchings, Luo, & Ji, 2002). Consequently, they also can be used to automate ham quality control and evaluation by implementing efficient image processing algorithms in industrial computer vision systems (CVS).

Principle of operation of the device
Guard dogs, safe traps with bait inside, and other folk methods were usually used to scare Minks away from fish traps on the farm.
Animals are more sensitive to a certain range of ultrasonic vibrations. It depends on belonging to a particular species. That is why it is necessary to use the range of oscillation, which has the most irritating effect on mammals. Also, it should be a short term not to be addictive.
Basing on statistic data of the establishment, every day at night, traces of a mink were fixed in the immediate vicinity of the fish traps, also there were fixed traces of damage to valuable fish species (minks every night caught fish from traps leaving the traces of dragging fish in the snow). Monthly damage of valuable fish species was estimated at up to 300 individuals. The developed repellent system is based on turning ultrasonic speakers on in the moment when a mink is close to fish traps. A carried board Nvidia Jetson TX1, a highresolution webcam and a series of ultrasonic speakers were used for the realization of the project.
Also, a deep learning neural network and the platform for the development of Jetson TX1 were developed to recognize minks and turn the repellent system on.
As a result, the high-resolution webcam realizes the monitoring of the territory (from the side of the forest, where minks come from). If there is some movement, it sends 7 photos with a resolution 640х480, the carrier board processes them and passes ones through the deep learning neural network of the InceptionV3 architecture. If the neural network finds out an object that looks like a mink, that through the carrier board it sends a signal on a relay and the ultrasonic speaker turns on 30 seconds. Consequently, a mink is repelled and a threat to animal life and damage from its impact on farmed fish are excluded.
The neural network is trained on images of minks from the Internet. Also, the computer vision library with an open-source code OpenCV was used.

The research
As a result, the research was conducted on the possibility of repelling minks using an automatic ultrasonic repelling system. During the research, the automatic system makes ultrasonic vibrations with a frequency of 2050 kHz and with sound pressure level at a distance of up to 2 meters up and to 100 dB. The scheme of the device is shown in Fig. 4.;  The device consists of a video camera with a built-in motion sensor (1), a microcomputer that processes the incoming video signal (2), and a speaker that broadcasts low-frequency ultrasound of the required frequency (3).
The device was placed near to fish traps to get all required data about the influence of ultrasonic signals on the mink behavior. It worked during the dark hours of the day. The video camera was placed in this way to fix any fact of approaching the mink to the traps for possible damage to the fish.
Additionally, during the day, the camera could fix mink prints near the fish traps. Measurements of the maximum removal of mink prints from the device were made Производились замеры максимального удаления следов от прибора.
As you can see, there are no mink prints near the fish traps. There are mink prints only outside the territory of the fish traps. Also, there are not any traces of dragging fish in the snow. During the experimental researches, a mink approached at a distance of no more than 1,4 meters from the fish traps when the device had the ultrasound frequency was from 20 kHz to 50 kHz. Increasing the frequency level of the device was looped, that is, the device constantly raised the frequency from 20 kHz to 50 kHz, then reset to 20 kHz and the cycle was repeated. It was not possible to fix the changes in the mink approach distance to the device depending on the signal frequency. The low temperature in diapason from -30 0 С to -35 0 C did not allow to fix mink behavior depending on the ultrasound frequency. Generally, we would like to say that the indicators of the device during the process of repelling mink were changing slightly.

Conclusions
In this article, we defined the relevance and applicability of computer vision technology in fish farming. It is necessary to implement the algorithms of computer vision and the models of deep learning that can the accuracy, generalization and easy customization to the needs of some enterprise. It can increase the efficiency of their application. As an example, the presented device should accurately distinguish a mink from other animals and respond accordingly. The usage of computer vision is not popular in fish farming yet, but this industry has applied problems that can be solved only by deep learning neural networks.