Dynamic texture recognition under adverse lighting and weather conditions for outdoor environments

. Recognizing dynamic patterns based on visual processing is significant for many applications. In this paper dynamic texture recognition focuses on outdoor scenarios where a crisis event might occur (i.e. fire in a forest, floods/flooding etc.) Real outdoor scenes may include the objects with dynamic behaviour due to illumination, blurring, or weather conditions effects. Under bad weather conditions the imaging systems is degraded and produce low visibility images. In this work precipitation artefacts and lightning effects for dynamic texture analysis were studied. Experimental results show that the proposed method of weather and adverse lighting effects compensation is feasible and effective for video-based dynamic texture analysis under bad weather conditions.


Introduction
Nowadays dynamic textures (DT) recognition plays an important role in different computer vision community tasks in a variety of fields such as urban and forest scenes. Most of the existing methods in the field of computer vision are based on the assumption that the weather in outdoor images or videos is clear. Real outdoor scenes may include the objects with dynamic behaviour because of possible varying illumination, blurring, or weather conditions. Under bad weather conditions the imaging systems is degraded and produce low visibility images. Such effects may significantly degrade the performance of outdoor vision systems which relies on image/video. The removal of these effects provides accuracy increasing of video based computer vision algorithms.
The goals of the dynamic textures recognition can be different. The dynamic scenes analysis in video is a very useful task especially for the detection and monitoring of natural hazards such as floods and fires. The analysis of such videos is considered of utmost importance during natural disasters, since it can improve situational awareness by providing early detection of floods and fires to prevent or reduce damage from such emergencies.
In this paper we target the problem of classifying such DTs as water, smoke and flame.
Video based detection algorithms rate significantly depends of the varying illumination or weather conditions. One of the most important factors is illumination level. The influence of shadows and time of day on performance of video detection systems at signalized intersections can provide from 2 to 36% false alert errors [1]. Lighting variations are a challenge in face recognition. In paper [2], focusing on the illumination change challenge, a deep network model is proposed which takes into account both visible light image and near-infrared image to perform face recognition. Decreasing of computer vision algotytms accuracy due to illumination changes is shown for various tasks [3,4]. Rain and snow detection algorithms are usually classified into three categories: optical approaches [5,6], spatial-domain approaches [7], and frequency-domain approaches [8]. Usually rain and snow compensation method consists of two stages: precipitation detection and precipitation removal from video images. For precipitation artifacts detection Kim et al proposed a method based on a Gaussian distribution model and block-matching algorithm [9]. Weather detection stage may extract weather cues such as sky, shadow, reflection, contrast and haze [10,11]. Shen et al label snow in videos and extract the alpha value of the snow particles by alpha matting and general image contrast [12]. As one of the state-of-theart motion-based rain removal algorithms, Chen et al. proposed a rain-pixel recovery algorithm based on motion segmentation of dynamic scene [13].

Detecting and extracting precipitation and lightning artifacts
In this section, the most suitable approaches for decetion and extraction weather and lightning artefacts are presented. One may refer the weather effects (Section 3.1) and adverse lightning (Section 3.2).

Weather conditions
There are the following precipitation types: haze, fog, rain and snow. Weather effects due to precipitations presistance in outdoor scenes may gives various effects in video images accoding to its type and intensity. For example, a rainy scene is typically composed of many raindrops -each with a size much larger than visible wave lengths, which causes the light to spread in a specific way. Closer raindrops, which have a projected size larger than a pixel, are individually visible, and further ones can only be seen in a macroscopic view as participating media [14]. Weather scenes include several effects such as streaks due to the velocity of raindrops and camera's exposure time, individual snow or rain particles and visibility is decreasing until details blur into a fog like appearance. To compensate the weather effects at the first step, the block-matching algorithm for each block is performed between adjacent frames in order to find blocks similar to the block that has weather artifact pixels. Size of blocks is 15-30 pixels with search radius defended in 5-20 pixels according to rain and snow particles speed. For moving blocks HOG -features of rain and snow are estimated. In weather compensation stage the video images were prossesed by VirtualDub, a free and open-source video-capture and video-processing utility for Microsoft Windows written by Avery Lee. Temporal Smoother, 2D Cleaner, MSU Denoiser, Smart Smoother, MSU Smart Sharpen, Sharpen filters were used for weather artifacts removal. The proposed removal algorithm is based on many filters, so it is very efficient for particles and tracks deleting. After filters application Dark Channel prior was used for haze removal. Dark Channel prior is based on atmospheric scattering model [15]. So haze-free image is recovered by: where I C (x, y) is the observed intensity in pixel with coordinates (x, y) in RGB-channel C, A C is the global atmospheric light in each of RBG channel, map(x, y) is the medium transmission describing the portion of the light that is not scattered and reaches the camera computed by: where =0,75 is the scattering coefficient of the atmosphere in pixel (x, y), (x, y) is a local patch centered at pixel (x, y). Finally, median filtering operator is performed by Eq. 3 for visibility enhancement and flickering artifacts removal.
The above precipitation effects removal algorithm, has achieved its high efficiency for snow, rain, haze and other weather artifacts, deleting them from video images.

Adverse lightning compensation
The adverse image histogram shown in Fig. 1 provides identification of adverse lightning images properties such as: most of the pixels have small intensity values, intensity range constriction and low contrast. For illumination effects compensation and visual quality enhancement images need to average pixel intensity increase, expanding the range of brightness, increasing image contrast and eliminating influence of the additive noise. The proposed algorithm employs Multi Scale Retinex with Color Restoration (MSRCR) calculated using Eq. 3 [16]. ,

DT recognition algorithm 4.1 Dynamic textures
In the context of surveillance, recognizing dynamic patterns is of significance to isolate activities of interest (e.g. fire) from distracting background (e.g. windblown vegetation and changes in scene illumination). Dynamic textures can be defined as visual phenomena that exhibit spatial and temporal regularity. A complex approach for early detection of floods and fires by prevent or reduce the damage from such emergencies primarily requires the detection of such categories of DT like flame, smoke and water.

DT recognition algorithm
In detail DT recognition algorithm was considered by the authors at previous work [17]. The first step of the proposed algorithm is motion estimation based on Block-matching SAD criteria and the optical flow for the information about the local and global motion vectors. Block-matching algorithm evolves 2 to 5 frames of video sequences according to experiments for objects in different Categories show various motion speed. The next algorithm step is colour, facial, entropy features estimation. The fourth step is Laws energy approach for edges, spots, ripples and wave texture features detection. The next step is ELBP descriptors computing. Then histogram approach was applied. Chi-square distance, histogram intersection distance, Kullback-Leibler divergence, and G-statistic are usually used during classification stage. In this research, the histogram intersection was chosen for histogram comparison. Regions clustering based on boosted random forests.

Experimentation
For experimentation results Dyntex dataset [18] and VMOTE dataset [19] with such DT objects like smoke, flame and water were used. Examples of video frames are depicted in Fig. 2. In this video sequences precipitation and lightning effects were added artificially. For adverse lightning imitation in this work Gamma correction, additive Gaussian noise and impulse noise were applied successively. Precipitation images imitation was generated using Gaussian filter. Video sequences with natural snow, rain and adverse lighting from the real surveillance cameras were downloaded from YouTube. In addition, the representativeness of the test set is increased by a video sequence, which the authors themselves recorded at night. Compensation algorithm experimental results are shown in Fig. 3.  True recognition for smoke in adverse lighting conditions is degraded to 70%. False rare rejection and false alert errors significantly increase to 29.7% and 22.5% respectively. After adverse lighting and weather effects compensation algorithm work true recognition of DT regions increases to 94,3%. This accuracy provides the influence of adverse lighting and weather effects on a quality of DT detection is studied. Experimental results show that the proposed method is feasible and effective for video-based dynamic texture analysis in varying illumination and weather conditions.

CONCLUSIONS
Real outdoor scenes may include the objects with dynamic behavior because of possible varying illumination, blurring, or weather conditions. Due to weather and lightning effects persistence averaged accuracy of DT recognition is decreasing to 70%. The minimum detection accuracy is 58 %. To compensate weather effect for damaged blocks the video images were processed by VirtualDub using a series of filters to suppress additive noise, blur, sharpness. After filters employment Dark Channel prior was used for haze removal. For illumination effects compensation and visual quality enhancement images the proposed algorithm employs MSRCR, Laplacian, Gaussian and median filters. This work focuses on detecting of dynamic textures in video sequences captured by surveillance cameras. In the aspect of preventing natural disasters, the work is focused on the detection of such types of DT as water, fire and smoke. After adverse lighting and weather effects compensation algorithm work true recognition of DT regions increases to 94,3%. This accuracy provides the influence of adverse lighting and weather effects on a quality of DT detection is studied. Experimental results show that the proposed method is feasible and effective for videobased dynamic texture analysis under bad weather conditions.