A pseudo Karnaugh mapping approach for datasets imbalance

The problem of dataset imbalance has raised a wide concern in many machine learning areas, but not in non-intrusive load monitoring, or load disaggregation. In this study, a pictorial evaluation method is proposed to representation the imbalance class distribution in datasets. We colored a Karnaugh maps according to the quantities of different variables combination to offer a visual impact to the whole dataset. After utilizing this method on a public dataset and its testing result, a clear imbalanced abundance in the dataset and an exciting performance have been found. A preliminary Python package to realize this mapping method has been uploaded on GitHub.


Introduction
In the field of Non-Intrusive Load Monitoring (NILM), most researchers focus on extracting the wave signal of a single appliance from aggregate public datasets, or recognizing the belongings of a single wave signal. There exists a vacancy of disaggregating a private-collected appliance from aggregate datasets.
NILM can easily and quickly identify load and decompose energy and monitor the use of appliances in a room. Ideally, it only needs data from smart meters to decompose the power consumption of appliances. The goal of NILM algorithm is to identify the power consumption of a single appliance based on the measurement data of the integrated power of the whole room. NILM can help the National Grid respond better to demand, store energy and develop and use more renewable energy, which is closely related to smart grid technology.

Unbalanced classification
As a multi-tag classification problem, NILM aims to determine whether a device is on or off within a given time step. However, in daily life, some electrical appliances are in a state of long-term switching on, and some electrical appliances are in a state of long-term switching off, which will be reflected in the data set and have a certain impact on the training model. There is always an imbalance in the data set in NILM, but appropriate techniques can be used to improve the robustness and effectiveness of the system. Zhang [1] adopts a stochastic sensitivity measure-based noise filtering and oversampling (SSMNFOS) training base classifier to form a multi-class set of neural network and perform multi-label classification for NILM. Compared with the famous oversampling method, the synthetic minority oversampling technique (SMOTE) has reached remarkable performance in the actual noninvasive load monitoring application. Fang [2] uses SMOTE for data expansion and for alleviating the severe imbalance of switch events to achieve better performance than the current approach, especially in multi-appliance switching scenario.

Variant Karnaugh Map utilizations
Apart from logistics analysis, a few attempts have utilized weighted Karnaugh Map [4] , and multi-valued Karnaugh Map [5] in medical data statistics.
Hassans [4] proposed a meta-data presentation approach that relies on human's pictorial perception capability. He present a methodological process of converting its equaling binary representation followed by furnishing values on to each cell in a Karnaugh Map instead of Boole. This pictorial graphing as shown in Figure 1, is expected to helped in simplifying the overview of meta-analysis and systemic reviews.
Rushdi [5] advocated and demonstrated a similar mapping approach of a multi-valued Karnaugh map, as Figure 2 shown, in qualitative comparative analysis for analyzing the clinical data in medical studies.
Other contributions like variable-entered Karnaugh map has been proposed in Banzhaf voting problems [6] and multi-state system analysis [7][8][9] also proposed a numbered Karnaugh Map to assist their researches.
These studies have greatly expanded the utilization of Karnaugh Map, but are still restricted to the quantity of variables included. The occupation of the map will be exponentially enlarged as variables increasing. Furthermore, human are far less sensitive to recognize numbers in compare to pure color.

Advanced mapping approach
While investigations on the presentation require of class imbalance in NILM dataset is still pending, and latest variant Karnaugh Maps fails encountering multiple variables, the pseudo Karnaugh Map (PKMap), a graph approach based on multi-valued Karnaugh Map, is proposed with the aim of filling this vacuum.
By counting, mapping and coloring the different appliance combination in a NILM dataset, the distribution of classes can be easily told with a single glance the colordisparate cells in the PKMap. And the arrangement of Karnaugh Map will help in pointing out to what appliances a specific cell in the PKMap referring.
Among the growing number of public datasets in NILM, the REFIT[10] offered by D. Murray is selected for demonstration since it has a long enough collecting period of two years, and have a cleaned edition released for easy acquiring. Especially, REFIT has 20 household for enough diversity, and involves 9 sub-metered power data in each house as well as the aggregate power data, which is exactly we need for mapping.
Along with the utility of PKMap in dataset, we also use this tool to mapping the testing consequence of metric, and some interesting discovers about the relationships between testing consequence and appliance combinations have been found. This implies the probability that characters in test dataset may impact the performance evaluation of a model. This may cause prejudice, bias or errors when cross-comparing in multiple models or datasets.
This study is expected to warn about the imbalance inherent in datasets to those who training, testing, or using NILM techniques for other purposes.

Assumptions and simplification
Some hypotheses are widely used in NILM researches, and others are arose for adaptive of the Karnaugh Map.  Power consumers are parallelly connected, which means all of them share almost the same voltage value at any timestamp. This assumption is naturally right as none electrical appliance is connected in series, and is the crucial prerequisite for the establishment of formula (1).
Where denotes the aggregate power of the timestamp , and ( ) denotes the individual power of the th appliance among appliances. This is also the common way how researchers treat aggregate power data and individual power data, but no one mention its precondition.
 Each appliance is treated as binary-state for simplify, despite the working pattern of appliance in NILM analysis has been categorized [11,12]. In this paper, a constant preset power threshold is applied to transfer valued power data into Boolean. Each appliance is treated as ON state while its power value surpass the threshold and OFF state for the rest situation. Multi-state will be considered in future works.  The plugging in or out behaviors during the dataset's collecting period are neglected. There is no guarantee of the complete non-occurrence of such incident, and such occurance do affect a lot in training and testing process. But we'll still treat any individual data channel as one identical appliance since this is beyond our discussion.

Pseudo Karnaugh Map
An overview of the main operational phases of PKMap generation is shown in Figure 3, and is briefly summarized as follows.

Binarization
As been expounded in Section 2, the power value will be converted into a Boolean value as we concern nothing but the ON/OFF states of appliances. Table 1 is a binarization example in which value 1, True in Boole, refers to `ON' state, while value 0, False in Boole, refers to `OFF' state. A pre-set threshold of 8 watts is taken because a general sleep power of 2 to 3 watts have been detected on some appliance, and the minimal working power value is around 30 watts. A small threshold will help in classifying an appliance into the appropriate statement.
In the 4th column in Table 1, the power value of appliance 3 is lower than 8, so it will be treated as 'OFF' state and be encoded as '0'.

Encoding
Each one of the 2^ possible cases of n appliances' state combinations is encoded to a n-digit binary number, and the value of th ( = 1,2, … , ) position starting from the left, the kth digit, represents the state of appliance at the corresponding timestamp. In another way, the encodings of each timestamp mean the combination of the binarized states of all n appliances.
The last column of Table 1 is the corresponding encodings of the former binarized load states in the same lines.

Counting
After encoding, we count the encoded data line by line, and store the statistical result into a dictionary, an embedded Python class. As can be told, a n-appliance-involved dataset will be transferred into a one-dimensional data of n-digits binary numbers with a length of 2 .
For advanced illustration of the PKMap, a fully controlled statistical result of a smaller scale would be welcomed. The Artificial Dataset, AD as being simplified, is manually created for this purpose. As been expanded in Table 2, it's a counting consequence of a hypothetical dataset with 4 appliances, which means it has 2 4 = 16 encodings of 4-digits each.  encodings  statistical  result  BIN HEX DEC  0000  0  00  0  0001  1  01  0  0011  3  03  0  0010  2  02  0  0110  6  06  0  0111  7  07  0  0101  5  05  0  0100  4  04  0  1100  C  12  255  1101  D  13  0  1111  F  15  604  1110  E  14  58852  1010  A  10  2771  1011  B  11  38  1001  9  09  40272  1000  8  08 1282001 The values in AD by each encoding are set according to the following details we found in REFIT datasets:  The "everything shut-down" state, which is encoded as 0 in decimal (whatever the quantity of appliances is), tends to obtain the highest frequency.  Some state combinations never show up through the whole collecting period of the dataset.
 Some appliances' working periods show a high correlation, while some do not.  The counts of different combinations varies at exponent level, and can differ from hundreds to millions.

Drawing
Inheriting the weighted Karnaugh map [4] and the multivalued Karnaugh map [5], we map the data with marker of variables on all the four margins. This is a more intuitionistic way than using the table-style headers. Like the Karnaugh Map, the PKMap comprises 2 cells, where n represent the number of concerned appliances in a dataset. Each cell in PKMap corresponds to a minterm denoted as , ∈ {0,1,2, … , 2 } in Boolean algebra. And it is made up of all the n variables meaning a specific state combination of all the n appliances.
The color map is select because it has multiple colors and is also capable in black and white printing without any loss suffered.
An example of PKMap with AD is shown in Figure 5, where we use a Karnaugh Map structure and fill in a heatmap instead of a textual table. Other visualization techniques of 2D array such as Hinton diagram are also welcomed.
This change is made to support large scale of data with far more than 2 6 = 64 cells. In fact, the data we explored from REFIT in the latter section has 2 9 = 512 cells. It's unnecessary and disorientating to expose a human before a textual table of that scale. These data will be observed visually with the help of visualization tools such as heatmap as we used.
From the PKMap presented in Figure 5, we can easily learn the basic distribution of the appliance working pattern and their relationship. All the three hypotheses put forward in Section 2 can be told.

Results & Discussion
To get an impression of PKMap to be expected in real datasets, we map the data in a public dataset REFIT [10] using the PKMap toolkit. The dataset is chosen since it has a clean edition released, and sub-metered power data included. The REFIT records 20 houses with 1 aggregate power sensor and 9 individual Appliance Monitors (IAMs) in each, and includes active power data in Watts with a 8second interval [10].

Logical expressions
Two Index Expression are introduced for better acknowledge the PKMap. One is the Peak Index Expression (PIE) representing the bright part of the map, and implies the abundant appliance combination in a dataset which is recommended for researching. The other is the Valley Index Expression (VIE) representing the dark part of the map, and implies the lack of appliance combinations in a dataset which require more attention. Concerning it's REFIT we are using, the PIE usually refers to a magnitude of millions (equivalent to the quantity of total number of the dataset), whereas the VIE usually refers to a magnitude lower than hundreds or even zero.

Anaylsis of House4
The PKMap of House4 in REFIT is presented in Figure 6 with cyan-square-highlighted PIE, from which we can get: VIE = A 4 ∩(A 8 ∪A 9 ) (2) The 4th appliance is always 'OFF' unless both the 8th appliance and 9th appliance are 'OFF'. They are mutually exclusive.

Analysis of House 7
The PKMap of House7 in REFIT is presented with PIE highlighted in Figure 7a, and with VIE highlighted in Figure 7b. We can get: (4) This indicates that the th ( ∈ {3,7,8}) appliance seldom work through the collecting period.
As can be noticed, the PIE and the VIE are roughly mutually Boole complementation as: VIE ≈ P̅ I̅ E̅ (5)

Analysis on House12
The PKMap of House12 in REFIT is presented in Figure 8, from where we can get: Extreme examples like this is not common. But it also warns us that will there be another one?

PKMap utilizations on testing result
When mapping the testing result, we bottom up the metric, because in PKMap, the bigger mean better.

Evaluation Metric
The Mean Absolute Error (MAE), a widely used metric in signal processing, as well as NILM, which is defined as: In which ( ) represents the estimated power consumption of target appliance on timestamp belonging to the appliance combination , while ( ) denoting the ground-truth power consumption offered by the original dataset. And is the total length of an single appliance combination .
Generally, metrics are applied to continue wave signals. Whereas, In this case, a test result will crack and be regathered by each appliance combination, and then be passed to evaluate the performance.

on appliance 1 in house7
By comparing Figure 9 and Figure 10, an exciting and counterintuitive discovery that the lack of data do not obstacle the performance of a model has been told. In most cases, the model performance evenly nice regardless the disappearance frequency of the relative combination. But in some classes, the metric become worse.
For further analysis, Figure 11 demonstrating the relationship between the class frequency (counts) and the model performances is brought out. Apart of the discovery above, it seems that those worse performances only happens when the counts is below 10 2 magnitude.

Conclusion
In this paper, we have introduced the PKMap, a Karnaugh Map arranged heatmap mapping approach, as an indicator of dataset imbalance and a navigator of dataset synthesis.
Benefitting from its Karnaugh Map arrangement, peaks and valleys of a PKMap can be easily pointed out, and be named by a Boolean expression. Then, load combination cases that appear more frequently or less frequently throughout a dataset can be directly told.
According to peaks and valleys in the PKMap, some traps in the dataset have been found while selecting loads to recognize and slicing dataset to training and testing in general NILM analyzing.
In the latest experiment, an exciting discovery has been found that the performance may not concern with the abundance of the input dataset, and the model do perform worse in some combinations. These findings deserve further studies.