Fuzzy model for assessing acquired competencies

. The recent pandemic and economic crises demonstrated that traditional educational approaches cannot face the today’s world challenges. A creative student-centered approach competency-based approach is becoming the leading educational trend around the world. In this regard, it is necessary to improve digital tools used at university to solve a wide range of new educational tasks. In this paper, a model that allows generating automatically the final testing of competencies is designed, along with the model of results evaluation with help of fuzzy logic. The presented model helps to assess the level of competencies through its components (understanding, having skill, ability), makes the assessment more flexible and adaptive and reduces teacher stress.


Introduction
Recent changes in the global economy present new challenges, among which digitalization, to the education system.Traditional educational methods cannot meet the demands of modern society, which was demonstrated by urgent situations such as the COVID-19 pandemic or today's economic turbulence.A creative student-centered approach that encourages critical thinking and the ability to solve non-standard problems is becoming the basis of the so-called competency-based approach, the leading educational trend around the world.Digital interactive tools, including test systems, widely promoted within this approach, are an integral part of our lives and have great potential in overcoming contemporary crises.
In this study, some terms are used, such as 'competence' (a know-how-to-act process [3], 'competency' (a dynamic characteristic that includes a complex set of formed competencies that allow a person to carry out professional activities and become an active member of society), 'competency-based approach' (an approach aimed at preparing graduates that possess a list of pre-defined measurable competencies based on knowledge, having skill, and ability).We use the working definitions of these concepts, to clarify all aspects of the definitions, as well as discussions about the classification of competencies and their elements, see articles [1,2,11,15,16], as well as numerous publications of researchers from around the world [12,17,25,30] and international organizations [4,13,21,27,29].
It is presumed that the multimedia learning environment is more flexible and attractive to students, and the digitalization of the educational process helps to minimize teacher stress and improves education quality.However, the potential of digital education is still not fully used, and often digitalization is interpreted as a mechanical process of replacing paper copies of educational materials with electronic ones.In this regard, it is necessary to improve digital tools, especially those used at university, in order to solve a wide range of new tasks that remain unresolved by classical educational approaches.

Methodology
This article presents a model for competence assessment based on fuzzy logic that helps to solve lots of complex tasks, as well as to make the learning process more efficient and save time and intellectual resources of teachers.First, it is important to consider the basic principles of the fuzzy logic approach.
In this research, the methodology for preparing educational material and the principles for the organization of test sets and specific techniques for the assessment of the level of competencies are described.To directly assess the level of competencies and assign a score to students, it is proposed a Fuzzy approach.
Since Fuzzy theory was introduced [35], Fuzzy logic is seen as a useful approach for dealing with ambiguity and uncertainty, and it has been successfully used in a variety of industries.Fuzzy Inference Systems (FIS) have been widely used by scholars in particular in the field of Decision Support Systems, and some scholars [23,24,28,31] have also used a Fuzzy logic approach to deal with students' degree of preparation assessment, so in this research it is been decided to explore FIS for the assessment the students' competencies as described.
A Fuzzy Inference System (FIS) consists of four main components, shown in Figure 1: ▪ Fuzzifier: acquires input crisp values from the real world and converts them into fuzzy numbers; ▪ Inference Engine: interacts with input data using a series of Rules; ▪ Defuzziyfier: converts the fuzzy values computed by Inference Engine in crisp outputs using IF…THEN rules.There are many degrees of freedom in the implementation of a FIS, such as the number and shape of member functions in both fuzzification and defuzzification phases and the inference model (Mamdani or Takagi-Sugeno).It has been chosen the Mamdani inference model because of the wide use of this model in Decision Support Systems [2].

Work programs
The content of any educational discipline is reflected in the work programs of the course.The program must not only indicate topics and activities but also give clear definitions of the competencies formed during the course with the help of particular educational activities (during a lecture/seminar/laboratory class/nontrivial problem solving on a specific topic, etc.).
In this paper, the concepts of final knowledge and competence (CL) are synonymous considered as a three-component model that includes: UNderstanding (UN), Having SKill (HSK), and ABility (AB), and the level of competence is a function (F) of these three components [8,9].
The educational process and, consequently, the work programs consist of theoretical and practical parts.For each of them, various tests/tasks are prepared to assess the level of competence development.In this paper, the term 'testing' is considered in a loose sense and includes tasks with open detailed answers.In theoretical testing, the levels of understanding and abilities are assessed.In the practical part, having skills is assessed (see Table 1).To generate the tests automatically when forming educational materials based on the work program, it is necessary to highlight the following sections: topic, text fragment, questions, question difficulty, answers (correct, there may be several), incorrect answers, comments (why only this or that answer is correct), the list of terms, the definition of terms, the list of references [10].The selected fragments are stored in the DB.Some parts of the cells of this database can be used later for automatic test generation (see Table 2).Questions/tasks should be assigned to each component of the competency.

Testing
For each component of competencies, different types of testing can be designed (see Table 3).The general test consists of tests for each competence component (UN, AB, HSK).This model is mainly aimed at assessing the final knowledge of the discipline, which implies a sufficiently large number of questions for testing.However, the model can also be used for midterm assessments with a sufficient number of questions.The test results for multiple-choice questions and one-word answers (a word or a phrase) are checked according to the pattern matching (correct answers) principle [11].Detailed open answers are checked with the help of scientific text analysis [6], namely the presence of keywords/phrases and logical-semantic graphs.Each correct answer gives a score (see Table 2), and some questions give a range of scores, for example, questions with multiple choice or open answers.
Each test, except for a topic (several topics at the same time), has the following indicators as well (see Figure 2): 1. Number of questions; 2. The percentage of questions for a particular competence (the total percentage should be 100%, i.e.UN% + AB% + HSK% = 100%); 3.For HSK tests, the time factor is involved.
The time factor is included when forming questions in the DB for each question for HSK (it is possible to indicate no time limit).The time factor can also be set for the entire test.After the end of the allocated time, the test form is closed, and it is not possible to answer the question(s) of the test.

Formation of test sets
The number of questions for each competence component is calculated as follows: The total number of questions * % of the competence component = number of questions for this part of the competence in the test.
The list of questions for tests is selected with the help of the RANDBETWEEN function.Then it is necessary to check if this question has not yet been used in the current testing [11].
An example of the formation of a part of the understanding test (other parts of the test are formed according to the same principle): Input conditions: 200 questions on given topics are stored for understanding in the DB, and the required percentage of questions is 25%.

Calculation of test results with help of fuzzy logic
To calculate the results of the part of the test, a percentage of the maximum possible score is calculated (see Table 7).

Question number in the test set for understanding
Score for correct answer Scores gained 1-5 4 Maximum score: 60 Total: 45; 75%

The fuzzy model
In the proposed model, CL, as described in the previous Section, is defined as a knowledge of the subject area, and it is represented as an F of three variables: ▪ Understanding (UN), ▪ Having Skill (HSK) and ▪ Ability (AB),

Results
The simulation has been conducted using the Fuzzy toolbox of MATLAB in the following configuration: ▪ MATLAB ver.9.13; ▪ Fuzzy Logic Toolbox, ver.3.0; ▪ PC HP Spectre Intel(R) Core(TM) i7-1195G7 @ 2.90GHz, 16 GB Ram; ▪ O.S. Windows 11 Home ed.The result of the Rules application on input data and the following defuzzification phase are visualized in Figure 8.It can be seen, that using as an example input data AB=70 (Medium), UN=80 (High), and HSK=80 (High), the result is CL=65 (Medium).It is also possible to display the FIS behaviour in a 3D diagram to highlight the sensitivity of the result for input variables (see Figure 9).Looking at Figure 9 it can be seen that, given this state of the parameters, it is possible to obtain high ratings only if the three separate scores are also high.
At this point, it is possible to test the FIS competency assessment system with the students' real data to analyse the results and tune the parameters.To obtain a smooth integration of the Assessment function into the University Information Systems, it could be interesting to investigate the use of Fuzzy Tahani methods [22,18] to perform Fuzzy analysis directly on a standard Relational DBMS, using regular SQL queries on University DB looking at recorded tests data.

Discussion
This paper presents a model for assessing competencies based on fuzzy logic, which solves many problems.Despite the growing number of works that study competencies and describe the competency-based approach to education, the question of the competencies content remains open.Nowadays, there are many approaches: some authors include only knowledge, skills, and abilities as components of competence, and others add one or more additional characteristics, for example, experience or personal characteristics [15].Another important issue is the way the level of competence development is assessed [1].This issue is relevant to all types of educational institutions.Many research papers offer an introspection (questionnaires) [5,19] (with quite a subjective result), assessment by experts (a very resource-consuming method) [17], other studies focus on a particular competence [26] (such approaches are not universal), and these methods do not help to solve the problem of general competence assessment.As discussed earlier, although there is no generally accepted opinion about the competence components, all authors agree that competence is a complex unit, which causes difficulties in assessment.The use of fuzzy logic to assess complex units is an alternative solution and can be applied to assess competencies using logical variables.Fuzzy logic tools have already been successfully used to solve complex problems, such as making administrative decisions in a university or selecting the best students [20].It is also important to consider the adaptability of the designed systems to the online environment.The recent pandemic has shown that linking and the ability to transfer processes to the online environment is necessary and very important [16], so most all modern systems need to be developed with the ability to adapt to the online environment.

Conclusion
Fuzzy logic tools are successfully used to solve problems in various areas, including when making decisions about final assessment of students.In this research, a system for assessing the formation of digital competencies was developed with the help of fuzzy logic algorithms.The developed model allows for assessment of the level of competencies in general through its components (understanding, having skill, ability).The presented model is implemented by the method of preparing educational material which highlights the main attributes of the text (title, key definitions, main text, questions, etc.) and also indicates the part of the competence to which the tasks on a given topic belong.Using the simplest functions (which are present even in MS Excel), test sets are formed.Within the sets, all questions are unique and do not repeat; in addition, it is possible to set the specific percentage of questions in the test (% questions for understanding, % questions for skill, % questions for ability), and for some questions, you can set a time limit.The calculation of the final result of the formation of competencies is implemented using fuzzy logic, which makes the assessment more flexible and adaptive, taking into account the individual characteristics of students.Another clear advantage of the presented system is that it helps to reduce teacher stress.The whole process and the results of educational activities are recorded in the DB, which makes it possible to analyze the system operation, including the educational process.

Fig. 1 .
Fig. 1.The main components of a fuzzy inference system.

Table 1 .
An example of a fragment of a work program.

Table 2 .
DB fragment of a work program.

Table 3 .
Test types used to identify the level of competencies.

Table 5 .
Control the question numbers that may repeat in the test set.